Loading...
2 results
Search Results
Now showing 1 - 2 of 2
- Artificial Vision for HumansPublication . Gomes, João Gaspar Ramôa; Alexandre, Luís Filipe Barbosa de Almeida; Mogo, Sandra Isabel PintoAccording to the World Health Organization and the The International Agency for the Prevention of Blindness, 253 million people are blind or vision impaired (2015). One hundred seventeen million have moderate or severe distance vision impairment, and 36 million are blind. Over the years, portable navigation systems have been developed to help visually impaired people to navigate. The first primary mobile navigation system was the white-cane. This is still the most common mobile system used by visually impaired people since it is cheap and reliable. The disadvantage is it just provides obstacle information at the feet-level, and it isn’t hands-free. Initially, the portable systems being developed were focused in obstacle avoiding, but these days they are not limited to that. With the advances of computer vision and artificial intelligence, these systems aren’t restricted to obstacle avoidance anymore and are capable of describing the world, text recognition and even face recognition. The most notable portable navigation systems of this type nowadays are the Brain Port Pro Vision and the Orcam MyEye system and both of them are hands-free systems. These systems can improve visually impaired people’s life quality, but they are not accessible by everyone. About 89% of vision impaired people live in low and middleincome countries, and the most of the 11% that don’t live in these countries don’t have access to a portable navigation system like the previous ones. The goal of this project was to develop a portable navigation system that uses computer vision and image processing algorithms to help visually impaired people to navigate. This compact system has two modes, one for solving specific visually impaired people’s problems and the other for generic obstacle avoidance. It was also a goal of this project to continuously improve this system based on the feedback of real users, but due to the pandemic of SARS-CoV-2 Virus I couldn’t achieve this objective of this work. The specific problem that was more studied in this work was the Door Problem. This is, according to visually impaired and blind people, a typical problem that usually occurs in indoor environments shared with other people. Another visually impaired people’s problem that was also studied was the Stairs Problem but due to its rarity, I focused more on the previous one. By doing an extensive overview of the methods that the newest navigation portable systems were using, I found that they were using computer vision and image processing algorithms to provide descriptive information about the world. I also overview Ricardo Domingos’s work about solving the Door Problem in a desktop computer, that served as a baseline for this work. I built two portable navigation systems to help visually impaired people to navigate. One is based on the Raspberry Pi 3 B+ system and the other uses the Nvidia Jetson Nano. The first system was used for collecting data, and the other was the final prototype system that I propose in this work. This system is hands-free, it doesn’t overheat, is light and can be carried in a simple backpack or suitcase. This prototype system has two modes, one that works as a car parking sensor system which is used for obstacle avoidance and the other is used to solve the Door Problem by providing information about the state of the door (open, semi-open or closed door). So, in this document, I proposed three different methods to solve the Door Problem, that use computer vision algorithms and work in the prototype system. The first one is based on 2D semantic segmentation and 3D object classification, it can detect the door and classify it. This method works at 3 FPS. The second method is a small version of the previous one. It is based on 3D object classification, but it works at 5 to 6 FPS. The latter method is based on 2d semantic segmentation, object detection and 2d image classification. It can detect the door, and classify it. This method works at 1 to 2 FPS, but it is the best in terms of door classification accuracy. I also propose a Door dataset and a Stairs dataset that has 3D information and 2d information. This dataset was used to train the computer vision algorithms used in the proposed methods to solve the Door Problem. This dataset is freely available online for scientific proposes along with the information of the train, validation, and test sets. All methods work in the final prototype portable system in real-time. The developed system it’s a cheaper approach for the visually impaired people that cannot afford the most current portable navigation systems. The contributions of this work are, the two develop mobile navigation systems, the three methods produce for solving the Door Problem and the dataset built for training the computer vision algorithms. This work can also be scaled to other areas. The methods developed for door detection and classification can be used by a portable robot that works in indoor environments. The dataset can be used to compare results and to train other neural network models for different tasks and systems.
- Real-time 2D–3D door detection and state classification on a low-power devicePublication . Ramôa, João Gaspar; Lopes, Vasco; Alexandre, Luís; Mogo, SandraIn this paper, we propose three methods for door state classifcation with the goal to improve robot navigation in indoor spaces. These methods were also developed to be used in other areas and applications since they are not limited to door detection as other related works are. Our methods work ofine, in low-powered computers as the Jetson Nano, in real-time with the ability to diferentiate between open, closed and semi-open doors. We use the 3D object classifcation, PointNet, real-time semantic segmentation algorithms such as, FastFCN, FC-HarDNet, SegNet and BiSeNet, the object detection algorithm, DetectNet and 2D object classifcation networks, AlexNet and GoogleNet. We built a 3D and RGB door dataset with images from several indoor environments using a 3D Realsense camera D435. This dataset is freely available online. All methods are analysed taking into account their accuracy and the speed of the algorithm in a low powered computer. We conclude that it is possible to have a door classifcation algorithm running in real-time on a low-power device.