Resumen de 3d scene understanding for vision and robotics
In order to have autonomous agents, we need to extract information from the raw, sensor, data.
This is perhaps the theme around which fields like robotics, computer vision and artificial intelligence revolve. This raw sensor data does not exist in isolation and is bounded by the physics of the environment in which the agent exists. Scene geometry is one of the fundamental component for extracting physically plausible information from the raw sensor data. There is still a long way to go for extracting the geometry for all types of scenes (underwater, blood stream, caves, jungles). However, the man-made world (cities, indoors) we live is highly structured. This work follows the research line of solving various robotics and computer vision tasks keeping in view this man-made world.
• How to generate novel views from a single image without having the 3D model? We show that, at least in man-made world, metrically rectified novel views are generated without requiring the scene geometric layout and even for the non-cuboid scenes. These novel views help training visual classifiers given few training examples (small data domain).
They also enable already trained classifiers to cope with the missing views (blind spots) at training time. Finally, these novel views lead to unsupervised 3D scene understanding.
• Does hearing help better seeing? We fuse audio acoustic cues with visual data to estimate the scene layout of the cuboid rooms.
• We introduce this room layout reasoning in classic robot mapping (SLAM) method. This ability to segment entities in different rooms helps to avoid the confusion of looking for occluded entities, which are currently in the other room. This leads to efficient performance.
• Finally we introduce the scene characteristics (priors) to extract dense 3D model of the scene in complex robotics scenarios of low-parallex and low-texture.
