86	S. Se and N. Pears

Fig. 2.25 3D reconstruction of a building on the ground using video (left) and using infra-red video (right) captured by an UAV (Unmanned Aerial Vehicle). Figure courtesy of [50]

ological features and consecutive 3D models of a mine tunnel created as the mine advances [48].

Airborne surveillance and reconnaissance are essential for successful military missions. Unmanned Aerial Vehicles (UAVs) are becoming the platform of choice for such surveillance operations and video cameras are among the most common sensors onboard UAVs. Photo-realistic 3D models can be generated from UAV video data to provide situational awareness as it is easier to understand the scene by visualizing it in 3D. The 3D model can be viewed from different perspectives and allow distance measurements and line-of-sight analysis. Figure 2.25 shows a 3D reconstruction of a building on the ground using video and infra-red video captured by an UAV [50]. The photo-realistic 3D models are geo-referenced and can be visualized in 3D Geographical Information System (GIS) viewers such as Google Earth.

2.9.3 Mobile Robot Localization and Mapping

Mobile robot localization and mapping is the process of simultaneously tracking the position of a mobile robot relative to its environment and building a map of the environment. Accurate localization is a prerequisite for building a good map and having an accurate map is essential for good localization. Therefore, Simultaneous Localization and Mapping (SLAM) is a critical underlying capability for successful mobile robot applications. To achieve a SLAM capability, high resolution passive vision systems can capture images in milliseconds, hence they are suitable for moving platforms such as mobile robots.

Stereo vision systems are commonly used on mobile robots, as they can measure the full six degrees of freedom (DOF) of the change in robot pose. This is known as visual odometry. By matching visual landmarks between frames to recover the robot motion, visual odometry is not affected by wheel slip and hence is more accurate than the wheel-based odometry. For outdoor robots with GPS receivers, visual odometry can also augment the GPS to provide better accuracy, and it is also valuable in environments where GPS signals are not available.

2 Passive 3D Imaging

87

Fig. 2.26 (a) Autonomous rover on a gravel test site with obstacles (b) Comparison of the estimated path by SLAM, wheel odometry and DGPS (Differential GPS). Figure courtesy of [1]

Unlike in 3D modeling where correlation-based dense stereo matching is typically performed, feature-based matching is sufficient for visual odometry and SLAM; indeed, it is preferable for real-time robotics applications, as it is computationally less expensive. Such features are used for localization and a feature map is built at the same time.

The MERs Opportunity and Spirit are equipped with visual odometry capability [32]. An update to the rover’s pose is computed by tracking the motion of autonomously-selected terrain features between two pairs of stereo images. It has demonstrated good performance and successfully detected slip ratios as high as 125 % even while driving on slopes as high as 31 degrees.

As SIFT features [28] are invariant to image translation, scaling, rotation, and fairly robust to illumination changes and affine or even mild projective deformation, they are suitable landmarks for robust SLAM. When the mobile robot moves around in an environment, landmarks are observed over time but from different angles, distances or under different illumination. SIFT features are extracted and matched between the stereo images to obtain 3D SIFT landmarks which are used for indoor SLAM [49] and for outdoor SLAM [1]. Figure 2.26 shows a field trial of an autonomous vehicle at a gravel test site with obstacles and a comparison of rover localization results. It can be seen that the vision-based SLAM trajectory is much better than the wheel odometry and matches well with the Differential GPS (DGPS).

Monocular visual SLAM applications have been emerging in recent years and these only require a single camera. The results are up to a scale factor, but can be scaled with some prior information. MonoSLAM [14] is a real-time algorithm which can recover the 3D trajectory of a monocular camera, moving rapidly through a previously unknown scene. The SLAM methodology is applied to the vision domain of a single camera, thereby achieving real-time and drift-free performance not offered by other structure from motion approaches.

Apart from localization, passive 3D imaging systems can also be used for obstacle/hazard detection in mobile robotics. Stereo cameras are often used as they can recover the 3D information without moving the robot. Figure 2.27 shows the stereo