R$^3$LIVE: A Robust, Real-time, RGB-colored, LiDAR-Inertial-Visual tightly-coupled state Estimation and mapping package

1. Introduction

R3LIVE is a novel LiDAR-Inertial-Visual sensor fusion framework, which takes advantage of measurement of LiDAR, inertial, and visual sensors to achieve robust and accurate state estimation. R3LIVE is built upon our previous work R2LIVE, is contained of two subsystems: the LiDAR-inertial odometry (LIO) and the visual-inertial odometry (VIO). The LIO subsystem (FAST-LIO) takes advantage of the measurement from LiDAR and inertial sensors and builds the geometric structure of (i.e. the position of 3D points) global maps. The VIO subsystem utilizes the data of visual-inertial sensors and renders the map’s texture (i.e. the color of 3D points).

1.1 Our accompanying videos

Our accompanying videos are now available on YouTube (click below images to open) and Bilibili^{1, 2}.

1.2 Our associate dataset: R3LIVE-dataset

Our associate dataset R3LIVE-dataset that use for evaluation is also available online. You can access and download our datasets via this Github repository.

1.3 Our open-source hardware design

All of the mechanical modules of our handheld device that use for data collection are designed as FDM printable, with the schematics of the design are also open-sourced in this Github repository.

2. R3LIVE Features

2.1 Strong robustness in various challenging scenarios

R3LIVE is robust enough to work well in various of LiDAR-degenerated scenarios (see following figures):

And even in simultaneously LiDAR degenerated and visual texture-less environments (see Experiment-1 of our paper).

2.2 Real-time RGB maps reconstruction

R3LIVE is able to reconstruct the precise, dense, 3D, RGB-colored maps of surrounding environment in real-time (watch this video).

2.3 Ready for 3D applications

To make R3LIVE more extensible, we also provide a series of offline utilities for reconstructing and texturing meshes, which further reduce the gap between R3LIVE and various 3D applications (watch this video).

3. Acknowledgments

In the development of R3LIVE, we stand on the shoulders of the following repositories:

1. R2LIVE: A robust, real-time tightly-coupled multi-sensor fusion package.
2. FAST-LIO: A computationally efficient and robust LiDAR-inertial odometry package.
3. ikd-Tree: A state-of-art dynamic KD-Tree for 3D kNN search.
4. livox_camera_calib: A robust, high accuracy extrinsic calibration tool between high resolution LiDAR (e.g. Livox) and camera in targetless environment.
5. LOAM-Livox: A robust LiDAR Odometry and Mapping (LOAM) package for Livox-LiDAR.
6. openMVS: A library for computer-vision scientists and especially targeted to the Multi-View Stereo reconstruction community.
7. VCGlib: An open source, portable, header-only Visualization and Computer Graphics Library.
8. CGAL: A C++ Computational Geometry Algorithms Library.