A General Framework For Lifelong Localization and Mapping in Changing Environment
A General Framework For Lifelong Localization and Mapping in Changing Environment
Environment
Min Zhao1 , Xin Guo1 , Le Song1 , Baoxing Qin1 , Xuesong Shi2 , Gim Hee Lee3 , Guanghui Sun4
The short-term memory map is updated after each localiza- method with submap trimming and resulting in the lifelong
tion step and the long-term map is stored and evaluated only SLAM pipeline as below.
after a run or after a day. Walcott et al. [10] presents Dynamic
Pose Graph SLAM (DPG-SLAM), an algorithm designed to III. S YSTEM OVERVIEW
remove the inactive scans and add the new scans to the map
A. System Structure
by detecting the label changes after each pass. However,
they maintain a pose graph which is comprised of all the This section describes the architecture of our proposed
scan nodes that will not be optimized in real-time because framework as shown in Fig. 2. Our system consists of three
of high computation complexity. Ding et al. [11] proposes subsystems: local LiDAR odometry (LLO), global LiDAR
a joint framework for vehicle localization that adaptively matching (GLM) and pose graph refinement (PGR). The
incorporate results from both the LiDAR inertial odometry role of LLO is the same with [2]: building a succession of
and global matching modules. To overcome the failure of submaps which are locally consistent. The GLM subsystem
global matching, they also propose an environmental change is responsible for calculating the relative constraint between
detection method to find out when and what portion of the the incoming scans and global submaps, as well as inserting
map should be promptly updated. the submaps and constraints into PGR. PGR is the most
In contrast to them, for the purpose of decreasing com- important part of our system. It collects the submaps from
putational complexity, our method do not include the map LLO and constraints from GLM, trims the old submaps saved
change detection module. The map data structure of ours in old session, and executes pose graph sparsification and
is comprised of several submaps and we replace the old optimization. These modules will be discussed in detail in
submaps with new ones directly. Section IV.
B. Pose Graph Sparsification
B. Notation
Another task for lifelong SLAM is to sparse the pose graph
while minimizing the loss of information when the robots In this paper, we use X , M, Z to represent the node
revisit already mapped area. The computation complexity poses, submap poses and relative constraints, respectively.
and memory usage should be proportional to the scale of Due to multi-session localization, each session consists of
explored space rather than to the localization duration. To several nodes, submaps and constraints. X is the collection
this end, pose graph sparsification methods are proposed. of xsi , which i is the index of the node pose and s means
Kretzschmar et al. [12] uses the expected information gain, the session ID. Analogously, M is composed by msi . It
which is defined as the expected reduction of uncertainty in should be emphasized that there are three types of constraints
the belief of the robot caused by an observation, to reduce the in our system: node-to-submap zmix ij , node-to-node zij
node
redundant nodes and keep constant size pose graph. In [13], and submap-to-submap zmap ij . These constraints take the
the authors propose the use of a Chow-Liu tree (CLT) [14] form of relative poses and associated covariance matrice
to approximate the individual elimination cliques as sparse Ωmix
ij , Ωij
node
and Ωmap mix
ij . For example, zis j s0 is the relative
0
tree structures and seek to minimize the loss of information, transform between the node pose xsi and submap pose msj .
restricting the size of the pose graph. Carlevaris et al. [14] The adjacent two node poses are related by the odometry
0
introduces a generic factor-based method for node removal constraint, e.g. zodom
i s j s0
controls xsi and xsj . Specifically, the
in factor-graph SLAM, which is referred to as generic linear pose and constraint is represented by a rotation R ∈ SO(2)
constraint(GLC). It operates on the Markov blanket of a and a translation t ∈ R2 because of the flat terrain of
marginalized node and compute a sparse approximation of indoor environment. Nonetheless, our framework can be
the blanket. In this work, we combine the CLT sparsification easily extended to SE(3).
C. Problem Definition graph, the relevant stale submaps, nodes and constraints are
Given the poses and relative measurement constraints, removed. The remaining submaps from PGR are transmitted
the pose graph optimization is formulated as a maximum to the global submap database for the subsequent localization
a posteriori probability (MAP) estimation problem. We then task. We call this process "Map Updating".
convert the MAP problem into a least square problem with
the following cost function:
X ∗ , M∗ = argmin J(X , M, Z). (1)
X ,M
! "#$%'
! "#$%(
)*+%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%),+%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%)-+
!"#$%&'()*
+(,*&'()*&
-,(#*./0&1(2)./$32.
1(2)./$32.&"*.4**2&+(,*&$2,& !"#$%
1(2)./$32.&"*.4**2&+(,*)
1(2)./$32.&"*.4**2& !"#$%)
-/3532&'()*
%%%).+%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%)!+
Fig. 4. An example to illustrate the graph sparsification based on the Chow–Liu maximum-mutual-information spanning tree. (a): the original pose graph
without submap removal. (b): the central submap m11 is selected to be trimmed along with its nodes: x11 and x12 (blue dotted rectangle). The red dotted
circles highlight the relevant submaps and nodes that have constraints with blue rectangle. (c): Belief after having marginalized out m11 , x11 and x12 . The
former neighbors of them form an elimination clique making the graph dense. (d): Belief resulting from Chow-Liu tree approximation of the elimination
clique. (e): Merge the elimination clique into original pose graph.
Fig. 6. Results of map change after one month running in market. (a): The pre-built map. (b): The updated map. (c): Comparison(define to execute
such operation) of (a) and (b). The blue points are obstacles that remain unchanged. The light green points are obstacles which have been removed and
red points are newly added obstacles.
3 ! 4 5 6 7 8 9 3: 33 3! 34 35 36 37 38 39 3 !: !3 !! !4 !5 !6
37;!6"
#$#%&'()
87;99"
1#-%./-0/$.'+12
:;47" 8!"
*+',-%./-0/$.'+12
!"
Fig. 7. Localization testing results with 25 sets data from market. The percentage value on the top left of each algorithm is MRCL and on the bottom
right is CRI.
TABLE I
AVERAGE MATCHING SCORE COMPARISON
Data Sequence 1 2 3 4 5 6 7 8 9 10 11 12 13 14 15 16 17 18 19 20 21 22 23 24 25
[2] 0.73 0.72 0.76 0.76 0.72 0.72 0.68 0.67 0.71 0.62 0.52 - 0.68 0.18 - 0.70 0.43 - 0.16 - 0.44 - - 0.42 -
Ours 0.74 0.70 0.78 0.75 0.74 0.74 0.73 0.72 0.72 0.72 0.69 0.68 0.77 0.48 - 0.69 0.59 0.74 0.68 0.68 0.72 0.26 - 0.72 0.58
from t0 to tmax , we define: represents the start of a data sequence. The blue line
correct_mileage(t0 , tmax ) segments start from the blue dots means robot initializes
M RCL = , (8) successfully and runs for a while. The length of blue
total_mileage(t0 , tmax )
line segments is proportional to a correct mileage. We
where correct_mileage() is used to calculate the tested 25 data sequences with three algorithms: original
mileage of correct trajectory and total_mileage() is odometry output, method from [2] and ours. The CRI
used to calculate the total mileage. and MRCL from Fig. 7 demonstrate that our method is
Average Matching Score(AMS). Our method of PGR more robust enough for localization in the challenging
module is the same with “Global SLAM” module in [2], environment compared with [2]. However, it should be
where building constraints between nodes and submaps noted that, in sequence 15 and 23, our method failed
relied on a “branch and bound (BNB)” mechanism to to do initialization. This is because the environment
work at different grid resolutions. Therefore, we use is full of pedestrians and the score of scan-matching
the BNB score to define whether a node is correctly does not exceed initialization BNB score. The AMS
matched against a submap. Usually, we set a minimum comparison result is listed in Table I. The results show
score(in our experiment, the score is 0.5) to accept a that with longtime map updating, our method achieves
good enough proposal and fed it into PGR. Otherwise, higher matching score for robust constraints building
we discard this constraint candidate. Formally, we de- based on fresh map.
fine a average score of matching as: We display the trajectory comparison from 17th data
PM −1 sequence as shown in Fig. 8. The blue line plots the
si
AM S = i=0 , (9) fusion result of wheel encoder and IMU. The green line
M
is from our method and the red line is the result of [2]. It
where M is the number of matching. si is the i-th BNB
demonstrates that the trajectory of ours is smoother than
score.
[2]. The reason is that, with map updating and higher
The results are visualized in Fig. 7, with blue line
matching score, the valid constraint measurements are
segments indicating the correct localization trajectory
much more dense than [2], which only computes con-
and blue dots indicating whether the initialization is
straints based on the stale map.
successful or not. Each black dot on the top line
odometry
no map updating[2]
20 with map updating
15
y[m]
10
−15 −10 −5 0 5 10 15 Fig. 10. Lifelong SLAM graph complexity. The number of nodes and
x[m] submaps remains roughly constant after 15 times of map updating.
Fig. 12. Examples of environment change and corresponding map updating experiment. The result in (a), (b) and (c) are collected from market. And (d),
(e) and (f) are from garage.
IEEE/RSJ International Conference on Intelligent Robots and Systems “Generic node removal for factor-graph SLAM,” IEEE Transactions
(IROS 2003)(Cat. No. 03CH37453). Vol. 3. IEEE, 2003. on Robotics 30.6 (2014): 1371-1385.
[2] W. Hess, D. Kohler, H. Rapp, and D. Andor, “Real-time loop closure [15] Bürki, Mathias, et al. “Map management for efficient long-term visual
in 2D LiDAR SLAM,” in Proceedings of the IEEE International Con- localization in outdoor environments,” 2018 IEEE Intelligent Vehicles
ference on Robotics and Automation (ICRA), 2016, pp. 1271–1278. Symposium (IV). IEEE, 2018.
[3] Censi, Andrea. "An ICP variant using a point-to-line metric." 2008 [16] Tipaldi, Gian Diego, Daniel Meyer-Delius, and Wolfram Burgard.
IEEE International Conference on Robotics and Automation. Ieee, “Lifelong localization in changing environments,” The International
2008. Journal of Robotics Research 32.14 (2013): 1662-1678.
[4] Olson, Edwin B. "Real-time correlative scan matching." 2009 IEEE [17] Labbé, Mathieu, and François Michaud. “Long-term online multi-
International Conference on Robotics and Automation. IEEE, 2009. session graph-based SPLAM with memory management,” Au-
[5] Cadena C, Carlone L, Carrillo H, Latif Y, Scaramuzza D, Neira J, tonomous Robots 42.6 (2018): 1133-1150.
Reid I and Leonard JJ, “Past, present, and future of simultaneous [18] Egger, Philipp, et al. “Posemap: Lifelong, multi-environment 3d lidar
localization and mapping: Toward the robustperception age),” IEEE localization,” 2018 IEEE/RSJ International Conference on Intelligent
Transactions on Robotics 32(6): 1309–1332. Robots and Systems (IROS). IEEE, 2018.
[19] S. Agarwal, K. Mierle, and Others, “Ceres solver,” http://ceres-
[6] Shan, Tixiao, and Brendan Englot. “Lego-loam: Lightweight and
solver.org.
ground-optimized lidar odometry and mapping on variable terrain,”
2018 IEEE/RSJ International Conference on Intelligent Robots and
Systems (IROS). IEEE, 2018.
[7] Mur-Artal, Raul, and Juan D. Tardós. “Orb-slam2: An open-source
slam system for monocular, stereo, and rgb-d cameras." IEEE Trans-
actions on Robotics 33.5 (2017): 1255-1262.
[8] P. Biber and T. Duckett, “Dynamic maps for long-term operation of
mobile service robots,”in RSS, 2005.
[9] Biber, Peter, and Tom Duckett. “Experimental analysis of sample-
based maps for long-term SLAM,” The International Journal of
Robotics Research 28.1 (2009): 20-33.
[10] Walcott-Bryant, Aisha, et al. “Dynamic pose graph SLAM: Long-term
mapping in low dynamic environments,” 2012 IEEE/RSJ International
Conference on Intelligent Robots and Systems. IEEE, 2012.
[11] Ding, Wendong, et al. “LiDAR Inertial Odometry Aided Robust
LiDAR Localization System in Changing City Scenes,” Proceedings
of the IEEE International Conference on Robotics and Automation
(ICRA). IEEE (May 2020).
[12] Kretzschmar, Henrik, Giorgio Grisetti, and Cyrill Stachniss. “Life-
long map learning for graph-based slam in static environments,” KI-
Künstliche Intelligenz 24.3 (2010): 199-206.
[13] Kretzschmar, Henrik, and Cyrill Stachniss. “Information-theoretic
compression of pose graphs for laser-based SLAM,” The International
Journal of Robotics Research 31.11 (2012): 1219-1230.
[14] Carlevaris-Bianco, Nicholas, Michael Kaess, and Ryan M. Eustice.