Combined Lane Mapping Using a Mobile Mapping System
Abstract
:1. Introduction
- Road-surface extraction: After the point cloud is gridded, the sudden change in the normal vector is used to locate the curb grids on both sides of the trajectory. Road edges are fitted by the curb grids and used to segment the road points. The inconsistency of the reflective intensity of the road points is corrected to facilitate the subsequent lane extraction.
- Lane-marking extraction: The 3D road points are mapped into a 2D image. A self-adaptive thresholding method is then developed to extract lane markings from the image.
- Lane mapping: Lane markings in a local section are clustered and fitted into lane lines. LiDAR-guided textural saliency analysis is proposed to validate the intensity contrast around the lane lines in the MMS images. Global post-processing is finally adopted to complement the missing lane lines caused by local occlusion.
2. Methods
2.1. Road-Surface Extraction
2.1.1. Preprocessing
2.1.2. Road Extraction Based on the Normal Vector
2.1.3. Intensity Correction of Road Points
2.2. Lane-Marking Extraction
2.2.1. 3D-2D Correspondence
2.2.2. Self-Adaptive Thresholding
Algorithm 1 Self-adaptive thresholding |
|
2.2.3. Shape Analysis: Removal of False Positives
- A single marking of a zebra crossing is similar to a lane marking, but a whole zebra crossing can be distinguished by its crossing orientation, as shown in Figure 11a,b. First, a closing operation is performed over the binary image to obtain the mask image . Next, connected component labeling is performed on , and the connected components are denoted . Finally, the minimum area rectangle of each is calculated and the acute angle between the long axis and the trajectory line is obtained. with greater than will be regarded as a zebra crossing and removed, i.e., .
- Road arrows and other road markings, according to the Standards of Road Traffic Marking (2009), have specific lengths and widths. First, connected component labeling is performed on the binary image , and the connected components are denoted . Second, the length and width of the minimum area rectangle of each are compared with the length and the width of the standard road markings , respectively. If both conditions of the length and width, i.e., and , where and are the tolerance of the length and width, respectively, are satisfied, is regarded as and removed . In the experiment, we set (Figure 11f,g).
2.3. Lane Mapping
2.3.1. Trajectory-Based Local Lane Line Fitting
- The binary image is first skeletonized to locate the center of the lane markings and reduce the computational burden.
- The trajectory points (n points) are connected sequentially to form line segments .
- As shown in Figure 12a, for each , a rectangular buffer is generated and moved along the orthogonal direction of with a certain step length. The step count and the number of pixels that fall within , which are denoted d and s, respectively, are recorded.
- If the lane-marking pixels fall into , s will be larger than the threshold and exhibit a peak. The peak buffer is denoted , where j is the number of peaks.
- For all , we find with its step count d and the pixels that fall within it.
- The DBSCAN algorithm is used to cluster all peak buffers according to their step counts d. The peak buffers of the same lane line will be clustered into a group due to their similar deviation from the trajectory. Figure 12b shows the clustering in an image.
- The pixels in the peak buffers of a group are fitted by a quadratic polynomial. The 3D points of lane lines are sampled from the polynomial every 0.5 m.
2.3.2. Local Lane Line Inference
2.3.3. LiDAR-Guided Textural Saliency Analysis
2.3.4. Global Post-Processing
3. Results
3.1. Experimental Data
3.2. Evaluation Method
3.3. Experimental Results and Evaluation
3.3.1. Typical Cases
3.3.2. Overall Mapping Results
3.3.3. Evaluation and Error Analysis
4. Discussion
4.1. Discussion of Textural Saliency Analysis
4.2. Discussion of Global Post-Processing
5. Conclusions
Author Contributions
Funding
Acknowledgments
Conflicts of Interest
References
- Levinson, J.; Askeland, J.; Becker, J.; Dolson, J.; Held, D.; Kammel, S.; Kolter, J.Z.; Langer, D.; Pink, O.; Pratt, V.; et al. Towards fully autonomous driving: Systems and algorithms. In Proceedings of the 2011 IEEE Intelligent Vehicles Symposium (IV), Baden-Baden, Germany, 5–9 June 2011; pp. 163–168. [Google Scholar]
- Urmson, C.; Anhalt, J.; Bagnell, D.; Baker, C.; Bittner, R.; Clark, M.N.; Dolan, J.; Duggins, D.; Galatali, T.; Geyer, C.; et al. Autonomous driving in Urban environments: Boss and the Urban Challenge. J. Field Robot. 2008, 25, 425–466. [Google Scholar] [CrossRef]
- Levinson, J.; Montemerlo, M.; Thrun, S. Map-Based Precision Vehicle Localization in Urban Environments. In Proceedings of the Robotics: Science and Systems 2007, Atlanta, GA, USA, 27–30 June 2007; p. 1. [Google Scholar]
- Levinson, J.; Thrun, S. Robust vehicle localization in urban environments using probabilistic maps. In Proceedings of the 2010 IEEE International Conference on Robotics and Automation, Anchorage, AK, USA, 3–8 May 2010; pp. 4372–4378. [Google Scholar]
- Tao, Z.; Bonnifait, P.; Fremont, V.; Ibañez-Guzman, J. Mapping and localization using GPS, lane markings and proprioceptive sensors. In Proceedings of the IEEE/RSJ International Conference on Intelligent Robots and Systems (IROS 2013), Tokyo, Japan, 3–7 November 2013; pp. 406–412. [Google Scholar]
- Schreiber, M.; Knoppel, C.; Franke, U. LaneLoc: Lane Marking based Localization using Highly Accurate Maps. In Proceedings of the 2013 IEEE Intelligent Vehicles Symposium (IV), Gold Coast, Australia, 23–26 June 2013; pp. 449–454. [Google Scholar]
- Gu, Y.; Wada, Y.; Hsu, L.; Kamijo, S. Vehicle self-localization in urban canyon using 3D map based GPS’ positioning and vehicle sensors. In Proceedings of the 2014 International Conference on Connected Vehicles and Expo (ICCVE), Vienna, Austria, 3–7 November 2014; pp. 792–798. [Google Scholar]
- Oliver, N.; Pentland, A.P. Driver Behavior Recognition and Prediction in a SmartCar. Proc. SPIE 2015, 4023, 280–290. [Google Scholar]
- Doshi, A.; Trivedi, M.M. Tactical driver behavior prediction and intent inference: A review. In Proceedings of the 14th Internetional IEEE Conference on Intelligent Transpotation Systems, Washinton, DC, USA, 5–7 October 2011; pp. 1892–1897. [Google Scholar]
- Ziegler, J.; Bender, P.; Schreiber, M.; Lategahn, H.; Strauss, T.; Stiller, C.; Dang, T.; Franke, U.; Appenrodt, N.; Keller, C.G.; et al. Making bertha drive-an autonomous journey on a historic route. IEEE Intell. Transp. Syst. Mag. 2014, 6, 8–20. [Google Scholar] [CrossRef]
- Li, X.; Sun, Z.; Cao, D.; He, Z.; Zhu, Q. Real-Time Trajectory Planning for Autonomous Urban Driving: Framework, Algorithms, and Verifications. IEEE/ASME Trans. Mechatron. 2016, 21, 740–753. [Google Scholar] [CrossRef]
- Rogers, S. Creating and evaluating highly accurate maps with probe vehicles. In Proceedings of the 2000 IEEE Intelligent Transportation Systems, Dearborn, MI, USA, 1–3 October 2000; pp. 125–130. [Google Scholar] [Green Version]
- Betaille, D.; Toledo-Moreo, R. Creating enhanced maps for lane-level vehicle navigation. IEEE Trans. Intell. Transp. Syst. 2010, 11, 786–798. [Google Scholar] [CrossRef]
- Chen, A.; Ramanandan, A.; Farrell, J.A. High-precision lane-level road map building for vehicle navigation. In Proceedings of the IEEE/ION Position, Location and Navigation Symposium, Indian Wells, CA, USA, 4–6 May 2010; pp. 1035–1042. [Google Scholar]
- Schindler, A.; Maier, G.; Janda, F. Generation of high precision digital maps using circular arc splines. In Proceedings of the 2012 IEEE Intelligent Vehicles Symposium, Alcala de Henares, Spain, 3–7 June 2012; pp. 246–251. [Google Scholar]
- Biagioni, J.; Eriksson, J. Inferring Road Maps from Global Positioning System Traces. Transp. Res. Rec. J. Transp. Res. Board 2012, 2291, 61–71. [Google Scholar] [CrossRef]
- Sehestedt, S.; Kodagoda, S.; Alempijevic, A.; Dissanayake, G. Efficient Lane Detection and Tracking in Urban Environments. In Proceedings of the 3rd European Conference on Mobile Robots, Freiburg, Germany, 19–21 September 2007; pp. 78–83. [Google Scholar]
- Aly, M. Real time detection of lane markers in urban streets. In Proceedings of the 2008 IEEE Intelligent Vehicles Symposium, Eindhoven, The Netherlands, 4–6 June 2008; pp. 7–12. [Google Scholar]
- Borkar, A.; Hayes, M.; Smith, M.T. Robust lane detection and tracking with Ransac and Kalman filter. In Proceedings of the 2009 16th IEEE International Conference on Image Processing (ICIP), Cairo, Egypt, 7–10 November 2009; pp. 3261–3264. [Google Scholar]
- Borkar, A.; Hayes, M.; Smith, M.T. A novel lane detection system with efficient ground truth generation. IEEE Trans. Intell. Transp. Syst. 2012, 13, 365–374. [Google Scholar] [CrossRef]
- Bar Hillel, A.; Lerner, R.; Levi, D.; Raz, G. Recent progress in road and lane detection: A survey. Mach. Vis. Appl. 2014, 25, 727–745. [Google Scholar] [CrossRef]
- Nedevschi, S.; Oniga, F.; Danescu, R.; Graf, T.; Schmidt, R. Increased Accuracy Stereo Approach for 3D Lane Detection. In Proceedings of the 2006 IEEE Intelligent Vehicles Symposium, Tokyo, Japan, 13–15 June 2006; pp. 42–49. [Google Scholar]
- Petrovai, A.; Danescu, R.; Nedevschi, S. A stereovision based approach for detecting and tracking lane and forward obstacles on mobile devices. In Proceedings of the IEEE Intelligent Vehicles Symposium, Seoul, Korea, 28 June–1 July 2015; pp. 634–641. [Google Scholar]
- Nex, F.; Remondino, F. UAV for 3D mapping applications: A review. Appl. Geomat. 2014, 6, 1–15. [Google Scholar] [CrossRef]
- Colomina, I.; Molina, P. ISPRS Journal of Photogrammetry and Remote Sensing Unmanned aerial systems for photogrammetry and remote sensing: A review. ISPRS J. Photogramm. Remote Sens. 2014, 92, 79–97. [Google Scholar] [CrossRef]
- Barber, D.; Mills, J.; Smith-Voysey, S. Geometric validation of a ground-based mobile laser scanning system. ISPRS J. Photogramm. Remote Sens. 2008, 63, 128–141. [Google Scholar] [CrossRef]
- Puente, I.; González-Jorge, H.; Martínez-Sánchez, J.; Arias, P. Review of mobile mapping and surveying technologies. Measurement 2013, 46, 2127–2145. [Google Scholar] [CrossRef]
- Li, R. Mobile Mapping: An Emerging Technology for Spatial Data Acquisition. Photogramm. Eng. Remote Sens. 1997, 63, 1085–1092. [Google Scholar]
- Ma, L.; Li, Y.; Li, J.; Wang, C.; Wang, R.; Chapman, M.A. Mobile laser scanned point-clouds for road object detection and extraction: A review. Remote Sens. 2018, 10, 1531. [Google Scholar] [CrossRef]
- Ogawa, T.; Takagi, K. Lane Recognition Using On-vehicle LIDAR. In Proceedings of the 2006 IEEE Intelligent Vehicles Symposium, Tokyo, Japan, 13–15 June 2006; pp. 540–545. [Google Scholar]
- Smadja, L.; Ninot, J.; Gavrilovic, T. Road extraction and environment interpretation from Lidar sensors. IAPRS 2010, 38, 281–286. [Google Scholar]
- Hanyu, W.; Huan, L.; Chenglu, W.; Jun, C.; Peng, L.; Yiping, C.; Cheng, W.; Jonathan, L. Road Boundaries Detection Based on Local Normal Saliency from Mobile Laser Scanning Data. IEEE Geosci. Remote Sens. Lett. 2015, 12, 2085–2089. [Google Scholar] [CrossRef]
- Rodríguez-Cuenca, B.; García-Cortés, S.; Ordóñez, C.; Alonso, M.C. An approach to detect and delineate street curbs from MLS 3D point cloud data. Autom. Constr. 2015, 51, 103–112. [Google Scholar] [CrossRef]
- Weiss, T. Automatic detection of traffic infrastructure objects for the rapid generation of detailed digital maps using laser scanners. In Proceedings of the 2007 Intelligent Vehicles Symposium, Istanbul, Turkey, 13–15 June 2007; pp. 1271–1277. [Google Scholar]
- Boyko, A.; Funkhouser, T. Extracting roads from dense point clouds in large scale urban environment. ISPRS J. Photogramm. Remote Sens. 2011, 66, S2–S12. [Google Scholar] [CrossRef] [Green Version]
- Zhao, G.; Yuan, J. Curb detection and tracking using 3D-LIDAR scanner. In Proceedings of the 2012 19th IEEE International Conference on Image Processing, Orlando, FL, USA, 30 September–3 October 2012; pp. 437–440. [Google Scholar]
- Wang, H.; Cai, Z.; Luo, H.; Wang, C.; Li, P.; Yang, W.; Ren, S.; Li, J. Automatic road extraction from mobile laser scanning data. In Proceedings of the 2012 International Conference on Computer Vision in Remote Sensing, Xiamen, China, 16–18 December 2012; pp. 136–139. [Google Scholar]
- Toth, C.; Paska, E.; Brzezinska, D. Using Road Pavement Markings as Ground Control for LiDAR Data. Int. Arch. Photogramm. Remote Sens. Spat. Inf. Sci. 2008, 37, 189–195. [Google Scholar]
- Hata, A.; Wolf, D. Road marking detection using LIDAR reflective intensity data and its application to vehicle localization. In Proceedings of the 17th International IEEE Conference on Intelligent Transportation Systems (ITSC), Qingdao, China, 8–11 October 2014; pp. 584–589. [Google Scholar]
- Chen, X.; Kohlmeyer, B.; Stroila, M.; Alwar, N.; Wang, R.; Bach, J. Next generation map making. In Proceedings of the 17th ACM SIGSPATIAL International Conference on Advances in Geographic Information Systems, Seattle, WA, USA, 4–6 Novermber 2009; pp. 488–491. [Google Scholar]
- Kumar, P.; McElhinney, C.P.; Lewis, P.; McCarthy, T. Automated road markings extraction from mobile laser scanning data. Int. J. Appl. Earth Obs. Geoinf. 2014, 32, 125–137. [Google Scholar] [CrossRef] [Green Version]
- Guan, H.; Li, J.; Yu, Y.; Wang, C.; Chapman, M.; Yang, B. Using mobile laser scanning data for automated extraction of road markings. ISPRS J. Photogramm. Remote Sens. 2014, 87, 93–107. [Google Scholar] [CrossRef]
- Soilán, M.; Riveiro, B.; Martínez-Sánchez, J.; Arias, P. Segmentation and classification of road markings using MLS data. ISPRS J. Photogramm. Remote Sens. 2017, 123, 94–103. [Google Scholar] [CrossRef]
- Yu, Y.; Li, J.; Guan, H.; Jia, F.; Wang, C. Learning Hierarchical Features for Automated Extraction of Road Markings From 3-D Mobile LiDAR Point Clouds. IEEE J. Sel. Top. Appl. Earth Obs. Remote Sens. 2015, 8, 709–726. [Google Scholar] [CrossRef]
- Ishikawa, K.; Takiguchi, J.I.; Amano, Y.; Hashizume, T. A mobile mapping system for road data capture based on 3D road model. In Proceedings of the IEEE International Conference on Control Applications, Munich, Germany, 4–6 October 2007; pp. 638–643. [Google Scholar]
- Hůlková, M.; Pavelka, K.; Matoušková, E. Automatic classification of point clouds for highway documentation. Acta Polytech. 2018, 58, 165–170. [Google Scholar] [CrossRef]
- Huang, P.; Cheng, M.; Chen, Y.; Luo, H.; Wang, C.; Li, J. Traffic Sign Occlusion Detection Using Mobile Laser Scanning Point Clouds. IEEE Trans. Intell. Transp. Syst. 2017, 18, 2364–2376. [Google Scholar] [CrossRef]
- Hu, X.; Rodriguez, F.S.A.; Gepperth, A. A multi-modal system for road detection and segmentation. In Proceedings of the 2014 IEEE Intelligent Vehicles Symposium, Dearborn, MI, USA, 8–11 June 2014; pp. 1365–1370. [Google Scholar]
- Xiao, L.; Dai, B.; Liu, D.; Hu, T.; Wu, T. CRF based road detection with multi-sensor fusion. In Proceedings of the 2015 IEEE Intelligent Vehicles Symposium (IV), COEX, Seoul, Korea, 28 June–1 July 2015; pp. 192–198. [Google Scholar]
- Premebida, C.; Carreira, J.; Batista, J.; Nunes, U. Pedestrian detection combining RGB and dense LIDAR data. In Proceedings of the 2014 IEEE/RSJ International Conference on Intelligent Robots and Systems, Chicago, IL, USA, 14–18 September 2014; pp. 4112–4117. [Google Scholar]
- Schoenberg, J.R.; Nathan, A.; Campbell, M. Segmentation of dense range information in complex urban scenes. In Proceedings of the 2010 IEEE/RSJ International Conference on Intelligent Robots and Systems, Taipei, Taiwan, 18–21 October 2010; pp. 2033–2038. [Google Scholar]
- Tan, K.; Cheng, X.; Ding, X.; Zhang, Q. Intensity Data Correction for the Distance Effect in Terrestrial Laser Scanners. IEEE J. Sel. Top. Appl. Earth Obs. Remote Sens. 2016, 9, 304–312. [Google Scholar] [CrossRef]
- Rahtu, E.; Heikkila, J. A Simple and efficient saliency detector for background subtraction. In Proceedings of the 2009 IEEE 12th International Conference on Computer Vision Workshops, Kyoto, Japan, 27 September–4 October 2009; pp. 1137–1144. [Google Scholar]
/m | /m | /m | ||||
---|---|---|---|---|---|---|
six-lane road | 3302.65 | 46.99 | 42.03 | 0.986 | 0.987 | 0.987 |
roundabout | 309.98 | 88.11 | 48.58 | 0.779 | 0.865 | 0.819 |
overall | 3612.63 | 135.10 | 90.61 | 0.964 | 0.976 | 0.970 |
© 2019 by the authors. Licensee MDPI, Basel, Switzerland. This article is an open access article distributed under the terms and conditions of the Creative Commons Attribution (CC BY) license (http://creativecommons.org/licenses/by/4.0/).
Share and Cite
Wan, R.; Huang, Y.; Xie, R.; Ma, P. Combined Lane Mapping Using a Mobile Mapping System. Remote Sens. 2019, 11, 305. https://doi.org/10.3390/rs11030305
Wan R, Huang Y, Xie R, Ma P. Combined Lane Mapping Using a Mobile Mapping System. Remote Sensing. 2019; 11(3):305. https://doi.org/10.3390/rs11030305
Chicago/Turabian StyleWan, Rui, Yuchun Huang, Rongchang Xie, and Ping Ma. 2019. "Combined Lane Mapping Using a Mobile Mapping System" Remote Sensing 11, no. 3: 305. https://doi.org/10.3390/rs11030305