Abstract
Data Matrix patterns imprinted as passive visual landmarks have shown to be a valid solution for the self-localization of Automated Guided Vehicles (AGVs) in shop floors. However, existing Data Matrix decoding applications take a long time to detect and segment the markers in the input image. Therefore, this paper proposes a pipeline where the detector is based on a real-time Deep Learning network and the decoder is a conventional method, i.e. the implementation in libdmtx. To do so, several types of Deep Neural Networks (DNNs) for object detection were studied, trained, compared, and assessed. The architectures range from region proposals (Faster R-CNN) to single-shot methods (SSD and YOLO). This study focused on performance and processing time to select the best Deep Learning (DL) model to carry out the detection of the visual markers. Additionally, a specific data set was created to evaluate those networks. This test set includes demanding situations, such as high illumination gradients in the same scene and Data Matrix markers positioned in skewed planes. The proposed approach outperformed the best known and most used Data Matrix decoder available in libraries like libdmtx.
Article PDF
Similar content being viewed by others
Avoid common mistakes on your manuscript.
References
Li, D., Ouyang, B., Wu, D., Wang, Y: Artificial intelligence empowered multi-AGVs in manufacturing systems. arXiv:1909.033731909.03373 (2019)
Shi, Q., Zhang, Y.-L., Li, L., Yang, X., Li, M., Zhou, J.: Safe: Scalable automatic feature engineering framework for industrial tasks (2020)
Oyekanlu, E.A., Smith, A.C., Thomas, W.P., Mulroy, G., Hitesh, D., Ramsey, M., Kuhn, D.J., Mcghinnis, J.D., Buonavita, S.C., Looper, N.A., Ng, M., Ng’oma, A., Liu, W., Mcbride, P.G., Shultz, M.G., Cerasi, C., Sun, D.: A review of recent advances in automated guided vehicle technologies: Integration challenges and research areas for 5g-based smart manufacturing applications. IEEE Access 8, 202312–202353 (2020)
Bhosekar, A., Isik, T., Eksioglu, S., Gilstrap, K., Allen, R.: Simulation-optimization of automated material handling systems in a healthcare facility. arXiv: Optimization and Control (2020)
Cui, W., Wu, C., Zhang, Y., Li, B., Fu, W.: Indoor robot localization based on multidimensional scaling. Int. J. Distrib. Sens. Netw. 11(8), 719658 (2015)
Sabattini, L., Digani, V., Secchi, C., Cotena, G., Ronzoni, D., Foppoli, M., Oleari, F.: Technological roadmap to boost the introduction of agvs in industrial applications. In: 2013 IEEE 9th International Conference on Intelligent Computer Communication and Processing (ICCP), pp. 203–208 (2013)
Fellan, A., Schellenberger, C., Zimmermann, M., Schotten, H.D.: Enabling communication technologies for automated unmanned vehicles in industry 4.0. In: 2018 International Conference on Information and Communication Technology Convergence (ICTC), pp. 171–176 (2018)
Bergamin, M.: Indoor localization using visual information and passive landmarks. Master Thesis, Universities of Padova and Aveiro. http://tesi.cab.unipd.it/50395/1/bergamin_marco.pdf (2015)
Betke, M., Gurvits, L.: Mobile robot localization using landmarks. IEEE Trans. Robot. Autom. 13(2), 251–263 (1997)
Okuyama, K., Kawasaki, T., Kroumov, V.: Localization and position correction for mobile robot using artificial visual landmarks. In: The 2011 International Conference on Advanced Mechatronic Systems, pp. 414–418 (2011)
Zhang, X., Zhu, S., Wang, Z., Li, Y.: Hybrid visual natural landmark-based localization for indoor mobile robots. Int. J. Adv. Robot. Syst. 15(6) (2018)
Almeida, T., Santos, V., Lourenço, B., Fonseca, P.: Detection of data matrix encoded landmarks in unstructured environments using deep learning. In: 2020 IEEE International Conference on Autonomous Robot Systems and Competitions (ICARSC), pp. 74–80 (2020)
Babinec, A., Jurišica, L., Hubinský, P., Duchoň, F.: Visual localization of mobile robot using artificial markers. Procedia Eng. 96, 1–9 (2014). Modelling of Mechanical and Mechatronic Systems
Mutka, A., Miklic, D., Draganjac, I., Bogdan, S.: A low cost vision based localization system using fiducial markers. IFAC Proc. Vol. 41(2), 9528–9533 (2008). 17th IFAC World Congress
Romero-Ramirez, F.J., Muñoz-Salinas, R., Medina-Carnicer, R.: Speeded up detection of squared fiducial markers. Image Vis. Comput. 76, 38–47 (2018)
Mantha, B., Garcia de Soto, B.: Designing a reliable fiducial marker network for autonomous indoor robot navigation. In: Al-Hussein, M. (ed.) Proceedings of the 36th International Symposium on Automation and Robotics in Construction (ISARC), pp 74–81. International Association for Automation and Robotics in Construction (IAARC), Banff (2019)
Annusewicz, A., Zwierzchowski, J.: Marker detection algorithm for the navigation of a mobile robot. In: 2020 27th International Conference on Mixed Design of Integrated Circuits and System (MIXDES), pp 223–226 (2020)
Zhong, X., Zhou, Y., Liu, H.: Design and recognition of artificial landmarks for reliable indoor self-localization of mobile robots. Int. J. Adv. Robot. Syst. 14(1), 1729881417693489 (2017)
Karrach, L., Pivarčiová, E.: The analyse of the various methods for location of data matrix codes in images. In: 2018 ELEKTRO, pp 1–6 (2018)
Dai, Y., Liu, L., Song, W., Du, C., Zhao, X.: The realization of identification method for datamatrix code. In: 2017 International Conference on Progress in Informatics and Computing (PIC), pp 410–414 (2017)
Hansen, D.K., Nasrollahi, K., Rasmusen, C.B., Moeslund, T.B.: Real-time barcode detection and classification using deep learning. In: Proceedings of the 9th International Joint Conference on Computational Intelligence - Volume 1: IJCCI, pp 321–327. SciTePress (2017)
Jiao, L., Zhang, F., Liu, F., Yang, S., Li, L., Feng, Z., Qu, R.: A survey of deep learning-based object detection. IEEE Access 7, 128837–128868 (2019)
Zhang, H., Shi, G., Liu, L., Zhao, M., Liang, Z.: Detection and identification method of medical label barcode based on deep learning. In: 2018 Eighth International Conference on Image Processing Theory, Tools and Applications (IPTA), pp 1–6 (2018)
Yang, Q., Golwala, G., Sundaram, S., Lee, P., Allebach, J.: Barcode detection and decoding in on-line fashion images. Electron. Imaging 2019(8), 413–1–413–7 (2019)
Zharkov, A., Zagaynov, I.: Universal barcode detector via semantic segmentation. CoRR arXiv:1906.06281 (2019)
Ren, S., He, K., Girshick, R.B., Sun, J.: Faster R-CNN: towards real-time object detection with region proposal networks. CoRR arXiv:1506.01497 (2015)
Liu, W., Anguelov, D., Erhan, D., Szegedy, C., Reed, S.E., Fu, C.-Y., Berg, A.C.: SSD: single shot multibox detector. CoRR arXiv:1512.02325 (2015)
Redmon, J., Divvala, S.K., Girshick, R.B., Farhadi, A.: You only look once: Unified, real-time object detection. CoRR arXiv:1506.02640 (2015)
Bochkovskiy, A., Wang, C-Y, Liao, H-: Yolov4: Optimal speed and accuracy of object detection. arXiv:2004.10934 (2020)
Benali Amjoud, A., Amrouch, M.: Convolutional neural networks backbones for object detection. In: El Moataz, A., Mammass, D., Mansouri, A., Nouboud, F. (eds.) Image and Signal Processing, pp 282–289. Springer International Publishing, Cham (2020)
Lin, T.-Y., Dollár, P., Girshick, R.B., He, K., Hariharan, B., Belongie, S J: Feature pyramid networks for object detection. CoRR arXiv:1612.03144 (2016)
Cao, G., Xie, X., Yang, W., Liao, Q., Shi, G., Wu, J.: Feature-fused SSD: fast detection for small objects. CoRR arXiv:1709.05054 (2017)
Huang, Z., Wang, J.: DC-SPP-YOLO: dense connection and spatial pyramid pooling based YOLO for object detection. CoRR arXiv:1903.08589 (2019)
Lin, T-Y, Goyal, P., Girshick, R B, He, K., Dollár, P.: Focal loss for dense object detection. CoRR arXiv:1708.02002 (2017)
Chen, Y., Yang, T., Zhang, X., Meng, G., Xiao, X., Sun, J.: Detnas: Backbone search for object detection. In: Wallach, H., Larochelle, H., Beygelzimer, A., d’Alché-Buc, F., Fox, E., Garnett, R. (eds.) Advances in Neural Information Processing Systems, vol. 32, pp 6642–6652. Curran Associates, Inc. (2019)
Simonyan, K., Zisserman, A.: Very deep convolutional networks for large-scale image recognition. arXiv:1409.1556 (2014)
He, K., Zhang, X., Ren, S., Sun, J.: Deep residual learning for image recognition. CoRR arXiv:1512.03385 (2015)
Huang, G., Liu, Z., Weinberger, K.Q.: Densely connected convolutional networks. CoRR arXiv:1608.06993 (2016)
Howard, A.G., Zhu, M., Chen, B., Kalenichenko, D., Wang, W., Weyand, T., Andreetto, M., Adam, H.: Mobilenets: Efficient convolutional neural networks for mobile vision applications. CoRR arXiv:1704.04861 (2017)
Iandola, F.N., Moskewicz, M.W., Ashraf, K., Han, S., Dally, W.J., Keutzer, K.: Squeezenet: Alexnet-level accuracy with 50x fewer parameters and < 1mb model size. CoRR arXiv:1602.07360(2016)
Zhang, X., Zhou, X., Lin, M., Sun, J.: Shufflenet: An extremely efficient convolutional neural network for mobile devices. CoRR arXiv:1707.01083 (2017)
Redmon, J., Farhadi, A.: Yolov3: An incremental improvement. arXiv:1804.02767 (2018)
Girshick, R.B.: Fast R-CNN. CoRR arXiv:1504.08083 (2015)
Wang, C.-Y., Liao, H.-Y.M., Yeh, I.-H., Wu, Y.-H., Chen, P.-Y., Hsieh, J.-W.: Cspnet: A new backbone that can enhance learning capability of cnn. arXiv:1911.11929 (2019)
Liu, S., Qi, L., Qin, H., Shi, J., Jia, J.: Path aggregation network for instance segmentation. CoRR arXiv:1803.01534 (2018)
Tan, M., Pang, R., Le, Q.: Efficientdet: Scalable and efficient object detection. arXiv:1911.09070 (2019)
Zhou, X., Zhuo, J., Krähenbühl, P.: Bottom-up object detection by grouping extreme and center points. CoRR arXiv:1901.08043 (2019)
Funding
Open access funding provided by Örebro University. This work was partially supported by Project SeaAI-FA_02_2017_011, Project PRODUTECH II SIF- POCI-01-0247-FEDER-024541, by the Wallenberg AI, Autonomous Systems and Software Program (WASP) funded by the Knut and Alice Wallenberg Foundation, and by the Spanish Ministerio de Ciencia, Innovación y Universidades under project RobWell (RTI2018-095599-A-C22).
Author information
Authors and Affiliations
Contributions
– Tiago Almeida: Coding and writing
– Vitor Santos: Writing and review
– Oscar Martinez Mozos: Review
– Bernardo Lourenço: Test set acquisition and review
Corresponding author
Ethics declarations
Competing Interests
The authors declare that they have no conflict of interest.
Additional information
Consent to Participate
The authors consent to participate in this work.
Availability of Data and Material
Code at github.com/tmralmeida/data-matrix-detection-benchmark
Publisher’s Note
Springer Nature remains neutral with regard to jurisdictional claims in published maps and institutional affiliations.
Rights and permissions
Open Access This article is licensed under a Creative Commons Attribution 4.0 International License, which permits use, sharing, adaptation, distribution and reproduction in any medium or format, as long as you give appropriate credit to the original author(s) and the source, provide a link to the Creative Commons licence, and indicate if changes were made. The images or other third party material in this article are included in the article's Creative Commons licence, unless indicated otherwise in a credit line to the material. If material is not included in the article's Creative Commons licence and your intended use is not permitted by statutory regulation or exceeds the permitted use, you will need to obtain permission directly from the copyright holder. To view a copy of this licence, visit http://creativecommons.org/licenses/by/4.0/.
About this article
Cite this article
Almeida, T., Santos, V., Mozos, O.M. et al. Comparative Analysis of Deep Neural Networks for the Detection and Decoding of Data Matrix Landmarks in Cluttered Indoor Environments. J Intell Robot Syst 103, 13 (2021). https://doi.org/10.1007/s10846-021-01442-x
Received:
Accepted:
Published:
DOI: https://doi.org/10.1007/s10846-021-01442-x