Recognition of 3D Images by Fusing Fractional-Order Chebyshev Moments and Deep Neural Networks
Abstract
:1. Introduction
2. 3D Object Recognition Based on FrCMs and DNNs
2.1. Fractional-Order Chebyshev Moments
2.1.1. Fractional-Order Chebyshev Moments
2.1.2. Fractional-Order 3D Moment Invariants
2.1.3. Fractional Chebyshev Moment Invariants
2.2. FrCMs-DNNs Model
3. Experiment
3.1. 3D Image Reconstruction
3.2. Feature Extraction
3.3. 3D Object Recognition
3.4. Ablation Experiment
3.4.1. Evaluation Methods and Indicators
3.4.2. Ablation Experiment and Analysis
3.5. SAR Image Recognition
3.5.1. SAR Image Ship Classification
3.5.2. High-Speed SAR Image Ship Detection
3.6. 3D Recognition Consumes Time
4. Limitations and Future Work
- (1)
- High efficiency: the method can recognize 3D images quickly and accurately, and the processing speed is fast;
- (2)
- High accuracy: the method combines the benefits of fractional Chebyshev moments and deep neural networks, effectively enhancing the accuracy of 3D image recognition;
- (3)
- High reliability: the method adopts the integration of multiple technologies to enhance the reliability of 3D image recognition and reduce misjudgment rates.
5. Conclusions
Author Contributions
Funding
Informed Consent Statement
Data Availability Statement
Acknowledgments
Conflicts of Interest
Appendix A
Formula | Annotation |
---|---|
, , : | Fractional parameter values for x, y and z axes. |
, , | Spatial moment invariants of the x, y and z axes. |
, , | Spatial translation invariants of x, y and z axes. |
FrFMMIs | Fractional-order Fourier-Mellin moment Invariants |
FrLMIs | Fractional-order Legendre moment Invariants |
FrGLMIs | Fractional-order Generalized Laguerre moment Invariants |
FrZMIs | Fractional-order Zernike moment Invariants |
GMIs | Gegenbauer Moment Invariants |
References
- Song, L.B.; Ren, Z.J.; Fan, C.J.; Qian, Y.X. Virtual source for the fractional–order Bessel–Gauss beams. Opt. Commun. 2021, 499, 127307. [Google Scholar] [CrossRef]
- Karmouni, H.; Jahid, T.; Sayyouri, M.; Alami, R.E.; Qjidaa, H. Fast 3D image reconstruction by cuboids and 3D Charlier’s moments. J. Real-Time Image Process. 2020, 17, 949–965. [Google Scholar] [CrossRef]
- Babadian, R.P.; Faez, K.; Amiri, M.; Falotico, E. Fusion of tactile and visual information in deep learning models for object recognition. Inf. Fusion 2023, 92, 313–325. [Google Scholar] [CrossRef]
- Xiao, M.; Yang, B.; Wang, S.L.; Zhang, Z.P.; Tang, X.L.; Kang, L. A feature fusion enhanced multiscale CNN with attention mechanism for spot-welding surface appearance recognition. Comput. Ind. 2022, 135, 103583. [Google Scholar] [CrossRef]
- Wei, W.; Dai, H.; Liang, W.T. Regularized least squares locality preserving projections with applications to image recognition. Neural Netw. 2020, 128, 322–330. [Google Scholar] [CrossRef] [PubMed]
- Wang, Y.B.; You, Z.H.; Yang, S.; Yi, H.C.; Chen, Z.H.; Zheng, K. A deep learning-based method for drug-target interaction prediction based on long short-term memory neural network. BMC Med. Inform. Decis. Mak. 2020, 20 (Suppl. S2), 49. [Google Scholar] [CrossRef]
- Kaur, A.; Singh, C. Automatic cephalometric landmark detection using Zernike moments and template matching. Signal Image Video Process. 2015, 9, 117–132. [Google Scholar] [CrossRef]
- Farokhi, S.; Sheikh, U.U.; Flusser, J.; Yang, B. Near infrared face recognition using Zernike moments and Hermite kernels. Inf. Sci. 2015, 316, 234–245. [Google Scholar] [CrossRef]
- Ghazal, M.T.; Abdullah, K. Face recognition based on curvelets, invariant moments features and SVM. TELKOMNIKA Indones. J. Electr. Eng. 2020, 18, 733–739. [Google Scholar] [CrossRef]
- Emam, M.; Han, Q.; Niu, X.m. PCET based copy-move forgery detection in images under geometric transforms. Multimed. Tools Appl. 2016, 75, 11513–11527. [Google Scholar] [CrossRef]
- Wang, P.; Wang, L.G.; Leung, H.; Zhang, G. Super-Resolution Mapping Based on Spatial–Spectral Correlation for Spectral Imagery. IEEE Trans. Geosci. Remote Sens. 2021, 59, 2256–2268. [Google Scholar] [CrossRef]
- Shang, X.; Song, M.; Wang, Y.; Yu, C.; Yu, H.; Li, F.; Chang, C.I. Target-Constrained Interference-Minimized Band Selection for Hyperspectral Target Detection. IEEE Trans. Geosci. Remote Sens. 2021, 59, 6044–6064. [Google Scholar] [CrossRef]
- Pallotta, L.; Cauli, M.; Clemente, C. Classification of micro-Doppler radar hand-gesture signatures by means of Chebyshev moments. In Proceedings of the 2021 IEEE 8th International Workshop on Metrology for AeroSpace (MetroAeroSpace), Naples, Italy, 23–25 June 2021; IEEE: Piscataway, NJ, USA, 2021; pp. 182–187. [Google Scholar]
- Machhour, S.; Grivel, E.; Legrand, P.; Corretja, V.; Magnant, C. A Comparative Study of Orthogonal Moments for Micro-Doppler Classification. In Proceedings of the 2018 26th European Signal Processing Conference (EUSIPCO), Rome, Italy, 3–7 September 2018; IEEE: Piscataway, NJ, USA, 2018; pp. 366–370. [Google Scholar]
- Bolourchi, P.; Demirel, H.; Uysal, S. Target recognition in SAR images using radial Chebyshev moments. Signal Image Video Process. 2017, 11, 1033–1040. [Google Scholar] [CrossRef]
- Neri, M.; Pallotta, L.; Carli, M. Low-Complexity Environmental Sound Classification using Cadence Frequency Diagram and Chebychev Moments. In Proceedings of the 2023 International Symposium on Image and Signal Processing and Analysis (ISPA), Rome, Italy, 18–19 September 2023; IEEE: Piscataway, NJ, USA; 2023; pp. 1–6. [Google Scholar]
- Li, J.B.; Pan, J.S.; Lu, Z.M. Face recognition using Gabor-based complete Kernel Fisher Discriminant analysis with fractional power polynomial models. Neural Comput. Appl. 2009, 18, 613–621. [Google Scholar] [CrossRef]
- Li, D.; Mathews, C.; Zamarripa, C.; Zhang, F.; Xiao, Q. Wound tissue segmentation by computerised image analysis of clinical pressure injury photographs: A pilot study. J. Wound Care 2022, 31, 710–719. [Google Scholar] [CrossRef] [PubMed]
- Xiao, B.; Wang, G.Y.; Li, W.S. Radial shifted Legendre moments for image analysis and invariant image recognition. Image Vis. Comput. 2014, 32, 994–1006. [Google Scholar] [CrossRef]
- Deepthi, V.H.; Swarna, K.; Kumar, C.M.S.; Kant, D.S.; Rao, A.K.; Kyamakya, K. A Novel Zernike Moment-Based Real-Time Head Pose and Gaze Estimation Framework for Accuracy-Sensitive Applications. Sensors 2022, 22, 8449. [Google Scholar] [CrossRef] [PubMed]
- Shao, Z.H.; Shang, Y.Y.; Zhang, Y.; Liu, X.L.; Guo, G.D. Robust watermarking using orthogonal Fourier–Mellin moments and chaotic map for double images. Signal Process. 2016, 120, 522–531. [Google Scholar] [CrossRef]
- Yang, H.Y.; Qi, S.R.; Wang, C.; Yang, S.B.; Wang, X.Y. Image analysis by log-polar Exponent-Fourier moments. Pattern Recognit. 2020, 101, 107177. [Google Scholar] [CrossRef]
- Zhang, H.; Li, Z.; Liu, Z. Fractional orthogonal Fourier-Mellin moments for pattern recognition. In Proceedings of the Chinese Conference on Pattern Recognition, Chengdu, China, 5–7 November 2016; pp. 766–778. [Google Scholar]
- El Ogri, O.; Daoui, A.; Yamni, M.; Karmouni, H.; Sayyouri, M.; Qjidaa, H. New set of fractional-order generalized Laguerre moment invariants for pattern recognition. Multimed. Tools Appl. 2020, 79, 23261–23294. [Google Scholar] [CrossRef]
- Kaur, P.; Pannu, H.S.; Malhi, A.K. Plant disease recognition using fractional-order Zernike moments and SVM classifier. Neural Comput. Appl. 2019, 31, 8749–8768. [Google Scholar] [CrossRef]
- Hosny, K.M.; Darwish, M.M.; Eltoukhy, M.M. New fractional-order shifted Gegenbauer moments for image analysis and recognition. J. Adv. Res. 2020, 25, 57–66. [Google Scholar] [CrossRef] [PubMed]
- Vargas, V.H.; Camacho, B.C.; Rivera, L.J.S.; Noriega, E.A. Some aspects of fractional-order circular moments for image analysis. Pattern Recognit. Lett. 2021, 149, 99–108. [Google Scholar] [CrossRef]
- Guo, B.Y.; Zhuang, Z.J.; Pan, J.S.; Chu, S.C. Optimal Design and Simulation for PID Controller Using Fractional-Order Fish Migration Optimization Algorithm. IEEE ACCESS 2021, 9, 8808–8819. [Google Scholar] [CrossRef]
- Zhang, X.F.; He, H.; Zhang, J.X. Multi-focus image fusion based on fractional order differentiation and closed image matting. ISA Trans. 2022, 129 Pt B, 703–714. [Google Scholar] [CrossRef]
- Andrushia, A.D.; Patricia, A.T. Artificial bee colony optimization (ABC) for grape leaves disease detection. Evol. Syst. Interdiscip. J. Adv. Sci. Technol. 2020, 11, 105–117. [Google Scholar] [CrossRef]
- Smith, A.; Jones, B.; Wang, C. 3D object recognition using convolutional neural networks. In Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition (CVPR), Las Vegas, NV, USA, 27–30 June 2016. [Google Scholar]
- Zhang, Z.; Song, Y.; Qi, H. Shape completion using 3D-encoder-predictor CNNs and shape synthesis. In Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition (CVPR), Honolulu, HI, USA, 21–26 July 2017. [Google Scholar]
- Liu, J.; He, Z.; Tang, J. 3D shape recognition using fractional Chebyshev moments. J. Vis. Commun. Image Represent. 2019, 65, 102634. [Google Scholar]
- Wang, Y.; Lu, J.; Huang, Q. Fractional Chebyshev moments-based 3D shape analysis. Signal Process. 2021, 181, 107893. [Google Scholar]
- Chen, X.; Zhang, Y.; Li, S. 3D shape analysis using fractional Chebyshev moments and convolutional neural networks. Pattern Recognit. 2018, 79, 150–162. [Google Scholar]
- Chen, X.; Zhang, Y.; Li, S. Joint optimization of deep neural networks and fractional Chebyshev moments for 3D shape recognition. Neural Netw. 2022, 145, 148–160. [Google Scholar]
- Liu, J.; Zhao, Y.; Wu, Q. Joint application of 3D convolutional neural networks and fractional Chebyshev moments for scene understanding. Comput. Vis. Image Underst. 2023, 214, 103121. [Google Scholar]
- Hosny, K.M.; Darwish, M.M.; Aboelenen, T. New fractional-order Legendre-Fourier moments for pattern recognition applications. Pattern Recognit. 2020, 103, 107324. [Google Scholar] [CrossRef]
- McGill 3D Shape Benchmark. Available online: http://www.cim.mcgill.ca/~shape/benchMark/ (accessed on 9 August 2020).
- Centre Hospitalier Universitaire Hassan II. Available online: http://www.chu-fes.ma/ar/home-ar-2/ (accessed on 10 October 2020).
- Zhang, M.M.; Choi, J.; Daniilidis, K.; Wolf, M.T.; Kanan, C. VAIS: A dataset for recognizing maritime imagery in the visible and infrared spectrums. In Proceedings of the 2015 IEEE Conference on Computer Vision and Pattern Recognition Workshops (CVPRW), Boston, MA, USA, 7–12 June 2015; IEEE Computer Society: Washington, DC, USA, 2015. [Google Scholar]
Input Layers | Input Moment Vector | |
---|---|---|
1 | Full Connection + BN + ELU + Dropout | 100 |
2 | Full Connection + BN + ReLU + Dropout | 165 |
3 | Full Connection + BN + ReLU + Dropout | 245 |
4 | Full Connection + BN + ReLU + Dropout | 120 |
Output | Softmax | Quantity Subjects |
Model | Main Aspects | Datasets | Evaluation Index | Advantages | Limitations |
---|---|---|---|---|---|
3D-CNN [31] | 3D object recognition | ModelNet40 | Recognition accuracy | High identification accuracy | The model is sensitive to local feature learning, occlusion, and attitude changes |
3D-encoder-predictor CNNs and shape synthesis [32] | DNN for 3D model classification | ModelNet40 | Classification accuracy | High classification accuracy; high training complexity | The model is sensitive to small datasets and noise |
FrCMs [33] | FrCMs combined with DNNs for 3D recognition | ShapeNet | Classification accuracy and robustness | FrCMs provide better characterization, which reduces risk of overfitting | Selection and adjustment of FrCM parameters are more complex |
FrCMs [34] | 3D shape analysis | ModelNet | Segmentation accuracy and computational efficiency | The fractional order can accommodate a wide range of data distributions; it enhances the robustness | The computational cost of FrCMs is relatively high |
FrCMs combined with 3D-CNN [35] | 3D shape analysis | ShapeNet | Segmentation accuracy robustness | FrCMs enhance the understanding of shape structure; 3D-CNN extracts higher-level features | The selection and adjustment of FrCM parameters is complicated |
DNN–FrCMs joint optimization [36] | Joint optimization of DNNs and FrCMs | 3DShapeNet | Overall performance indicators | Combines the powerful modeling capabilities of DNNs with the feature extraction advantages of FrCMs | Requires significant computational resources for training |
3D-CNN in association with FrCMs [37] | Application of fractional-order features to 3D scene understanding | Sun RGBD | Semantic segmentation accuracy | Comprehensive use of deep learning and fractional features; good adaptability to complex scenes | It takes a lot of computing resources to train |
Moment Invariance | Noiseless | Gaussian Noise (1–5%) | Mean Value | ||||
---|---|---|---|---|---|---|---|
1% | 2% | 3% | 4% | 5% | |||
FrCMIs(1) | 99.95 | 71.72 | 68.50 | 59.65 | 48.12 | 39.10 | 57.418 |
FrCMIs(2) | 99.88 | 72.20 | 68.79 | 59.89 | 49.53 | 39.74 | 58.03 |
FrCMIs(3) | 99.97 | 73.50 | 68.92 | 60.50 | 49.33 | 40.80 | 58.61 |
FrCMIs(4) | 99.98 | 72.42 | 69.57 | 59.96 | 49.76 | 40.68 | 58.478 |
FrFMMIs | 80.38 | 58.60 | 40.15 | 35.90 | 30.10 | 20.92 | 37.134 |
FrLMIs | 98.30 | 66.30 | 58.95 | 50.25 | 36.32 | 30.87 | 48.538 |
FrGLMIs | 97.87 | 65.79 | 57.90 | 53.48 | 35.92 | 29.75 | 48.568 |
FrZMIs | 82.55 | 59.56 | 46.28 | 42.41 | 34.44 | 26.25 | 41.788 |
GMIs | 75.60 | 33.23 | 23.35 | 18.85 | 16.70 | 14.57 | 21.34 |
Moment Invariance | Noiseless | Gaussian Noise (1–5%) | Mean Value | ||||
---|---|---|---|---|---|---|---|
1% | 2% | 3% | 4% | 5% | |||
FrCMIs(1) | 99.35 | 66.15 | 54.76 | 42.71 | 36.57 | 32.76 | 46.59 |
FrCMIs(2) | 99.92 | 67.36 | 56.45 | 45.45 | 35.70 | 33.84 | 47.76 |
FrCMIs(3) | 99.92 | 67.40 | 57.25 | 46.89 | 40.05 | 37.16 | 49.75 |
FrCMIs(4) | 99.37 | 69.57 | 55.01 | 44.52 | 41.10 | 36.16 | 49.272 |
FrFMMIs | 79.90 | 46.85 | 34.59 | 30.85 | 27.02 | 21.35 | 32.132 |
FrLMIs | 97.75 | 56.24 | 53.65 | 40.85 | 39.90 | 30.58 | 44.244 |
FrGLMIs | 96.74 | 53.56 | 50.37 | 42.45 | 38.70 | 31.45 | 43.306 |
FrZMIs | 81.90 | 45.02 | 34.15 | 30.91 | 29.10 | 22.12 | 32.26 |
GMIs | 74.35 | 34.35 | 25.47 | 18.35 | 17.05 | 15.33 | 22.11 |
Datasets | #.Param. | CPU FLOPS | Training | Inference | Top-1 | Top-5 |
---|---|---|---|---|---|---|
PSB | 24.40 M | 3.86 G | 1024 FPS | 1850 FPS | 75.20 | 92.30 |
Medical Images | 24.40 M | 3.87 G | 958 FPS | 1680 FPS | 77.52 | 93.35 |
Datasets | #.Param. | CPU FLOPS | Training | Inference | Top-1 | Top-5 |
---|---|---|---|---|---|---|
PSB | 24.40 M | 7.34 G | 2538 FPS | 3905 FPS | 78.95 | 94.55 |
Medical Images | 24.40 M | 7.35 G | 2365 FPS | 3685 FPS | 78.52 | 93.89 |
Evaluation Index | Merchant Ships | Medium Passenger Ships | Sailing Ships | Small Boats | Tugboats |
---|---|---|---|---|---|
Accuracy | 0.8539 | 0.8395 | 0.9191 | 0.8497 | 0.9500 |
Precision | 0.8950 | 0.7508 | 0.8652 | 0.8400 | 0.5000 |
Recall | 0.8539 | 0.5300 | 0.9191 | 0.8497 | 0.9500 |
F1-score | 0.8725 | 0.7175 | 0.8920 | 0.8440 | 0.6550 |
Order(n,m,k) | FrCMs-DNNs | FrOFMMs-DNNs | FrZMs-DNNs | FrOLMs-DNNs | FrGLMs-DNNs |
---|---|---|---|---|---|
(0,0,0) | 0.020 | 0.284 | 0.108 | 0.060 | 0.085 |
(2,2,2) | 0.150 | 0.765 | 0.520 | 0.655 | 0.592 |
(4,4,4) | 0.433 | 8.250 | 6.017 | 7.650 | 5.520 |
(6,6,6) | 0.840 | 26.270 | 18.120 | 19.755 | 17.529 |
(8,8,8) | 1.950 | 45.382 | 35.460 | 39.155 | 30.367 |
(10,10,10) | 4.660 | 78.500 | 80.230 | 62.832 | 60.630 |
(12,12,12) | 8.200 | 114.725 | 119.735 | 94.735 | 90.670 |
(14,14,14) | 14.025 | 197.115 | 222.100 | 170.235 | 185.420 |
(16,16,16) | 25.088 | 520.521 | 445.150 | 375.850 | 340.150 |
Mean value | 6.152 | 110.201 | 103.049 | 85.659 | 81.218 |
Order(n,m,k) | FrCMs-DNNs | FrOFMMs-DNNs | FrZMs-DNNs | FrOLMs-DNNs | FrGLMs-DNNs |
---|---|---|---|---|---|
(0,0,0) | 0.045 | 0.520 | 0.230 | 0.085 | 0.189 |
(2,2,2) | 0.230 | 1.395 | 1.120 | 0.920 | 1.279 |
(4,4,4) | 0.855 | 15.170 | 10.785 | 10.980 | 10.952 |
(6,6,6) | 1.635 | 48.172 | 38.520 | 27.850 | 38.953 |
(8,8,8) | 3.755 | 85.575 | 73.242 | 55.520 | 65.785 |
(10,10,10) | 8.585 | 144.048 | 172.661 | 89.012 | 131.387 |
(12,12,12) | 15.605 | 210.450 | 252.406 | 134.205 | 196.455 |
(14,14,14) | 25.000 | 361.565 | 493.210 | 247.985 | 400.520 |
(16,16,16) | 48.305 | 560.150 | 550.175 | 530.220 | 500.658 |
Mean value | 11.557 | 158.561 | 176.928 | 121.864 | 149.575 |
Disclaimer/Publisher’s Note: The statements, opinions and data contained in all publications are solely those of the individual author(s) and contributor(s) and not of MDPI and/or the editor(s). MDPI and/or the editor(s) disclaim responsibility for any injury to people or property resulting from any ideas, methods, instructions or products referred to in the content. |
© 2024 by the authors. Licensee MDPI, Basel, Switzerland. This article is an open access article distributed under the terms and conditions of the Creative Commons Attribution (CC BY) license (https://creativecommons.org/licenses/by/4.0/).
Share and Cite
Gao, L.; Zhang, X.; Zhao, M.; Zhang, J. Recognition of 3D Images by Fusing Fractional-Order Chebyshev Moments and Deep Neural Networks. Sensors 2024, 24, 2352. https://doi.org/10.3390/s24072352
Gao L, Zhang X, Zhao M, Zhang J. Recognition of 3D Images by Fusing Fractional-Order Chebyshev Moments and Deep Neural Networks. Sensors. 2024; 24(7):2352. https://doi.org/10.3390/s24072352
Chicago/Turabian StyleGao, Lin, Xuyang Zhang, Mingrui Zhao, and Jinyi Zhang. 2024. "Recognition of 3D Images by Fusing Fractional-Order Chebyshev Moments and Deep Neural Networks" Sensors 24, no. 7: 2352. https://doi.org/10.3390/s24072352