Optical-Based Foot Plantar Pressure Measurement System for Potential Application in Human Postural Control Measurement and Person Identification †
Abstract
:1. Introduction
2. Optical-Based Plantar Pressure Measurement System
2.1. Foot Plantar Pressure Acquisition Process
2.1.1. Distortion Correction
2.1.2. Intensity Offset and Contrast Offset Adjustment
- -
- Intensity Offset AdjustmentTo ensure that the camera has the same intensity or brightness level, we compute the mean intensity in each camera and then find the difference in the average intensity. Then, we subtract the intensity of one camera from the difference in the average intensity so that the intensity offset of the two cameras is equalized.
- -
- Contrast Offset AdjustmentAfter intensity offset equalization, the histogram centroid will be at the same intensity level, but the maximum and minimum intensity in the images captured by the two cameras may be the different. This results in a contrast difference. To ensure that the cameras have the same contrast level, we perform a contrast stretching procedure [32] such that the maximum and minimum of the intensity are the same in both cameras. As a result, the contrast offset is equalized.
2.1.3. Mosaicking Process
2.1.4. Smoothening Transition Region
2.2. Calibration
2.3. Foot Plantar Pressure Acquisition
3. Human Posture Balance Measurement
4. Personal Identification and/or Recognition
4.1. Static Plantar Pressure Feature
4.2. Gaiting Plantar Pressure Feature
4.3. Personal Identification and/or Recognition Evaluation
5. Discussion
- (i)
- In the proposed technique of person identification and/or recognition, we have defined the total feature vector Si as the combination of the dynamic feature vector ((Fd}i) and static feature vector ({Fs}i). In practice, the combination can be weighted asSi = α (Fd}i + (1 − α) {Fs}i
- (ii)
- Our optical-based plantar pressure measurement system uses multiple USB cameras to capture foot plantar images. The height of the measurement platform depends on the focal length of the USB camera. Using the OKER 386 USB camera, the height of the platform is approximately 18 cm. The top surface of the system, which is not level with the nearby floor, makes the system inconvenient to use. To alleviate this problem, a wooden platform was built surrounding our measurement system such that the top surface of our system is level with the top of the wooden platform.
- (iii)
- One unit of our optical-based plantar pressure measurement system requires eight USB cameras and can measure gaiting plantar pressure for up to three steps. The active area of the measurement system is 30 cm × 72 cm. The number of pixels in the active area is 600 pixels × 1440 pixels. The resolution of our system is hence 400 pixel/cm2. The resolution of our optical-based plantar pressure measurement system is much higher than that of a commercial pressure sensor mat, which has an average resolution of only 25 sensor/cm2 [39]. To speed up the acquisition process, our proposed system can decrease the resolution to 200 pixel/cm2. Nevertheless, the resolution is still higher than that of the commercial system. In the event that a greater active area is needed, our measurement system can be concatenated to create a longer walkway. This makes our system expandable and its construction affordable.
- (iv)
- To achieve the best performance, our system is designed to be used with a barefoot subject. In the case that the subject is wearing shoes, the robustness of the static feature will be degraded. In such a case, the α in Equation (7) can be adjusted to 1 and only the dynamic feature is included in the total feature.
- (v)
- All the digital image processing used in our optical-based plantar pressure measurement system is based on OpenCV library [40]. The OpenCV digital image processing can run on many platforms, e.g., a personal computer with a Windows operating system or a Raspberry Pi microcontroller with the LINUX operating system. A personal computer provides the best performance, with a fast acquisition process. The Raspberry Pi microcontroller can be used for a portable system. Our system uses the number of USB cameras connected to personal computer or Raspberry Pi microcontroller. A USB hub is required. In general, the frame rate of a typical USB camera is 30 frames per second (fps) which is sufficient for gaiting plantar pressure acquisition. In the case of using a USB hub, however, the frame rate will be decreased to some extent as the bandwidth of the USB port is shared among all connected cameras. The problem, however, can be lessened using USB 2.0.
- (vi)
- The application of our optical-based plantar pressure measurement for static plantar pressure measurement including human postural balance and foot classification can be performed in real time. For the person recognition/identification application, since it requires the acquisition of the gaiting plantar pressure feature, which requires the subject to walk for three steps, there will be a delay of a few seconds in reporting the recognition/identification results. In addition, the static plantar pressure image is captured when subject fully steps onto the platform. The area of the plantar pressure image is the maximum.
- (vii)
- The overall person identification performance using plantar pressure features is promising, with a recognition rate of 98.396% and an equal error rate (EER) of 6.5808%. Compared with Connor [27], who also has studied the underfoot pressure features for barefoot gait biometrics on a dataset of 92 subjects, our proposed method achieves a better result on a walking dataset of 90 subjects. In Connor’s work, a recognition rate of 93% is reported. Our proposed algorithm’s person identification performance achieved a recognition rate similar to that of Pataky’s work [41], which also evaluated the performance of a large number of barefoot subjects. Both Connor’s and Pataky’s data were recorded on a commercial pressure mat with a fine resolution of approximately 5 mm. Our optical-based pressure sensor provides a finer resolution of 0.5 mm, which is ten times greater than the commercial mat used by Connor and Pataky. Using a commercial pressure mat that provides a sampling rate of 100 Hz, Connor’s and Pataky’s data include both a normal walking pace and a fast pace. Our proposed method relies on multiple USB cameras, where the USB bandwidth has to be shared, so it can study only the normal walking pace dataset. Compared with a non-plantar-pressure-based system, our personal identification system can also achieve a higher personal identification accuracy than that of Sinha et al. [22], which uses Kinect to perform area and distance feature recognition, with a recognition rate of 90%. Personal identification from the three-dimensional range information of the target, as performed by Yamada et al. [23], used a real-time multi-line light detection and ranging (LiDAR) camera and the collected data were trained and classified by a convolution neural network, with a recognition rate of only 60%. In addition, Uhl and Wild [42] designed a footprint-based feature extraction system including geometry, shape, and texture, with a recognition rate of 85%.
- (viii)
- Our human posture balance measurement system is based on plantar pressure measurement, which is similar to the work of Hernandez et al. [4] and Gopalai et al. [8]. Hernandez et al. [4] used a ground-level six-channel force plate (AccuGait, AMTI, Watertown, MA, USA) to measure the human posture balance of 11 subjects. The data were collected in various situations, including with eyes open, eyes closed, bipedal, and unipedal stance. To estimate human posture balance, Hernandez et al. used the COP, velocity autocorrelation function (COP-VAF), and stabilogram diffusion analysis (SDA). Due to the intensive data processing, the posture balance results were reported off-line after collecting and processing data for a period of time. Compared with the work of Hernandez et al., the resolution of our human posture balance system is higher, while our sampling rate is poorer. Human posture balance measurement, however, does not require a high sampling rate, which makes our optical-based human balance system feasible to be used for human posture balance measurement. Our human posture balance index (PBI) is based on an ellipsoidal fit to the trajectory of COP. The advantage of PBI is that the index contains both magnitude and direction. The magnitude of the major and minor axis of the fitted ellipsoid reflects the magnitude of human posture balance, while the principal axis of the ellipsoid reflects the direction. In term of data collection, Hernandez et al. collected data for more possible factors, which is required for human posture balance evaluation. Although we collected data for a large number of subjects and also included a subject under the influence of alcohol, we need further studies to test the reliability of the system for other factors affecting human balance control. In Gopalai et al.’s [8] work, eighteen force-sensing resistors (FSR) were used to measure human posture balance in 18 subjects. The COP was used for human posture balance evaluation. Both our system and that of Gopalai et al. are capable of real-time measurement. The 40 mm resolution of Gopalai et al.’s measurement system is, however, significantly lower than that of our optical-based measurement system.
6. Conclusions
Author Contributions
Funding
Institutional Review Board Statement
Informed Consent Statement
Data Availability Statement
Conflicts of Interest
References
- Guoand, Y.; Lanmei, W. Recent advance in plantar pressure measurement system. Sens. Transducers 2012, 145, 96–105. [Google Scholar]
- Racic, V.; Pavic, A.; Brownjohn, J.M.W. Modern facilities for experimental measurement of dynamic loads induced by humans: A literature review. Shock Vib. 2013, 20, 53–67. [Google Scholar] [CrossRef]
- Wafai, L.; Zayegh, A.; Woulfe, J.; Aziz, S.M.; Begg, R. Identification of foot pathologies based on plantar pressure asymmetry. Sensors 2015, 15, 20392–20408. [Google Scholar] [CrossRef] [PubMed] [Green Version]
- Hernandez, M.E.; Snider, J.; Stevenson, C.; Cauwenberghs, G.; Poizner, H. A Correlation-Based Framework for Evaluating Postural Control Stochastic Dynamics. IEEE Trans. Neural Syst. Rehabil. Eng. 2016, 24, 551–561. [Google Scholar] [CrossRef] [PubMed]
- Ma, R.; Hu, F. An Intelligent Thermal Sensing System for Automatic, Quantitative Assessment of Motion Training in Lower-Limb Rehabilitation. IEEE Trans. Syst. ManCybern. Syst. 2018, 48, 661–669. [Google Scholar] [CrossRef]
- Kharboutly, H.; Ma, J.; Benali, A.; Thoumie, P.; Pasqui, V.; Bouzit, M. Design of Multiple Axis Robotic Platform for Postural Stability Analysis. IEEE Trans. Neural Syst. Rehabil. Eng. 2015, 23, 93–103. [Google Scholar] [CrossRef] [PubMed]
- Ma, J.; Kharboutly, H.; Benali, A.; Amar, F.B.; Bouzit, M. Design of Omnidirectional Mobile Platform for Balance Analysis. IEEE/ASME Trans. Mechatron. 2014, 19, 1872–1881. [Google Scholar] [CrossRef]
- Gopalai, A.A.; Senanayake, S.M.N.A.; Gouwanda, D. Determining Level of Postural Control in Young Adults Using Force-Sensing Resistors. IEEE Trans. Inf. Technol. Biomed. 2011, 15, 608–614. [Google Scholar] [CrossRef]
- Kuo, A.D. An optimal control model for analyzing human postural balance. IEEE Trans. Biomed. Eng. 1995, 42, 87–101. [Google Scholar] [CrossRef]
- Boukhenous, S.; Attari, M.; Remram, Y. Force platform for postural balance analysis. In Proceedings of the 2012 11th International Conference on Information Science, Signal Processing and Their Applications (ISSPA), Montreal, QC, Canada, 2–5 July 2012. [Google Scholar]
- Zago, M.; Luzzago, M.; Marangoni, T.; De Cecco, M.; Tarabin, M.; Galli, M. 3D Tracking of Human Motion Using Visual Skeletonization and Stereoscopic Vision. Front. Bioeng. Biotechnol. 2020, 8, 181. [Google Scholar] [CrossRef]
- Ramirez-Bautista, J.A.; Huerta-Ruelas, J.A.; Chaparro-Cárdenas, S.L.; Hernández-Zavala, A. A Review in Detection and Monitoring Gait Disorders Using In-Shoe Plantar Measurement Systems. IEEE Rev. Biomed. Eng. 2017, 10, 299–309. [Google Scholar] [CrossRef]
- Kim, K.; Shin, S.; Kong, K. An Air-Filled Pad with Elastomeric Pillar Array Designed for a Force-Sensing Insole. IEEE Sens. J. 2018, 18, 3968–3976. [Google Scholar] [CrossRef]
- Senanayake, C.M.; Senanayake, S.M.N.A. Computational Intelligent Gait-Phase Detection System to Identify Pathological Gait. IEEE Trans. Inf. Technol. Biomed. 2010, 14, 1173–1179. [Google Scholar] [CrossRef] [PubMed]
- Domingues, M.F.; Alberto, N.; Leitão, C.S.J.; Tavares, C.; de Lima, E.R.; Radwan, A.; Sucasas, V.; Rodriguez, J.; André, P.S.B.; Antunes, P.F.C. Insole Optical Fiber Sensor Architecture for Remote Gait Analysis—An e-Health Solution. IEEE Internet Things J. 2019, 6, 207–214. [Google Scholar] [CrossRef] [Green Version]
- Ferreira, J.P.; Crisóstomo, M.M.; Coimbra, A.P. Human Gait Acquisition and Characterization. IEEE Trans. Instrum. Meas. 2009, 58, 2979–2988. [Google Scholar] [CrossRef]
- Kwon, J.; Park, J.H.; Ku, S.; Jeong, Y.H.; Paik, N.J.; Park, Y.L. Soft Wearable Robotic Ankle-Foot-Orthosis for Post-Stroke Patients. IEEE Robot. Autom. Lett. 2019, 4, 2547–2552. [Google Scholar] [CrossRef]
- Mohamed Refai, M.I.; van Beijnum, B.F.; Buurke, J.H.; Veltink, P.H. Gait and Dynamic Balance Sensing Using Wearable Foot Sensors. IEEE Trans. Neural Syst. Rehabil. Eng. 2019, 27, 218–227. [Google Scholar] [CrossRef] [PubMed] [Green Version]
- Stone, E.E.; Skubic, M. Evaluation of an inexpensive depth camera for passive in-home fall risk assessment. In Proceedings of the 2011 5th International Conference on Pervasive Computing Technologies for Healthcare (PervasiveHealth) and Workshops, Dublin, Ireland, 23–26 May 2011. [Google Scholar]
- Stone, E.E.; Skubic, M. Passive in-home measurement of stride-to-stride gait variability comparing vision and Kinect sensing. In Proceedings of the 2011 Annual International Conference of the IEEE Engineering in Medicine and Biology Society, Boston, FL, USA, 30 August–3 September 2011. [Google Scholar]
- Gabel, M.; Gilad-Bachrach, R.; Renshaw, E.; Schuster, A. Full Body Gait Analysis with Kinect. In Proceedings of the 2011 Annual International Conference of the IEEE Engineering in Medicine and Biology Society, San Diego, CA, USA, 28 August–1 September 2012. [Google Scholar]
- Sinha, A.; Chakravarty, K.; Bhowmick, B. Person Identification using Skeleton Information from Kinect. In Proceedings of the ACHI 2013: The Sixth International Conference on Advances in Computer-Human Interactions, Nice, France, 24 February–1 March 2013. [Google Scholar]
- Yamada, H.; Ahn, J.; Mozos, O.M.; Iwashita, Y.; Kurazume, R. Gait-based person identification using 3D LiDAR and long short-term memory deep networks. Adv. Robot. 2020, 34, 1201–1211. [Google Scholar] [CrossRef]
- Nguyen, N.D.; Bui, D.T.; Truong, P.H.; Jeong, G.M. Classification of Five Ambulatory Activities Regarding Stair and Incline Walking Using Smart Shoes. IEEE Sens. J. 2018, 18, 5422–5428. [Google Scholar] [CrossRef]
- Agrawal, T.K.; Thomassey, S.; Cochrane, C.; Lemort, G.; Koncar, V. Low-Cost Intelligent Carpet System for Footstep Detection. IEEE Sens. J. 2017, 17, 4239–4247. [Google Scholar] [CrossRef]
- Takeda, T.; Taniguchi, K.; Asari, K.; Kuramoto, K.; Kobashi, S.; Hata, Y. Biometric Personal Authentication by One Step Foot Pressure Distribution Change by Load Distribution Sensor. In Proceedings of the 2009 IEEE International Conference on Fuzzy Systems, Jeju, Korea, 20–24 August 2009. [Google Scholar]
- Connor, P.C. Comparing and Combining Underfoot Pressure Features for Shod and Unshod Gait Biometrics. In Proceedings of the 2015 IEEE International Symposium on Technologies for Homeland Security, Waltham, MA, USA, 14–16 April 2015. [Google Scholar]
- Heravi, H.; Ebrahimi, A.; Nikzad, S.; Olyaee, E.; Zamani, Y.S. Low Price Foot Pressure Distribution Screening Technique: Optical Podoscope with Accurate Foot Print Segmentation using Hidden Markov Random Field Model. J. Biomed. Phys. Eng. 2020, 10, 523. Available online: https://jbpe.sums.ac.ir/article_44624.html (accessed on 26 February 2021). [CrossRef]
- Keatsamarn, T.; Visitsattapongse, S.; Pintavirooj, C.; Aoyama, H. Optical-Based Foot Plantar Pressure Measurement with Application in Human Postural Balance, Gait and Recognition Analysis. In Proceedings of the 2020 6th International Conference on Engineering, Applied Sciences and Technology (ICEAST), Chiang Mai, Thailand, 1–4 July 2020. [Google Scholar]
- Fetić, A.; Jurić, D.; Osmanković, D. The procedure of a camera calibration using Camera Calibration Toolbox for MATLAB. In Proceedings of the 2012 35th International Convention MIPRO, Opatija, Croatia, 5–21 May 2012. [Google Scholar]
- Bradski, G.; Kaehler, A. Learning OpenCV: Computer Vision with the OpenCV Library; O’Reilly Media, Inc.: Sebastopol, CA, USA, 2008; pp. 370–381. [Google Scholar]
- Radha, N. Comparison of Contrast Stretching methods of Image Enhancement Techniques for Acute Leukemia Images. Int. J. Eng. Res. Technol. 2012, 1, 332–335. [Google Scholar]
- Srisuk, N.; Aootaphao, S.; Chanwimalueang, T.; Pintavirooj, C. Indirect X-Ray Detector Panel Using Multiple Camera. In Proceedings of the 2008 International Symposium on Communications and Information Technologies, Vientiane, Laos, 21–23 October 2008. [Google Scholar]
- Takeda, T.; Taniguchi, K.; Asari, K.; Kuramoto, K.; Kobashi, S.; Hata, Y. Biometric Personal Identification by Dynamics of Sole Pressure at Walking. In Proceedings of the 2010 World Automation Congress, Kobe, Japan, 19–23 September 2010. [Google Scholar]
- Ye, H.; Kobashi, S.; Hata, Y.; Taniguchi, K.; Asari, K. Biometric System by Foot Pressure Change Based on Neural Network. In Proceedings of the 2009 39th International Symposium on Multiple-Valued Logic, Naha, Japan, 21–23 May 2009. [Google Scholar]
- Wong, C.K.; Weil, R.; de Boer, E. Standardizing Foot-Type Classification Using Arch Index Values. Physiother. Can. 2012, 64, 280–283. [Google Scholar] [CrossRef] [PubMed] [Green Version]
- Proakis, J.G.; Manolakis, D.G. Digital Signal Processing: Principles, Algorithms, and Applications, 3rd ed.; Prentice-Hall, Inc.: Hoboken, NJ, USA, 1996. [Google Scholar]
- Khor, J.H.; Ismail, W.; Rahman, G. Prevention and Detection Methods for Enhancing Security in an RFID System. Int. J. Distrib. Sens. Netw. 2012, 8, 1–8. [Google Scholar] [CrossRef]
- Medical Sensor 7101E. Available online: https://www.tekscan.com/products-solutions/medical-sensors/7101e (accessed on 25 February 2021).
- Releases. Available online: https://opencv.org/releases/ (accessed on 26 February 2021).
- Pataky, T.C.; Mu, T.; Bosch, K.; Rosenbaum, D.; Goulermas, J.Y. Gait recognition: Highly unique dynamic plantar pressure patterns among 104 individuals. J. R. Soc. Interface 2011, 9, 790–800. [Google Scholar] [CrossRef]
- Uhl, A.; Wild, P. Footprint-based biometric verification. J. Electron. Imaging 2008, 17, 1–11. [Google Scholar] [CrossRef] [Green Version]
Female | Male | |
---|---|---|
n | 50 | 40 |
Age (years) | 20.98 (1.778) | 21.575 2.934) |
Mass (kg) | 54.7 (8.758) | 69.2 (8.618) |
Height (cm) | 160.02 (5.192) | 173.075 (6.455) |
Publisher’s Note: MDPI stays neutral with regard to jurisdictional claims in published maps and institutional affiliations. |
© 2021 by the authors. Licensee MDPI, Basel, Switzerland. This article is an open access article distributed under the terms and conditions of the Creative Commons Attribution (CC BY) license (https://creativecommons.org/licenses/by/4.0/).
Share and Cite
Keatsamarn, T.; Visitsattapongse, S.; Aoyama, H.; Pintavirooj, C. Optical-Based Foot Plantar Pressure Measurement System for Potential Application in Human Postural Control Measurement and Person Identification. Sensors 2021, 21, 4437. https://doi.org/10.3390/s21134437
Keatsamarn T, Visitsattapongse S, Aoyama H, Pintavirooj C. Optical-Based Foot Plantar Pressure Measurement System for Potential Application in Human Postural Control Measurement and Person Identification. Sensors. 2021; 21(13):4437. https://doi.org/10.3390/s21134437
Chicago/Turabian StyleKeatsamarn, Tanapon, Sarinporn Visitsattapongse, Hisayuki Aoyama, and Chuchart Pintavirooj. 2021. "Optical-Based Foot Plantar Pressure Measurement System for Potential Application in Human Postural Control Measurement and Person Identification" Sensors 21, no. 13: 4437. https://doi.org/10.3390/s21134437