AUTISMVISION: PREDICTIVE ANALYTICS FOR AUTISM

SPECTRUM DISORDER
Dr.M.Praneesh
Assistant Professor
PG & Research Department of Computer Science
Sri Ramakrishna College of Arts and Science
[email protected]

Ms.C.V.Shrinithi
UG-Student Final year
PG & Research Department of Computer Science
Sri Ramakrishna College of Arts and Science
[email protected]

Abstract:
Autism Spectrum Disorder (ASD) is a complex neurodevelopmental condition
characterized by challenges in social interaction, communication, and repetitive behaviors. Early
diagnosis and intervention significantly improve outcomes for individuals with ASD. In this study,
we propose the development of an autism detection system leveraging facial expression analysis
and artificial intelligence (AI) techniques. The system aims to analyze facial expressions captured
through images or videos to identify potential indicators of ASD. Data collection involves
assembling a diverse dataset comprising facial expressions from individuals diagnosed with ASD
and neurotypical individuals. Preprocessing techniques are applied to enhance data quality and
prepare it for training. Feature extraction methods extract relevant facial features, such as key
points and expression patterns. A machine learning or deep learning model, particularly
Convolutional Neural Networks (CNNs), is developed to analyze facial expressions and classify
them as indicative of ASD or neurotypical behavior. The model undergoes training using the
preprocessed dataset, followed by validation and testing to assess its generalization ability and
real-world performance. The trained model is integrated into a user-friendly system capable of
receiving input in the form of facial images or videos and providing output indicating the
likelihood of ASD. Overall, the proposed autism detection system represents a promising approach
to facilitating early diagnosis and intervention for individuals with ASD, potentially leading to
improved outcomes and quality of life.
Keywords: Autism, VGG19, Convolutional Neural Network(CNN), Feature Extraction

Introduction:
Autism Spectrum Disorder (ASD) presents a complex array of challenges, spanning social
interaction difficulties, communication impairments, and repetitive behaviors. Identifying ASD
early is crucial for facilitating timely interventions that can profoundly influence an individual's
developmental trajectory and overall well-being. Yet, conventional diagnostic methods often rely
on subjective clinical assessments and standardized evaluations, which can be time-consuming,
resource-intensive, and susceptible to various biases. However, recent strides in artificial
intelligence (AI) and computer vision offer a promising avenue for revolutionizing ASD
diagnostics, promising more objective and efficient detection methods. This work proposes an
innovative approach to ASD detection, harnessing the power of facial expression analysis
alongside AI algorithms. Facial expressions serve as fundamental cues for social interaction and
emotional expression, rendering them invaluable for detecting ASD-related behavioral patterns.
By employing advanced AI techniques to scrutinize facial expressions, our methodology aims to
discern subtle cues indicative of ASD, thus facilitating early intervention and support. Automated
facial expression analysis holds significant potential for enhancing diagnostic accuracy,
streamlining evaluation processes, and expediting access to interventions and services for
individuals with ASD.

Related work:
Srividhya Ganesan, Dr. Raju and Dr. J. Senthil(2021) proposed Autism Prediction system
that focuses on classification models using VGG16 algorithm of SVM, CNN, Haar Cascade using
OpenCV.

Madison Beary, Alex Hadsell, Ryan Messersmith, Mohammad-Parsa Hosseini(2020)

proposed “Diagnosis of Autism in Children using Facial Analysis and Deep Learning “. This deep
learning model uses MobileNet and two dense layers for feature extraction and image
classification.

Amrita Budarapu, Nara Kalyani, Seetha Maddala (2021) proposed “Early Screening of
Autism among Children Using Ensemble Classification Method”. This model is based on Image
and Video processing and capable of classifying emotions like Happy, Sad. This model also tracks
the eye gaze to know the focus of the child.

Suman Raj and Sarfaraz Mazood (2020) “Analysis and Detection of Autism Spectrum
Disorder Using Machine Learning Techniques” focuses on early diagnosis of Autism in Adult,
Children and Adolescents. The proposed model uses various algorithms like Naïve Bayes, Support
Vector Machine, Logistic Regression, KNN and CNN.

M. S. Mythili, A. R. Mohamed Shanavas (2014) conducted a study on Autism Spectrum

Disorder using Classification techniques. This paper deals with Neural Networks, Support Vector
Machine and Fuzzy techniques with WEKA tools are used to analyze the child behavior and their
social interaction.

Fadi Thabtah, Firuz Kamalov, Khairan Rajab (2018) proposed “A new computational
intelligence approach to detect autistic features for autism screening” which involves a
computational intelligence method called Variable Analysis (VA) to reduce feature-to-feature
correlations.

D P Wall, J Kosmicki, T F DeLuca, E Harstad & V A Fusaro (2012) proposed “Use of

Machine Learning to shorten observation-based screening and diagnosis of autism” in which series
of machine learning algorithms are used to study the scores of Autism Diagnostic Observation
Schedule-Generic (ADOS).

Muhammed Shoaib Farooq, Rabia Tehseen, Maidah Sabir and Zabihullah Atal proposed
“Detection of autism spectrum disorder (ASD) in children and adults using Machine Learning”.
This model uses Federated Learning (FL) technique to train two different ML classifiers including
logistic regression and support vector machine for detection of ASD in children and adults.

Proposed Methodology:
The proposed system leverages the VGG19 model combined with a Convolutional Neural
Network (CNN) architecture, incorporating max-pooling layers. This hybrid model aims to
enhance feature extraction from neuroimaging data, enabling more accurate and nuanced detection
of autism spectrum disorder (ASD) by capturing subtle patterns in images. The proposed system
further improves accuracy by integrating multi-modal information, combining structural and
functional neuroimaging data. This holistic approach enables a comprehensive analysis, capturing
diverse aspects of brain activity and structure associated with autism spectrum disorder.

The proposed system aims to address the limitations of existing ASD diagnostic methods
by leveraging advanced AI techniques and facial expression analysis. At its core, the system will
utilize a comprehensive dataset comprising a diverse range of facial expressions and behaviors
exhibited by individuals with ASD and neurotypical controls. This dataset will be meticulously
curated to ensure representativeness and inclusivity, encompassing various age groups, ethnicities,
and cultural backgrounds. Data preprocessing techniques will be employed to enhance data quality
and mitigate biases, including normalization, augmentation, and rigorous quality control measures.

The system will employ state-of-the-art machine learning algorithms, including deep
learning approaches such as Convolutional Neural Networks (CNNs), to extract meaningful
features from facial images. These algorithms will be trained on the curated dataset to learn
intricate patterns and associations between facial expressions and ASD-related traits. Model
development will involve iterative refinement and validation to optimize performance metrics such
as accuracy, sensitivity, and specificity.

To enhance interpretability and transparency, the system will incorporate explainable AI

techniques that provide insights into the decision-making process of the model. This will enable
clinicians and caregivers to understand the rationale behind the system's diagnoses, fostering trust
and acceptance in clinical practice. Ethical considerations will be paramount throughout the
development and deployment of the system. Data privacy and security measures will be
implemented to safeguard sensitive health information, ensuring compliance with relevant
regulations and guidelines. Additionally, efforts to mitigate algorithmic biases and promote
fairness will be prioritized, including diverse dataset curation, bias detection, and mitigation
strategies.

VGG19 AND CNN

The proposed system integrates the VGG19 model with a Convolutional Neural Network
(CNN) architecture, strategically incorporating max-pooling layers to optimize feature extraction
from neuroimaging data. This hybrid model represents a sophisticated approach aimed at
improving the accuracy and sensitivity of autism spectrum disorder (ASD) detection by capturing
subtle and intricate patterns inherent in neuroimages. By leveraging the VGG19 model's
established performance in image classification tasks and the adaptability of CNNs in processing
complex visual data, our system endeavors to enhance the efficacy of identifying ASD-related
features within neuroimages.

Facial ASD Diagnostic

Expression Data Detection Reports and
Source System Feedback

medical
imaging
Input databases
Output

Figure(i): Context Diagram

The context diagram illustrates the ASD detection system's interactions with external entities. At
the core of the system lies the ASD detection module responsible for processing neuroimaging
and facial expression data to generate diagnostic predictions. External entities include sources of
neuroimaging and facial expression data, clinicians, and researchers who input data and review
diagnostic reports. Training and validation data are used to train and evaluate the ASD detection
model, while regulatory and ethical guidelines govern data handling.
System Flow:

Start

Image Data Collection

Greyscale Conversion

Image preprocessing

Noise Filtering using Gaussian

Object Detection (ROI) using the Viola Jones Structure

Edge, Shape and Histogram Feature Extraction

Formation of Model Architecture

Creating multiple Hidden layer in VGG19

Applying back propagation and error reduction in CNN

Autism classification and Analysis

Stop

The flow outlines the process for autism detection from image data, starting with image data
collection, likely from sources such as MRI scans or facial images. The collected images undergo
greyscale conversion for standardized processing. Subsequently, image preprocessing techniques,
including noise filtering using Gaussian filters, are applied to enhance image quality and remove
unwanted artifacts. Object detection using the Viola-Jones structure is employed to identify regions
of interest (ROI) within the images, potentially focusing on facial regions for expression analysis.
Feature extraction techniques such as edge detection, shape analysis, and histogram computation
are then utilized to capture relevant visual features from the ROIs.

The process continues with the formation of the model architecture, possibly involving the
creation of multiple hidden layers within the VGG19 model, a deep convolutional neural network
(CNN) known for its effectiveness in image classification tasks. Backpropagation and error
reduction techniques are applied to optimize the model's parameters and improve its ability to
classify images accurately. Finally, the trained model is used for autism classification and analysis,
where it predicts whether the input image exhibits signs of autism spectrum disorder based on the
extracted features. This flow represents a comprehensive pipeline for leveraging image data and
advanced machine learning techniques to aid in autism detection and analysis.

Result and discussion

When comparing with other similar algorithms,

96

94

92

90

88

86

84
OpenFace VGG16 VGG19

Accuracy % precision recall F1 score

VGG19 achieved promising results in classifying facial expressions and eye movements. This
model also demonstrated robust performance in analyzing diverse datasets comprising facial
images of both autistic and non-autistic children. Through iterative training and validation, the
VGG19 algorithm effectively extracted and analyzed features that indicates autism in children.
Further training could enhance its accuracy for early diagnosis of Autism Spectrum Disorder.
Conclusion
In conclusion, the development and testing of the ASD detection system represent a
significant advancement in leveraging AI and facial expression analysis for early diagnosis of
autism spectrum disorder. Through meticulous design and implementation, the system offers a
comprehensive solution for processing neuroimaging and facial image data, extracting relevant
features, and making accurate diagnostic predictions. The integration of advanced machine
learning techniques, such as convolutional neural networks (CNNs) and the VGG19 model,
ensures robustness and efficiency in analyzing complex image data. Additionally, the system's
modular architecture allows for seamless integration of individual components, facilitating
flexibility and scalability in adapting to diverse clinical and research environments. Overall, the
ASD detection system holds immense promise in revolutionizing early diagnosis and intervention
strategies for individuals with autism spectrum disorder, ultimately leading to improved outcomes
and quality of life.

References
1. Bours, C.C.A.H., Bakker-Huvenaars, M.J., Tramper, J., Bielczyk, N., Scheepers, F., Nijhof,
K.S., Baanders, A.N., Lambregts-Rommelse, N.N.J., Medendorp, P., Glennon, J.C. and
Buitelaar, J.K., 2018. Emotional face recognition in male adolescents with autism spectrum
disorder or disruptive behavior disorder: an eye-tracking study. European child & adolescent
psychiatry, 27, pp.1143-1157.
2. Almourad, M.B. and Bataineh, E., 2020, February. Visual attention toward human face
recognizing for autism spectrum disorder and normal developing children: An eye tracking
study. In Proceedings of the 2020 the 6th international conference on e-business and
applications (pp. 99-104).
3. Kang, J., Han, X., Song, J., Niu, Z. and Li, X., 2020. The identification of children with autism
spectrum disorder by SVM approach on EEG and eye-tracking data. Computers in biology and
medicine, 120, p.103722.
4. Elbattah, M., Guérin, J.L., Carette, R., Cilia, F. and Dequen, G., 2022, February. Vision-based
Approach for Autism Diagnosis using Transfer Learning and Eye-tracking. In HEALTHINF
(pp. 256-263).
5. Ahmed, I.A., Senan, E.M., Rassem, T.H., Ali, M.A., Shatnawi, H.S.A., Alwazer, S.M. and
Alshahrani, M., 2022. Eye tracking-based diagnosis and early detection of autism spectrum
disorder using machine learning and deep learning techniques. Electronics, 11(4), p.530.
6. Kanhirakadavath, M.R. and Chandran, M.S.M., 2022. Investigation of eye-tracking scan path
as a biomarker for autism screening using machine learning algorithms. Diagnostics, 12(2),
p.518.
7. Black, M.H., Chen, N.T., Iyer, K.K., Lipp, O.V., Bölte, S., Falkmer, M., Tan, T. and Girdler,
S., 2017. Mechanisms of facial emotion recognition in autism spectrum disorders: Insights from
eye tracking and electroencephalography. Neuroscience & Biobehavioral Reviews, 80, pp.488-
515.
 8. Yi, L., Fan, Y., Quinn, P.C., Feng, C., Huang, D., Li, J., Mao, G. and Lee, K., 2013. Abnormality
in face scanning by children with autism spectrum disorder is limited to the eye region:
Evidence from multi-method analyses of eye tracking data. Journal of vision, 13(10), pp.5-5.

9. Zhao, Z., Tang, H., Zhang, X., Qu, X., Hu, X. and Lu, J., 2021. Classification of children with
autism and typical development using eye-tracking data from face-to-face conversations:
Machine learning model development and performance evaluation. Journal of Medical Internet
Research, 23(8), p.e29328.
10. Tang, J., Falkmer, M., Horlin, C., Tan, T., Vaz, S. and Falkmer, T., 2015. Face recognition and
visual search strategies in autism spectrum disorders: Amending and extending a recent review
by Weigelt et al. PloS one, 10(8), p.e0134439.
11. Sterling, L., Dawson, G., Webb, S., Murias, M., Munson, J., Panagiotides, H. and Aylward, E.,
2008. The role of face familiarity in eye tracking of faces by individuals with autism spectrum
disorders. Journal of Autism and Developmental Disorders, 38, pp.1666-1675.
12. Kollias, K.F., Syriopoulou-Delli, C.K., Sarigiannidis, P. and Fragulis, G.F., 2021, July. The
contribution of machine learning and eye-tracking technology in autism spectrum disorder
research: a review study. In 2021 10th International Conference on Modern Circuits and
Systems Technologies (MOCAST) (pp. 1-4). IEEE.
13. Carette, R., Cilia, F., Dequen, G., Bosche, J., Guerin, J.L. and Vandromme, L., 2018. Automatic
autism spectrum disorder detection thanks to eye-tracking and neural network-based approach.
In Internet of Things (IoT) Technologies for Healthcare: 4th International Conference,
HealthyIoT 2017, Angers, France, October 24-25, 2017, Proceedings 4 (pp. 75-81). Springer
International Publishing.
14. Oliveira, J.S., Franco, F.O., Revers, M.C., Silva, A.F., Portolese, J., Brentani, H., Machado-
Lima, A. and Nunes, F.L., 2021. Computer-aided autism diagnosis based on visual attention
models using eye tracking. Scientific reports, 11(1), p.10131.