Psychometric Properties and Clinical Utility of The Executive Function Inventory For Children and Adolescents - A Large Multistage Populational Study Including Children With ADHD.

Applied Neuropsychology: Child
ISSN: 2162-2965 (Print) 2162-2973 (Online) Journal homepage: https://www.tandfonline.com/loi/hapc20
Psychometric properties and clinical utility of

the executive function inventory for children and
adolescents: a large multistage populational study
including children with ADHD
Marco Antônio Arruda, Renato Arruda & Luis Anunciação
To cite this article: Marco Antônio Arruda, Renato Arruda & Luis Anunciação (2020): Psychometric
properties and clinical utility of the executive function inventory for children and adolescents: a large
multistage populational study including children with ADHD, Applied Neuropsychology: Child, DOI:
10.1080/21622965.2020.1726353
To link to this article: https://doi.org/10.1080/21622965.2020.1726353
Published online: 02 Mar 2020.
Submit your article to this journal
Article views: 125
View related articles
View Crossmark data
Full Terms & Conditions of access and use can be found at

https://www.tandfonline.com/action/journalInformation?journalCode=hapc20
APPLIED NEUROPSYCHOLOGY: CHILD
https://doi.org/10.1080/21622965.2020.1726353
Psychometric properties and clinical utility of the executive function inventory

for children and adolescents: a large multistage populational study including
children with ADHD
^nio Arrudaa, Renato Arrudab, and Luis Anunciaç~aoc
Marco Anto
a
Glia Institute, Ribeir~ao Preto, Brazil; bDepartment of Neuroscience and Behavioral Sciences, Ribeir~ao Preto Medical School, University of S~ao
Paulo, Ribeir~ao Preto, Brazil; cDepartment of Psychometrics, Federal University of Rio de Janeiro, Rio de Janeiro, Brazil
ABSTRACT KEYWORDS
Executive functions (EF) are a set of high order mental abilities that regulate cognition, emotions, Psychometrics; executive
and behavior. This study aims to report the construction and validation of a rating scale instru- functions; ADHD; child
ment for EF in children and adolescents aged from 5 to 18 years (EFICA), as well as to report the development
results of a comparison between children with ADHD and their peers without it. Thus, we con-
ducted a population-based cross-sectional study relying on a sample composed of 3,284 typical
children and adolescents accessed to study the psychometric properties of the parents’ inventory
(EFICA-P) and the teacher’s inventory (EFICA-T) within a Structural Equation Modeling framework
(SEM). Exploratory and confirmatory analyses were fitted, as well as the Cronbach’s alpha and the
McDonald’s omega reliability indices. The known-groups method was carried out by independent
Welch t-tests between untreated ADHD children and their peers. We concluded that the parents’
inventory is composed of three dimensions (Cool Index 1, Cool Index 2, and Hot Index): v2 (1,649)
¼ 4,607.852 p < .01, CFI ¼ .965, TLI .963, RMSEA ¼ .053, whereas the teachers inventory is com-
posed of two dimensions (Cool Index and Hot Index): v2 (1,273) ¼ 5,158.240, p < .01, CFI ¼ .991,
TLI ¼ .991, RMSEA ¼ .077. The internal consistency of both inventories was >.9. Significant differ-
ences between the ADHD groups were found in all domains accessed. These findings indicate
that both inventories have a high degree of validity regarding their internal structures, as well as
supporting their clinical utility.
Introduction multiple results which are not entirely related or convergent

(Barkley, 2001; Sijtsma, 2013). Broad definitions are often
Psychology in general and psychometrics in particular deals
reported in the specific literature, and when one defines EF
with latent process (factors, abilities), a class of variables not
as a broad cluster of frontal-lobe dependent high order cogni-
directly observed from the data, but supposed to exercise a tive and emotional abilities necessary for regulating and man-
causal influence to the observed behavior (Sijtsma, 2006). aging ongoing and goal-directed behaviors, despite being an
Analogous to other scientific disciplines, the theories within umbrella term, this explanation offers an important link
psychological science aim to enable one to explore, system- which brings together fields such as Psychology, Psychiatry,
atize, and predict observational experience by constructing and Neurology (Best & Miller, 2010; Blair & Raver, 2012;
models which mimic the phenomena of interest (Figueroa, Diamond & Lee, 2011; Poon, 2018; Prencipe et al., 2011).
2016). On the other hand, there has been an increasing However, the accommodation of distinct theoretical roots
awareness that several theories suffer from a contradictory, does not always fit as expected. From a historical and aca-
incomplete, insufficient, or even a lack of evidence, even demic perspective, EF as a concept comes with the tough
when considering theories as a fundamental piece of any sci- mission of filling out or providing answers to some gaps on
entific discipline within the psychological field, especially how specific cognitive processes are controlled and coordi-
when the goal encompasses the univocal/specific definition nated during the performance of complex cognitive tasks
of the objects intended to study (Boring, 1953; (Miyake et al., 2000). At least three broad questions emerge
Muthukrishna & Henrich, 2019). within the development of EF as a concept, and they can be
This statement holds true for many psychological proc- further detailed. First, (i) which psychological processes
esses, including in this group Executive Functions (EF). EF comprise EF; second, (ii) how can these processes be
are elusive to define, therefore allowing the existence of dif- grouped differently, and (iii) what are the roles of emotional
ferent conceptualizations. This consequently nurtures the and motivational aspects in EF? (Barkley, 2012; Gioia,
development of dissimilar standardized measures with Isquith, Retzlaff, & Espy, 2003; Goldstein & Brooks, 2013).
CONTACT Luis Anunciaç~ao [email protected] Department of Psychometrics, Federal University of Rio de Janeiro, Pasteur Avenue, 150, Rio de Janeiro 22290-
902, Brazil.
This article has been republished with minor changes. These changes do not impact the academic content of the article.
ß 2020 Taylor & Francis Group, LLC
2 M. A. ARRUDA ET AL.
The processes which are involved in EF have undergone conditions to a broader and more comprehensive construct
some metamorphoses throughout the years (Jurado & (Peterson & Welsh, 2014). The importance of emotion regu-
Rosselli, 2007). The first description was made by Baddeley lation on EF performance in different contexts and demands
and Hitch in 1974 within a framework in which the aim has led to consensus in the literature (Bechara, 2004; Zelazo,
was to check whether the hypothesis of a “central executive” 2007), also overcoming a previous understanding that EF
component was reasonable for a working memory model could be represented as a unidimensional structure (Gioia
(Baddeley & Hitch, 1974). Next, in 1982, Lezak proposed to et al., 2003; Goldstein & Naglieri, 2014). In this same regard
cluster EF into four major dimensions of human behavior and according to Luria’s postulate and current evidence, the
which deal with “how” or whether behavior is expressed, in optimal interaction between neurobiological processes and
contrast to cognitive functions concerned with “what and environmental stimuli involving cultural, educational, and
how much knowledge, skill, and intellectual equipment a social factors is responsible for the cortical efficiency to pro-
person may possess” (Lezak, 1982, p. 282). According to cess cognitive skills such as language, attention, memory,
Lezak, these dimensions were capacities: (1) necessary for intelligence, and EF (Luria, 1966; Barkley, 2012; Gioia et al.,
formulating goals; (2) involved in planning; (3) having to do 2003; Goldstein & Brooks, 2013). Concurrently with individ-
with carrying out plans to reach the goals; and (4) for per- ual biological factors, culture modulates patterns of human
forming these activities effectively. knowledge development, belief system, and consequent
The Supervisory Attentional System (SAS) was proposed behaviors. The impact of cultural and educational aspects on
in virtually in the same period by Norman and Shallice executive functioning has been shown in studies comparing
(1986), wherein the authors explored the role of attention in children and adults from Western and Eastern countries
behavior control. The interaction of all conceptualizations (Goldstein & Brooks, 2013; Lan, Legare, Ponitz, Li, &
led to a contemporary and almost well-accepted EF core Morrison, 2011; Luria, 1966).
composed of inhibitory control, working memory, and cog- Several reasons justify the importance of assessing EF in
nitive flexibility (Diamond, 2013, Miyake et al., 2000). children and in adolescents, since EF are likely to have a sig-
Despite this core, the attempt to explore the psychological nificant impact on the development of everyday life skills.
processes related to EF is still appealing nowadays, and has Since EF plays an essential role in cognitive development, its
originated a multifaceted mental concept, including more relationship with emotional regulation, and inhibitory con-
than 30 different components assumed to be interrelated trol is regarded as part of the development of key outcomes
(Barkley, 2001). of child development (Sun & Buys, 2012), as well for mental
Within this context, a new theoretical EF framework has health (Han et al., 2016; Kertz, Belden, Tillman, & Luby,
been recently proposed in the literature aiming at grouping 2016; Shanmugan et al., 2016; White et al., 2017; Wilson,
EF processes, as represented by cool and hot EF (Peterson & Andrews, Hogan, Wang, & Shum, 2018), socioemotional
Welsh, 2014; Poon, 2018; Prencipe et al., 2011; Schreiber, skills (Holmes, Kim-Spoon, & Deater-Deckard, 2016; Walk,
Possin, Girard, & Rey-Casserly, 2014). Cool EF are mainly Evers, Quante, & Hille, 2018), and school achievement
composed of goal-directed and future-oriented behaviors. It (Cantin, Gnaedinger, Gallaway, Hesson-McInnis, & Hund,
comprises skills such as inhibition, flexibility, planning, 2016; Oberer, Gashaj, & Roebers, 2018; Viterbori, Usai,
working memory, and monitoring which are required/or Traverso, & De Franchis, 2015).
emerge under analytical testing conditions, relatively decon- The assessment of EF in children is challenging
textualized, and low- to non-emotional (Peterson & Welsh, (Chevignard, Catroppa, Galvin, & Anderson, 2010; Miyake
2014). On the other hand, hot EF is also related to goal- & Friedman, 2012), especially in terms of psychological
directed and future-oriented cognitive skills, but the main instrumentation. Assuming Neuropsychology as the area
difference is the tension between immediate gratification interested to study the brain-behavior relationships, clinical
and long-term reward, which elicits an interplay with emo- observation and psychometrically defined behavioral exam-
tion and motivation (Poon, 2018; Prencipe et al., 2011). ination go hand in hand (Barth et al., 2003). The develop-
Furthermore, other complex processes have been supple- ment and use of neuropsychological testing has become a
mented in addition to the classic tripod EF components, cornerstone for clinicians. These instruments are based on
such as planning, organizing, attention, initiation, self- the assumption of an intrinsic relationship between cogni-
regulation, and self-monitoring (Gioia et al., 2003; Goldstein tion and behavior, and rely on the comparison between
& Naglieri, 2014). groups as well the performance obtained via single-subjects
The role of emotion and motivation in conceptualizing design. Thus, they enable to check how well various cogni-
EF has been neglected in the past, despite the evidence com- tive functions are operating, in addition to exploring which
ing from animal models and human frontal lobe lesions areas of the brain are functioning normally and which areas
which have been well-described in the seminal works of are having problems in helping to determine if a “person’s
Luria, Damasio, and others (Damasio, 1994; Fuster, 1973; decision-making capacity may be impaired and why this
Spence, 1995). However, evidence coming from neuro- might be happening” (Zasler, Katz & Zafonte, 2007,
physiological, neuropsychological, and clinical studies has p. 1211).
changed the traditional concept of EF in the last two deca- Within assessment built on instrument-based strategies,
des from purely cognitive skills which are elicited under test-based and questionnaire-based approach coexists and
relatively abstract, decontextualized, and non-affective both play a pivotal role in clinical neuropsychology
APPLIED NEUROPSYCHOLOGY: CHILD 3
(Batrancourt et al., 2010). With regard to test-based frame- above-mentioned instruments, the BRIEF (Carim, Miranda,
work, performance-based neuropsychological tests comprise & Bueno, 2012) and the CHEXI (Trevisan, Dias, Berberian,
systematic procedures for their application, scoring, and & Seabra, 2017) have been translated, but only partially
interpretation of structured tasks. We call these instruments adapted to the Brazilian population. The data relied on con-
ecological measures (Woods et al., 2019) when these tools venience sampling with a lack of standardization or norma-
simulate or mimic daily life activities, or make this evalu- tive data. All analyses only achieved exploratory results, and
ation possible through behavioral observation with naturalis- the use of these tools nowadays additionally depends on
tic stimuli. Finally, functional instruments refer to rating payment due to copyrights, and consequently prevent their
scales fulfilled by the parents, caregivers and/or teachers to use, especially within the primary care area in develop-
evaluate child behaviors that mirror their executive functioning countries.
ing (Gillen, 2010; Lezak, Howieson, Bigler, & Tranel, 2012). Given the above, this study has two specific goals. First,
A lack of reliability, predictive value and ecological valid- by taking advantage of a large multistage populational study
ity of performance-based tools have been argued in the lit- in which students were identified in the school system and
erature (Barkley, 2012; Lezak et al., 2012). One of the most the parents and teachers were directly interviewed, we aimed
important limitations is the structured nature of the tasks, to develop an EF rating scale instrument for children and
assigning the child’s executive control, orienting, organizing, adolescents aged from 5 to 18 years. Second, we also sought
and self-monitoring to the examiner, and therefore masking to increase the clinical utility of this new tool by developing
the child’s executive dysfunction (Barkley, 2012). Although a percentile-based norm which can help professionals screen
some authors impute the lack of inter-observers and (inter- for the development of executive functions within children
setting agreement) of functional scales and questionnaires as and adolescents.
a cause of the limited validity and reliability of (functional
assessment) (Chevignard et al., 2010), these instruments are
able to provide useful clinical information on the discrep- Methods
ancy of child executive functioning in (different) contexts. Overview
There are some functional rating scales to assess EF in
children and adolescents (Goldstein & Naglieri, 2014), with The present study is a population-based cross-sectional
the most cited being: the Behavior Rating Inventory of study based on a multi-stage probability sample survey con-
Executive Function (BRIEF) (Gioia et al., 2003), Barkley ducted in two cities located in the state of Minas Gerais,
Deficits in Executive Functions Scale for Children and southeast region of Brazil. This is a stratified sampling
Adolescents (BDEFS-CA) (Barkley, 2012), Delis Rating of method that takes into account the number of children in
Executive Functions (D-REF) (Diamond & Lee, 2011), the each place to adjust the final sample to the population
Comprehensive Executive Function Inventory (CEFI) information.
(Goldstein & Naglieri, 2014) and the Childhood Executive The first stage was conducted in 2013 in the city of S~ao
Function Inventory (CHEXI) (Thorell & Nyberg, 2008). Sebasti~ao do Paraıso, which covers an area with 64,980
Although reliable and valid instruments for functional inhabitants, 92.3% of them living in the urban area.
assessment of EF in children and adolescents are mainly According to the 2010 Brazilian demographic census, the
available in the United States, significant gaps still exist for Human Development Index (HDI) was 0.722, life expect-
Portuguese-speaking countries (such as Brazil, Portugal, and ancy 75.2 years, and the fecundity rate was 1.9 (IBdGeE,
Mozambique), and communities within specific states in the 2010c). The HDI is a summary measure for assessing long-
United States. term progress in three basic dimensions (a long and healthy
First, the strong influence of cultural and socioeconomic life, access to knowledge, and a decent standard of living).
aspects on EF requires enormous efforts to develop new Its scale ranges from 0 to 1, and the higher the score, the
instruments or adapt and validate the available ones across greater the development.
different languages and diverse realities, particularly in The second stage was carried out in 2016 in the city of
low and middle-income countries (Willoughby, Piper, Delfinopolis which has a total of 6,830 inhabitants, with
Kwayumba, & McCune, 2019). Secondly, from the psycho- 70.9% of them living in the urban area, HDI of 0.740, life
metric point of view, a very different methodology has been expectancy of 77.3 years, and a fecundity rate of 2.3
used to evaluate the construction, standardization, reliability, (IBdGeE, 2010b).
and validity of the available instruments, thereby making it The Federal Government in Brazil regulates the country’s
difficult to compare them (Goldstein & Naglieri, 2014). educational system through the Ministry of Education,
Thirdly, diverse approaches are applied in the standardized which is equipped with public and private sector schools.
samples used to create norms, for example, some authors While private schools are privately financed, public schools
only provide percentile scores (BDEFS-CA), others provide are funded by taxes and do not demand any direct payment.
T-scores (BRIEF) or mean of 100 and standard deviation of City governments coordinate the elementary schools and
15 (CEFI) (Goldstein & Naglieri, 2014). middle and high schools are managed by the state govern-
To the best of our knowledge, there is no specifically ment. Because of the consortium characteristic of this study,
developed, validated and standardized functional instrument access in S~ao Sebasti~ao do Paraıso was only permitted for
to evaluate EF in Brazilian children and adolescents. Of the elementary schools, whereas we could only evaluate children
in middle school in Delfin opolis. School is compulsory for Table 1. Target sample, consented, and final sample according to demo-
graphic features.
all children between seven and 14 years of age.
Age group N %
5 and 6 years old 1,017 31
Participants 7 and 8 years old 983 29.9
9 and 10 years old 807 24.6
Two different groups with responses of children and adoles- 11 and 12 years old 273 8.31
13 and 14 years old 88 2.68
cent behaviors were composed for the analytical procedures of 15 and 16 years old 72 2.19
this research. The first group was mainly used to explore the 17 and 18 years old 44 1.34
internal structure of the instrument (its psychometric mod- X2(6) ¼ 2,559, p < 0.01
Gender
els), and therefore, we called it the validation group. The Female 1,638 49.9
second group was called the clinical group, being useful to Male 1,646 50.1
X2(7) ¼ 0.02, p ¼ 0.9
promote the clinical utility of the instruments and to demon- Race (self-reported)
strate the relation with other variables and response processes. White 2,169 66
An independent sample of children from two cities (S~ao Nonwhite 863 26.3
Non-respondents 252 7.7
Sebasti~ao do Paraıso, and Delfin opolis) were assembled to X2(2) ¼ 1,752, p < 0.01
compose this first group. In S~ao Sebasti~ao, we aimed to Area
assess all children from 5 to 12 years registered in any of the Urban 364 11.1
Rural 2,920 88.9
12 public elementary schools, which are managed by the city X2(1) ¼ 1,989, p < 0.01
government (1st to 5th grades in the American educational Income levelsa
system). The target sample within these children was 3,658 A 42 1.3
B 1,027 31.3
participants, in which parental consents were obtained from C 1,925 58.6
3,581 (97.9%), and 501 (14%) children were excluded due to D, E 290 8.8
the following criteria: diagnosis of Attention-Deficit/ X2(3) ¼ 2,619, p < .01
House-hold head educational status
Hyperactivity Disorder (ADHD) (n ¼ 160) or ASD (n ¼ 7), Illiterate or incomplete middle school 1,602 48.8
visual impairment (n ¼ 94), hearing impairment (n ¼ 47), Completed middle school but incomplete high school 651 19.8
intellectual disability (n ¼ 37), children using psychoactive Completed high school or more 956 29.1
Not determined 75 2.3
drugs (n ¼ 40), or with more than 10% of missing responses X2(3) ¼ 40,540, p <.01
in the parent and/or teacher forms (n ¼ 116). Thus, the final Total 3,284 100
a
sample of this stage was 3,080 (84.2%) children. Income levels defined according to the Brazilian Economic Classification
In Delfinopolis, all children from 11 to 18 years registered Criteria (A-ABdEd., 2015).
in the only public school of the city with elementary and
middle school (6th to 12th grades) (n ¼ 401) were taken into
account. Thus, the target sample in this city was 270 adoles-
cents, in which parental consents were obtained from 253
(93.7%), and 49 (18.1%) were excluded due to the following
criteria: diagnosis of ADHD (n ¼ 5), visual impairment
(n ¼ 8), hearing impairment (n ¼ 3), intellectual disability
(n ¼ 4), psychoactive drug usage (n ¼ 2), or those with more
than 10% of missing responses (n ¼ 27). The final sample of
this stage was 204 (75.6%) adolescents.
With that being said, the validation group was composed
of 3,284 questionnaire responses from typical children and
adolescents aged from 5 to 18 years of age (Mean age ¼ 8.2,
SD ¼ 2.0), 50.1% boys, 11.1% living in rural area, 66%
white, and 58.6% from middle income level (class C)
(Table 1). As both scales were developed for parent-report,
all data were reported by guardians or 173 school teachers
(96% females).
Within this group, 79.8% (n ¼ 2,621) of the parents’ ques-
tionnaires were completed by the mother, 12.7% by the
father, and 7.5% by another child’s guardian. Children
included in this sample were subjected to the following cri-
teria: I. no visual, hearing, or intellectual impairment; II. no Figure 1. Flow chart shows sample population selection, elegible and excluded
history of special education; III. no psychoactive drug usage; participants.
IV. Questionnaires with less than 10% of missing responses;
and V. not fulfilling the last version of the Diagnostic and or Autism Spectrum Disorder (ASD). The number of chil-
Statistical Manual of Mental Disorders (5th ed.; DSM-5) cri- dren and adolescents of both stages selected in the different
teria for Attention Deficit Hyperactivity Disorder (ADHD) phases of the eligible process is shown in Figure 1.
The second group (clinical group) was formed by 165 including BRIEF and CHEXI. Thus, the item development
children previously diagnosed with ADHD and not conduct- of the EFICA started with a comprehensive review of the lit-
ing any treatment at the time of the research. This group erature and authors’ clinical and research experience based
was formed of 126 (76%) of boys, with the mean age of on best-practices recommendations (Roth, Isquith, & Gioia,
7.7 ± 2.16 years, and about 78% from families with low socio- 2014; Streiner, Norman, & Cairney, 2015). The first author
economic status. The data from these children were firstly has 30 years of clinical experience in evaluating and treating
collected through a semi-structured questionnaire based on children and adolescents with executive dysfunction caused
the Swanson, Nolan and Pelham Questionnaire (SNAP-IV) by a broad variety of causes.
Teacher 18-Item Rating Scale (Swanson, 2003), and filled All items were created to assess the traditional compo-
out by their parents. We also asked the parents if their son/ nents of executive functioning, and the broad range of abil-
daughter has had a diagnosis of ADHD given by a special- ities dependent and/or related to executive functioning.
ized professional (physician or psychologist), and a specific Therefore, the content was created to assess the patient per-
question asking “Has your son/daughter taken Ritalin, formance on real world behavior, i.e., daily routine tasks, as
Vyvanse, or Concerta in the past?”. These children were well to explore typical situation in which children should
reevaluated via consultation with a medical doctor (pediatri- have an adequate emotional regulation or response with
cian) or a psychologist with Specialization in neuropsych- peers, and to plan, organize or modify their behavior to
ology, and their diagnosis was confirmed if the diagnostic achieve the desired goals.
criteria present in DSM-5 were present. The Parents’ version (EFICA-P) encompassed behaviors
This study and the surveys received full approval from especially performed at home, such as “Leaves the light on,
the Human Research Committee at the S~ao Jose do Rio door open or wet towels on top of the bed, even after being
Preto School of Medicine (State of S~ao Paulo). Written told several times”, “Explodes or gets angry when he/she is
informed consent was discussed and obtained from the contradicted” and/or “Interrupts others, doesn’t know how
parents (or from the child’s guardian). to wait for his/her turn to talk.” The Teachers’ version
(EFICA-T) had some particularities. As they are able to rec-
ognize child behaviors in the classroom, this version
Measures
explored typical situations, asking if the child “gets up from
Executive function inventory for children and adoles- their seat when they are supposed to be sitting”, “Loses their
cents (EFICA) snack/sandwich, or their snack money and their homework”
Both EFICA-P (parents’ version) and EFICA-T (teachers’ and/or “Leaves their belongings scattered everywhere they
version) comprises a standardized assessment tool developed go throughout the classroom and the school”.
to comprehensively assess dysfunctional behaviors related to The initial pool of 180 items (95 items for the EFICA-P
EF. Their data are collected by individualized assessment and 85 items for the EFICA-T) were reviewed and screened
only, in which it is expected that test users have to identify redundancy, idiosyncrasy, and low specificity of
knowledge of child development and are able to perform EF domains according to the authors. As these are intensely
some mathematical computations in order to score and debated in the literature, the fine differences and nuances
interpret the results adequately. These characteristics include which separate executive function deficits from ADHD and
EFICA-P and EFICA-T on the Level B test, following learning disorders can often be confusing, but since both
the traditional recommendations of the (American clinical conditions are related, items with content which
Psychiatric Association, 2013). might overlap with other conditions were not summarily
Parents and teachers are instructed to use a 0-2 rating excluded in this part of the study (Barkley, 2001, 2012;
scale indicating if each specific behavior occurs: never or Thorell & Wåhlstedt, 2006). This first step also intended to
almost never, (0), sometimes (1), or always/almost always keep items immediately comprehensible to the target popu-
(2). All items are summed to produce one final score in lation, unambiguous, assertive, easy to use and culturally
order to preserve its clinical utility, being inspired by other related (Streiner et al., 2015). A total of 63 items were
tools designed to assess child development, and higher removed, resulting in 65 items for the EFICA-P and 52 for
scores indicate/suggest more problems (Gioia et al., 2003; the EFICA-T. The remaining 117 items were randomly com-
Squires & Bricker, 2009; Squires, Bricker, Heo, & Twombly, bined in one pool and were submitted to two independent
2001). Percentile cutoffs were used to aid clinicians to iden- and licensed child neuropsychologists with more than
tify children needing further evaluation. Raw scores located 10 years of clinical practice. They were asked to identify the
at the 90th percentile are considered in the monitoring best fit EF domains for each item. When more than one
zone, and raw score located at the 95th percentile considers domain was recognized, they were asked to rank them as
that the child could be at risk of developmental delay or dis- primary or secondary domains.
ability in developing executive functions. The agreement between raters was measured by the
Similar to other measurement instruments within psych- Cohen’s kappa and reached a high value (k or r 0.75)
ology and neuropsychology areas, the EFICA elaboration with .79 for EFICA-p (p < .001) and also .79 for
was based on a theoretical paradigm (Hot and Cold EF) and EFICA-t (p < .001).
its relationship with other traditional standardized functional As recommended when developing new tools especially
instruments used to assess EF in children and adolescents, designed to assess child abilities, engaging small groups of
local key informants (i.e., parents and teachers working with cases in which missing values were lower than or equal to
young children) is an ideal way to collect information on 10%, whereas observations in which missing values were
the test content and procedures (Fernald, Prado, Kariger, & higher than this threshold were not included in the analysis
Raikes, 2017). Thus, the evidence based on test content (i.e., (Di Zio & Guarnera, 2010).
evidence associated with item content and item format or The data was described using plots, and descriptive statis-
face and content validity) was also examined by two focus tics. The participants’ characteristics were analyzed in terms
group discussions with the first author. They were not of their age, gender, race, area of living, income level, and
organized to generate the items, but with the objective for household head educational status. Race was divided into
the research team to receive feedback on understanding the three categories: white, nonwhite, and non-respondents. The
items from those with the same profile which EFICA was five income levels were defined according to the Brazilian
built to assess. Economic Classification Criteria, which are determined
The first group was composed by six pairs of parents based on a combination of variables such as consumption
(EFICA-P) and the second one by eight teachers (EFICA-T) of certain goods, and household characteristics
who were randomly selected from the 12 public elementary (A-ABdEd, 2015).
schools in Sao Sebasti~ao, and from the only public school of A dataset composed of typical children was used
the city with elementary and middle school (Delfin opolis) (n ¼ 3,284) to investigate the psychometric properties of
(see participants section for more details). A two-thirds both forms of EFICA (described in Participants section).
majority consensus opinion was required. The recommended The data was randomly divided into two subsets to execute
changes were mostly related to wording of the items and the exploratory and confirmatory analyses in accordance
were implemented. with best practices in psychometrics literature. The explora-
In summary, both questionnaires were developed aiming tory analyses were conducted on one part of the sample,
at capturing daily life behaviors related to executive function- and the confirmatory analyses were performed on the
ing of children and adolescents aged from 5 to 18 years old, second one (Costello & Osborne, 2011). Sample-splitting is
asking their parents and/or teachers to indicate if each spe- generally considered a cross-validation method (Brown,
cific behavior happens: never (or almost never), sometimes, 2015; Osborne, 2011).
or always (or almost always), on a three-point rating scale. The results of the Kaiser–Meyer–Olkin (KMO) and
Bartlett’s test were interpreted before performing an
Exploratory Factor Analysis (EFA). A sample is deemed
Procedure
adequate if the value obtained by KMO is equal or greater
One of the parents or the child’s guardians was invited to than 0.6 in addition to a significant result on Bartlett’s Test
attend a school meeting when the study was presented, and of Sphericity. This latter result indicates that the matrix is
they completed the questionnaires while being supervised by not an identity matrix, and is therefore suitable for EFA
the teachers and the first author. They also replied to a (Watson, 2017). Next, the EFA was performed to identify
structured general questionnaire assessing sociodemographic and determine the number and nature of the latent dimen-
characteristics, neuropsychomotor development, current and sions of the scales, as well as to reduce the number of varia-
past health and medical history of the child, and other psy- bles being dealt with. All items were treated as categorical,
chometric tools. and the estimator/extraction method used was the Weighted
The teachers were trained by the first author on how to Least Squares Means and Variance adjusted (WLSMV), as it
supervise the parents and also on how to fill out their ques- makes no distributional assumptions about the observed var-
tionnaires. Each school teacher in Brazil is responsible for iables (Suh, 2015). Models with 2–5 oblique dimensions
approximately 20–30 children in their classroom, and there- were estimated using a polychoric correlation. We note that
fore this was the number each teacher assessed. CFA of categorical variables and Item Response Theory 2-
Demographic characteristics of teachers were only partially PL models yield similar results.
collected since there are strict laws protecting their informa- The decision of how many factors to retain is a critical
tion, and therefore it was not possible to nest responses at component of Factor Analysis procedures. In this study, the
the teacher level. results of the eigenvalues scree-plot and the parallel analysis
Children from urban and rural areas were assessed as with 1,000 iterations were analyzed. The Confirmatory
long as they were enrolled in the school system, which is Factor Analysis (CFA) was then performed using the items
required by law in Brazil. from EFA which factor loadings were higher or equal to 0.3
(which represents about 9% of overlap in variance). Items
with factor loadings in two or more factors simultaneously
Analytical plan
were preserved in its higher factor.
Different steps of the data analysis process were carried out The CFA fit tends to be poorer than those achieved by
in order to achieve the aims proposed in the present EFA because constraints are implemented in CFA,
research. The dataset was initially explored aiming at finding whereas all indicators are loaded on all factors in EFA.
the presence of inconsistencies and missing data. No outliers Therefore, the CFA solution was considered appropriate if its
were excluded from the analysis. Multiple imputation pro- Goodness of Fit (GoF) index met certain criteria and its
cedure via Predictive Mean Matching (PMM) was used for interpretation was reasonable to the theoretical model. The
GoF criteria was met when the Comparative Fit Index (CFI) was .95 for both indices. All the items were positively corre-
or the Tucker-Lewis Index (TLI) results were higher or equal lated to the total score. However, the items 30 (“Can’t pay
than 0.9, and the Root Mean Square Error of Approximation attention to video games or computer games for a long
(RMSEA) or the Standardized Root Mean Square Residual time”), 33 (“Doesn’t do a final check on his/her tasks or
(SRMR) were less than 0.1 (Furr, 2011; Hu & Bentler, 1999). activities to look for possible mistakes”), 39 (“Doesn’t know
The reliability of the results was accessed via Cronbach’s how to keep a secret”), 51 (“Recognizes they have a problem
alpha, McDonald’s omega, and the Average Variance to solve”), and 64 (“Tries to solve a problem before asking
Extracted within the confirmatory framework. A one-way someone for help”) had an item-total correlation lower
ANOVA was carried out to check whether the difference than 0.3.
between the results from children’s guardians was significant. The results obtained by the KMO and the Bartlett test
Normative data and its respective tables were developed were checked before performing the Exploratory Factor
for both EFICA questionnaires based on the entire sample Analysis. The EFA was performed in half of the randomly
within a Classical Test Theory (CTT) framework specifically
divided data as the Bartlett test was significant at 5% prob-
addressed to emphasize the clinical utility of the question-
ability level and the KMO was .97. A solution between 2 to
naires and provide a useful resource to clinicians or primary
5 dimensions was explored and a solution with 3 dimen-
care professionals. Thus, a linear model was fitted for
sions had acceptable indexes in the confirmatory procedure:
EFICA-P and EFICA-T to check whether the gender, age,
and the interaction between these two factors had an effect v2 (1,649) ¼ 4,607.852 p < .01, CFI ¼.965 (Robust ¼
on the children’s total results, being composed of raw scores 0.902), TLI .963 (Robust ¼ 0.898), RMSEA ¼ .053 (Robust
of all the summed factors. A normative table was then cre- ¼ 0.044). The correlation between all dimensions were sig-
ated based on significant outcomes, being inspired by other nificant. Cool Index 1 and Hot Index correlated at .73, and
questionnaires commonly used for assessing child develop- Cool Index 1 with Cool index 2 correlated at .80. The Cool
ment (Gioia et al., 2003; Squires & Bricker, 2009). Since the index 2 correlated at .70 with the Hot Index. Item properties
higher the score on the questionnaires, the greater the child- are reported in Table 2.
ren’s dysfunctional executive behaviors, the score related to The difference between the respondents/guardians total
90th percentile was considered in the monitoring zone, and scores on the EFICA-P was evaluated and no statistically sig-
the score related to 95th percentile was considered that the nificant difference was found (F(2, 3,214) ¼ 2.12, p ¼ .12).
child could be at risk of developmental delay or disability in
developing executive functions, and so a further assessment
is recommended to determine the need for intervention EFICA-T
(Crawford & Garthwaite, 2009; Fonseca et al., 2006). The same process previously described was performed with
Finally, the known-groups method was used to further the data gathered by the EFICA-T. The KMO was 97 and
explore some features related to the validity evidence based the Bartlett test was significant at 5% probability level. The
on relations with other variables, as well as the clinical utility Cronbach’s alpha and McDonald’s omega were .98. The
of both forms. Independent Welch t-tests between ADHD high Cronbach’s alpha results are a consequence of the
and non-ADHD groups were carried out in all investigated majority of the respondents having checked the “never hap-
domains by both forms from a sample of 165 children previ- pens (or almost never)” category, as well as the high number
ously diagnosed with ADHD according to the DSM-5 criteria, of items in both questionnaires.
while Cohen’s d was used as the measure of the effect size of All items were positively correlated to the total score and
the difference. The known-groups technique is considered a the item-total correlation was higher than 0.3. The CFA
component of the traditional concept of validity studies and revealed an adequate fit for the two-factor model: v2 (1,273)
checks the extent to which a measure predicts different ¼ 5,158.240, p < .01, CFI ¼ .991 (Robust ¼ .954), TLI ¼
results between groups that are theoretically/previously .991 (Robust ¼ .952), RMSEA ¼ .077 (Robust ¼ .061). The
known to differ (Hattie & Cooksey, 1984). The Welch test correlation between both factors was .80 (p < .01). The
is more robust when compared to the traditional Student’s results are reported in Table 3.
t-test, and prevents type 1 error. All analyses were performed
in R (R Core Team, 2019) with tidyverse (Wickham, 2016),
lavaan (Rosseel, 2012), psych (Revelle, 2015), and ApaTables Normative data
(Stanley & Spence, 2018) packages.
EFICA-P and EFICA-T
A key feature of developing a new standardized measure is
Results its normative data. Raw scores are often difficult to interpret
and/or meaningless in the absence of further information.
Reliability and validity Therefore, normative data (and tables) provide information
EFICA-P which enable drawing useful inferences about the person’s
As described in the methods section, the EFICA-P data were relative performance. However, as EFICA-P and EFICA-T
accessed to check its internal structure (mainly its reliability results were completed with data from boys and girls aged
and validity). The internal consistency of all data obtained from 5 to 18 years old, testing the age and gender effects
by Cronbach’s alpha and the McDonald’s omega coefficient must be carried out to develop specific reference normative
Table 2. Items analysis, descriptive, and CFA results of EFICA-P (parents).

Confirmatory Factor Analysis Item description for the entire scale
k Standardized (loadings) k Unstandardized M SD r
Cool index 1
8 Leaves the light on, door open or wet towels on 0.715 1.082 1.08 0.71 0.53
top of the bed, even after being told
several times.
20 Forgets to bring homework to school, even if 0.699 1.058 0.32 0.54 0.44
they had done it.
17 Is lazy. 0.667 1.011 0.83 0.65 0.49
12 Messy, not able to keep his/her room, closet, or 0.66 1 0.90 0.73 0.5
desk organized.
26 Leaves his/her clothes spread around the house. 0.656 0.994 1.00 0.74 0.51
48 When asked to go do something, he/she forgets 0.655 0.993 0.59 0.63 0.51
what to do.
7 Makes careless mistakes 0.637 0.964 0.83 0.52 0.48
1 Leave everything until the last minute. 0.632 0.957 0.65 0.67 0.47
47 Loses their belongings or forgets where they 0.617 0.935 0.83 0.68 0.53
put them.
10 It takes too long to do tasks such as bathing, 0.581 0.88 0.96 0.76 0.48
changing clothes, and eating.
55 If you ask him/her to get three things from 0.573 0.868 0.62 0.67 0.43
someplace, they will forget to bring one
of them.
60 Has difficulty getting ready for school on time 0.568 0.86 0.75 0.79 0.45
and ends up being late if someone isn’t
pushing them to get ready.
57 Their notebooks, agendas and binders are 0.529 0.802 0.72 0.74 0.41
always disorganized.
6 Always arrives late to appointments or 0.497 0.752 0.27 0.5 0.35
commitments.
Cronbach’s a ¼.89
McDonald’s x ¼.73
Average variance extracted ¼.39
Hot Index
16 Instructions need to be repeated many times to 0.785 1.114 1.08 0.73 0.64
be obeyed.
4 Acts without thinking, has difficulty controlling 0.716 1.016 0.55 0.66 0.59
his/her behavior and reactions.
21 “Explodes”/gets angry when he/she is 0.705 1 0.88 0.71 0.55
contradicted.
45 Loses his/her control easier than others. 0.703 0.997 0.53 0.66 0.55
19 Stubborn, everything has to be his/her 0.697 0.989 0.80 0.72 0.56
way, inflexible.
23 Gets angry for no reason. 0.69 0.978 0.88 0.75 0.53
3 Overreacts excessively to little problems. 0.676 0.959 0.72 0.67 0.53
28 Mood changes for no reason. 0.658 0.934 0.70 0.68 0.53
25 Interrupts others, doesn’t know how to wait for 0.655 0.929 0.90 0.7 0.54
his/her turn to talk.
59 He/she has difficulty waiting their turn. 0.654 0.928 0.85 0.68 0.54
52 Resists accepting an alternative way to solve a 0.646 0.916 0.83 0.7 0.53
problem, isn’t flexible.
14 Excessively restless, never stops. 0.642 0.911 0.78 0.73 0.54
24 He/She becomes upset with changes in plans or 0.619 0.878 0.70 0.68 0.53
new situations to them.
18 Is repetitive, speaks or asks the same 0.615 0.873 0.84 0.74 0.53
things repeatedly.
50 Wants to have fun and get attention from 0.606 0.86 0.77 0.71 0.51
others at all times.
53 Focuses on unimportant details and does not 0.603 0.855 0.65 0.63 0.5
perceive “the big picture.”
42 Doesn’t tolerate frustrations, doesn’t accept 0.568 0.806 0.92 0.68 0.46
when something he/she wants goes wrong
or doesn’t work.
22 Talks too much. 0.565 0.802 1.17 0.71 0.44
32 Not able to wait for rewards, prefers to get 5 0.545 0.773 0.83 0.78 0.46
Reais today (5 Dollars) than wait a week to
get 10 Reais (10 Dollars).
11 Reckless/fearless, gets involved in 0.54 0.766 0.39 0.6 0.41
risky situations.
54 Gets frustrated and cries easily. 0.53 0.753 0.88 0.75 0.48
58 His/her moments of anger and crying are 0.444 0.63 0.98 0.74 0.34
intense but end quickly.
Average variance extracted ¼ .40
(continued)
Table 2. Continued.
Cool Index 2
46 Misses deadlines to deliver homework or meet 0.734 1.988 0.35 0.56 0.47
other commitments.
27 Even when knowing the answer, makes mistakes 0.712 1.928 0.78 0.61 0.5
due to being careless, impulsive, or from a
lack of attention.
37 Doesn’t finish what they start and quits soon 0.679 1.838 0.69 0.68 0.56
after encountering any difficulty.
44 Needs a “push” to start an activity, someone to 0.675 1.829 0.77 0.71 0.55
take initiative for him/her.
5 When speaking, he/she doesn’t finish one 0.633 1.715 0.42 0.6 0.47
subject and moves on to another.
62 Has difficulty staying focused on homework and 0.613 1.662 0.70 0.67 0.51
in the classroom.
31 Not good at solving problems without help 0.607 1.645 0.73 0.64 0.44
from another person.
2 It takes time to understand things. 0.606 1.641 0.63 0.62 0.5
35 Doesn’t perceive their own performance very 0.602 1.631 0.57 0.63 0.43
well, leaves an exam thinking it went well,
when in fact it didn’t.
13 Not able to evaluate if a problem has been 0.579 1.569 0.69 0.6 0.45
solved well.
38 Doesn’t know how to act/overcome new 0.563 1.525 0.81 0.63 0.42
situations or when confronting a problem.
34 Doesn’t think before speaking or acting. 0.547 1.481 0.86 0.64 0.46
43 Needs help from an adult to finish an activity. 0.537 1.455 0.94 0.62 0.45
40 Doesn’t have a good notion of the time 0.517 1.399 0.84 0.67 0.41
necessary to perform a determined task.
36 Doesn’t notice that his/her behavior is bothering 0.5 1.355 0.77 0.68 0.41
other people.
9 It takes time to understand jokes. 0.499 1.352 0.56 0.61 0.41
56 If you make plans to pick them up at a certain 0.458 1.241 0.60 0.7 0.33
time, they are never ready at the
scheduled time.
41 Doesn’t have creativity to solve day-to- 0.452 1.225 0.71 0.67 0.37
day problems.
65 Understands things very literally, has difficulty 0.437 1.183 0.81 0.66 0.34
understanding expressions with double/
idiomatic meanings.
39 Doesn’t know how to keep a secret. 0.415 1.125 0.88 0.74 0.28
29 Not able to pay attention to movies or cartoons 0.376 1.017 0.44 0.66 0.33
for prolonged periods.
30 Can’t pay attention to video games or computer 0.369 1 0.38 0.64 0.28
games for a long time.
33 Doesn’t do a final check on his/her tasks or 0.354 0.959 0.83 0.7 0.28
activities to look for possible mistakes.
Average variance extracted ¼.31
Cronbach’s a (total) ¼ .96
McDonald’s x(total) ¼ .90
Average variance extracted (total)¼ .36
v2 (1,649) ¼ 4,607.852p < .01, CFI ¼.965 (Robust ¼ 0.902), TLI .963 (Robust ¼ 0.898), RMSEA ¼ .053 (Robust ¼ 0.044)
r means the correlation of the item with the scale composed of the remaining item.
table norms to assist in the classification or description of On the other hand, the results of EFICA-T had a differ-
new examinees. ent pattern. Despite statistically null results for effects of age
A linear model was fitted for EFICA-P results considering (F(1, 3,280) ¼ .46, p ¼ .497), and gender (F(1, 3,280) ¼ .46,
the results of all the summed items, and no interaction p ¼ .500), an interaction effect was found between gender
effect between age and gender (F(1, 3,280) ¼ 1.06, p ¼ and age (F(1, 3,280) ¼ 17.35, p < .01, gp2 .01). When an
.302), nor gender alone (F(1, 3,280) ¼ .52, p ¼ .470) was interaction term is significant in a model, the interpretation
found. Contrasting these first results, a simple main effect of main effects is no longer the average effect but rather a
revealed that children’s age had a significant effect on the conditional effect, and therefore this result reflected the
overall results, but with a small to null effect size (F(1, development of the normative table for EFICA-T. Table 4
3,280) ¼ 9.47, p ¼ .002, gp2 0.01). reports the results.
Table 3. Item analysis and CFA results – EFICA-T (Teachers).

Cool index 1
8 Has difficulty focusing on school tasks and work. 0.92 1.02 0.66 0.69 0.79
25 Doesn’t have a good notion of the time required to 0.91 1 0.56 0.66 0.73
perform a determined task.
23 Has difficulty completing tasks and activities. 0.91 1 0.54 0.67 0.75
19 Needs a “push” to start an activity, someone to take 0.9 0.99 0.47 0.67 0.75
initiative for him/her.
48 Has difficulty maintaining concentration for 0.89 0.98 0.67 0.7 0.78
prolonged periods.
14 Has difficulty performing tasks that require more than 0.88 0.97 0.65 0.68 0.75
one step for their execution.
21 Does not finish homework and class activities. 0.88 0.97 0.5 0.65 0.72
30 Doesn’t know how to find adequate resources to solve 0.88 0.97 0.61 0.64 0.71
a problem.
17 Needs help and monitoring by an adult to keep doing a 0.88 0.96 0.61 0.72 0.74
task or another activity
49 Doesn’t finish an activity on time. 0.87 0.96 0.56 0.68 0.73
29 Starts their tasks at the last minute, puts things off/ 0.87 0.96 0.33 0.59 0.7
procrastinates.
36 Doesn’t take notes well. 0.87 0.95 0.54 0.64 0.71
12 They are easily distracted by sounds and visual 0.85 0.94 0.82 0.69 0.73
stimulations.
52 Doesn’t understand simple instructions. 0.84 0.93 0.45 0.61 0.69
9 Doesn’t use creativity to solve problems. 0.84 0.93 0.66 0.66 0.67
1 Doesn’t perceive their own performance very well, leaves 0.83 0.92 0.65 0.65 0.66
an exam thinking it went very well, when in fact it
was very bad.
13 Forgets to bring homework to school, even if they had 0.83 0.92 0.51 0.63 0.68
done it.
32 Doesn’t try to solve a problem before asking the teacher 0.83 0.92 0.5 0.64 0.68
for help.
33 Has difficulty perceiving their abilities and weaknesses. 0.82 0.91 0.58 0.64 0.71
3 Loses their snack/sandwich, or their snack money and 0.82 0.91 0.26 0.49 0.61
their homework.
43 Has difficulty thinking about a different way to solve 0.82 0.9 0.64 0.66 0.66
a problem.
39 Can’t find what they need in the classroom or in 0.82 0.9 0.36 0.57 0.65
their desk.
37 Has difficulty remembering things, even after only a 0.81 0.89 0.41 0.59 0.66
few minutes.
46 Makes mistakes on tests even when knowing the answer 0.81 0.89 0.8 0.64 0.67
for being careless, impulsive, or inattentive.
18 Focuses on unimportant details and does not perceive 0.79 0.87 0.45 0.62 0.68
“the big picture.”
38 Doesn’t take initiative. 0.79 0.87 0.57 0.66 0.61
4 Doesn’t bring their homework, agenda, or 0.77 0.85 0.53 0.66 0.6
school materials.
6 Doesn’t do a final check on their tasks or activities to 0.76 0.84 0.78 0.68 0.62
look for possible mistakes.
7 Has good ideas, but can’t organize them, plan them, or 0.75 0.83 0.69 0.64 0.6
put them to paper.
47 Is sloppy on schoolwork and handwriting. 0.75 0.82 0.45 0.66 0.63
10 If you give them 3 things to go get, they forget one. 0.74 0.81 0.47 0.61 0.61
15 Doesn’t ask for help when they need it. 0.69 0.76 0.55 0.64 0.56
Cronbach’s a ¼ .99
McDonald’s x ¼ .94
Hot Index
44 Leaves everything/everywhere messy and others need to 0.92 1.1 0.24 0.5 0.64
clean up.
51 Is a disaster with their belongings, pens, pencils, erasers 0.92 1.1 0.29 0.56 0.68
always falling on the floor.
28 Loses their temper more than their friends. 0.91 1.09 0.25 0.52 0.6
40 Leaves their belongings scattered everywhere they go 0.9 1.08 0.26 0.52 0.65
throughout the classroom and the school.
35 Has problems when not supervised by an adult. 0.88 1.05 0.4 0.62 0.69
41 Doesn’t think about the consequences of their actions. 0.87 1.05 0.5 0.63 0.67
11 Their desk is a mess. 0.86 1.04 0.34 0.59 0.65
22 Doesn’t think before speaking or acting. 0.85 1.02 0.54 0.66 0.65
27 Gets up from their seat when they are supposed to 0.85 1.02 0.44 0.64 0.61
be sitting.
34 Speaks and plays loudly and at inappropriate moments. 0.85 1.02 0.43 0.63 0.64
(continued)
Table 3. Continued.
31 Resists or has difficulty accepting an alternative form of 0.84 1.01 0.3 0.55 0.62
solving a problem with their friends, tasks, or school
work, “it has to be done their way.”
45 Has difficulty waiting their turn. 0.84 1.01 0.42 0.6 0.61
26 Interrupts others, doesn’t wait for their turn to speak. 0.83 1 0.43 0.62 0.58
2 “Makes a mountain out of a molehill”, overreacts 0.83 1 0.36 0.59 0.57
excessively to little problems.
5 Gets upset over changing teachers or classes. 0.83 0.99 0.31 0.52 0.57
16 Has frequent mood swings for no reason. 0.83 0.99 0.34 0.58 0.57
50 Is repetitive, repeatedly speaks and asks the same thing. 0.82 0.99 0.25 0.51 0.6
42 Explodes/Becomes furious when they are contradicted. 0.81 0.97 0.37 0.6 0.55
24 Does not accept disappointments or reprimands. 0.81 0.97 0.48 0.64 0.59
20 Doesn’t notice that their behavior is bothering 0.79 0.95 0.57 0.67 0.6
other people.
Cronbach’s a ¼ .98
McDonald’s x ¼ .91
Cronbach’s a (total)¼ .99
McDonald’s x(total) ¼.96
Average variance extracted (total)¼ .71
v2 (1,273) ¼ 5,158.240, p < .01, CFI ¼ .991 (Robust ¼ .954), TLI ¼ .991 (Robust ¼ .952), RMSEA ¼ .077 (Robust ¼ .061)
Table 4. Normative table for EFICA-p and EFICA-t results.

EFICA-P (Parents’ form) EFICA-T (Teachers’ form)
Age group M SD P90 P95 N Gender M SD P90 P95 N
5 and 6 yo 44 18.9 69 77 1,017 Girls 21 20 50 61 498
Boys 27 22 58 70 519
7 and 8 yo 45 19.7 71 79 983 Girls 22 20 51 60 492
Boys 29 23 61 74 491
9 and 10 yo 43 19.6 70 76 807 Girls 20 19 46 55 414
Boys 29 23 65 72 393
11 and 12 yo 43 20.5 71 77 273 Girls 21 18 42 55 128
Boys 34 24 65 77 145
13 and 14 yo 42 21.9 72 81 88 Girls 20 23 56 65 46
Boys 38 29 78 92 42
15 and 16 yo 40 19.3 67 72 72 Girls 20 22 52 53 35
Boys 45 26 75 98 37
17 and 18 yo 31 18 57 60 44 Girls 24 20 52 58 25
Boys 29 32 70 101 19
General 44 19.6 70 77 3,284 25 22 56 68 3,284

yo: years old.
We recommend using the suggested cutoff points as pre- variance heterogeneity and therefore robust in terms of
liminary, incipient or exploratory for groups, whereas the Type I errors, and more preferable than the traditional
sample size is fewer than 50 participants. Student’s t-test.
The results from the parent form revealed a significant
Known-Groups difference between the groups in all domains assessed. The
When the different validity sources of a new tool are Cool Index 1 was higher for ADHD participants (M ¼ 17 .9,
explored, an important method to access one of these sour- SD ¼ 5.23) than non-ADHD participants (M ¼ 10.3, SD ¼
ces is the “Known-Groups” method. The evidence of the test 5.50; p < .01, d ¼ 1.4 95% CI [1.2, 1.5]), as well as for Hot
validity is seen within this procedure, as its scores signifi- Index (ADHD participants: M ¼ 30.00, SD ¼ 8.1; Non-
cantly discriminate across groups which are theoretically ADHD participants: 17.6, SD ¼ 9.21, p < .01, d ¼ 1.3 95%
known to differ. Consequently, this method potentiates the CI [1.20, 1.5]), and Cool Index 2 (ADHD participants:
clinical utility of an instrument, which is not different for M ¼ 27.1, SD ¼ 7.67; Non-ADHD participants: 15.7, SD ¼
the scales studied in this research. Therefore, the clinical 7.62, p < .01, d ¼ 1.5 95% CI [1.3, 1.7]). Taking into account
utility of both scales was then explored from these results. that the higher the value of these indexes, the greater the
The full sample (n ¼ 3,284) was used to obtain informa- difficulties presented by the child in these EF domains.
tion about its means and standard deviation to compare The same pattern of results was found regarding the
these results with those obtained from the ADHD sample teacher questionnaires. Cool Index results were higher in the
(n ¼ 165). All comparisons were carried out via an inde- ADHD group (M ¼ 44.55, SD ¼ 13.10) than the non-ADHD
pendent Welch Two Sample t-test. This test is robust for group (M ¼ 17.74, SD ¼ 15.14; t(172.84) ¼ 24.51, p < .01,
Figure 2. Known-Group results (EFICA-P).
Figure 3. Known-Group results (EFICA-T).
d ¼ 1.78 95% CI [1.61, 1.95]), and the Hot Index results Cantin et al., 2016; Han et al., 2016; Meltzer, 2007;
were also higher in the ADHD group (M ¼ 24.47, SD ¼ Shanmugan et al., 2016). Although the importance of study-
9.66) than the non-ADHD group (M ¼ 7.48, SD ¼ 8.54; ing EF is evident, there is a scarcity of standardized instru-
t(165.9) ¼ 21.49, p < .01, d ¼ 1.98 95% CI [1.81, 2.14]). ments to perform a functional EF evaluation in Brazil and
Figures 2 and 3 show the density differences between in other low and middle-income countries (Willoughby
ADHD (light grey) vs. non-ADHD (dark grey) participants et al., 2019). To a lesser degree, this lack of a standardized
according to the EFICA-P and EFICA-T, respectively. As tool is also evident in Portuguese-speaking communities in
previously explained, the higher the result, the higher the the United States and other countries.
risk for any of the EF problems. Thus, in the present study we describe the developmental
process and the psychometric properties of a new functional
instrument to evaluate EF within a large population sample
Discussion of Brazilian children and adolescents aged from 5 to 18 years-
old. Normative tables were developed considering age groups
Previous studies indicate that EF plays an important role in and children gender, as these factors were significantly related
cognition, emotions, and behavior, in addition to being cor- to the outcomes. Percentiles were used as reference since they
related with key outcomes in child development, mental are easy to understand and have an intuitive interpretation as
health, socio-emotional skills, and school achievement they report the percentage of observations lying below or
(Brock, Rimm-Kaufman, Nathanson, & Grimm, 2009; equal to the raw score obtained by the child.
After successive analytical approaches, the present study ¼ .46–.72) in a sample of 40 children (Jarratt, Riccio, &
shows robust evidence of the different sources of validity for Siekierski, 2005).
both EFICA-P and EFICA-T. The internal structure of the Discrepancies between parent and teacher ratings and dif-
scales was submitted to an empirical analysis within the ferences in the inter-rater reliability of specific scales may
framework of internal consistency reliability and categorical partly reflect the consistency with which behavioral and emo-
factor analysis, which is comparable to 2PL IRT model. tional difficulties are expressed in disparate environments and
Cronbach’s alpha and the McDonald’s omega coefficient demands. In addition, the results obtained in the normative
met the required rules of thumb for their acceptance. The study also accounted for these differences. For example, prob-
resulting coefficients range from 0 to 1 in providing this lems with disinhibition may be more readily observed across
overall assessment of an instrument’s reliability. A high settings than some other aspects of executive dysfunction.
Cronbach’s alpha means that all of the items have high Differences with respect to degrees of environmental structure
covariances, ensuring high internal consistency (Clark & and demand in home and school settings may also contribute
Watson, 1995). to inconsistencies between raters.
When exploring the dimensionality aspects of both scales, The relationship between EF and gender also depends on
the solution obtained from the factor analysis processes the definition of EF, and consequently the procedures relied
encompasses a multidimensional structure. Therefore, it upon for its assessment. Some academic dispute is present
points out the complexity of EF, and allows growth to the in the field which partially happens because the evidence is
conclusion of the multidimensional nature of EF instead of not concise, whether there exists a difference only in certain
a unidimensional nature (Gioia et al., 2003; Goldstein & behaviors (such as attention or working memory), or if the
Naglieri, 2014). Although these results support a multidi- differences found are direct consequences of methodological
mensional structure, the high correlation between the differ- choices, and not as a function of gender at all.
ent behaviors observed to execute different tasks related to As an example, results from a recent meta-analysis about
EF also allows understanding that all EF are linked or mutu- visual-spatial working memory concluded a significant but
small advantage for males when compared to females (mean
ally dependent. This result is in accordance with other con-
d ¼ 0.155, 95% confidence interval ¼ 0.087–0.223), with the
clusions reported in the literature (Castellanos,
expectation of memory for location, in which a female
Kronenberger, & Pisoni, 2018; Gioia et al., 2003; Skogan
advantage emerged. Thus, these results support the idea of
et al., 2016). There is no restriction to model loadings dur-
the existence of cognitive gender (Voyer, Voyer, & Saint-
ing exploratory analyses, therefore they are obtained for all
Aubin, 2017; Wang & Carr, 2014).
items among all factors. In contrast, the confirmatory analy-
In contrast, another meta-analysis considering both
ses can be modeled using the exploratory results, but adding
humans and other animals found little support for signifi-
restrictions regarding which variables load into spe-
cant gender differences in executive function (deemed as
cific factors.
attention and focus, impulse control, decision making, and
An interesting solution was obtained after analyzing the
working memory). The authors considered that age of test-
interface between the exploratory and confirmatory analysis.
ing, as well as the specific testing modality were collapsing
The EFICA-P was formed by three different factors, from the gender effect (Grissom & Reyes, 2019).
which two factors are composed of distinct cool EF. The In this study, we found that the age effect was significant
Cool Index 1 was composed of items pointing to goal- for both parents and teachers, but the gender effect was
directed behavior, planning, organizing, initiating, attention, only associated with the teachers’ version of the EFICA.
and working memory. Persistency and self-monitoring were Some explanations can be made from this result and previ-
mainly found in the Cool Index 2. The factor related to hot ous literature suggests a gender difference in externalizing
EF was composed of items such as inhibiting, flexibility, and behaviors, arguing that boys are more susceptible to atten-
emotional regulation. These results are convergent to a tion problems than girls. Also, girls get higher teacher rat-
model to other findings in the literature (Diamond & ings for approaches to learning than boys when they are
Lee, 2011). 6–7 years old (Berthelsen, Hayes, White, & Williams, 2017).
Some considerations should be taken into account for The relationship between all factors was positive, but the
better understanding of the results of the two related, yet correlation between Cool Index 1 and 2 was higher than
independent factors reflecting cool EF obtained by the their association with Hot Index, supporting a current para-
EFICA-P, in contrast to just one factor to the same theoret- digm in the literature of a dual dimensionality of EF with a
ical construct obtained by the EFICA-T. Although the cool and hot components (Peterson & Welsh, 2014).
results obtained by parents and teachers are in the same dir- The second cool EF index distinguished by the parents is
ection, when considering the set of instruments that inspired predominantly composed by two EF (self-monitoring and
the development of the EFICA, but especially the BRIEF, goal persistency), which can perhaps be better observed with
parents tend to rate their children, both boys and girls, as a larger observation time, explaining why parents were able
having more problems on the BRIEF-P and BRIEF as com- to identify them and teachers were not. The different per-
pared to teachers. This is consistent with other literature on ceptions about EF between parents and teachers in func-
parent-teacher discrepancies (Offord et al., 1996). Jarratt tional instruments are reported in the literature, such as
et al. reported an average correlation of .58 (range different kinds of demand and time. In addition, teachers
spend less time with the child when compared to parents, importance for the adaptation of the tool within the United
and therefore it is expected they will have less knowledge States and other English-speaking countries/communities.
about their behavior, abilities and difficulties (Germano, Secondly, our study lacks test-retest and criterion (con-
Brito, & Capellini, 2017; Jarratt et al., 2005). current) validity analyzes. However, we have to keep in
Another result that must be dealt with is the statistical mind that establishing concurrent validity is particularly dif-
difference between the performance of the ADHD and non- ficult for an EF rating scale given the broad variability in
ADHD groups. The capacity to discriminate between the the conceptualization, measurement approach, and psycho-
groups expected to have different scores adds an important metric characteristics (Goldstein & Naglieri, 2014). Thirdly,
psychometric value to EFICA. However, clinical utility is the neuropsychological literature for defining an ideal sam-
also composed of sensitivity and specificity studies, which ple size for normative group from which the normative table
were not carried out in this study, and therefore, the use of will be developed varies from n ¼ 5 to n ¼ 500. Therefore,
EFICA to identify participants with (or in-risk of having) we suggest using the results from cell in which the sample
ADHD should be done with caution and in an exploratory size is equal than 50 as exploratory purposes only when
context only (Lohr, 2002). sample size is fewer than 50 participants. However, some
It is easy to recognize that items of both instruments characteristics are often common. Finally, this study did not
address the common behaviors seen in children with count on a random sample of children who do not have
ADHD, revealing the deep interplay between cognitive and clinical diagnoses, which could distort the interpretation of
behavioral demands, as well the reciprocal relation between the results for these children.
attentional control and (Drigas and Karyotaki, 2017). Numerous reasons are responsible for referring children
ADHD has been associated with a broad range of negative for neuropsychological evaluations. Difficulties in achieving
outcomes for affected subjects on academic, occupational, scholastic achievement, and problems with hyperactivity and
and (Miller et al., 2013), as well as being associated with attention are frequently dependent on executive functions
other mental and substance use disorders, criminality, and and lead to significant limitations in the daily routines of
unemployment (Jensen & Steinhausen, 2015), causing a ser- children. Clinical neuropsychologists often rely on clinical
ious financial burden to families and society, which charac- and parental interviews to assess everyday functioning.
terizes it as a major public health problem (Nasol, Lindly, Despite the utility of the method, its unstructured nature
Chavez, & Zuckerman, 2019; Reus-Smit & Snidal, 2009). leaves it vulnerable to the pitfalls of clinical judgment.
The results found in this study are in line with the literature Therefore, clinicians might benefit from having a new
which reports the central role of executive dysfunction in screening test with adequate psychometric properties for
ADHD diagnosis (Drigas and Karyotaki, 2016; Barkley, their daily use. Thus, this study provides a new instrument
2012; Schreiber et al., 2014). to be used which can be helpful in the first or preliminary
The strengths of this study include the populational- process of identifying children and adolescents with a risk of
based/robust sample size, the strong reliability and validity poor EF development, thereby enabling preventive and
evidence, and the quite similar distribution of household- remedial interventions and better school support for those
head education level compared to Brazilian standards children who are facing difficulties.
(IBdGeE, 2010a). We emphasize the considerable influence The validity process of a neuropsychological measure is
of parents’ educational status on the perception of executive related to the inferences and interpretations obtained by the
functioning in children and adolescents (Goldstein & use of the test, but not the test itself. As a result, because it
Naglieri, 2014). We also achieved some of the main criteria is inadequate to state that a test is valid (or invalid) given a
for normative tables. Both EFICA questionnaires have a set of fixed data, another study is currently ongoing aiming
clear definition of their objective, their items were retained to check the differential item function of the EFICA-P and
after a continuous approach of studies of evidence of its val- EFICA-T items and to build more refined normative data.
idity, and the standardization of both test administration
and scoring increase their clinical utility.
Acknowledgment
However, this study also presents some limitations. First,
the studied sample was composed of children and adoles- We would like to acknowledge and thank both Dr. Landeira-Fernandez
cents enrolled in the Brazilian public-school system. (PUC-Rio) and Dr. Cristiano Fernandes (PUC-Rio) for the fruitful dis-
cussions about statistics, psychometrics, and measurement.
Although this is a well-accepted method to investigate dis-
ease epidemiology in the pediatric population, children from
higher income levels are less likely to study in public schools ORCID
in Brazil. Adjustments would not perfectly address this
Luis Anunciaç~ao http://orcid.org/0000-0001-5303-5782
topic, since the socioeconomic variables would be shifted
toward the lower incomes. Therefore, a generalization of the
results achieved in this study is limited to children. In this References
regard, because the participants were not native English-
A-ABdEd, P. (2015). Brazilian economic classification criteria. Retrieved
speaking children, this tool should be used for exploratory from http://www.abep.org/Servicos/Download.aspx?id=11.
purposes or just as preliminary tools outside of this context. American Psychiatric Association. (2013). DSM-5. Diagnostic and stat-
Despite its limitation, these results could be of significant istical manual of mental disorders.
Baddeley, A. D., & Hitch, G. (1974). Working memory. In: G. H. Damasio, A. R. (1994). Descartes’ error: Emotion, rationality and the
Bower, G.H. (Ed.), The psychology of learning and motivation: human brain. New York, NY: Putnam.
Advances in research and theory (pp. 47-89). New York: Academic Di Zio, M., & Guarnera, U. (2010). A multiple imputation approach to
Press. Retrieved from http://dx.doi.org/10.1016/S0079- deal with the unity measure error. Statistical Methods &
7421(08)60452-1 Applications, 19(3), 431–444. doi:10.1007/s10260-010-0132-1.
Barkley, R. A. (2001). The executive functions and self-regulation: An Diamond, A., & Lee, K. (2011). Interventions shown to aid executive
evolutionary neuropsychological perspective. Neuropsychology function development in children 4 to 12 years old. Science,
Review, 11(1), 1–29. 333(6045), 959–964. doi:10.1126/science.1204529
Barkley, R. A. (2012). Executive functions what they are, how they work, Diamond, A. (2013). Executive Functions. Annual Review of
and why they evolved. New York, NY: The Guilford Press. Psychology, 64(1), 135–168. doi:10.1146/annurev-psych-113011-
Barth, J. T., Pliskin, N., Axelrod, B., Faust, D., Fisher, J., Harley, J. P., 143750
… Silver, C. (2003). Introduction to the NAN 2001 Definition of a Drigas, A., & Karyotaki, M. (2017). Attentional Control and other
Clinical Neuropsychologist NAN Policy and Planning Committee. Executive Functions. International Journal of Emerging Technologies
Archives of Clinical Neuropsychology, 18, 551–555. doi:10.1093/ in Learning (Ijet), 12(03), 219 doi:10.3991/ijet.v12i03.6587.
arclin/18.5.551 Drigas, A., & Karyotaki, M. (2016). Online and other ICT-based
Batrancourt, B., Dojat, M., Gibaud, B., Kassel, G. (2010). A core ontol- Assessment Tools for Problem-solving Skills. International Journal
ogy of instruments used for neurological, behavioral and cognitive of Emerging Technologies in Learning (Ijet), 11(04), 56 doi:10.3991/
assessments. International conference on formal ontology in infor- ijet.v11i04.5339.
mation systems, May 2010, Toronto, Canada. pp. 185–198, Fernald, L. C. H., Prado, E., Kariger, P., & Raikes, A. (2017). A toolkit
ff10.3233/978-1-60750-535- 8-185ff. ffinserm-00519801f for measuring early childhood development in low and middle
Bechara, A. (2004). The role of emotion in decision-making: Evidence income countries (English). Washington, D.C.: World Bank Group.
from neurological patients with orbitofrontal damage. Brain and Retrieved from http://documents.worldbank.org/curated/en/
Cognition, 55(1), 30–40. doi:10.1016/j.bandc.2003.04.001 384681513101293811/A-toolkit-for-measuring-early-childhood-devel-
Berthelsen, D., Hayes, N., White, S. L. J., & Williams, K. E. (2017). opment-in-low-and-middle-income-countries
Executive function in adolescence: Associations with child and fam- Figueroa, A. (2016). The Alpha-Beta Method in Economics. In: Rules
ily risk factors and self-regulation in early childhood. Frontiers in for Scientific Research in Economics. Cham: Palgrave Macmillan.
Psychology, 8. doi:10.3389/fpsyg.2017.00903 Retrieved from https://doi.org/10.1007/978-3-319-30542-4_4
Best, J. R., & Miller, P. H. (2010). A developmental perspective on Fonseca, A. C. C. F., Fonseca, M. T. M., Rodrigues, M. E. S. M.,
executive function. Child Development, 81(6), 1641–1660. doi:10. Lasmar, L. M. L. B. F., & Camargos, P. A. M. (2006). Pico do fluxo
1111/j.1467-8624.2010.01499.x expiratorio no acompanhamento de crianças asmaticas. Jornal De
Blair, C., & Raver, C. C. (2012). Child development in the context of Pediatria, 82(6), 465–469. doi:10.1590/S0021-75572006000800012.
adversity: Experiential canalization of brain and behavior. American Furr, R. M. (2011). Scale construction and psychometrics for social and
Psychologist, 67(4), 309–318. doi:10.1037/a0027493 personality psychology. London: SAGE Publications Ltd. doi:10.4135/
Boring, E. G. (1953). The role of theory in experimental psychology. 9781446287866
The American Journal of Psychology, 66(2), 169. doi:10.2307/1418724 Fuster, J. M. (1973). Unit activity in prefrontal cortex during delayed-
Brock, L. L., Rimm-Kaufman, S. E., Nathanson, L., & Grimm, K. J. response performance: Neuronal correlates of transient memory.
(2009). The contributions of ‘hot’ and ‘cool’ executive function to Journal of Neurophysiology, 36(1), 61–78. doi:10.1152/jn.1973.36.1.61
children’s academic achievement, learning-related behaviors, and Germano, G. D., Brito, L. B., & Capellini, S. A. (2017). The opinion of
engagement in kindergarten. Early Childhood Research Quarterly, parents and teachers of students with learning disorders regarding
24(3), 337–349. doi:10.1016/j.ecresq.2009.06.001. executive function skills. Revista Cefac, 19(5), 674–682. doi:10.1590/
Brown, T. A. (2015). Confirmatory Factor Analysis for Applied Research 1982-0216201719510817
(2nd ed.). New York, NY: Guilford Publications. Gillen, R. W. (2010). Neuropsychology of everyday functioning.
Cantin, R. H., Gnaedinger, E. K., Gallaway, K. C., Hesson-McInnis, Archives of Clinical Neuropsychology, 25(7), 685–686. doi:10.1093/
M. S., & Hund, A. M. (2016). Executive functioning predicts read- arclin/acq055
ing, mathematics, and theory of mind during the elementary years. Gioia, G. A., Isquith, P. K., Retzlaff, P. D., & Espy, K. A. (2003).
Journal of Experimental Child Psychology, 146, 66–78. doi:10.1016/j. Confirmatory factor analysis of the Behavior Rating Inventory of
jecp.2016.01.014 Executive Function (BRIEF) in a clinical sample. Child
Carim, D. D. B., Miranda, M. C., & Bueno, O. F. A. (2012). Traduç~ao Neuropsychology, 8(4), 249–257. doi:10.1076/chin.8.4.249.13513.
e adaptaç~ao para o portugu^es do behavior rating inventory of execu- Goldstein, S., & Brooks, R. B. (2013). Handbook of resilience in chil-
tive function – BRIEF translation and adaptation into portuguese of dren. In S. Goldstein & R. B. Brooks (Eds.), Boston, MA: Springer
the behavior. Psicologia: Reflex~ao e Critica. US. doi:10.1007/978-1-4614-3661-4
Castellanos, I., Kronenberger, W. G., & Pisoni, D. B. (2018). Goldstein, S., & Naglieri, J. A. (2014). Handbook of executive function-
Questionnaire-based assessment of executive functioning: ing. New York, NY: Springer-Verlag. doi:10.1007/978-1-4614-8106-5
Psychometrics. Applied Neuropsychology: Child, 7(2), 93–109. doi:10. Grissom, N. M., & Reyes, T. M. (2019). Let’s call the whole thing off:
1080/21622965.2016.1248557 Evaluating gender and sex differences in executive function.
Chevignard, M. P., Catroppa, C., Galvin, J., & Anderson, V. (2010). Neuropsychopharmacology, 44(1), 86–96. doi:10.1038/s41386-018-0179-5
Development and evaluation of an ecological task to assess executive Han, G., Helm, J., Iucha, C., Zahn-Waxler, C., Hastings, P. D., &
functioning post childhood TBI: The children’s cooking task. Brain Klimes-Dougan, B. (2016). Are executive functioning deficits con-
Impairment, 11(2), 125–143. doi:10.1375/brim.11.2.125 currently and predictively associated with depressive and anxiety
Clark, L. A., & Watson, D. (1995). Constructing validity: Basic issues symptoms in adolescents? Journal of Clinical Child & Adolescent
in objective scale development. Psychological Assessment, 7(3), Psychology, 45(1), 44–58. doi:10.1080/15374416.2015.1041592
309–319. doi:10.1037/1040-3590.7.3.309. Holmes, C. J., Kim-Spoon, J., & Deater-Deckard, K. (2016). Linking
Costello, A., & Osborne, J. (2011). Best practices in exploratory factor executive function and peer problems from early childhood through
analysis: Four recommendations for getting the most from your ana- middle adolescence. Journal of Abnormal Child Psychology, 44(1),
lysis. Practical Assessment, Research & Evaluation, 10, 1–9. 31–42. doi:10.1007/s10802-015-0044-5
Crawford, J. R., & Garthwaite, P. H. (2009). Percentiles please: The Hu, L., & Bentler, P. M. (1999). Cutoff criteria for fit indexes in covari-
case for expressing neuropsychological test scores and accompanying ance structure analysis: Conventional criteria versus new alterna-
confidence limits as percentile ranks. The Clinical Neuropsychologist, tives. Structural Equation Modeling: A Multidisciplinary Journal,
23(2), 193–204. doi:10.1080/13854040801968450 6(1), 1–55. doi:10.1080/10705519909540118
IBdGeE. (2010a). Estatıstica da Populaç~ao Brasileira. Retrieved from Norman, D. A., & Shallice, T. (1986). Attention to action: Willed and
https://www.ibge.gov.br/estatisticas-novoportal/sociais/populacao/ automatic control of behaviour (Revised reprint of Norman and
9662-censo-demografico-2010.html?edicao=9753&t=resultados. Shallice (1980)). Consciousness and Self-Regulation: Advances in
IBdGeE. (2010b). Estatistica da Populacao Brasileira - Delfinopolis. Research and Theory. New York: Plenum Press.
Retrieved from https://cidades.ibge.gov.br/brasil/mg/delfinopolis/ Oberer, N., Gashaj, V., & Roebers, C. M. (2018). Executive functions,
panorama. visual-motor coordination, physical fitness and academic achieve-
IBdGeE. (2010c). Estatısticas da Populaç~ao Brasileira - S~ao Sebasti~ao do ment: Longitudinal relations in typically developing children. Human
Paraıso. Retrieved from https://cidades.ibge.gov.br/brasil/mg/sao- Movement Science, 58, 69–79. doi:10.1016/j.humov.2018.01.003
sebastiao-do-paraiso/panorama. Offord, D. R., Boyle, M. H., Racine, Y., Szatmari, P., Fleming, J. E.,
Jarratt, K. P., Riccio, C. A., & Siekierski, B. M. (2005). Assessment of Sanford, M., & Lipman, E. L. (1996). Integrating assessment data
Attention Deficit Hyperactivity Disorder (ADHD) Using the BASC from multiple informants. Journal of the American Academy of
and BRIEF. Applied Neuropsychology, 12(2), 83–93. doi:10.1207/ Child & Adolescent Psychiatry, 35(8), 1078–1085. doi:10.1097/
s15324826an1202_4 00004583-199608000-00019
Jensen, C. M., & Steinhausen, H.-C. (2015). Comorbid mental disorders Osborne, J. (2011). Best practices in quantitative methods. Thousand
in children and adolescents with attention-deficit/hyperactivity dis- Oaks, CA: Sage Publications. doi:10.4135/9781412995627
order in a large nationwide study. ADHD Attention Deficit and Peterson, E., & Welsh, M. C. (2014). The development of hot and cool
Hyperactivity Disorders, 7(1), 27–38. doi:10.1007/s12402-014-0142-1 executive functions in childhood and adolescence: Are we getting
Jurado, M. B., & Rosselli, M. (2007). The elusive nature of executive warmer?. In Handbook of executive functioning (pp. 45–65). New
functions: A review of our current understanding. Neuropsychology York, NY: Springer New York. doi:10.1007/978-1-4614-8106-5_4
Review, 17(3), 213–233. doi:10.1007/s11065-007-9040-z Poon, K. (2018). Hot and cool executive functions in adolescence:
Kertz, S. J., Belden, A. C., Tillman, R., & Luby, J. (2016). Cognitive Development and contributions to important developmental out-
control deficits in shifting and inhibition in preschool age children comes. Frontiers in Psychology, 8(2311). doi:10.3389/fpsyg.2017.
are associated with increased depression and anxiety over 7.5 years 02311
of development. Journal of Abnormal Child Psychology, 44(6), Prencipe, A., Kesek, A., Cohen, J., Lamm, C., Lewis, M. D., & Zelazo,
1185–1196. doi:10.1007/s10802-015-0101-0 P. D. (2011). Development of hot and cool executive function dur-
Lan, X., Legare, C. H., Ponitz, C. C., Li, S., & Morrison, F. J. (2011). ing the transition to adolescence. Journal of Experimental Child
Investigating the links between the subcomponents of executive Psychology, 108(3), 621–637. doi:10.1016/j.jecp.2010.09.008
function and academic achievement: A cross-cultural analysis of R Core Team. (2019). R: A language and environment for statistical
Chinese and American preschoolers. Journal of Experimental Child computing. R Foundation for Statistical Computing, Vienna,
Psychology, 108(3), 677–692. doi:10.1016/j.jecp.2010.11.001. Austria. Retrieved from http://www.R-project.org/.
Lezak, M. D. (1982). The problem of assessing executive functions. Reus-Smit, C., & Snidal, D. (2009). The Oxford handbook of inter-
International Journal of Psychology, 17(1–4), 281–297. doi:10.1080/ national relations. Oxford: Oxford University Press. doi:10.1093/
00207598208247445 oxfordhb/9780199219322.001.0001
Lezak, Muriel Deutsch, Howieson, D. B., Bigler, E. D., & Tranel, D. Revelle, W. (2015). Package “psych” - Procedures for Psychological,
(2012). Neuropsychological assessment (5th ed.). New York, NY: Psychometric and Personality Research. R Package, 1–358. Retrieved
Oxford University Press. from http://personality-project.org/r/psych-manual.pdf
Lezak, M. D., Howieson, D. B., Bigler, E. D., & Tranel, D. (2012). Rosseel, Y. (2012). lavaan: An R Package for Structural Equation
Neuropsychological assessment (5th ed.). New York: Oxford Modeling. Journal of Statistical Software, 48(2), 36. doi:10.18637/jss.
University Press. v048.i02.
Lohr, K. N. (2002). Assessing health status and quality-of-life instru- Roth, R. M., Isquith, P. K., & Gioia, G. A. (2014). Assessment of execu-
ments: attributes and review criteria. Quality of life research, 11(3), tive functioning using the Behavior Rating Inventory of Executive
193–205. doi: 10.1023/a:1015291021312 Function (BRIEF). In S. Goldstein & J. A. Naglieri (Eds.), Handbook
Luria, A. (1966). The higher cortical functions in man. Anuario De of executive functioning (p. 301–331). Springer Science þ Business
Psicologıa Jurıdica. Media. Retrieved from https://doi.org/10.1007/978-1-4614-8106-5_18
Luria, A. R. (1966). Higher cortical functions in man. New York: NY: Schreiber, J. E., Possin, K. L., Girard, J. M., & Rey-Casserly, C. (2014).
Basic Books. Executive function in children with attention deficit/hyperactivity
Marco Di Zio & Ugo Guarnera, 2008. “A multiple imputation method disorder: The NIH EXAMINER battery. Journal of the International
for non-Gaussian data,” Metron - International Journal of Statistics, Neuropsychological Society, 20(1), 41–51. doi:10.1017/
LXVI(1), 75–90. S1355617713001100
Meltzer, L. (Ed.). (2007). Executive function in education: From theory Shanmugan, S., Wolf, D. H., Calkins, M. E., Moore, T. M., Ruparel, K.,
to practice. Guilford Press. Hopson, R. D., … Satterthwaite, T. D. (2016). Common and dis-
Miller, A. C., Keenan, J. M., Betjemann, R. S., Willcutt, E. G., sociable mechanisms of executive system dysfunction across psychi-
Pennington, B. F., & Olson, R. K. (2013). Reading Comprehension atric disorders in youth. American Journal of Psychiatry, 173(5),
in Children with ADHD: Cognitive Underpinnings of the Centrality 517–526. doi:10.1176/appi.ajp.2015.15060725
Deficit. Journal of Abnormal Child Psychology, 41(3), 473–483. doi: Sijtsma, K. (2006). Psychometrics in Psychological Research: Role
10.1007/s10802-012-9686-8. Model or Partner in Science? Psychometrika, 71(3), 451–455. doi:10.
Miyake, A., & Friedman, N. P. (2012). The nature and organization of 1007/s11336-006-1497-9.
individual differences in executive functions. Current Directions in Sijtsma, K. (2013). Theory development as a precursor for test validity
Psychological Science, 21(1), 8–14. doi:10.1177/0963721411429458 (pp. 267–274). New York, NY: Springer.
Miyake, A., Friedman, N. P., Emerson, M. J., Witzki, A. H., Howerter, Skogan, A. H., Egeland, J., Zeiner, P., Øvergaard, K. R., Oerbeck, B.,
A., & Wager, T. D. (2000). The unity and diversity of executive Reichborn-Kjennerud, T., & Aase, H. (2016). Factor structure of the
functions and their contributions to complex “frontal lobe” tasks: A Behavior Rating Inventory of Executive Functions (BRIEF-P) at age
latent variable analysis. Cognitive Psychology, 41(1), 49–100. doi:10. three years. Child Neuropsychology, 22(4), 472–492. doi:10.1080/
1006/cogp.1999.0734 09297049.2014.992401
Muthukrishna, M., & Henrich, J. (2019). A problem in theory. Nature Spence, S. (1995). Descartes’ error: Emotion, reason and the human
Human Behaviour, 3(3), 221–229. doi:10.1038/s41562-018-0522-1. brain. BMJ, 310(6988), 1213–1213. doi:10.1136/bmj.310.6988.1213
Nasol, E., Lindly, O. J., Chavez, A. E., & Zuckerman, K. E. (2019). Squires, J., Bricker, D., Heo, K., & Twombly, E. (2001). Identification
Unmet need and financial impact disparities for US children with of social-emotional problems in young children using a parent-com-
ADHD. Academic Pediatrics, 19(3), 315–324. doi:10.1016/j.acap. pleted screening measure. Early Childhood Research Quarterly, 16(4),
2018.09.001 405–419. doi:10.1016/S0885-2006(01)00115-6.
Squires, J., & Bricker, D. (2009). Ages and stages questionnaires. 3rd ed. Voyer, D., Voyer, S. D., & Saint-Aubin, J. (2017). Sex differences in
(ASQ-3): A parent-completed child-monitoring system. Baltimore, visual-spatial working memory: A meta-analysis. Psychonomic
MD: Brooks Publishing Company. Bulletin and Review, 24(2), 307–334. doi:10.3758/s13423-016-1085-7
Stanley, D. J., & Spence, J. R. (2018). Reproducible tables in psychology Walk, L. M., Evers, W. F., Quante, S., & Hille, K. (2018). Evaluation of
using the apatables package. Advances in Methods and Practices in a teacher training program to enhance executive functions in pre-
Psychological Science, 1(3), 415–431. doi:10.1177/2515245918773743 school children. PLOS One, 13(5), e0197454. doi:10.1371/journal.
Streiner, D. L., Norman, G. R., & Cairney, J. (2015). Health measure- pone.0197454
ment scales: A practical guide to their development and use (5th ed.). Wang, L., & Carr, M. (2014). Working memory and strategy use con-
Oxford, UK: Oxford University Press. Retrieved from http://ovidsp. tribute to gender differences in spatial ability. Educational
ovid.com/ovidweb.cgi?T=JS&PAGE=reference&D=psyc11&NEWS= Psychologist, 49(4), 261–282. doi:10.1080/00461520.2014.960568
N&AN=2014-48611-000. Watson, J. C. (2017). Establishing evidence for internal structure using
Suh, Y. (2015). The performance of maximum likelihood and weighted exploratory factor analysis. Measurement and Evaluation in
least square mean and variance adjusted estimators in testing differ- Counseling and Development, 50(4), 232–238. doi:10.1080/07481756.
ential item functioning with nonnormal trait distributions. 2017.1336931
Structural Equation Modeling, 22(4), 568–580. doi:10.1080/10705511. White, L. K., Moore, T. M., Calkins, M. E., Wolf, D. H., Satterthwaite,
2014.937669 T. D., Leibenluft, E., … Gur, R. E. (2017). An evaluation of the spe-
Sun, J., & Buys, N. (2012). Early executive function deficit in preterm cificity of executive function impairment in developmental psycho-
pathology. Journal of the American Academy of Child & Adolescent
children and its association with neurodevelopmental disorders in
Psychiatry, 56(11), 975–982.e3. doi:10.1016/j.jaac.2017.08.016
childhood: A literature review. International Journal of Adolescent
Wickham, H. (2016). tidyverse: Easily Install and Load “Tidyverse”
Medicine and Health, 24(4), 291–299. doi:10.1515/ijamh.2012.042
Packages. R package version 1.0.0. Retrieved from https://cran.r-pro-
Swanson, J. M. (2003) SNAP-IV Teacher and Parent Ratings Scale. In:
ject.org/package=tidyverse
F. Aykr (Ed.), Therapist’s Guide to Learning and Attention Disorders
Willoughby, M. T., Piper, B., Kwayumba, D., & McCune, M. (2019).
(pp. 487–500). New York: Academic Press. Retrieved from https:// Measuring executive function skills in young children in Kenya.
doi.org/10.1016/B978-012256430-7/50022-3 Child Neuropsychology, 25(4), 425–444. doi:10.1080/09297049.2018.
Thorell, L. B., & Nyberg, L. (2008). The Childhood Executive 1486395
Functioning Inventory (CHEXI): A new rating instrument for Wilson, J., Andrews, G., Hogan, C., Wang, S., & Shum, D. H. K.
parents and teachers. Developmental Neuropsychology, 33(4), (2018). Executive function in middle childhood and the relationship
536–552. doi:10.1080/87565640802101516 with theory of mind. Developmental Neuropsychology, 43(3),
Thorell, L. B., & Wåhlstedt, C. (2006). Executive functioning deficits in 163–182. doi:10.1080/87565641.2018.1440296
relation to symptoms of ADHD and/or ODD in preschool children. Woods, S. P., Kordovski, V. M., Tierney, S. M., & Babicz, M. A.
Infant and Child Development, 15(5), 503–518. doi:10.1002/icd.475 (2019). The neuropsychological aspects of performance-based
Trevisan, B. T., Dias, N. M., Berberian, A. D. A., & Seabra, A. G. Internet navigation skills: A brief review of an emerging literature.
(2017). Childhood executive functioning inventory: Adaptaç~ao e The Clinical Neuropsychologist, 33(2), 305–326. doi:10.1080/
propriedades psicometricas da vers~ao Brasileira. Psico-USF, 22(1), 13854046.2018.1503332.
63–74. doi:10.1590/1413-82712017220106 Zasler N., Katz D., Zafonte R., Eds. (2007). Brain Injury Medicine:
Viterbori, P., Usai, M. C., Traverso, L., & De Franchis, V. (2015). How Principles and Practice. New York, NY: Demos Medical Publishing.
preschool executive functioning predicts several aspects of math achieve- Zelazo Cunningham, W. P. D. (2007). Executive function: Mechanisms
ment in Grades 1 and 3: A longitudinal study. Journal of Experimental underlying emotion regulation. Handbook of emotion regulation.
Child Psychology, 140, 38–55. doi:10.1016/j.jecp.2015.06.014 New York, NY: Guilford. doi:10.1177/1545968309354536

Psychometric Properties and Clinical Utility of The Executive Function Inventory For Children and Adolescents - A Large Multistage Populational Study Including Children With ADHD.

Uploaded by

Copyright:

Available Formats

Psychometric Properties and Clinical Utility of The Executive Function Inventory For Children and Adolescents - A Large Multistage Populational Study Including Children With ADHD.

Uploaded by

Document Information

Original Title

Copyright

Available Formats

Share this document

Share or Embed Document

Sharing Options

Did you find this document useful?

Is this content inappropriate?

Copyright:

Available Formats

Psychometric Properties and Clinical Utility of The Executive Function Inventory For Children and Adolescents - A Large Multistage Populational Study Including Children With ADHD.

Uploaded by

Copyright:

Available Formats

Applied Neuropsychology: Child

ISSN: 2162-2965 (Print) 2162-2973 (Online) Journal homepage: https://www.tandfonline.com/loi/hapc20

Psychometric properties and clinical utility of

Marco Antônio Arruda, Renato Arruda & Luis Anunciação

To link to this article: https://doi.org/10.1080/21622965.2020.1726353

Published online: 02 Mar 2020.

Submit your article to this journal

Article views: 125

View related articles

View Crossmark data

Full Terms & Conditions of access and use can be found at

Psychometric properties and clinical utility of the executive function inventory

Introduction multiple results which are not entirely related or convergent

Table 2. Items analysis, descriptive, and CFA results of EFICA-P (parents).

Table 3. Item analysis and CFA results – EFICA-T (Teachers).

Table 4. Normative table for EFICA-p and EFICA-t results.

General 44 19.6 70 77 3,284 25 22 56 68 3,284

Figure 2. Known-Group results (EFICA-P).

Figure 3. Known-Group results (EFICA-T).

You might also like