Epidemiology and Psychiatric Sciences

http://journals.cambridge.org/EPS

Additional services for Epidemiology and Psychiatric Sciences:

Email alerts: Click here

Subscriptions: Click here
Commercial reprints: Click here
Terms of use : Click here

Behavioural/emotional problems in Brazilian children: ndings from

parents' reports on the Child Behavior Checklist

M. M. Rocha, L. A. Rescorla, D. R. Emerich, E. F. M. Silvares, J. C. Borsa, L. G. S. Araújo, M. H. S. M. Bertolla, M. S.

Oliveira, N. C. S. Perez, P. M. Freitas and S. G. Assis

Epidemiology and Psychiatric Sciences / Volume 22 / Issue 04 / December 2013, pp 329 - 338
DOI: 10.1017/S2045796012000637, Published online: 27 November 2012

Link to this article: http://journals.cambridge.org/abstract_S2045796012000637

How to cite this article:

M. M. Rocha, L. A. Rescorla, D. R. Emerich, E. F. M. Silvares, J. C. Borsa, L. G. S. Araújo, M. H. S. M. Bertolla, M. S.
Oliveira, N. C. S. Perez, P. M. Freitas and S. G. Assis (2013). Behavioural/emotional problems in Brazilian children: ndings
from parents' reports on the Child Behavior Checklist. Epidemiology and Psychiatric Sciences, 22, pp 329-338 doi:10.1017/
S2045796012000637

Request Permissions : Click here

Downloaded from http://journals.cambridge.org/EPS, IP address: 159.178.22.27 on 05 Nov 2015

Epidemiology and Psychiatric Sciences (2013), 22, 329–338. © Cambridge University Press 2012 ORIGINAL ARTICLE
doi:10.1017/S2045796012000637

Behavioural/emotional problems in Brazilian

children: findings from parents’ reports on the Child
Behavior Checklist

M. M. Rocha1*, L. A. Rescorla2, D. R. Emerich1, E. F. M. Silvares1, J. C. Borsa3, L. G. S. Araújo4,

M. H. S. M. Bertolla1, M. S. Oliveira5, N. C. S. Perez6, P. M. Freitas7 and S. G. Assis8
1
Universidade de Sao Paulo, Sao Paulo, Brazil
2
Bryn Mawr College, Bryn Mawr, Pennsylvania, USA
3
Universidade Federal do Rio Grande do Sul, Porto Alegre, Brazil
4
Faculdade Ciência da Vida, Sete Lagoas, Minas Gerais, Brazil
5
Pontifícia Universidade Católica do Rio Grande do Sul, Porto Alegre, Brazil
6
Universidade Federal do Rio Grande do Norte, Campus Universitário Lagoa Nova, Natal, Brazil
7
Universidade Federal do Recôncavo Bahiano, Santo Antonio de Jesus, Brazil
8
Fundação Oswaldo Cruz, Manguinhos, Rio de Janeiro, Brazil

Background. To compare Child Behavior Checklist (CBCL) findings for a large Brazilian general population sample
with those for US children considering: (a) mean problem item ratings; (b) fit of the US-derived CBCL 8-syndrome
model; (c) scale internal consistency measured by Cronbach’s alphas; (d) effects of society, age, gender on CBCL pro-
blem scores; and (e) ability to discriminate referred from non-referred children.

Methods. Parents of 1228 non-referred 6-to-11-year-olds from three different regions of Brazil and 247 referred 6-to-11-
year-olds from one clinic rated their children’s behavioural and emotional problems using the CBCL/6–18.

Results. Results for mean item ratings and scale internal consistencies were very similar to those found in the US and in
Uruguay. Confirmatory factor analysis indicated that Brazilian data showed the best fit to the US 8-syndrome model of
all countries studied to date. Gender patterns were comparable to those reported in other societies, but mean problem
scores for non-referred Brazilian children were higher than those for US children. Therefore, the CBCL discriminated
less well between non-referred and referred children in Brazil than in the US.

Conclusions. Overall, our findings replicated those reported in international comparisons of CBCL scores for 31
societies, thereby providing support for the multicultural robustness of the CBCL in Brazil.

Received 19 May 2012; Revised 23 September 2012; Accepted 26 September 2012; First published online 27 November 2012

Key words: CBCL, confirmatory factor analysis, psychological assessment, psychometric, multicultural.

Introduction emotional problems are typical in children in the gen-

eral population and what level of problems necessi-
To meet the mental health needs of children in a
tates referral for intervention or prevention efforts
society, it is necessary to identify those children who
(Zwirs et al. 2007). Professionals in developing
have behavioural/emotional problems severe enough
countries face major challenges in collecting such epi-
to warrant intervention. Identification of children in
demiological data. These challenges may include lack
need of mental health referral thus requires a reliable
of resources, lack of valid, reliable and cost-effective
and valid assessment instrument that is able to dis-
instruments, and lack of agreement about how to
criminate between typically developing children and
define impairment (Belfer, 2008). For these reasons,
children with high-enough levels of problems to need
epidemiological research on child mental health in
mental health services. Epidemiological data are
developing countries is most feasible when an instru-
needed to determine what level of behavioural/
ment is available that can be administered by non-
professionals or is self-administered, can be under-
stood by people with varying levels of education, is
* Address for correspondence: Professor M. M. Rocha, Instituto de
Psicologia, Universidade de São Paulo, Bloco F, sala 30, Av. Prof. Melo
inexpensive and simple to use, can be easily scored
Moraes 1721, 05508-030 São Paulo, SP, Brazil. and interpreted, and has been shown to work well in
(Email: [email protected]) many different societies.
330 M. M. Rocha et al.

The Child Behavior Checklist (CBCL/6–18; sufficiently broad general population sample were not
Achenbach & Rescorla, 2001), developed in the US available at that time. However, numerous CBCL studies
and translated into more than 85 languages, has been have been conducted in Brazil, starting with the work of
successfully used to assess behavioural/emotional pro- Bordin and co-workers, who conducted validation
blems in many societies, including many developing studies using the CBCL/4–18 (Achenbach, 1991) with
countries. Rescorla et al. (2007) and Ivanova et al. clinic sample. For example, Bordin et al. (1995) reported
(2007) conducted multicultural comparisons of CBCL that the CBCL had high sensitivity for predicting
scores obtained for >50 000 children from general popu- ICD-10 psychiatric diagnoses for 49 low-income pedi-
lation samples in 31 societies. Both studies took an etic atric outpatients, and Brasil & Bordin (2010) reported
approach to research (Pike, 1967), whereby the same high sensitivity in a sample of 78 children seen for intake
instrument was used to obtain data in different societies at a mental health outpatient clinic. The Portuguese
so that the findings could be compared. Etic research is CBCL/4–18 has also been widely used in Brazilian
often contrasted with emic research, whereby the mean- studies conducted by Rohde and co-workers (e.g.
ing of constructs is examined within each society. Lampert et al. 2004; Roessner et al. 2007; Petresco et al.
When Ivanova et al. (2007) used confirmatory factor 2009), primarily to examine children with Attention def-
analysis (CFA) to test the fit of CBCL data obtained icit hyperactivity disorder (ADHD). Additionally, the
from the 30 non-US societies to the CBCL’s 8-syn- CBCL has been used by several Brazilian researchers to
drome factor model, fit indices strongly supported screen children for behavioural and emotional problems
the correlated 8-syndrome structure in each of the 30 (e.g. Alvarenga & Piccinini, 2001; Silvares et al. 2006;
societies. Rescorla et al. (2007) reported that internal Tanaka & Lauridsen-Ribeiro, 2006; Schneider &
consistencies of scales, mean item ratings, and age Ramires, 2007; Moraes & Enumo, 2008; Garzuzi et al.
and gender patterns were very similar across the 31 2009; Mota et al. 2010), and to assess treatment outcomes
societies. Although mean total problems scores for (e.g. Bolsoni-Silva et al. 2008; Pereira et al. 2009).
most of the societies fell within one standard deviation Some larger scale studies have also used the CBCL
(S.D.) of the omnicultural mean, 12 societies had mean in Brazil. Benvegnú et al. (2005) assessed 3139 children
scores outside this range (six above and six below). and adolescents from the Southern area of the country
This led Achenbach & Rescorla (2007) to establish and reported that 13.5% of the sample achieved scores
low-, middle-, and high-scoring norm groups for mul- in the clinical range based on the American norms.
ticultural scoring of the CBCL. Assis et al. (2007), in a similar study done with 500 stu-
Although societies from Asia, Africa, Australia, the dents from Sao Gonçalo/Rio de Janeiro/Brazil, found
Caribbean, Europe, the Middle East, and North that 15.7% scored in the clinical range, also based on
America were included in the Rescorla et al. (2007) the American norms. Paula et al. (2007) reported that
and Ivanova et al. (2007) studies, no South American 14% did so in a sample from Sao Paulo, another state
societies were included. More recently, Viola et al. from Brazil. These studies highlighted the need for
(2011), who used a Spanish translation of the CBCL, mental health services for Brazil’s children.
reported CBCL findings for 1364 6- to 11-year olds Although all of these studies constituted important
recruited through 65 schools nationwide in Uruguay. steps in developing CBCL research in Brazil, the exist-
Mean item ratings, factor structure, and scale internal ing studies had various limitations. Many of these
consistencies were very similar to findings reported studies used the 1991 rather than the 2001 version of
by Rescorla et al. (2007) and Ivanova et al. (2007) for the CBCL, many had small, clinical, and/or non-
31 societies. Uruguay’s mean total problems score representative samples, and all were limited to a par-
was significantly higher than the US mean, but ticular region of the country. Most importantly, none
Uruguay’s score still fell in the middle-scoring group reported statistical comparisons between CBCL find-
of the Achenbach & Rescorla (2007) multicultural ings for Brazil and those for other societies. We
norms when children with diagnosed disabilities addressed this limitation by comparing 2001 CBCL
and/or documented clinical or special education ser- findings from a large Brazilian sample with findings
vices were excluded, following the practice of from the US normative sample. Furthermore, because
Achenbach & Rescorla (2001) in the US. The present we used exactly the same methods employed by
study adds to the growing international CBCL litera- Ivanova et al. (2007) and Rescorla et al. (2007) in their
ture by reporting epidemiological findings from international comparisons of 31 societies and by and
Brazil, a much larger and more diverse South Viola et al. (2011) in comparing Uruguayan and US
American country than Uruguay and one in which data, we were able to examine the degree to which
Portuguese, rather than Spanish, is spoken. Brazilian findings replicated those reported for other
Brazil was not included in the Rescorla et al. (2007) and societies in the world as well as those reported for
Ivanova et al. (2007) studies because data from a another South American society.
 Child Behavior Checklist findings on Brazilian children 331

The present study had several aims, each of which children who had received mental health or special
involved comparing Brazilian data with data from education services in the past year. After we excluded
the US: (a) to compare mean problem item ratings children with an identified condition, we had 1891 par-
for Brazil and the US (b); to use CFA to test the fit of ticipants. This exclusion process resulted in some sub-
the Brazilian data to the CBCL 8-syndrome model samples being <90 children. These subsamples were
derived in the US; (c) to compare Cronbach’s alphas then excluded, leaving 1757 participants.
for the CBCL’s scales with US alphas reported by Since the majority (87.9%) of the 1757 participants
Rescorla et al. (2007); (d) to test effects of society were in the age group of 6–11, children >11 years
(Brazil v. US), age, gender on CBCL problem scores were excluded, resulting in 1494 participants. Finally,
for the non-referred and referred samples separately; following Achenbach & Rescorla’s (2001) procedure,
and (e) to test the ability of the Brazilian CBCL to dis- cases with more than eight blank items on the CBCL
criminate referred from non-referred children. were excluded. This left a final sample of 1475 chil-
dren, aged 6–11. Within this sample of 1475, there
was a group of 247 children referred for psychological
Method treatment at a clinic in Porto Alegre/RS. These 247 chil-
dren were retained as a separate ‘referred’ sample. The
Participants
remaining 1228 children were recruited through
Since resources did not permit national probability schools in six cities by researchers studying children’s
sampling for this study, we opted to use a convenience behaviour. These 1228 children comprised our ‘non-
sample recruited from different areas of Brazil via col- referred’ general population sample. Because infor-
leagues who had purchased the CBCL’s computer mation about receipt of mental health and special
scoring software from the Brazilian distributor during educational services was incomplete for the non-
the past 5 years. Of the 47 colleagues contacted via referred sample, it is possible that some of the children
email, 32 responded and 16 had samples with more were receiving mental health or special educational
than 90 participants. Of these 16 invited to collaborate treatment. Table 1 presents details about each of the
on the research, 11 colleagues from nine Brazilian cities six sub-samples comprising the non-referred sample
agreed to share their data, for a total of 2369 potential of 1228 children, as well as about the referred sample
participants. Each investigator who provided a school comprising 247 children.
sample had recruited it for one’s own research project. The non-referred sample comprised 608 girls
For example, the sample for Belo Horizonte/MG was (49.5%) and 620 boys (50.5%), with a mean age of 8.1
obtained in order to analyse if families who lived in years (S.D. = 1.3). The referred sample comprised
less developed areas of the city reported more behav- 38.5% girls and 61.5% boys, with a mean age of 8.8
ioural/emotional problems than families from more (S.D. = 1.6). The non-referred sample had children
developed areas, whereas the Sao Gonçalo/RJ sample from three different areas of Brazil: 66.2% from the
was obtained to analyse the prevalence of behaviour Southeast, 26.9% from the Northeast and 6.9% from
problems among students exposed to violence. the South, while the referred sample was composed
From the initial pool of 2369 children, we excluded only of children from Porto Alegre/RS, a big city in
children identified with any genetic disease, neurologi- the southern region of the country.
cal damage, or diagnosed psychiatric disorder, follow- To compare Brazilian children with US children, we
ing the procedure of Achenbach & Rescorla (2001) selected all children with ages of 6–11 from the US
used of excluding from their national survey sample referred and non-referred samples. These two US

Table 1. Sample characteristics

Samples N Age mean (S.D.) Age range Male (%) Response rate (%) Sampling frame

Rio de Janeiro 475 7.9 (1.0) 6–11 51.4% 99% School-based

Belo Horizonte 244 8.3 (1.7) 6–11 51.2% NI School-based
Natal 159 7.9 (1.2) 6–11 52.2% NI School-based
Porto Alegre 85 9.6 (.7) 8–11 42.4% 100% School-based
Salvador 172 7.8 (1.4) 6–11 48.3% NI School-based
São Paulo 93 7.7 (.6) 7–9 52.7% 85% School-based
Porto Alegre 247 8.8 (1.6) 6–11 61.5% NI Clinic-based

Note. NI = No information.
332 M. M. Rocha et al.

samples had previously been matched on age, sex, eth- scores in the two societies. Second, to test the CBCL
nicity, and Socioeconomic status (SES) (Achenbach & 8-syndrome model in Brazil, CFA was implemented
Rescorla, 2001). As noted above, the US non-referred using the same procedure employed by Achenbach
sample excluded children with special needs. The US & Rescorla (2001) and Ivanova et al. (2007). Third,
referred group was composed of children from the the internal consistency of the CBCL scales was calcu-
same age group who had been seen in mental health lated using Cronbach’s alphas. Fourth, the effects of
clinics. The US referred and non-referred samples society, age, and gender on CBCL scores were tested
each had 733 children, 52% boys and 48% girls. using analyses of variance (ANOVAs and MANOVAs).
Fifth, decision statistics were used to test the ability
of CBCL scales to differentiate referred and non-
Measure
referred children in the Brazilian sample. For this last
The 2001 version of the CBCL/6–18 (Achenbach & analysis, 247 children were chosen from the 1228 chil-
Rescorla, 2001) comprises 118 problem items that dren in the Brazilian non-referred sample to demo-
parents rate 0 = not true, 1 – somewhat or sometimes graphically match the referred sample in age and
true, or 2 = very true or often true, based on the past gender, following the procedure of Achenbach &
6 months. The competence items of the CBCL were Rescorla (2001). For all analyses, p < 0.001 was used
not used in the present study, as too many children to determine statistical significance, due to the large
had missing data. The 2001 scales for the CBCL sample sizes and the number of tests in each set of ana-
(Achenbach & Rescorla, 2001) were computed from lyses (17, one for each problem scale). Effect sizes for
these 118 problem items, following the methodology ANOVAs and MANOVAs are represented by η2.
of Rescorla et al. (2007). These included eight syn-
dromes, the three broadband scales (Internalizing,
Externalizing, and Total Problems), and six Results
DSM-oriented scales. High test–retest reliability (e.g.,
Mean item ratings
mean r = 0.90 for empirically based scales) and strong
internal consistency (e.g., alpha = 0.97 for total pro- In order to verify the comparability between Brazil and
blems score) have been reported by Achenbach & the US with respect to which items tended to receive
Rescorla (2001). The CBCL also contains some open- high, medium, or low ratings, a correlation was com-
ended items on which respondents can choose to puted between the 118 mean item ratings for the
write in physical problems, concerns, and strengths 1228 Brazilian non-referred children and for the 733
of the child. Although qualitative data provided by US non-referred children. The correlation coefficient
these items can be very useful in the clinical assess- was 0.84, higher than the mean correlation of 0.79 for
ment context, they were not analysed in the present the US and 30 other societies reported by Rescorla
study because written comments were not included et al. (2007). This very high correlation indicates strong
in the data sets we received due to the fact that they comparability between Brazil and the US regarding
are not amenable to quantitative analysis. which items received high, medium, or low ratings.
The Brazilian Portuguese version of the 2001 CBCL/
6–18 was developed in a series of steps. Initially, it was
CFA results
translated into Portuguese by Silvares et al. (2007). To
make it similar to the US version, the Portuguese ver- The fit of the Brazilian data to the 2001 CBCL 8-syn-
sion was written at about a fifth grade reading level. drome model was tested using the robust weighted
This Portuguese version was then back-translated to least squares (WLSMV) estimator on tetrachoric corre-
English by a professional bilingual translator originally lations (ratings of 0 v. 1 and 2) for the 102 items com-
from the US and blind to the American version of the prising those syndromes, following Ivanova et al.’s
instrument. The back-translation was reviewed by the (2007) procedures. The root-mean-squared error of
original authors, some minor changes in item wording approximation (RMSEA) was chosen as the primary
were made, and the translation was approved for use. index of the model’s fit (values ≤ 0.06 indicate good
fit), and the Tucker–Lewis index (TLI) and the com-
parative fit index (CFI) were used as additional
Data analysis
measures of model fit (values ≥ 0.90 indicate good fit).
MPlus 5.0 (Muthén & Muthén, 2007) was used for the The RMSEA index was 0.023 for the Brazilian data,
CFA, whereas PASW Statistics 18 was used for all indicating very good fit. This index was even below
other data analyses. First, mean item ratings were cor- the range reported by Ivanova et al. (2007) for 30
related for the non-referred Brazilian and US samples, societies (0.026–0.055). The TLI and CFI also indicated
in order to verify the comparability between the items that the Brazilian data fit the US 8-syndrome model
 Child Behavior Checklist findings on Brazilian children 333

quite well (TLI = 0.900 and CFI = 0.903). All Brazilian mean scores of the Brazilian and US referred and non-
items loaded significantly on their predicted factor, referred samples for all CBCL problem scales. For non-
with the following mean item loadings: anxious/ referred children, the gender × age (6–8, 9–11 years)
depressed = 0.51, withdrawn/depressed = 0.59, somatic ×society ANOVA on Total Problems indicated main
complaints = 0.62, social problems = 0.51, thought pro- effects for society (Brazil > US, ES = 16%) and gender
blems = 0.57, attention problems = 0.63, rule-breaking (boys > girls, ES = 1%), but not for age. For referred
behaviour = 0.56, and aggressive behaviour = 0.65. To children, the ANOVA indicated main effects for gen-
verify the consistency between the Brazilian loadings der (boys > girls, ES = 1%) and age (older > younger,
and the average of the loadings for 30 other societies, ES = 1%) but not for society. As shown in Table 4, the
the differences between the Brazilian mean factor load- mean total problems score for the non-referred sample
ings and the mean loadings reported by Ivanova for 30 was much higher in Brazil (43.34) than in the US
societies were calculated. They ranged from 0.02 to (23.12), and sufficiently high to place Brazil in the high-
0.08 (mean of 0.05), indicating great consistency. scoring norm group for the CBCL (Achenbach &
Rescorla, 2007). In contrast, scores for the referred
groups were very similar in the two societies (62.88
Internal consistency of CBCL scales in Brazil and 63.22 in the US).
Table 2 presents the Cronbach’s alphas of the CBCL To test the effects of society, gender, and age on
scales for both referred and non-referred samples in internalizing and externalizing scores, 2 × 2 × 2
Brazil and the US. The highest alphas in both countries ANOVAs were conducted separately by referral status
were found for the three broadband scales groups. For internalizing, the society effect was signifi-
(Internalizing, Externalizing, and Total Problems), with cant in both samples but much larger in the non-
all alphas ≥ 0.80. Alphas for syndromes and referred sample (ES = 16%) than the referred sample
DSM-oriented scales ranged from 0.56 (anxiety pro- (ES = 1%). Older children scored significantly higher
blems) to 0.81 (conduct problems), very similar to than younger children in the referred sample (ES =
what was found for the US sample. The correlation 2%) but not the non-referred sample. The gender effect
between alphas for the 17 problem scales obtained for was not significant in either sample. For externalizing,
the referred v. non-referred samples were 0.92 in Brazil Brazilian children had significantly higher scores than
and 0.89 in the US. The Brazil–US correlation was 0.93 US children in the non-referred sample (ES = 12%),
for non-referred samples and 0.88 for referred samples. whereas Brazilian children had significantly lower
Both correlations are close to the mean bi-society corre- scores than US children in the referred sample (ES =
lation of 0.88 reported by Rescorla et al. (2007). 2%). In both referred and non-referred samples, boys
scored significantly higher than girls on externalizing
(both ESs = 1%). No significant age effect was found
Effects of gender, age, and country on CBCL scores for externalizing problems in either sample.
A 2 (gender) × 2 (age) × 2 (society) MANOVA was
Since the Brazilian referred sample was only 20% as
conducted on the eight CBCL syndromes for the non-
large as the non-referred sample, whereas the US
referred and referred samples separately. For the non-
referred and non-referred groups were the same size,
referred sample, the significant ESs for society ranged
effects of gender, age, and society were analysed sep-
from 4% for thought problems to 15% for anxious/
arately for referral status groups. Table 3 shows the
depressed, with Brazilian scores higher on all scales.
Boys had significantly higher scores on two syn-
Table 2. Demographic characteristics of non-referred and referred dromes: Attention problem and rule-breaking behav-
samples iour (both ESs = 1%). No significant age or interaction
effects were found. For the referred sample, US chil-
Non-referred sample Referred sample dren obtained significantly higher scores on
(n = 1228) (n = 247) rule-breaking behaviour (ES = 2%) and aggressive be-
haviour (ES = 1%), whereas Brazilian children obtained
Gender higher scores on Somatic complaints (ES = 2%). Boys
Boys 620 (50.5%) 152 (61.5%)
obtained significantly higher scores on attention pro-
Girls 608 (49.5%) 95 (38.5%)
blems and rule-breaking behaviour (ES = 2%). Older
Mean age 8.1 (1.3) 8.8 (1.6)
children had significantly higher scores than younger
Region
Southeast 66.2% 0% children on withdrawn/depressed, and rule-reaking
Northeast 26.9% 0% behaviour (ESs = 2 and 1%, respectively).
South 6.9% 100% The 2 × 2 × 2 MANOVA on the six DSM-oriented
scales for the non-referred sample yielded significant
334 M. M. Rocha et al.

Table 3. Cronbach’s Alpha for CBCL for Brazil and US samples for ages 6–11

Brazil Brazil US US
Non-referred (n = 1228) Referred (n = 247) Non-referred (n = 733) Referred (n = 733)

Total problems 0.95 0.95 0.93 0.96

Internalizing 0.83 0.86 0.81 0.89
Externalizing 0.89 0.92 0.86 0.93
Syndromes
Anxious/depressed 0.72 0.72 0.72 0.82
Withdrawn/depressed 0.67 0.76 0.64 0.75
Somatic complaints 0.70 0.76 0.63 0.75
Social problems 0.68 0.75 0.70 0.76
Thought problems 0.70 0.77 0.54 0.78
Attention problems 0.76 0.83 0.73 0.79
Rule-breaking behaviour 0.62 0.77 0.58 0.79
Aggressive behaviour 0.87 0.91 0.84 0.92
DSM-oriented scales
Affective problems 0.69 0.70 0.59 0.78
Anxiety problems 0.56 0.62 0.51 0.69
Somatic problems 0.70 0.71 0.64 0.72
Attention deficit problems 0.78 0.79 0.70 0.79
Oppositional problems 0.74 0.79 0.72 0.81
Conduct problems 0.81 0.88 0.73 0.80

society effects for all the scales (ESs from 2 to 16%), defiant problems, and DSM-conduct problems (all
with Brazilian scores higher than US scores. Boys ESs = 1%). For the referred sample, Brazilian children
obtained higher scores than girls on DSM-attention obtained higher scores on affective problems (ES =
deficit hyperactivity problems, DSM-oppositional 3%), whereas US children obtained higher scores for

Table 4. Mean CBCL scores for Brazil and US samples for ages 6–11

Brazil Brazil US US
Scale Non-referred (n = 1228) Referred (n = 247) Non-referred (n = 733) Referred (n = 733)

Total problems 43.34 (25.25)a 62.88 (32.10) 23.13 (16.59) 63.22 (32.55)
Internalizing 11.43 (7.53)a 16.19 (9.50)a 5.51 (4.90) 13.86 (9.52)
Externalizing 12.35 (8.82)a 17.89 (11.92)b 6.32 (5.82) 21.55 (12.59)
Syndromes
Anxious/depressed 6.21 (4.07)a 7.88 (4.52) 3.05 (2.83) 7.07 (5.05)
Withdrawn/depressed 2.85 (2.61)a 4.57 (3.49) 1.28 (1.67) 3.98 (3.20)
Somatic complaints 2.37 (2.69)a 3.75 (3.69)a 1.18 (1.69) 2.81 (3.04)
Social problems 4.87 (3.42)a 7.36 (4.44) 2.46 (2.58) 6.74 (4.24)
Thought problems 2.87 (3.11)a 4.85 (4.53) 1.72 (1.90) 5.32 (4.58)
Attention problems 6.07 (4.37)a 9.15 (4.82) 3.52 (3.28) 9.12 (4.72)
Rule-breaking behaviour 2.80 (2.74)a 4.55 (4.09)b 1.74 (1.94) 5.94 (4.62)
Aggressive behaviour 9.55 (6.66)a 13.34 (8.40)b 4.58 (4.29) 15.61 (8.84)
DSM-oriented scales
Affective problems 3.79 (3.38)a 6.09 (4.22) 1.44 (1.88) 5.34 (4.34)
Anxiety problems 3.44 (2.29)a 4.43 (2.63)a 1.56 (1.59) 3.67 (2.68)
Somatic problems 1.15 (1.82)a 2.00 (2.45) 0.71 (1.29) 1.74 (2.19)
Attention deficit problems 5.73 (3.56)a 7.75 (3.74) 3.36 (2.86) 7.67 (3.91)
Oppositional problems 3.37 (2.47)a 4.80 (2.83)b 2.31 (1.96) 5.81 (2.86)
Conduct problems 3.12 (3.73)a 5.43 (5.79)b 1.63 (2.25) 8.17 (6.45)

a
Indicates that Brazilian mean was significantly higher than US mean ( p < 0.001) by simple effects analysis.
b
Indicates that US mean was significantly higher than Brazilian mean ( p < 0.001) by simple effects analysis.
 Child Behavior Checklist findings on Brazilian children 335

conduct problems (ES = 6%). Boys obtained higher non-referred group, not that much higher than the
scores than girls on DSM-attention deficit hyperactiv- 41% who were from the referred group. This indicates
ity problems, DSM-oppositional defiant problems, a specificity of 59% (i.e., the percentage of non-deviant
and DSM-conduct problems (all ESs = 2%). children who were from the non-referred group).
Receiver operating curve (ROC) analysis (Swets,
1996) results were consistent with these cross-
Decision statistics analysis
tabulation findings. The area under the curve (AUC)
In order to verify the ability of the Brazilian CBCL to was 62%, indicating only moderately good prediction.
discriminate between referred and non-referred chil- Discrimination between the referred and non-referred
dren, a random sample of 247 non-referred children, samples based on Total Problems score was much
matched by age and gender with the referred sample, stronger in the US sample than in the Brazilian sample.
was selected. Both samples were composed by 61.5% Nevertheless, the odds ratio for a deviant Total
of boys, and were equivalent in age (mean of 8.7 in Problems score being from a referred child was 2.86
the non-referred and 8.8 in the referred group). Prior (CI 1.94–4.21), indicating that children who scored in
to the decision statistics analyses, group differences the deviant range on the CBCL Total problems scale
in mean CBCL problem scores were tested using were almost three times more likely to be in the
ANOVAs and MANOVAs. Referred children had sig- referred group than the non-referred group.
nificantly higher scores on all scales, as would be
expected. For example, mean Total problems scores
were 44.28 in the non-referred group and 62.88 in the
Discussion
referred group. Effect sizes ranged from to 0.02 for
Anxious/depressed to 0.08 for Total problems scale. The goal of the present study was to examine the psy-
To determine deviance in the Brazilian sample, we chometric properties of the Brazilian CBCL for ages 6–
used a 90th percentile cutpoint (by gender) on Total 11 by conducting statistical comparisons between
Problems score, which is the threshold for the clinical Brazilian findings and those from the US. In addition
range. We identified the raw score corresponding to to discussing the findings of our Brazil–US compari-
the 90th percentile using Achenbach & Rescorla’s sons, we also discuss our findings with respect to find-
(2007) multicultural norms. Specifically, we used the ings reported by Rescorla et al. (2007) and Ivanova et al.
cutpoints for Achenbach & Rescorla’s (2007) high- (2007) for 31 societies, as well as with respect to find-
scoring norm group (77 for boys and 78 for girls), ings reported by Viola et al. (2011) for Uruguay.
because Brazil’s Total roblems score placed it in the Although there are many cultural differences
high-scoring group of societies. For our decision stat- between Brazil and the US, our analysis revealed
istics analysis, we classified children scoring > 90th per- many similarities between parents’ reports in these
centile as ‘deviant’ and children scoring ≤ 90th two societies. It is interesting to note that these simi-
percentile as ‘non-deviant’. Following Achenbach & larities are the same as those found by Viola et al.
Rescorla (2001), we then looked at the cross-tabulation (2011) in a Uruguayan sample. For example, the corre-
of deviance and referral status. lation between the 118 mean item ratings for the 1228
Within the non-referred sample, 23% obtained Brazilian non-referred children and the 733 US non-
scores in the deviant range, more than twice as many referred children was 0.84, higher than the mean corre-
as would be expected using a 90th percentile cutpoint. lation of 0.79 for the US and 30 other societies reported
This indicates a negative predictive value of only 77% by Rescorla et al. (2007) and comparable with the
(i.e., the percentage of non-referred children who were Uruguay–US correlation of 0.82. Thus, Brazilian and
not deviant). Additionally, 67% of the children scoring US parents were very similar with respect to which
in the deviant range were from the referred group, but items, on average, received high, medium, or low rat-
33% were from the non-referred group, further con- ings. Additionally, CFA confirmed that the Brazilian
firming the high scores in the non-referred sample. data showed excellent fit to the 8-syndrome model
This corresponds to a sensitivity of 67% (i.e., the per- derived in the US structure. The RMSEA of 0.023 indi-
centage of deviant children who were from the cates better fit than found in any of the 30 societies
referred sample). Although 46% of the referred group compared by Ivanova et al. (2007) (range 026–0.055)
scored in the deviant range, 54% did not, indicating or than found in Uruguay (RMSEA of 0.037) (Viola
that the referred group did not have exceedingly et al. 2011). Furthermore, Cronbach’s alphas were
high scores. This indicates a positive predictive value very similar to US values. Correlations between alphas
of 46% (i.e., the percentage of referred children who for the 17 problem scales obtained in Brazil and the US
scored in the deviant range). Finally, only 59% of the were very high (0.93 for non-referred and 0.88 for
children scoring below the cutpoint were from the referred children), close to the mean bi-society
336 M. M. Rocha et al.

correlation of 0.88 reported by Rescorla et al. (2007) and different regions of Brazil who had conducted individ-
comparable with those reported for the Uruguay–US ual projects to assess behavioural problems in school
comparison (0.92 for non-referred and 0.93 for settings and then shared their data with us for this
referred). study. Since the data were not collected expressly for
In both referred and non-referred groups, boys this study, information was incomplete with respect
scored higher than girls on Externalizing, consistent to whether participants were receiving mental health
with Viola et al. (2011) in Uruguay and with Rescorla or special education services, as well as with respect
et al.’s (2007) report that boys had significantly higher to SES level of each child. Furthermore, we had to
Externalizing scores than girls for 19 of 28 societies restrict our study to ages 6–11. Additionally, the
with samples ages 6–11. Higher scores on Attention referred sample was rather small, was drawn from
Problem, Rule-Breaking Behaviour, DSM-attention only one clinic, and was most likely not very represen-
deficit hyperactivity problems, DSM-oppositional defi- tative of children attending mental health clinics in
ant problems, and DSM-conduct problems found for Brazil. A final limitation is that no other measure of
boys in many societies help explain the higher rate of children’s emotional/behavioural difficulties was avail-
boys referred for mental health care. Brazilian girls able for the children, thus preventing a cross-
did not have significantly higher Internalizing scores validation analysis for this study. These limitations
than boys, consistent with the Uruguayan findings must be considered when interpreting the findings of
and with Rescorla et al.’s (2007) report that none of our study.
the 28 societies with samples of children ages 6–11
showed a significant gender difference on
Internalizing.
Conclusions
Since we had a referred sample of 247 children, we
were able to examine the CBCL’s ability to discrimi- Despite these limitations, our findings provide an
nate referred from non-referred children in Brazil. We important addition to the literature on the use of the
used the procedure employed by Achenbach & CBCL in South America. Our psychometric findings
Rescorla (2001) of comparing equal-sized groups on mean item ratings, factor structure, Cronbach’s
matched on age and gender, although we could not alphas, and gender effects replicated findings reported
also match on SES because we lacked SES data. As by Viola et al. (2011) for Uruguay and were comparable
noted above, the Brazilian non-referred group had with those reported by Rescorla et al. (2007) and
much higher scores than the US non-referred group. Ivanova et al. (2007) for 31 societies in Europe, Asia,
Furthermore, given the high scores in the Brazilian and other parts of the world. Therefore, with respect
non-referred group, the scores in the Brazilian referred to mean item rating, factor structure, scale internal
group were not sufficiently high to obtain good dis- consistency, and gender patterns, few differences
crimination. This pattern most likely explains why were found between Brazil and the many other
Brazilian decision statistics results were so much societies that have been studied.
weaker than US results, where referred and non- In Rescorla et al. (2007), the omnicultural mean for
referred group mean scores were very different. A Total Problems score for the 31 societies was 22.5
similar pattern of poor discrimination was found for (S.D. = 5.7.) The six societies scoring > 1 S.D. above the
Uruguay, where Viola et al. (2011) did not have a clinic omnicultural mean were Puerto Rico, Portugal,
sample as their ‘referred’ group but rather used chil- Ethiopia, Greece, Lithuania, and Hong Kong, with
dren from their school-based sample who had docu- Puerto Rico having the highest score (34.7). The
mented disabilities or confirmed mental health mean of 43.34 found for Brazil in the current study is
treatment/special education status. therefore higher than the means of all 31 societies as
compared by Rescorla et al. (2007). Brazil’s mean
Ttotal problems score placed it within the range for
the high-scoring norm group specified by Achenbach
Limitations
& Rescorla (2007), but it fell at the high end of that
Our sample was the largest and most nationally repre- range.
sentative sample obtained to date in Brazil. Although we cannot say for certain why Brazilian
Nevertheless, it presented numerous limitations that non-referred children scored higher than American
must be considered in interpreting our findings. Ours non-referred children on virtually all CBCL scales, it
was a convenience school-based sample recruited in is likely that several factors played a role. First, the
several regions of Brazil, rather than a sample recruited fact that ours was a convenience sample may have
using a nationally representative general population resulted in over-representing children with relatively
survey. Our data were collected by researchers in high scores. It is possible that a sample obtained
 Child Behavior Checklist findings on Brazilian children 337

through probability sampling might have yielded 1995; Silvares et al. 2006; Brazil & Bordin, 2010) did
lower scores. Another factor may be ‘cultural,’ in that not report raw CBCL scores, it is difficult to determine
our CBCL findings for this Brazilian sample are con- if their referred groups obtained higher scores than the
sistent with pre-2001 CBCL findings for Argentina referred group in this study. However, these previous
(Samaniego, 2008) and Chile (Bralio et al. 1987), as studies generally reported higher sensitivity than we
well as with the Viola et al. (2011) findings based on obtained and demonstrated good discriminant validity
use of the 2001 CBCL in Uruguay. In all these studies, of the CBCL when cross-validated with diagnostic
parents of South American children, on average, measures. In sum, while it is possible that cultural fac-
reported more problems on the CBCL than parents tors contributed to the weak discrimination between
of US children, which might reflect a cultural tendency referred and non-referred children in our Brazilian
of having a lower threshold for reporting problems. study, we think that these sampling issues are more
However, it should be noted that Brazil’s mean Total likely to be the main explanation for our findings. It
problems score placed it in the high-scoring norm is therefore possible that our study may have underes-
group for multicultural scoring of the CBCL timated the ability of the CBCL to discriminate
(Achenbach & Rescorla, 2007), whereas Uruguay’s between referred and non-referred children in Brazil.
mean placed it in the middle-scoring group (when chil- Future Brazilian studies should address differences
dren with known diagnoses, mental health services, on CBCL scores in different regions of Brazil based
and developmental disabilities were removed from on a nationally representative general population sur-
the school-based sample). A third factor that may vey. Additionally, future Brazilian studies should
have contributed to higher scores in Brazil might be obtain information on referral status and SES, obtain
the SES of the children. Given that lower SES is associ- CBCL data for adolescents, and obtain CBCL data for
ated with somewhat higher CBCL problem scores in a larger sample of referred children drawn from a var-
all societies where this has been measured to date iety of mental health facilities in the country.
(Achenbach & Rescorla, 2007), it may be that a greater Furthermore, additional studies are needed from
preponderance of low SES children in the Brazilian other South American societies to test the generaliz-
sample relative to the American sample contributed ability of the findings reported here for Brazil and by
to the score differences. Although we did not have Viola et al. (2011) for Uruguay. Nonetheless, the pre-
SES levels for individual children, many of the schools sent study represents an important advance in the
from which the children were recruited served low- use of standardized assessment of behavioural/
income families, making it likely that the sample had emotional problems in Brazil.
a large low SES component. A final possible reason
is that scores in the Brazilian non-referred group may
have been high because the group actually contained Declaration of Interest
children who had been referred for problems, or
would have had services been available. Because refer- Dr Rocha and Ms Emerich provide training workshops
ral information was not consistently available, it can- on the Achenbach System of Empirically Based
not be known how significant this factor was in Assessment (ASEBA) in Brazil, from which they
accounting for US–Brazil score differences. receive financial remuneration. Dr Rescorla is remun-
Our study yielded poorer discrimination between erated by the Research Center for Children, Youth,
referred and non-referred groups for Brazil than was and Families (RCCYF), which publishes the ASEBA.
found in the US, but our findings were comparable Dr Silvares is the ASEBA distributor in Brazil. The
with those from Uruguay. Compared with the US, other authors have no conflict of interest to declare.
scores for referred children were rather low, whereas
scores for non-referred children were rather high, a
pattern also found in Uruguay. The non-referred
sample may have had high scores because it contained References
children who had received mental health or special
Achenbach TM (1991). Manual for the Child Behavior Checklist
education services of which we were unaware, or
and 1991 Profile. University of Vermont, Department of
who would have been good candidates for services
Psychiatry: Burlington, VT.
had they been available. The referred children prob-
Achenbach TM, Rescorla LA (2001). Manual for the ASEBA
ably had relatively low scores because they were School-Age Forms & Profiles. University of Vermont,
drawn from a single clinic and are probably not that Research Center for Children, Youth, & Families:
representative of children receiving mental health Burlington, VT.
treatment in Brazil more generally. Since previous Achenbach TM, Rescorla LA (2007). Multicultural Supplement
Brazilian studies using clinical samples (Bordin et al. to the Manual for the ASEBA School-Age Forms & Profiles.
338 M. M. Rocha et al.

University of Vermont, Research Center for Children, outskirts of Sao Paulo City: treatment needs and service
Youths, & Families: Burlington, VT. capacity evaluation. Revista Brasileira de Psiquiatria 29, 11–17.
Alvarenga P, Piccinini C (2001). Práticas educativas maternas Pereira RF, Costa NJD, Rocha MM, Arantes MC, Silvares
e problemas de comportamento em pré-escolares. EFM (2009). O efeito da terapia comportamental para
Psicologia: Reflexão e Crítica 14, 449–460. enurese sobre outros problemas de comportamento.
Assis SG, Avanci JQ, Pesce RP, Oliveira RVC, Furtado LX Psicologia: Teoria e Prática 25, 419–423.
(2007). A Violência Familiar Produzindo Reversos: Problemas de Pike KL (Ed.) (1967). Language in Relation to a Unified Theory of
Comportamento em Crianças Escolares (relatório de pesquisa). Structure of Human Behavior, 2nd edn. Mouton: The Hague,
Claves/Fiocruz: Rio de Janeiro. Netherlands.
Belfer ML (2008). Child and adolescent mental disorders: the Petresco S, Gutt EK, Krelling R, Neto FL, Rohde LAP,
magnitude of the problem across the globe. Journal of Child Moreno RA (2009). The prevalence of psychopathology in
Psychology and Psychiatry 49, 226–236. offspring of bipolar women from a Brazilian tertiary center.
Bordin IAS, Mari JJ, Caeiro MF (1995). Validação da versão Revista Brasileira de Psiquiatria 31, 1–7.
brasileira do ‘Child Behavior Checklist’ (CBCL) (Invetário Rescorla L, Achenbach T, Ivanova MY, Dumenci L,
de comportamentos da infância e adolescência): dados Almqvist F, Bilenberg N, Bird H, Chen W, Dobrean A,
preliminares. Revista ABP-APAL 17, 55–66. Döpfner M, Erol N, Fombonne E, Fonseca A, Frigerio A,
Bolsoni-Silva AT, Silveira FF, Marturano EM (2008). Grietens H, Hannesdottir H, Kanbayashi Y, Lambert M,
Promovendo habilidades sociais educativas parentais na Larsson B, Leung P, Liu X, Minaei A, Mulatu MS, Novik
prevenção de problemas de comportamento. Revista TS, Oh K, Roussos A, Sawyer M, Simsek Z, Steinhausen
Brasileira de Terapia Comportamental e Cognitiva 10, 125–142. H, Weintraub S, Weisz J, Metzke CW, Wolanczyk T, Yang
Bralio S, Seguel X, Montenegro H (1987). Prelavencia de H, Zilber N, Zukauskiene R, Verhulst F (2007). Behavioral
transtornos psíquicos em la población escolar de Santiago and emotional problems reported by parents of children
de Chile. Acta Psiquiatrica Pscycología America Latina 33, ages 6 to 16 in 31 societies. Journal of Emotional and
316–325. Behavioral Disorders 15, 130–142.
Brasil HH, Bordin IAS (2010). Convergent validity of Roessner V, Becker A, Rothenberger A, Rohde LA,
K-SADS-PL by comparison with CBCL in a Portuguese Banaschewski T (2007). A cross-cultural comparison
speaking outpatient population. BMC Psychiatry 10, 83. between samples of Brazilian and German children with
Garzuzi Y, Carreiro LR, Schwartzman JS, Mesquita ML, ADHD/HD using the Child Behavior Checklist.
Palma D, Ancona Lopez F, Moraes DE, Macedo E, European Archive of Psychiatry and Clinical Neuroscience 257,
Teixeira MC (2009). Perfil comportamental de crianças e 352–359.
adolescentes com síndrome de Prader-Willi e obesidade Samaniego VC (2008). El Child Behavior Checklist: Su
exógena. Psicologia: Teoria e Prática 11, 167–178. Estandarización en la Población Escolar de 6 a 11 Años de Edad.
Ivanova MY, Dobrean A, Dopfner M, Erol N, Fombonne E, Memorias de la XI Jornada de Investigación: Buenos Aires.
Fonseca AC, Frigerio A, Grietens H, Hannesdottir H, Schneider ACN, Ramires VRR (2007). Vínculo parental e
Kanbayashi Y, Lambert M, Achenbach TM, Larsson B, rede de apoio social: relação com a sintomatologia
Leung P, Liu X, Minaei A, Mulatu MS, Novik TS, Oh KJ, depressiva na adolescência. Aletheia 26, 95–108.
Roussos A, Sawyer M, Simsek Z, Dumenci L, Silvares EFM, Meyer SB, Santos EOL, Gerencer TT (2006).
Steinhausen HC, Metzke CW, Wolanczyk T, Yang HJ, Um estudo em cinco clínicas-escola brasileiras com a lista
Zilber N, Zukauskiene R, Verhulst FC, Rescorla LA, de verificação comportamental para crianças (CBCL). In
Almqvist F, Weintraub S, Bilenberg N, Bird H, Chen WJ Atendimento Psicológico em Clínicas-Escola (ed. EFM
(2007). Testing the 8-syndrome structure of the child Silvares), pp. 59–72. Alínea: Campinas.
behavior checklist in 30 societies. Journal of Clinical Child and Silvares EFM, Rocha MMR, Equipe Projeto Enurese (2007).
Adolescent Psychology 36, 405–417. Inventário dos Comportamentos de Crianças e Adolescentes de 6
Lampert TL, Polanczyk G, Tramontina S, Mardini V, Rohde a 18 anos. Versão brasileira do ‘Child Behavior Checklist’,
LA (2004). Diagnostic performance of the CBCL-Attention Not published.
Problem Scale as a screening measure in a sample of Swets JA (1996). Signal detection theory and ROC analysis
Brazilian children with ADHD. Journal of Attention Disorders in psychology and diagnostics. Hillsdale, NJ: Lawrence
8, 63–71. Erlbaum.
Moraes EO, Enumo SRF (2008). Estratégias de enfrentamento Tanaka OY, Lauridsen-Ribeiro E (2006). Desafio para a
da hospitalização em crianças avaliadas por instrumento atenção básica: incorporação da assistência em saúde
informatizado. PsicoUSF 13, 221–231. mental. Cadernos de Sáude Pública 22, 1845–1853.
Mota ER, Bertola DR, Kim C, Teixeira MCTV (2010). Viola L, Garrido G, Rescorla LA (2011). Testing multicultural
Alterações comportamentais na Síndrome de Noonan: robustness of the Child Behavior Checklist in a national
dados preliminares brasileiros. Revista Latinoamericana de epidemiological sample in Uruguay. Journal of Abnormal
Psicología 42, 87–95. Child Psychology 39, 897–908.
Muthén LK, Muthén BO (2007). MPlus User’s Guide, 5th edn. Zwirs B, Burger H, Schulpen T, Wiznitzer M, Fedder H,
Muthén & Muthén: Los Angeles, CA. Buitelaar J (2007). Prevalence of psychiatric disorders
Paula CS, Duarte CS, Bordin IA (2007). Prevalence of mental among children of different ethnic origin. Journal of
health problems in children and adolescents from the Abnormal Child Psychology 35, 556–566.