A Multilevel Analysis of Mathematically Low-Achieving Students in

Singapore
Dr. Qian Chen

Nanyang Technological University

Address: NIE5-B2-16, National Institute of Education, Nanyang Technological University, 1

Nanyang Walk, Singapore 637616.
Tel: (65) 6219 6157
Fax: (65) 6515 1458
Email: [email protected]

Abstract
In this study, TIMSS 2007 data were analysed with a particular focus on
mathematically low-achieving students at grade four in Singapore.
Specifically, the quantity and major characteristics of mathematically low-
achieving fourth graders were examined. Furthermore, eight variables at
the student level and nine variables at the school/class level were used to
build the two-level hierarchical generalized linear model so as to predict
the status of mathematically low-achieving fourth grader. The final model
suggested that six variables at the student level, i.e. student gender,
number of books at home, frequency of test language spoken at home,
frequency of mathematics homework, student’s self-confidence in
learning mathematics, student’s perception of school safety, and two
variables at the school/class level, i.e. teacher professional development
opportunity and principal’s perception of school climate significantly
predicted the status of mathematically low-achieving fourth grader.
Implications for educational research and practice are presented at the end
of this paper.

Key words: Low-achieving, Mathematics achievement, Grade four, TIMSS 2007, Singapore.
 INTRODUCTION

Over the past two decades, international comparative studies on educational achievement have

gained remarkable development. One of the most influential assessment projects is Trends in

International Mathematics and Science Study (TIMSS). It has constantly examined the

mathematics and science achievement of fourth and eighth graders in the participating countries

on a 4-year cycle since 1995. It is indicated that the East Asian countries and regions (Chinese

Taipei, Hong Kong SAR, Japan, Korea and Singapore) have consistently been the top achievers

among the participating countries over the world (Hensher & Johnson, 1981; Mullis, 2000, 2003;

Mullis, Martin, & Foy, 2008; OECD, 2004, 2010; Wilkins, 2004).

The admirable performance of the East Asian school systems has brought about

unprecedented attention from the wide community of educational researchers and policy makers

around the world. In order to reveal the keys to success, a fast-growing body of studies have

been intended to investigate the school education in these school systems from diverse

perspectives (Leung, 2001, 2006; Park, 2004; Wößmann, 2005; Zhu & Leung, 2011).

Nevertheless, with the widely acclaimed high performance in international assessments, some

important educational issues within the East Asian school systems, especially those regarding

achievement gaps seem to have been hardly explored. Needless to say, in any country and region,

attending to low-achieving students is crucial to the improvement of overall educational

achievement level as well as the realization of educational equity. Specifically, it is necessary to

develop a sound understanding of this group of students and factors related to their low

achievements. Therefore, this study aimed to use TIMSS 2007 data to conduct an in-depth

analysis of mathematically low-achieving students at grade four in Singapore. The specific

research questions for this study were formulated as follows:

 1) In Singapore, what was the quantity of mathematically low-achieving students at grade

four, defined as those whose benchmark scores were below the TIMSS Intermediate

international benchmark?

2) What were major characteristics of the mathematically low-achieving fourth graders in

Singapore?

3) What factors significantly predicted the status of mathematically low-achieving student at

grade four in Singapore?

THEORETICAL BACKGROUND

A large body of literature has discussed factors related to student achievement (Bloom, 1976;

Carroll, 1963; Coleman et al., 1966; Creemers, 1994; Lee & Shute, 2010; Walberg, 2003). These

factors can be divided into student-, classroom-, and school-level factors (Teodorović, 2011).

Student-level factors

According to Ismail (2009), several factors may help explain why some students attain higher

mathematics performance than others do, including students’ personal and home backgrounds,

resources for learning, time spent out of school in studying or doing homework in school

subjects, self-confidence in learning mathematics, motivation to learn mathematics, and

perceptions of being safe in school. In his study, multiple logistic regression analysis was

conducted on the data from Malaysian eighth graders in TIMSS 2003. It was found that having

self-confidence in learning mathematics, having a large number of books at home, regularly

using computers, and being non-Malay have a high positive association with mathematics

achievement among Malaysian students (Ismail, 2009).

Akyüz and Berberoğlu (2010) conducted a two-level analysis of TIMSS 1999 data from

nine European countries, i.e. Belgium, Slovak Republic, Czech Republic, Hungary, Italy,
Lithuania, Netherlands, Slovenia and Turkey. They found that at the student level, home

educational resources was significantly related to mathematics achievement in all the countries

except Netherlands (Akyüz & Berberoğlu, 2010). Ghagar, Othman and Mohammadpour (2011)

conducted multilevel analysis of mathematics achievement of Malaysian and Singaporean eighth

graders in TIMSS 2003. It was found that at the student level, mathematics self-concept was the

strongest predictor of mathematics achievement in both countries; attitude towards mathematics

was also a significant predictor in both countries; home educational resources was significantly

associated with mathematics achievement in Singapore but not in Malaysia. Wang, Osterlind and

Bergin (2012) used TIMSS 2003 grade eight data to build two-level mathematics achievement

model in four countries with varying level of achievement: the USA, Russia, Singapore and

South Africa. They found that student’s self-concept of ability in mathematics was a key

predictor of mathematics achievement in all the four countries; other student variables including

gender, parents’ highest education level, perception of school etc. differed in the magnitude of

relations to mathematics achievement across countries (Wang, et al., 2012).

Based on a review of school effectiveness research, Teodorović (2011) concluded that

“student-level variables are very important in determining student achievement in industrialised

and better-off developing countries, while their effect is less pronounced in poor developing

countries” (p. 220). It is further indicated that variance in student achievement can be mostly

attributed to student-level factors (Teodorović, 2012).

Classroom-level factors

Undoubtedly, teacher plays a key role at the classroom level. Teacher’s demographic

characteristics, perceptions/beliefs, attitudes, and classroom behaviours are widely examined in

relation to student achievement. In addition, two factors, i.e. class size and teacher professional
development opportunity are also discussed. As in the case with student factors, findings about

effects of these factors are mixed.

Akyüz and Berberoğlu (2010) found that after controlling for home educational resources,

teacher gender was not significantly related to mathematics achievement in Belgium, Slovak

Republic, Italy, Lithuania and Slovenia; however, female teacher was significantly related to

lower achievement in Czech Republic and Turkey, higher achievement in Hungary and

Netherlands. Wang et al. (2012) found that after controlling for student effects, teacher gender is

not significantly associated with students’ mathematics achievement in Russia, Singapore and

South Africa whereas male teacher is significantly related to higher mathematics achievement in

the USA. As far as the relationship between teaching experience and mathematics achievement is

concerned, a negative association was identified in Slovak Republic and Slovenia (Akyüz &

Berberoğlu, 2010), but a positive one in Netherlands, Turkey (Akyüz & Berberoğlu, 2010) and

South Africa (Wang, et al., 2012).

Furthermore, Akyüz and Berberoğlu (2010) found that after controlling other effects,

teacher’s perception of limitations to classroom teaching was negatively related to mathematics

achievement in Belgium, Italy, Netherlands and Slovenia, but not related in Slovak Republic,

Czech Republic, Hungary, Lithuania and Turkey. Wang et al. (2012) found that after controlling

for other variables, teacher’s perception of school climate is a significant predictor of eighth

graders’ mathematics achievement in Russia, South Africa and Singapore, but not in USA; and

teacher’s perception of school safety is not a significant predictor in all the four countries.

The nature of the relationship between homework and student achievement remains

disputed (Trautwein & Koller, 2003). However, the way teachers assign homework (e.g.

frequency, amount) is still an important consideration in many achievement studies (e.g. Akyüz
& Berberoğlu, 2010; De Jong, Westerhof, & Creemers, 2000; House, 2004). In the study by

Akyüz and Berberoğlu (2010), after controlling for other variables, teacher’s emphasis on

homework was found to be positively related to mathematics achievement in Belgium, Slovak

Republic and Lithuania, but not related in Czech Republic, Hungary, Italy, Netherlands, Slovenia

and Turkey.

Class size is also a commonly researched factor associated with student achievement

(Akyüz & Berberoğlu, 2010; Konstantopoulos, 2011; Nye, Hedges, & Konstantopoulos, 2001).

Although it is often assumed that being in small class is beneficial to all students, research

findings are inconclusive (Baker & Jones, 2005). Akyüz and Berberoğlu (2010) found that with

other variables controlled, class size was positively related to mathematics achievement in

Belgium, Hungary, Italy, Lithuania and Netherlands, but not related in Slovak Republic, Czech

Republic, Slovenia, and Turkey.

Lastly, teacher professional development opportunity is a potential factor related to

mathematics achievement. Goldhaber and Brewer (1997) concluded that teachers’ subject-

specific training has a significant impact on student test scores in mathematics and science.

Teachers who are trained and well-prepared are more effective in the classroom and therefore

have the greatest impact on student learning (Killion, 1999). However, other studies found that

professional development does not contribute to student achievement (Harbison & Hanushek,

1992; Jacob & Lefgren, 2004).

Based on review of effectiveness research, Teodorović (2011) concludes that “classroom-

level variables exhibit significant association with student achievement in industrialised and

better-off developing countries, and less so in poor developing countries, with composite

variables having a considerably larger effect than individual variables” (p. 220).
School-level factors

At the school level, good attendance at school, availability of school resources for mathematics

instruction and school climate as perceived by principals are usually considered as important

factors related to mathematics achievement. Again, findings about the association are

inconsistent across contexts.

Ghagar, Othman and Mohammadpour (2011) found that school location, availability of

school resources for mathematics instruction, good attendance at school, school climate as

perceived by principals were all significantly associated with eighth graders’ mathematics

achievement in Malaysia, after controlling for other variables; in Singapore, school climate as

perceived by principals was significantly related to mathematics achievement, but availability of

school resources for mathematics instruction and good attendance at school were not so, after

controlling for other student-level and classroom level variables. Wang et al (2012) found that

with other variables controlled for, school and class attendance significantly predicted eighth

graders’ mathematics achievement in Russia, but not so in the USA, Singapore and South Africa;

availability of school resources for mathematics instruction was significantly associated with

mathematics achievement in South Africa, but not in the USA, Russia and Singapore; principal’s

perception of school climate was a significant predictor of mathematics achievement in the USA

and Singapore, but not so in Russia and South Africa.

With regard to school-level factors in effectiveness research, Teodorović (2011)

concludes that they show the least consensus, “with their likely effect sizes ranging from null to

modest in industrialised and better-off developing countries, but they are very important for poor

developing countries” (p. 220).

 To sum up, research literature shows that many factors at different levels relate to

students’ mathematics achievement, however, findings about the relationships are yet

inconclusive, depending on research contexts and the set of variables involved in the analysis.

Therefore, it is necessary to continue context-specific secondary analysis of achievement data so

that new evidence and insights in this aspect can be produced. Moreover, multilevel modelling

has become a well-accepted approach to analysis of achievement data. Nevertheless, as

compared to the conventional linear modelling approach which uses achievement scores as

outcome variable, hierarchical generalized linear modelling (HGLM) or multilevel logistic

regression modelling has scarcely been used by achievement researchers. It is also noted that

some of the reviewed studies have been built on secondary analysis of TIMSS data (e.g.Louis &

Mistele, 2011; Thomson, 2008; Zuzovsky, 2008). Due to the use of nationally representative

samples in such large-scale assessments, these studies have obvious advantages in terms of

generalizability of research findings. However, it must be acknowledged that secondary analysis

studies are limited to a certain extent in selecting variables for analysis. In the next section,

research methods for this study are described in detail.

METHODS

Data Source and Sample

TIMSS is one of the largest and most complex cross-national data collection efforts of

educational achievement. In addition to assessing students’ mathematics and science proficiency,

TIMSS also gathers a lot of background information from students, their teachers and school

principals through questionnaire survey. Data for this study were drawn from grade four datasets

of Singapore in the TIMSS 2007 international database.

 According to Joncas (2008a), the TIMSS 2007 assessment generally employed a two-

stage stratified cluster sample design. At the first stage, schools were selected using probability-

proportional-to-size sampling; at the second stage, one or two classes were randomly sampled in

each school. Particularly, Singapore had a third sampling stage where students were sampled

within classes. It is suggested that in analysing TIMSS data, it is important to include a weight

variable reflecting the sampling scheme (Joncas, 2008b; Rutkowski, Gonzalez, Joncas, & von

Davier, 2010).

In Singapore, two classes were selected from each participating school. The numbers of

schools, classes, students and mathematics teachers, together with the rank and mean

mathematics achievement score for the two systems are given in Table 1.

TABLE 1
Sample Size, Rank and Mean Score of Singapore
Mathematics
School Class Student Rank Mean Score
Teacher
177 354 378 5041 2 599
Rank and mean scores were from TIMSS 2007 International Mathematics Report (Mullis, et al., 2008).

Due to the nested structure of the data and the sample design, three-level (student, class

and school) modelling was initially attempted, but found to be unsuccessful afterwards. Finally,

two-level modelling was chosen for this study. At level 1, variables were from students, and at

level 2, variables were related to schools and classes.

Measures and Preparation of Variables

The outcome variable for this study was a dummy variable (1=being mathematically low-

achieving student, 0=not being mathematically low-achieving student), which was created based

on fourth graders’ benchmark score in TIMSS 2007 (1=below 400, 2=at or above 400 but below

475, 3= at or above 475 but below 550, 4= at or above 550 but below 625, 5= at or above 625).

TIMSS 2007 used five plausible values to derive five benchmark scores so as to describe the
mathematics performance of each student in an international setting, where the score of 400, 475,

550 and 625 denoted the Low, Intermediate, High and Advanced international benchmark

respectively (More details about the four international benchmarks are given in Appendix 1). In

this study, if individual student’s benchmark scores were below 3 (i.e. the Intermediate

benchmark 475), he or she was treated as mathematically low-achieving student.

As far as the predictor variables are concerned, at the student level, a total of eight

variables were used, i.e. student gender (0=girl, 1=boy), student immigration background

(0=both parents were born in the country, 1=at least one parent was not born in the country),

number of books at home as proxy for family SES (1=0-10, 2=11-25, 3=26-100, 4=101-200,

5=over 200), frequency of test language spoken at home (1=never, 2=sometimes, 3=almost

always, 4=always), frequency of mathematics homework (1=never, 2=less than once a week,

3=1or 2 times a week, 4=3 or 4 times a week, 5=every day), positive affect to mathematics, self-

confidence in learning mathematics, and perception of school safety. At the school/class level, a

total of nine variables were used, i.e. teacher gender (0=female, 1=male), number of years of

teaching experience, class size for mathematics instruction (number of students in mathematics

class), teacher’s perception of few or no limitations on mathematics instruction due to student

factors, teacher’s perception of school climate, index of good attendance at school (1=low,

2=medium, 3=high), availability of school resources for mathematics instruction, principal’s

perception of school climate, and teacher professional development opportunity. A total of six

scales were developed to respectively measure student’s positive affect to mathematics, self-

confidence in learning mathematics, teacher’s perception of few or no limitations on

mathematics instruction due to student factors, teacher’s perception of school climate,

availability of school resources for mathematics instruction, and principal’s perception of school
climate. In addition, two index variables, i.e. student’s perception of school safety and teacher

professional development opportunity were created based on relevant items in the student

questionnaire and teacher questionnaire respectively. Details of scale and index development are

included in Appendix 2.

Particularly, in the HGLM analyses, dummy/dichotomous variables were not centred. As

a result, the coefficients for those variables were interpreted as the mean log-odds difference

between the two groups. All scale and index variables were transformed into z scores (M=0,

SD=1) so that results could be reported as standard deviation units.

For all the variables, the number of cases with incomplete data of each variable was

calculated. The highest percentage of cases with incomplete data for all student-level variables

was 2.29%, and that for the school/class level variables was 11.96%. The multiple imputation

(MI) method was used to impute missing data on the variables. Consequently, five imputed

datasets were produced for subsequent data analysis.

Data Analysis

In this study, all analyses involved use of the five imputed datasets. Each imputed dataset was

analysed separately and then the results were averaged across the five datasets. Furthermore, for

each imputed dataset, there were five outcome variables indicating whether students were

mathematically low-achieving students or not. Thus, when analysing each imputed dataset, all

analyses concerned with the outcome variable were actually replicated five times, using a

different outcome variable each time, and the results were combined into a single result that

included information on standard errors which incorporated both sampling and imputation error,

similar to the procedures suggested by Foy and Olson (2009).

 Multiple programs, i.e. SPSS 20, International Database (IDB) Analyser as well as HLM

7 were used to prepare and analyse the data. IDB Analyser software was developed by IEA

(2009) specifically for managing and analysing TIMSS international data files (Foy & Olson,

2009). In this study, student file, teacher file and school file for Singapore were merged into one

through IDB Analyser’s merge module for subsequent data preparation and analyses. SPSS 20

and IDB Analyser’s analysis module were used to conduct reliability analysis of scales and

indices, descriptive and between-group inferential analyses (independent samples T-test). HLM 7

was used for two-level HGLM analyses. It is noteworthy that because of the use of merged file,

the reported mathematics teacher weights MATWGT was involved in the analyses where

appropriate, which could correctly reflect the characteristics of the student population.

In the two-level HGLM analyses (Raudenbush & Bryk, 2002), full maximum likelihood

method was used to estimate the model parameters. A model building process was applied to

study the incremental contribution of variables in predicting the status of mathematically low-

achieving students. Specifically, student, class and school variables were used to build a model

for fourth graders in Singapore. The level 1 unit of analysis was students and the level 2 unit of

analysis was schools/classes. The model building process was first conducted at level 1 and then

at level 2. At each step in the model building, a model trimming procedure was employed to

achieve parsimony.

For each of the five imputed datasets, three sets of HGLM models were analyzed and

modified in the modelling process. Firstly, the fully unconditional model (Model A) was

examined, which did not contain any predictors. Secondly, student variables were added to the

model as level 1 predictors, and nonsignificant predictors were removed (model trimming).

Thirdly, class and school variables were added to model B as level 2 predictors. In order to
achieve model parsimony, the final model (Model C) only included statistically significant

variables. With α = 0.05, the conclusions on statistical significance of relationships between each

individual variable and outcome variable were the same across Model B and Model C. It

included significant level 1 and level 2 predictors.

RESULTS

Descriptive Statistics of Scales

As mentioned earlier, six scales were developed to respectively measure student’s positive affect

to mathematics, self-confidence in learning mathematics, teacher’s perception of few or no

limitations on mathematics instruction due to student factors, teacher’s perception of school

climate, availability of school resources for mathematics instruction, and principal’s perception

of school climate. Descriptive statistics together with the reliability coefficients for these scales

are included in Appendix 2.

Quantity of Mathematically Low-achieving Students at Grade Four

Table 2 displays the quantity and mean score of mathematically low-achieving students, as

compared to the rest of student population at grade four.

TABLE 2
Quantity and Mean Score of Mathematically Low-achieving Fourth Graders
Sample Size Population Size Percentage (se) Mean Score (se)
Mathematically
Low-achieving 457 4071 8.25 (0.9) 435 (6.1)
Students
Non-mathematically
Low-achieving 4584 45305 91.75 (0.9) 614 (3.1)
Students
se = standard error.

It is apparent from the table that there were 4071 mathematically low-achieving fourth

graders in Singapore, they accounted for 8.25% of the student population at grade four, and their

average mathematics score in TIMSS 2007 was 435.

Major Characteristics of Mathematically Low-achieving Students at Grade Four

Independent samples T-test was carried out to compare mathematically low-achieving students

with the rest of student population at grade four in terms of the considered variables, the

results are given in Table 3. In this way, the major characteristics of this particular group of

students were identified.

TABLE 3
Major Characteristics of Mathematically Low-achieving Fourth Graders
Mathematically Non-mathematically
Characteristics Low-achieving Low-achieving Sig.
Students Students
Student gender 0.60 0.51 ***
Student immigration background 0.39 0.37 -
Number of books at home 2.34 3.07 ***
Frequency of test language spoken at home 2.22 2.74 ***
Frequency of mathematics homework 3.63 3.93 ***
Positive affect to mathematics 8.54 9.70 ***
Self-confidence in learning mathematics 8.94 11.29 ***
Student’s perception of school safety 2.79 3.53 ***

Teacher gender 0.39 0.33 ***

Number of years of teaching experience 10.67 9.83 ***
Teacher’s perception of few or no limitations on
11.07 12.28 ***
mathematics instruction due to student factors
Teacher’s perception of school climate 27.12 27.78 ***
Teacher professional development opportunity 3.54 3.46 **
Class size for mathematics instruction 35.43 38.43 ***
Good attendance at school 2.57 2.56 -
Availability of school resources for mathematics
36.74 36.70 -
instruction
Principal’s perception of school climate 29.00 30.10 ***
**p<0.01, ***p<0.001, - denotes no significant difference.

According to the table above, mathematically low-achieving students were significantly

different from the rest of student population at grade four in the majority of variables.

Specifically, they consisted of bigger proportion of boys, had less number of books at home

(lower family SES), spoke test language at home and did mathematics homework less frequently,

had lower self-confidence in learning mathematics and worse perceptions of school safety.
Furthermore, they were taught mathematics by teachers who were comprised of larger proportion

of males, had longer teaching experience, enjoyed more teacher professional development

opportunities, handled smaller class, but perceived more limitations on mathematics instruction

due to student factors and worse school climate. Lastly, mathematically low-achieving fourth

graders attended schools whose principals had worse perceptions of school climate.

Factors Predicting the Status of Mathematically Low-achieving Students at Grade Four

In the HGLM analyses, only the results from population-average models (with robust standard

errors) were considered because population-average questions rather than school-specific

questions were of interest in this study (Raudenbush & Bryk, 2002). The unconditional model

(Model A) was as follows:

Level 1: Prob (Yij=1| βj) = φij

log [φij /(1- φij)] = β0j

Level 2: β0j = γ00 + u0j

where γ00 is the average log-odds of being mathematically low-achieving student across the

schools. The variance of u0j is the variance between schools in school-average log-odds of being

mathematically low-achieving student. The results from Model A are given in Table 4.

According to the table, in Singapore, the odds of being mathematically low-achieving student at

grade four was e (-2.40) = 0.09, corresponding to a predicted probability of 0.08.

Model building at the student level (level 1) began by adding all student-level predictors.

Specifically, the eight student-level variables were modelled as predictors of status of

mathematically low-achieving student. The effects of these predictors were modelled as fixed

coefficients while the intercept was treated as random. After model trimming, the results from

Model B are also given in Table 4. Later on, Model B was compared with Model A to examine
how much the student-level variables contributed to the prediction of status of mathematically

low-achieving student.

From Model B, student gender, number of books at home, frequency of test language

spoken at home, frequency of mathematics homework, self-confidence in learning mathematics,

and student’s perception of school safety were significantly associated with the log-odds of being

mathematically low-achieving student. Among the six predictors, student’s self-confidence in

learning mathematics was the strongest one. A one standard deviation increase in it reduced the

log-odds of being mathematically low-achieving student by 0.70, which was associated with a
(-0.70)
relative odds ratio of e = 0.5, after controlling for the other predictors in the model and the

random school effect u0j. That is, if two students were similar in other ways but differed by one

standard deviation in their self-confidence in learning mathematics, then the odds of being

mathematically low-achieving student for the student with higher self-confidence were 0.5 times

the odds for the student with lower self-confidence. Interestingly, the relationship between

student gender and the log-odds of being mathematically low-achieving student was positive.
0.35
The odds of being mathematically low-achieving student for boys were e = 1.42 times the

odds for girls. Surprisingly, student’s positive affect to mathematics was not significantly related

to the log-odds of being mathematically low-achieving student.

After model building and model trimming at the student level (level 1) and at the

school/class level (level 2), the final model (Model C) was obtained. The details of the final

model are also displayed in Table 4. According to Model C, teacher professional development

opportunity and principal’s perception of school climate were significantly related to the log-

odds of being mathematically low-achieving student. Unexpectedly, the relationship between

teacher professional development opportunity and the log-odds of being mathematically low-
achieving student was positive. Holding constant the other predictors in the model and the

random school effect u0j, a one standard deviation increase in teacher professional development

opportunity increased the log-odds of being mathematically low-achieving student by 0.29,

0.29
which was associated with a relative odds ratio of e =1.34. In simple words, student whose

mathematics teacher enjoyed more professional development opportunities was more likely to be

mathematically low-achieving student.

TABLE 4 Results for Population-Average Models with Robust Standard Errors

Unconditional Model Level-1 Model Final Model
(A) (B) (C)
Coeff. se Coeff. se Coeff. Se
Intercept -2.40*** 0.10 -3.17*** 0.12 -3.21*** 0.12
Level 1
Student gender 0.35** 0.11 0.35** 0.11
Student immigration background - - - -
Number of books at home -0.42*** 0.07 -0.42*** 0.07
Frequency of test language spoken at home -0.33*** 0.06 -0.33*** 0.07
Frequency of mathematics homework -0.18** 0.06 -0.18** 0.06
Positive affect to mathematics - - - -
Self-confidence in learning mathematics -0.70*** 0.05 -0.71*** 0.05
Student’s perception of school safety -0.39*** 0.06 -0.40*** 0.06
Level 2
Teacher gender - -
Number of years of teaching experience - -
Teacher’s perception of few or no limitations on - -
mathematics instruction due to student factors
Teacher’s perception of school climate - -
Teacher professional development opportunity 0.29** 0.10
Class size for mathematics instruction - -
Good attendance at school - -
Availability of school resources for mathematics - -
Principal’s perception of school climate
instruction -0.24** 0.08

Variance 1.10 0.73 0.64

Deviance 12310.46 11856.47 11843.46
Coeff. = Coefficient; se =standard error; - Not statistically significant; **p<0.01, ***p<0.001.

The final model can be formulated as below:

Level 1: log [φij/(1- φij)] = β0j + β1j (student gender) + β2j (number of books at home) + β3j

(frequency of test language spoken at home) + β4j (frequency of mathematics homework) + β5j

(self-confidence in learning mathematics) + β6j (student’s perception of school safety)

Level 2: β0j = γ00 + γ01 (teacher professional development opportunity) + γ02 (principal’s

perception of school climate) + u0j

β1j = γ10

β2j = γ20

β3j = γ30

β4j = γ40

β5j = γ50

β6j = γ60.

The model fit statistic McFadden’s ρ2 was calculated by computing the proportion

reduction in deviance obtained from the final model relative to the unconditional model, which

was equal to 0.04. Adding the student-level variables to the unconditional model reduced 34% of

the intercept variance in adjusted school log-odds of being mathematically low-achieving

students. After including the school/class-level variables in the student-level model, the variance

was further reduced by 12%. The final model explained a total of 42% of the intercept variance.
 CONCLUSION AND DISCUSSION

In this study, the quantity and major characteristics of mathematically low-achieving fourth

graders in high-performing Singapore were examined. It was found that in Singapore, there were

4071 mathematically low-achieving fourth graders, which accounted for 8.25% of the student

population at grade four. This particular group of students significantly differed from the rest of

the student population in many aspects, including gender composition, number of books at home

(as proxy for family SES), self-confidence in learning mathematics, mathematics teacher’s

teaching experience, teacher’s and principal’s perception of school climate etc.

Furthermore, HGLM techniques were used in this study to predict the status of

mathematically low-achieving student at grade four in Singapore. Among the six student-level

significant predictors, student’s self-confidence in learning mathematics was found to be the

strongest, which is essentially in accord with the previous findings that student’s self-confidence

or self-concept of ability in mathematics has the largest relation to mathematics achievement in

the USA, Russia, South Africa and Malaysia (Abd.Ghagar, et al., 2011; Wang, et al., 2012).

One unexpected finding is that student’s positive affect to mathematics was not

significantly associated with the log-odds of being mathematically low-achieving student. One

possible explanation is that Singaporean students do not think of learning mathematics as a

pleasurable activity; whether they like it or not, they are motivated to study mathematics hard by

instrumental reasons, e.g. pass various exams to attend the desired secondary schools and

universities. As noted by Leung (2001), the traditional view in East Asian countries has been

that “studying is a serious endeavour” (p. 41). Students need to study hard, but do not need to

enjoy the study.

 It is not surprising to find in this study that other student-level variables, particularly

number of books at home (as proxy for family SES) were significantly associated with the log-

odds of being mathematically low-achieving student, given plenty of evidence from previous

achievement research (e.g. Teodorović, 2012; Wößmann, 2005).

At the school/class level, teacher professional development opportunity was found to be

positively related to the log-odds of being mathematically low-achieving student. That is, student

whose mathematics teacher enjoyed more professional development opportunities was more

likely to be mathematically low-achieving student. This finding was surprising at first glance

because it was assumed that less teacher professional development opportunities would relate to

lower student achievement. However, on second thought, this finding seems reasonable and even

encouraging given that this group of students would benefit from mathematics teachers who

enjoyed more professional development opportunities.

IMPLICATIONS

This study provides several implications for educational research and practice. Firstly,

this study used cross-sectional observational data, thus claims about causal relationship are

inappropriate. This made impossible exploring the direction of the influence of students’

perceptions and the status of mathematically low-achieving students. Further research can

explore the possibilities in this regard. However, this study does provide evidence about the

strong association between student’s self-confidence in learning mathematics and mathematics

achievement.

Secondly, it was found that boys were more likely to be mathematically low-achieving

students than girls at grade four in Singapore. Clearly, reasons behind this phenomenon need to

be explored so that the gender-based achievement gap can be diminished.

 Thirdly, teacher professional development opportunity was found to be positively

associated with the log-odds of being mathematically low-achieving student in Singapore. Given

the great efforts and resources put into this area, it seems necessary for teacher educators and

other parties concerned in Singapore to examine current teacher professional development

activities from both quantitative and qualitative perspective so that effective strategies for closing

achievement gaps can be identified.

Fourthly, this study produced a predictive model of mathematically low-achieving

students at grade four in Singapore based on TIMSS 2007 data. The model fit statistic

McFadden’s ρ2 for this model was equal to 0.04. According to Hensher and Johnson (1981),

values in the 0.2 to 0.4 range are considered highly satisfactory. Therefore, the model from this

study needs to be improved so as to better predict the status of mathematically low-achieving

fourth grader in Singapore. Moreover, a replicate study with the soon available TIMSS 2011 data

will be able to provide new evidence and insights about the changes in the predictive model over

the period. Thereby, the high-low achievement gap and the particular group of mathematically

low-achieving students within high-performing Singapore can gain sustainable attention. At the

same time, relevant educational research and practices can be constantly revisited for

improvement.

Lastly, by focusing on Singapore, this study makes a good starting point to understand

the mathematically low-achieving students in the high-performing school systems. Similar

efforts can be extended to other high-performing systems, which will help us develop a more

comprehensive understanding of issues regarding achievement gap in the East Asian area. In

addition, relevant similarities and differences among these places can be identified.
 REFERENCES

Abd.Ghagar, M. N., Othman, R., & Mohammadpour, E. (2011). Multilevel analysis of achievement in
mathematics of Malaysian and Singaporean students. Journal of Educational Psychology and
Counseling, 2, 285-304.
Akyüz, G., & Berberoğlu, G. (2010). Teacher and classroom characteristics and their relations to
mathematics achievement of the students in the TIMSS. New Horizons in Education, 58(1), 77-
95.
Baker, R., & Jones, A. (2005). How can international studies such as the international mathematics and
science study and the programme for international student assessment be used to inform
practice, policy and future research in science education in New Zealand? International Journal
of Science Education, 27(2), 145-157. doi: 10.1080/0950069042000276695
Bloom, B. (1976). Human characteristics and school learning. New York, NY: McGraw-Hill.
Carroll, J. B. (1963). A model of learning. Teacher College Record, 64, 723-733.
Coleman, J., Campbell, E., Hobson, C., McPartland, J., Mood, A., & Weinfeld, F. (1966). Equality of
educational opportunity. Washington DC: U.S. Department of Health, Education and Welfare.
Creemers, B. P. M. (1994). The effective classroom. London: Cassell.
De Jong, R., Westerhof, K. J., & Creemers, B. P. M. (2000). Homework and student math achievement in
junior high schools. Educational Research & Evaluation, 6(2), 130-157. doi: 10.1076/1380-
3611(200006)6:2;1-E;F130
Foy, P., & Olson, J. F. (2009). TIMSS 2007 User Guide for the International Database. Chestnut Hill, MA:
International Study Center, Boston College.
Goldhaber, D., & Brewer, D. (1997). Evaluating the effect of teacher degree level on educational
performance. In W. Fowler (Ed.), Developments in School Finance, 1996 (pp. 197-210).
Washington, DC: NCES.
Harbison, R. W., & Hanushek, E. A. (1992). Educational performance of the poor: Lessons from rural
northeast Brazil. New York, NY: Oxford University Press.
Hensher, D., & Johnson, L. W. (1981). Applied discrete choice modelling. London: Croom Helm.
House, J. D. (2004). The effects of homework activities and teaching strategies for new mathematics
topics on achievement of adolescent students in Japan: Results from the TIMSS 1999
assessment. International Journal of Instructional Media, 31(2), 199-210.
Ismail, N. A. (2009). Understanding the gap in mathematics achievement of Malaysian students. Journal
of Educational Research, 102(5), 389-394.
Jacob, B. A., & Lefgren, L. (2004). The impact of teacher training on student achievement:
Quasiexperimental evidence from school reform efforts in Chicago. Journal of Human Resources,
39(1), 50-79.
Joncas, M. (2008a). TIMSS 2007 sample design. In J. F. Olson, M. O. Martin & I. V. S. Mullis (Eds.), TIMSS
2007 Technical Report (pp. 77-92). Chestnut Hill, MA: International Study Center, Boston
College.
Joncas, M. (2008b). TIMSS 2007 sampling weights and participation rates. In J. F. Olson, M. O. Martin & I.
V. S. Mullis (Eds.), TIMSS 2007 Technical Report (pp. 154-192). Chestnut Hill, MA: International
Study Center, Boston College.
Killion, J. (Ed.). (1999). What works in the middle: Results-based staff development. Oxford, OH: National
Staff Development Council.
Konstantopoulos, S. (2011). How consistent are class size effects? Evaluation Review, 35(1), 71-92. doi:
10.1177/0193841X11399847
Lee, J., & Shute, V. J. (2010). Personal and social-contextual factors in K-12 academic peferomance: An
integrative perspective on student learning. Educational Psychology, 45(3), 185-202. doi:
10.1080/00461520.2010.493471
Leung, F. K. S. (2001). In search of an East Asian identity in mathematics education. Educational Studies
in Mathematics, 47, 35-51.
Leung, F. K. S. (2006). Mathematics education in East Asia and the West: Does culture matter? In F. K. S.
Leung, K.-D. Graf & F. J. Lopez-Real (Eds.), Mathematics education in different culture: A
comparative study of East Asia and the West (pp. 21-46). New York: Springer.
Louis, R. A., & Mistele, J. M. (2011). The differences in scores and self-efficacy by student gender in
matehmatics and science. International Journal of Science and Mathematics Education, 1-28.
doi: 10.1007/s10763-011-9325-9
Mullis, I. V. S. (2000). TIMSS 1999 International Mathematcs Report: Findings from IEA's Repeat of the
Third International Mathematcs and Science Study at the Eighth Grade. Chestnut Hill, MA:
International Study Center, Lynch School of Education, Boston College.
Mullis, I. V. S. (2003). TIMSS 2003 International Mathematics Report : Findings from IEA's Trends in
International Mathematics and Science Study at the Fourth and Eighth grade. Chestnut Hill, MA:
International Study Center, Boston College, Lynch School of Education.
Mullis, I. V. S., Martin, M. O., & Foy, P. (2008). TIMSS 2007 International Mathematics Report: Findings
from IEA’s Trends in International Mathematics and Science Study at the Fourth and Eighth
Grades. Chestnut Hill, MA: TIMSS & PIRLS International Study Center, Boston College.
Nye, B., Hedges, L. V., & Konstantopoulos, S. (2001). The Long-term effects of small classes in early
grades: Lasting benefits in mathematics achievement at grade 9. Journal of Experimental
Education, 69(3), 245. doi: 10.1080/00220970109599487
OECD. (2004). First Results From PISA 2003: Executive Summary.
OECD. (2010). PISA 2009 Results: Executive Summary.
Park, K. (2004). Factors contributing to East Asian students'high achievement:Focusing on East Asian
teachers and their teaching Paper presented at the APEC Educational Reform Summit,
Beijing,China.
Raudenbush, S. W., & Bryk, A. S. (2002). Hierarchical Linear Models : Applications and Data Analysis
Methods (2nd ed.). Thousand Oaks, CA: Sage Publications.
Rutkowski, L., Gonzalez, E., Joncas, M., & von Davier, M. (2010). International large-acale assessment
data: Issues in secondary analysis and reporting. Educational Researcher, 39( 2), 142-151 doi:
10.3102/0013189X10363170
Teodorović, J. (2011). Classroom and school factors related to student achievement: What works for
students? School Effectiveness and School Improvement, 22(2), 215-236. doi:
10.1080/09243453.2011.575650
Teodorović, J. (2012). Student background factors influencing student achievement in Serbia.
Educational Studies, 38(1), 89-110. doi: 10.1080/03055698.2011.567027
Thomson, S. (2008). Examining the evidence from TIMSS: Gender differences in Year 8 science
achievement in Australia. Studies in Educational Evaluation, 34, 73-81. doi:
10.1016/j.stueduc.2008.04.003
Trautwein, U., & Koller, O. (2003). The relationship between homework and achievement--Still much of
a mystery. Educational Psychology Review, 15(2), 115-145.
Walberg, H. J. (2003). Improving educational productivity. Washington, DC: Education Resources
Information Center.
Wang, Z., Osterlind, S. J., & Bergin, D. A. (2012). Building mathematics achievement models in four
countries using TIMSS 2003. International Journal of Science and Mathematics Education, 10, 1-
28.
Wilkins, J. L. M. (2004). Mathematics and Science Self-Concept: An International Investigation. [Article].
Journal of Experimental Education, 72(4), 331-346.
Wößmann, L. (2005). Educational production in East Asia: The impact of family background and
schooling policies on student performance. German Economic Review, 6, 331-353. doi:
10.1111/j.1468-0475.2005.00136.x
Zhu, Y., & Leung, F. K. S. (2011). Motivation and achievement: Is there an East Asian model?
International Journal of Science and Mathematics Education, 9, 1189-1212. doi:
10.1007/s10763-010-9255-y
Zuzovsky, R. (2008). Closing achievement gaps between Hebrew-speaking and Arabic-speaking students
in Israel: Findings from TIMSS-2003. Studies in Educational Evaluation, 34, 105-117. doi:
10.1016/j.stueduc.2008.04.006

Appendix 1

TABLE 5
TIMSS 2007 International Benchmarks of Mathematics Achievement for Grade 4
Advanced International Benchmark – 625
Students can apply their understanding and knowledge in a variety of relatively complex situations and explain
their reasoning. They can apply proportional reasoning in a variety of contexts. They demonstrate a developing
understanding of fractions and decimals. They can select appropriate information to solve multi-step word
problems. They can formulate or select a rule for a relationship. Students can apply geometric knowledge of a
range of two- and three-dimensional shapes in a variety of situations. They can organize, interpret, and
represent data to solve problems.

High International Benchmark – 550

Students can apply their knowledge and understanding to solve problems. Students can solve multi-step word
problems involving operations with whole numbers. They can use division in a variety of problem situations.
They demonstrate understanding of place value and simple fractions. Students can extend patterns to find a
later specified term and identify the relationship between ordered pairs. Students show some basic geometric
knowledge. They can interpret and use data in tables and graphs to solve problems.

Intermediate International Benchmark – 475

Students can apply basic mathematical knowledge in straightforward situations. Students at this level
demonstrate an understanding of whole numbers. They can extend simple numeric and geometric patterns.
They are familiar with a range of two-dimensional shapes. They can read and interpret different representations
of the same data.
Low International Benchmark – 400
Students have some basic mathematical knowledge. Students demonstrate an understanding of adding and
subtracting with whole numbers. They demonstrate familiarity with triangles and informal coordinate systems.
They can read information from simple bar graphs and tables.
Source: TIMSS 2007 International Mathematics Report (Mullis, et al., 2008, p. 68)
 Appendix 2
Scale and Index Development

Six scales and two index variables were developed based on the recoded questionnaire items
with reference to TIMSS 2007 User Guide (Foy & Olson, 2009). When recoding the original
items for scale development, the general principle was that a higher code represented a more
favourable characteristic. The scale for student’s positive affect to mathematics was based on
student’s responses to three items in the student questionnaire, and each item was recoded on 4-
point scale (1=Disagree a lot, 2=Disagree a little, 3=Agree a little, 4=Agree a lot). The scale for
student’s self-confidence in learning mathematics was based on student’s responses to four items
in the student questionnaire, and each item was recoded on 4-point scale (1=Disagree a lot,
2=Disagree a little, 3=Agree a little, 4=Agree a lot). The scale for teacher’s perception of few or
no limitations on mathematics instruction due to student factors was based on teacher’s
responses to five items in the teacher questionnaire, and each item was recoded on 4-point scale
(1= A lot, 2=Some, 3= A little, 4=Not at all). The scale for teacher’s perception of school climate
was based on teacher’s responses to eight items characterizing his/her school in the teacher
questionnaire, and each item was recoded on 5-point scale (1=Very low, 2=Low, 3=Medium,
4=High, 5=Very high). The scale for availability of school resources for mathematics instruction
was based on principal’s responses to ten items characterizing the extent to which the school’s
instruction capacity is affected by shortage of resources, and each item was recoded on 4-point
scale (1=A lot, 2=Some, 3=A little, 4=None). The scale for principal’s perception of school
climate was based on principal’s responses to eight items characterizing his/her school in the
school questionnaire, and each item was recoded on 5-point scale (1= Very low, 2=Low,
3=Medium, 4=High, 5=Very high). Reliability analyses indicated that the six scales were fairly
reliable measures, with all Cronbach’s alpha values above 0.7. The items and descriptive
statistics are displayed in Table 6.
In addition, two indices were created by summating relevant items which were recoded
into dummy variables. The index of student’s perception of school safety was calculated by
adding up student’s responses to five items in the student questionnaire: ‘In school, did any of
these things happen during the last month? (a) something of mine was stolen; (b) I was hit or
hurt by other student(s); (c) I was made to do things at I didn’t want to do by other students; (d) I
was made fun of or called names; (e) I was left out of activities by other students (0=No, 1=Yes)’.
Reliability analysis revealed that Cronbach’s alpha value for this index was 0.58. Despite
relatively low reliability, this index was retained for use in data analysis. The index of teacher
professional development opportunity was created by summing six recoded items related to
professional development in the teacher questionnaire: ‘In the past two years, have you
participated in professional development in any of the following? (a) mathematics content, (b)
mathematics pedagogy/instruction, (c) mathematics curriculum, (d) integrating information
technology into mathematics, (e) improving students’ critical thinking or problem solving skills,
(f) mathematics assessment (0=No, 1=Yes)’. Reliability analysis revealed that this index variable
was acceptable, with Cronbach’s alpha value being 0.81.
The summated scores for each scale and index variable were standardized into z score for
subsequent HGLM analysis.
 TABLE 6
Items and Descriptive Statistics for Scale
Scale and Item M SE α
Positive affect to mathematics 0.86
I enjoy learning mathematics. 3.22 0.02
Mathematics is boring. 3.19 0.02
I like mathematics. 3.20 0.02
Self-confidence in learning mathematics 0.75
I usually do well in mathematics. 2.92 0.02
Mathematics is harder for me than for many of my classmates. 2.67 0.02
I am just not good at mathematics. 2.70 0.02
I learn things quickly in mathematics. 2.81 0.02
Teacher’s perception of few or no limitations on mathematics instruction due to
0.80
student factors
Students with different academic abilities. 2.11 0.04
Students who come from a wide range of backgrounds. 2.61 0.05
Student with special needs. 2.80 0.06
Uninterested students. 2.31 0.05
Disruptive students. 2.36 0.05
Teacher’s perception of school climate 0.82
Teachers’ job satisfaction. 3.46 0.04
Teachers’ understanding of the school’s curricular goals. 3.80 0.04
Teachers’ degree of success in implementing the school’s curriculum. 3.63 0.03
Teachers’ expectations for student achievement. 3.86 0.04
Parental support for student achievement. 3.29 0.04
Parental involvement in school activities. 3.21 0.04
Students’ regard for school property. 3.11 0.04
Students’ desire to do well in school. 3.37 0.03
Availability of school resources for mathematics instruction 0.90
Instructional materials. 3.92 0
Budget for supplies. 3.89 0
School buildings and grounds. 3.47 0
Heating/cooling and lighting systems. 3.77 0
Instructional space. 3.29 0
Computers for mathematics instruction. 3.71 0
Computer software for mathematics instruction. 3.61 0
Calculators for mathematics instruction. 3.81 0
Library materials relevant to mathematics instruction. 3.59 0
Audio-visual resources for mathematics instruction. 3.63 0
Principal’s perception of school climate 0.83
Teachers’ job satisfaction. 3.82 0
Teachers’ understanding of the school’s curricular goals. 3.97 0
Teachers’ degree of success in implementing the school’s curriculum. 3.81 0
Teachers’ expectations for student achievement. 3.93 0
Parental support for student achievement. 3.60 0
Parental involvement in school activities. 3.27 0
Students’ regard for school property. 3.79 0
Students’ desire to do well in school. 3.82 0