Profile Analysis via Multidimensional Scaling for the Revised

Two-Factor Learning Process Questionnaire

Alan Socha and Ellen A. Sigler
Western Carolina University



Author Note
Alan Socha, Office of Institutional Planning and Effectiveness, Western Carolina
University; Ellen A. Sigler, Department of Psychology, Western Carolina University.
We are grateful to David Kember and J ohn Biggs for allowing us to conduct research
with the Revised Learning Process Questionnaire. We are also grateful to them for allowing us to
reproduce their copyrighted questionnaire and scales in Appendices A and B.

Paper presented at the Annual Meeting of the National Council on Measurement in Education
(New Orleans, LA, April 9-11, 2011).

Abstract
The Revised Learning Process Questionnaire has been part of the development of a conceptual
understanding of how students learn and what motivates them to engage in particular tasks. We
obtained responses from 329 student volunteers at a mid-sized public university in the southeast
United States. While looking at the psychometric properties of this questionnaire in a different
educational context from which the instrument had been originally validated was one purpose of
the study, the main thrust of this research was to use Profile Analysis via Multidimensional
Scaling (PAMS) to improve the diagnostic functionality of the instrument as well as further
explore the validity of the questionnaire. We found that interpreting the latent structure in terms
of the dimensions of Strategy and Motive as opposed to the factors of Deep and Surface
approaches to be more appropriate for diagnostic use. We also found that PAMS has the inherent
ability to assess an individuals fit within the model, thereby acting as a measure of self-report
credibility. Both the Strategy and Motive dimensions were found to have ecological validity
through analyzing its relationship to academic performance.
Profile Analysis via Multidimensional Scaling for the Revised
Two-Factor Learning Process Questionnaire
Learning and study strategies have been an interest in higher education for decades. From
the earliest studies done by Entwistle and Ramsden (1982), looking at the self-perceptions of
learning and motivation in the academic environment, researchers have been interested in
recording how students study. In recent years, instruments have been developed to record and
analyze the phenomenon, to better understand how people study and what factors can be
attributed to success or failure in the academic setting. Kember, Biggs, and Leung (2004) and
Everson, Weinstein, and Laitusis (2000) are examples of those who have endeavored to evaluate
student learning goals, and in addition suggest the realignment of many general education and
core curricula across many higher education institutions to help better cater to learning styles and
study strategies of college students. Although some research has concentrated on more
economic concerns, such as retention of students who struggle academically (Kember & Leung,
2009), other studies have been focused on the development of instruments to delve more into the
construct of learning and study strategies (Kaplan, 2008).
The development of learning and studying as a construct has taken various paths. In the
early 1970s some of the earlier research was geared toward evaluating how students learn, and
within what context does that learning occur. Marton and Slj (1976a, 1976b), one of the first
studies to attempt to describe levels of processing and examine qualitative differences in how
students learn, has been the foundation for many studies attempting to develop instruments to
measure these constructs. Some of these researchers see learning to be more related to the
impetus or motivation of the learner, whether it be intrinsic or extrinsic (Grant & Dweck, 2001),
while others recognize the motivational aspects but combine it with cognitive factors (Lufi,
Parish-Plass, & Cohen,2003; Scraw, 1994; Tobias & Everson,1997) . Yet even combining these
aspects, there are differing opinions of how they are manifested. For example, Weinstein,
Palmer, and Schulte (2002) see the learning and study strategies to be a composite of constructs
related to skill, will, and self-regulation, while Biggs, Kember, and Leung (2001), conceptualize
learning as being composed of motive and strategy, which in either case is ether surface or deep
depending on the complexity of the particular attribute.
For this paper, however, our interest was not to debate the difference between theories, or
to champion one specific explanation, but to move forward in the evolution of the instrument
itself. That is, as the course of the history of researching learning and study strategies, numerous
studies have been conducted in order to determine a more perfect understanding of the
representation of these ideas from the data collected. In all cases research designs have been
constructed and analysis undertaken to better able to researcher to understand not only how the
construct manifests itself, but also how the instrument that assesses the construct operates. For
example, Entwistle and Ramsden (1982) looked at the data qualitatively to help first identify the
rudimentary questions to be utilized on a survey instrument, in the earliest development of the
Learning Process Questionnaire. Later, Kember et al. (2004) ran a variety of exploratory factor
analyses to determine the latent structure of the constructs, and later, confirmatory factor
analyses to see how well the instrument represents that construct.
More specifically, these instruments have been evaluated and validated. Utilizing factor
analytic procedures helped refine the constructs that emerged from the data collected. For
example, for the Learning Process Questionnaire (LPQ) the conceptualization of Deep Approach
and Surface Approach were realized. Biggs et al. (2001) went further to demonstrate that motives
and strategies are found within these learning approaches by the representation of the factor
structure. In the analysis of the LPQ and its subsequent revisions, Kember et al. (2004) were able
to clearly show the factors found in the instrument, and how those factors related to one another.
It is now imperative that we take yet another step forward, and that is to show how to
apply the instrument when describing samples of people. In short, the construct has been
confirmed, the instruments ability to detect that construct has been confirmed, and it is now time
to apply the instrument to help students. Factor analysis determines how the construct is
manifested within the instrument (exploratory) and whether the instrument consistently
demonstrates the existence of the factors for that construct (confirmatory), while PAMS looks at
the instrument in terms of what it tells us about people.
The current study was conducted with two major purposes. The first is to evaluate the
psychometric properties of the Revised Two-Factor Learning Process Questionnaire, a later
version of the LPQ (R-LPQ-2F; Kember et al., 2004). It is important to note that the sample used
to confirm the two-factor version of the questionnaire was comprised of secondary students in
Hong Kong, which required that the survey be translated into Chinese. As there may be some
subtle issues in translation from English to Chinese, as well as cross-cultural differences in
academic experiences, it is essential to determine if the Deep Approach and Surface Approach
structure can be replicated with a more rural, American population. In addition, in order to use
this survey for a sample of post-secondary students, it is important to replicate the findings with
that age group. This included evaluating the ecological validity of the instrument by looking at
relationships between the instrument factors and academic performance indicators such as GPA
and student retention. In addition, this study engaged in the next level of progression in the
analysis, by using Profile Analysis via Multidimensional Scaling (PAMS) in order to explore the
characteristics of respondents of the R-LPQ-2F.
As PAMS can be used to develop profiles for individual test takers, it can be used as a
means for improving the diagnostic functionality of the instrument. The two-factor structure,
identified by Kember et al. (2004), is indeed useful for describing the construct of the learning
process and identifying how the instrument operates in terms of Deep Approach and Surface
Approach. However, being able to use the instrument to interpret the data affords a richer
utilization of the instrument, both diagnostically and prescriptively. For example, which students
are more likely to be retained, maintain high GPAs, or finish within 4 years? Describing student
in terms of the dimensions of strategy and motive will allow us to identify those with him
motive low strategy or high strategy use, high motivation and be better prepared to associate
that with their academic successes or failures.
PAMS is a method that can be used to explore individual profiles of test takers for
assessments with multiple subtests. Profiles such as these can be instrumental in looking at
strengths and weaknesses of test takers within the subtests as well as affording the ability to
make an evaluation that can be used to remediate problem behaviors (Kim, Davison, & Frisby,
2007). Also, PAMS allows a researcher to identify students who develop in an idiographic
manner or are not explained by the model for whatever reason, whereas this capability is not
inherent in factor analysis.
Method
Participants
We distributed a web-based version of the R-LPQ-2F to students at a mid-sized public
university in the southeast United States. A total of 329 volunteers (227 women and 102 men)
responded. Ages ranged from 17 years old to 64 years old (M =27). Approximately 89.4% were
white and 31.3% were graduate students. Since the R-LPQ-2F and its psychometric properties
were originally produced by a large sample of secondary school students in Hong Kong, our
sample is important in determining whether those psychometric properties are consistent in a
university setting and across cultures.
Materials
The R-LPQ-2F consists of 22 items which can be summarized into two broad factors of
Deep Approach and Surface Approach. Within Deep Approach, there are subscales for Deep
Motive and Deep Strategy, and, within Surface Approach, there are subscales for Surface Motive
and Surface Strategy. Further, these subscales contain two subcomponents each. Items were
answered on a 5-point Likert-type scale ranging from 1 (this item is never or only rarely true of
me) to 5 (this item is always or almost always true of me). Standard R-LPQ-2F instructions were
presented to participants. Appendices A and B contain the questionnaire and scales, respectively
(Kember et al., 2004).
Results
Construct Validity
First, we conducted a confirmatory factor analysis using EQS (Version 6.1) to determine
if the full hierarchical model (see Figure 1) found by Kember et al. (2004) has good construct
validity with our sample. The model was fitted from the covariance matrix. The variances for the
Deep Approach and Surface Approach constructs were fixed to one, which is less restrictive than
constraining factor pattern coefficients (MacCallum, 1995). This presumes that the variables are
independently estimated for different groups and that the same model fits different groups
(Thompson, 2004). Hu and Bentler (1999) recommended that a Comparative Fit Index (CFI)
value greater than or equal to .96 in combination with a Standardized Root Mean Squared
Residual (SRMR) value less than .09 would minimize the errors of rejecting a model when it is
true and accepting a model when it is false. As a result, the CFI and SRMR were used to analyze
model fit.
The CFI fell below the cut-off (CFI =0.810), but the SRMR was right at the cut-off
(SRMR =0.092). These values were not as good as those found by Kember et al. (2004), where
the CFI was 0.967 and the SRMR was 0.036, but are still good. It is possible that our lower fit
indices are due to a smaller sample size (N =329 compared to N =801). It is important to note
that all of the paths were statistically significant at the =.05 level, indicating that all items and
lower-order factors have a significant and useful contribution to the model. Also, we found a
modest negative correlation between Deep Approach and Surface Approach whereas Kember et
al. (2004) found a modest positive correlation. Intuitively, it makes more sense for this
correlation to be negative because a positive correlation implies that a student who utilizes more
surface strategies and motives would be more likely to also be utilizing more deep strategies and
motives. The standardized solution for the hierarchical model is shown in Figure 1.
Next, multidimensional scaling (MDS), conducted in SAS (Version 9.1.3), was used to
further explore the latent variable model of the R-LPQ-2F. Dimensions in MDS are similar to
factors in factor analysis. In fact, if the data satisfy a simple structure factor model then those
factors can be seen within the MDS solution even though the MDS dimensions and factors do
not need to neatly map onto each other (Davison & Skay, 1991). The variable space in factor
analysis is representative of the underlying relationships of a set of attributes with respect to a
sample of individuals. MDS is different from factor analysis in that dimensions are interpreted as
characteristics of the objects to which the individual pays attention when he/she makes
judgments. In other words, MDS focuses on item differences more than individual differences.

MDS uses proximities, or measures of how similar or dissimilar different objects are, to
derive a geometric configuration of points representing the hidden structure of the data with
each point representing an object (Kruskal & Wish, 1978). These proximities were calculated
using the city-block, or Minkowski-1, metric from observed student responses of the R-LPQ-2F.
This metric is suitable for describing certain types of psychological data because it emphasizes
psychological judgmental processes (Weinberg, 1991).
These proximities are used to derive dimensions, or coordinate axes, representing
underlying characteristics of the objects under study. The goal is to obtain the lowest
dimensionality that best explains the underlying structure of the data (MacCallum, 1974).
Choosing the dimensionality is usually done through the use of goodness-of-fit statistics,
Krustals stress formula 1 (STRESS 1) being a common statistic. This stress value is the square
root of a normalized residual sum of squares, and indicates the level of fit for a specific
dimensionality (Kruskal & Wish, 1978). STRESS 1 values closer to 0 represent better
configurations. Stress values partially depend on the number of objects and the dimensionality,
so for the most accurate interpretation of stress the number of objects should be large compared
to the number of dimensions. One rule of thumb is that the number of objects should be more
than four times the number of dimensions (Kruskal & Wish, 1978). Since the R-LPQ-2F contains
22 items, the number of dimensions should be no more than five.


other and can be used to label the dimensions (Ding, 2001). The low end of Dimension 1 on this
graph is represented by items 4, 8, 12, 16, and 22. The items are all Surface Strategy items in the
hierarchical factor model. The high end of the dimension is represented by items 2, 10, 11, 14
and 15. Items 2, 10, and 14 are all Deep Strategy items in the hierarchical factor model and items
11 and 15 are Surface Motive items in the factor model. Based on the MDS solution, Dimension
1 appears to represent Strategy, with negative values representing Surface Strategy and positive
values representing Deep Strategy. The low end of Dimension 2 is represented by items 3, 7, 11
and 15, which are all Surface Motive items. The high end of Dimension 2 is represented by items
5, 13, and 17, which are Deep Motive items, and item 6, which is a Deep Strategy item. Based on
the MDS solution, Dimension 1 appears to represent Motive, with negative values representing
Surface Motive and positive values representing Deep Motive.
Table 1
Two-Dimensional MDS Solution
Item # Dimension 1 Dimension 2
1 0.088 0.736
2 1.281 0.266
3 0.733 -2.071
4 -1.230 -0.029
5 0.548 1.017
6 0.161 1.849
7 0.031 -2.154
8 -1.656 0.538
9 0.686 0.187
10 1.291 0.145
11 1.547 -1.300
12 -1.296 0.283
13 -0.685 1.075
14 1.015 0.085
15 1.217 -1.270
16 -1.572 0.306
17 0.027 1.279
18 -0.879 -0.759
19 0.207 -0.145
20 -0.611 -0.530
21 0.543 0.685
22 -1.444 -0.194
Figure 3. Configuration Obtained by Applying Multidimensional Scaling to the R-LPQ-2F

Figure 3. Configuration obtained by applying multidimensional scaling to the R-LPQ-2F. The dimensions were
scaled (i.e., normalized) to a root-mean-square value of 1, and the dimension coefficients were adjusted to
compensate. Dimension 1 appears to reflect Strategy and Dimension 2 appears to reflect Motive.

Although items do not need to neatly map onto the factors in the hierarchical factor
model (Davison & Skay, 1991), most of the R-LPQ-2F items in our study did. This is further
evidence of the construct validity of the latent model. Those that did not were items 6, 11, and
15. Item 6 is I like constructing theories to fit odd things together. The content of this question
can be interpreted as either a Strategy or a Motive, explaining a possible reason why it fell on the
Motive dimension instead of Strategy. Items 11 and 15 do fall under Surface Motive, but seem to
also fall in the MDS solution under Deep Strategy. These items have to do with doing well in
school in order to get a better job, so we do not know why they fall on both dimensions.
Although it is not the purpose of the current study to improve the R-LPQ-2F, it is possible that
removing items 6, 11, and 15 or changing the latent model to compensate for these items might
improve the model fit obtained from our factor analysis.
Profile Analysis
MDS cannot describe individual differences until the profile analysis is applied. PAMS
can be used to build profiles (i.e., interpret the MDS dimensions as latent profiles) through re-
parameterizing the linear latent variable model. These profiles can be used diagnostically
because PAMS can be used to determine group membership of people where the membership is
not known in advance of the analysis. A PAMS model calculates person parameters which are
essentially profile match indices that signify the direction and magnitude of the match between
the actual profile of the person and the dimension profile. Factor analysis emphasizes how tasks
vary in their sensitivity to person variates whereas MDS emphasizes how people vary in their
sensitivity to the task dimensions. A PAMS model therefore studies the latent person, that is
types among people as opposed to factors among variables (Ding, 2001), so that the latent
variables can be interpreted as profile patterns (Ding, 2006). This is essentially a variation of the
traditional factor model. PAMS studies clusters of people and each cluster is a hypothetical
prototypical person. Person parameters (i.e., dimension weights) are derived by linearly
regressing each persons observed scores onto the dimension scale values obtained from the
MDS analysis (Davison, Kim, & Ding, 2001).
A fit statistic is derived in this analysis (i.e., the R
2
from the regression) indicating the
proportion of variance in an individuals observed data that can be accounted for by the profiles
(Davison et al., 2001; Ding, 2006). This fit statistic is important to identify individuals who
develop in an idiographic manner or answered the instrument randomly and therefore do not fit
within the overall model. It can also represent the credibility of an individuals response as some
may over- or under-exaggerate responses or not take the assessment seriously. It can be used to
calculate the F-statistic and probability value used in regression to determine whether any of the
explanatory variables are statistically related to the dependent variable. This probability value
represents whether an individual fits within the overall model. PAMS goes beyond factor
analysis and allows a researcher to look at whether an individual can be accurately described in
the context of the latent model, such as is the case with unreliable respondents.
We computed person parameters and fit statistics for everyone in our sample. Table 2
contains the person parameters (i.e., dimension weights), level parameter (i.e., intercept from the
regression), fit and significance, cumulative GPA, and semester GPA at the university for eight
students from our sample (see Figure 4 for a plot of the person parameters). These eight students
are good illustrations of how PAMS can be used to assign profiles for diagnostic use. Student
198 is an example of one who has poor fit and poor probability of being accurately placed in a
profile. Since the R-LPQ-2F has been shown to have good reliability and construct validity, and
since the MDS solution maps fairly close to the hierarchical factor model, this student most
likely responded unreliably. It might be worthwhile to re-administer the R-LPQ-2F or collect
data from other sources for this student.
The other students significantly fit within the overall model and can be assigned to a
profile. Students 5 and 275 have positive weights for both dimensions and therefore have both
Deep Strategy and Deep Motive. Interestingly enough, these students also have the highest
cumulative GPA of those in the table. Student 275 also has the highest semester GPA (student 5
took only S/U courses so a GPA could not be calculated). Student 175 is the opposite, having
both negative weights for both dimensions and therefore have both Surface Strategy and Surface
Motive. This student has the lowest cumulative GPA of those in the table and one of the lowest
semester GPAs. Students 23, 81, and 99 have Deep Strategy and Surface Motive and student 58
has Surface Strategy and Deep Motive.
Table 2
PAMS Person Parameters for Eight Students: R-LPQ-2F Profiles
Student
Dimension 1 Wt.
(Strategy)
Dimension 2 Wt.
(Motive) Level Par. Fit (R
2
) p-value
Cumulative
GPA
Semester
GPA
5 1.036 0.267 3.136 .536 .004 4.000 S
23
0.358 -0.729 2.818 .503 .006 3.289 2.750
58 -0.854 0.145 1.864 .432 .015 2.726 2.879
81
0.543 -0.444 2.000 .358 .033 3.271 2.685
99 0.734 -0.534 3.227 .414 .018 3.750 4.000
175
-0.116 -0.748 3.000 .456 .011 1.629 2.018
198 0.090 -0.031 3.136 .004 .840 2.119 1.686
275
1.358 0.432 2.909 .905 <.001 4.000 4.000


Figure 4. Profile Plot for Eight Students

Figure 4. Configuration obtained by plotting the dimension weights for each respondent.

PAMS describes people in terms of continuous person profile indices that specify to what
extent people are mixtures of the various types whereas factor analysis describes people in terms
of discrete groupings. PAMS profile information can be utilized clinically to make differential
diagnoses and to design appropriate interventions based on an individuals profile pattern. To
illustrate this we have also computed the factor scores for the same eight students (see Table 3).
Since factor analysis describes people in terms of discrete groupings, the dimensions of Motive
and Strategy are represented as the factors Surface Motive, Deep Motive, Surface Strategy, and
Deep Strategy. Students 81, 99, 175, and 198 have almost identical factor scores for Surface
Motive and Deep Motive and students 81 and 99 also have almost identical factor scores for
Surface Strategy and Deep Strategy. These students cannot be profiled accurately using factor
scores. The PAMS model can classify each one though, as seen above in Table 2.
Table 3
Factor Scores for Eight Students
Student Surface Motive Deep Motive Surface Strategy Deep Strategy
5 13 26 12 18
23 18 13 17 14
58 4 8 22 7
81 13 13 9 9
99 20 19 16 16
175 17 17 22 10
198 14 13 23 19
275 13 25 8 18

Ecological Validity
We have discussed how PAMS can be used to assign individuals to profiles based on a
latent model. The next step is to determine whether these profiles have a relationship with
behavioral measures such as semester GPA and whether or not the student is retained the
following semester. To do this we conducted a multiple regression, regressing each students
semester GPA onto their level and dimension parameters. For retention we conducted a logistic
regression. We included gender (i.e., male or female), race (i.e., white or non-white), credit type
(i.e., resident credit or distance education), and class level (i.e., graduate or undergraduate) as
controls in the models. Of the 329 student volunteers, 29 had missing values for one or more of
the variables, leaving us with 300 for the regressions. The results for the semester GPA model
show that Strategy has a significant positive relationship with academic performance (see Table
4). This intuitively makes sense since surface strategies revolve around learning for
examinations, memorization, and devoting as little time to learning as possible and deep
strategies revolve around learning to understand the material and relating ideas. Motive was also
significant, but had a negative relationship with academic performance. This makes sense since
Deep Motive involves having an intrinsic interest and commitment whereas Surface Motive has
to do more with doing well in school and not wanting to get poor marks. These results indicate
that Strategy and Motive are significantly related to academic performance.
Since PAMS allows a researcher to identify students who develop in an idiographic
manner and are not explained by the model through the calculation of a fit statistic, we decided
to run the regression again, only this time removing students with a low fit. This capability is not
inherent in factor scores and is an important component of a PAMS analysis. Students were
removed based on the probability of their fit (using =.05), such as student 198. Doing this
allows us to look more closely at the relationship of each profile to semester GPA, which is
important because only those who fit within the model can be profiled and helped with the use of
an intervention. After removing those with poor fit we were left with 244 for the regression
analysis. Overall the results are exactly the same as when all students were left in (see Table 4).
This second regression is noteworthy, though, because it shows that the PAMS method can be
applied from a diagnostician viewpoint in order to make more accurate decisions and determine
which students we may need additional data for.
Table 4
Regression Parameters for Regressing Semester GPA onto PAMS Dimensions
All Observations Excluding Observations
Variable
Parameter
Estimate
Standardized
Estimate
Parameter
Estimate
Standardized
Estimate
Intercept 3.1594*** 0 2.9755*** 0
Gender (1 =Female, 0 =Male) -0.0609 -0.0310 -0.1903 -0.0983
Race (1 =White, 0 =Non-white) 0.3263* 0.1097 0.2797 0.0970
Credit Type (1 =Resident Credit, 0 =Distance Education) -0.1197 -0.0443 -0.1320 -0.0526
Class Level (1 =Graduate, 0 =Undergraduate) 0.4092*** 0.1987 0.3873*** 0.1992
Dimension 1 Wt. (Strategy) 0.4963*** 0.2102 0.6677*** 0.2538
Dimension 2 Wt. (Motive) -0.3242** -0.1485 -0.3774** -0.1866
PAMS Level -0.2144 -0.0849 -0.1578 -0.0644
Note. The model using all observations has an adjusted R
2
=.0749 and the model excluding observations has an
adjusted R
2
=.0825.
* p <.10. ** p <.05. *** p <.01.
As with most diagnostic tests and self-reports, some students are going to give invalid
results. The ability to eliminate unreliable respondents is not a new idea, as it is common in such
instruments as the MMPI. Hahn (2005) notes that when using a clients self-report, it is crucial
to determine the credibility of the individuals performancefor example, whether he or she has
cooperated fully with the evaluation (p. 65). The advantage with PAMS is that this can be done
without the need for an extra validity scale, as the fit statistic in PAMS is a measure of how well
an individuals item responses fit within the model. This ability allows a clinician to apply the
appropriate assessment and offer more relevant assistance to students.
We also ran a multiple regression, regressing each students semester GPA on the factor
scores, in order to show a comparison between PAMS and factor analysis. In this model none of
the subscales were significant, showing no evidence of ecological validity (see Table 5). Also,
whereas the PAMS model has only two dimensions as independent variables, the factor analysis
model has four subscales, thus increasing the complexity of drawing inference. It is possible that
the complexity of interpreting each side of each dimension as a discrete factor instead of
continuous dimensions is why this model has no significance. It makes sense that Deep Strategy
is related to Surface Strategy and Deep Motive is related to Surface Motive, and that looking at
each separately without including that relationship could affect the significance of each in the
model.
Table 5
Regression Parameters for Regressing Semester GPA onto Factor Scores
Variable Parameter Estimate Standardized Estimate
Intercept 3.0935*** 0
Gender (1 =Female, 0 =Male) 0.0025 0.0013
Race (1 =White, 0 =Non-white) 0.3137* 0.1055
Credit Type (1 =Resident Credit, 0 =Distance Education) -0.1216 -0.0450
Class Level (1 =Graduate, 0 =Undergraduate) 0.4267*** 0.2072
Deep Motive Subscale -0.0168 -0.0825
Deep Strategy Subscale 0.0139 0.0454
Surface Motive Subscale 0.0162 0.0584
Surface Strategy Subscale -0.0211 -0.0951
Note. This model has an adjusted R
2
=.0518.
* p <.10. ** p <.05. *** p <.01.
Next we conducted logistic regressions, regressing retention on the PAMS dimensions
and regressing retention on the factor scores. Retention was coded as 1 for students who enrolled
in the next semester and 0 for those who did not. Those students who graduated prior to the next
semester were removed prior to the analysis. Like before, we ran this regression including all
observations and again excluding those with low fit (see Table 6). The model with all
observations did not yield a significant relationship for Strategy and Motive. Strategy did have a
mildly significant negative relationship with retention in the model that removed those with poor
fit though. This indicates that those who use more surface strategies are for some reason more
likely to enroll again the next semester. This makes sense since Deep Strategy deals with trying
to understand the material and relate it with other subjects and Surface Strategy deals with
studying for examinations and to get by in school. Those utilizing deep strategies might be
transferring out or may be leaving because they are not concerned with learning solely to do well
in school. Finally, we conducted this logistic regression using the factor subscales instead of the
PAMS dimensions (see Table 7). We obtained similar results in that Surface Strategy has a
significant positive relationship with the probability of whether a student will return next
semester.
Table 6
Logistic Regression Parameters for Regressing Retention onto PAMS Dimensions
All Observations (N =277)
Removing Observations
(N =227)
Variable Estimate Odds Ratio Estimate Odds Ratio
Intercept -0.5222 0.593 0.5496 1.733
Gender (1 =Female, 0 =Male) -0.1019 0.903 -0.0820 0.921
Race (1 =White, 0 =Non-white) 0.4686* 1.598 0.5104* 1.666
Credit Type (1 =Resident Credit, 0 =Distance Education) 0.5516*** 1.736 0.4581** 1.581
Class Level (1 =Graduate, 0 =Undergraduate) -0.1018 0.903 -0.1391 0.870
Dimension 1 Wt. (Strategy) -0.9985 0.368 -1.5736* 0.207
Dimension 2 Wt. (Motive) 0.4705 1.601 0.6218 1.862
PAMS Level 0.8354 2.306 0.6734 1.961
* p <.10. ** p <.05. *** p <.01.
Table 7
Logistic Regression Parameters for Regressing Retention onto Factor Scores (N = 277)
Variable Estimate Odds Ratio
Intercept -0.7426 0.476
Gender (1 =Female, 0 =Male) -0.0871 0.917
Race (1 =White, 0 =Non-white) 0.4768** 1.611
Credit Type (1 =Resident Credit, 0 =Distance Education) 0.5414*** 1.718
Class Level (1 =Graduate, 0 =Undergraduate) -0.1148 0.892
Deep Motive Subscale 0.0357 1.036
Deep Strategy Subscale 0.0305 1.031
Surface Motive Subscale -0.0705 0.932
Surface Strategy Subscale 0.1232** 1.131
* p <.10. ** p <.05. *** p <.01.
Discussion
The LPQ and its subsequent revisions have been part of the development of a conceptual
understanding of how students learn, their motivation to engage in particular tasks, and the
strategies they utilize to reach their academic goals. It has been generally accepted throughout
the literature that these approaches to learning can be reduced to Deep Approach and Surface
Approach, which encompass the motives and strategies which follow suit. Even with a smaller
sample, the factor analysis accomplished in this study clearly confirmed the findings seen in
previous studies as the subscales of Deep Motive, Surface Motive, Deep Strategy, and Surface
Strategy emerged in the R-LPQ-2F.
Both factor analysis and PAMS techniques have advantages and disadvantages, and
depending on the instrument and the latent construct, one may be more appropriate than the other
for determining how to best use the instrument practically. In the case of the R-LPQ-2F, PAMS
definitely seems to provide a better understanding of the application of the constructs of
motive and strategy to students in higher education.
Diagnostically, these factors might be utilized to represent personality characteristics, but
they can be cumbersome and difficult to interpret. That is, in order to use this instrument as a
means for remediation, student support, or simply reflection, information concerning the student
needs to be easily attainable and useable. For example, if a student completes the assessment and
indicates utilization of both deep and surface learning approaches, whether he is more surface
than deep or more deep than surface will have bearing how that student might alter his learning
approach. The PAMS model does indeed help look at the individual test taker and a profile that
represents characteristics of that individual. That is, the PAMS model allows for a person-level
interpretation of the analysis.
PAMS basically allows for a conversion of the data into profiles so that overarching
behaviors can be more easily categorized. The analysis forms a representation of how these
behaviors play out in relationship to one another. A profile is basically a persons performance
on a set of scores (Ding, 2001). PAMS extends MDS by interpreting the MDS dimensions as
latent profiles (i.e., each dimension represents a group of individuals with similar characteristics;
Ding, 2001, 2006; Kim et al., 2007). PAMS represents what profiles of variables exist in the
population and how individuals differ in those profiles (Ding, 2006). In this particular case, the
dimensions Motive and Strategy and level of processing (deep and surface) allow for a learner
personality type to be developed. More specifically, the dimensions show how a person
functions in the academic environment which can then be tied to other variables.
In this study, the dimension of Strategy can clearly be seen as a continuum from surface
to deep. In addition, student profiles which fall along that dimension have been shown to relate
significantly to GPA. In short, the capacity to utilize this instrument diagnostically has become
readily apparent. A student profile is directly linked to the desired or undesired behaviors which
now can be addressed, altered, and remediated. Although it seems intuitively obvious for
students to use deep strategies because they are more efficacious, some students are unable to
assess their own strategy use, and they continue to use methods that do not work. These students
may be unable to see that their techniques are unsuccessful, or they may simply have a limited
repertoire of study skills from which to pull. Helping students become more aware of the
strategies they use, helping them monitor and regulate these strategies, and helping them choose
between more successful and less successful strategies is essential. Counselors and advisors
would be able to use these student profiles to determine whether it is indeed the strategies
employed by the student that are affecting the academic outcomes. Also inherent in PAMS is the
ability to identify individuals who do not fit within the model or who respond unreliably which
allows one to identify those individuals with which further data, possibly from other outside
sources, are needed. This might indicate the need for more diagnostic information or further
testing in other areas and allows for the maximization of resources for helping individual
students.
The dimension of Motive can also clearly be seen as a continuum from surface to deep.
Student profiles which fall along that dimension have been shown to relate negatively to GPA.
This makes sense since Deep Motive involves having an intrinsic interest and commitment
whereas Surface Motive has to do more with doing well in school and not wanting to get poor
marks. Unfortunately, this is less than ideal. Our mission is to develop students into lifelong
learners and yet get good grades. We may not want to intervene with students based on Motive,
but instead intervene with our curriculums to change the nature of Motives relationship with
academic performance.
As the current research offers a new perspective on the learning approach dimensions,
allows the development of profiles, and affords an opportunity to diagnostically assess students
concerning their strategies and motives, there is still the question of what other academic and
personal factors are related to both dimensions. We have clearly identified the continuums of
Strategy and Motive, surface to deep, and have demonstrated those characteristics to be strongly
related to semester GPA and mildly related to retention. For future research it might be beneficial
to collect drop/withdraw/failure statistics, learning outcomes, variables related to what the
students do after they graduate, and how possible interventions and curriculums improve
students strategies and motives.
References
Biggs, J ., Kember, D., & Leung, D. Y. P. (2001). The revised two-factor Study Process
Questionnaire: R-SPQ- 2F. British Journal of Educational Psychology, 71 (1), 133.
Davison, M. L., Kim, S., & Ding, S. (2001, April 10-14). Profile analysis via multidimensional
scaling (PAMS): Exploring the predominant profile patterns in data. Paper presented at
the Annual Meeting of the American Educational Research Association, Seattle,
Washington.
Davison, M. L., & Skay, C. L. (1991). Multidimensional scaling and factor models of test and
item responses. Psychological Bulletin, 110 (3), 551-556.
Ding, C. S. (2001). Profile analysis: Multidimensional scaling approach. Practical Assessment,
Research & Evaluation, 7 (16). Retrieved March 26, 2010 from
http://PAREonline.net/getvn.asp?v=7&n=16
Ding, C. S. (2006). Multidimensional scaling modeling approach to latent profile analysis in
psychological research. International Journal of Psychology, 41 (3), 226-238.
Entwistle, N. J ., & Ramsden, P. (1982). Understanding student learning. New York, NY:
Nichols Publishing Company.
EQS (Version 6.1) [Computer software]. Encino, CA: Multivariate Software. Available from
http://www.mvsoft.com/index.htm.
Everson, H. T., Weinstein, C. E., & Laitusis, V. (2000, April 24-28). Strategic learning abilities
as a predictor of academic achievement. Paper presented at the Annual Meeting of the
American Educational Research Association, New Orleans, Louisiana.
Grant, H., & Dweck, C. S. (2001). Cross-cultural response to failure: Considering outcome
attributions with different goals. In F. Salili, C.-Y. Chiu, & Y.-Y. Hong (Eds.), Student
motivation: The culture and context of learning (pp. 203-219). Dordrecht, Netherlands:
Kluwer Academic Publishers.
Hahn, J . (2005). Faking bad and faking good by college students on the Korean MMPI-2.
Journal of Personality Assessment, 85 (1), 65-73.
Hu, L., & Bentler, P. M. (1999). Cutoff criteria for fit indexes in covariance structure analysis:
Conventional criteria versus new alternatives. Structural Equation Modeling, 6 (1), 1-55.
Kaplan, A. (2008). Clarifying metacognition, self-regulation, and self-regulated learning: Whats
the purpose? Educational Psychology Review, 20 (4), 477-484.
Kember, D., Biggs, J ., & Leung, D. Y. P. (2004). Examining the multidimensionality of
approaches to learning through the development of a revised version of the Learning
Process Questionnaire. British Journal of Educational Psychology, 74 (2), 261-280.
Kember, D., & Leung, D. Y. P. (2009). Development of a questionnaire for assessing students'
perceptions of the teaching and learning environment and its use in quality assurance.
Learning Environments Research, 12 (1), 15-29.
Kim, S., Davison, M. L., & Frisby, C. L. (2007). Confirmatory factor analysis and profile
analysis via multidimensional scaling. Multivariate Behavioral Research, 42 (1), 1-32.
Kruskal, J . B., & Wish, M. (1978). Multidimensional scaling. Beverly Hills, CA: Sage
Publications, Inc.
Lufi, D., Parish-Plass, J ., & Cohen, A. (2003). Persistence in higher education and its
relationship to other personality variables. College Student Journal, 37 (1), 50-60.
MacCallum, R. C. (1974). Relations between factor analysis and multidimensional scaling.
Psychological Bulletin, 81 (8), 505-516.
MacCallum, R. C. (1995). Model specification: Procedures, strategies, and related issues. In R.
H. Hoyle (Ed.), Structural equation modeling (pp. 16-36). Thousand Oaks, CA: Sage
Publications, Inc.
Manly, B. F. J . (2004). Multivariate statistical methods: A primer (3 ed.). New York, NY:
Chapman & Hall/CRC.
Marton, F., & Slj, R. (1976a). On qualitative differences in learning: IOutcome and process.
British Journal of Educational Psychology, 46 (1), 4-11.
Marton, F., & Slj, R. (1976b). On qualitative differences in learning: IIOutcome as a
function of the learners conception of the task. British Journal of Educational
Psychology, 46 (2), 115-127.
SAS (Version 9.1.3) [Computer software]. Cary, NC: SAS Institute Inc.
Scraw, G. (1994). The effect of metacognitive knowledge on local and global monitoring.
Contemporary Educational Psychology, 19 (2), 143-154.
Thompson, B. (2004). Exploratory and confirmatory factor analysis: Understanding concepts
and applications. Washington, DC: American Psychological Association.
Tobias, S., & Everson, H. T. (1997). Studying the relationship between affective and
metacogntive variables. Anxiety, Stress & Coping: An International Journal, 10 (1), 59-
81.
Weinberg, S. L. (1991). An introduction to multidimensional scaling. Measurement and
Evaluation in Counseling and Development, 24 (1), 12-36.
Weinstein, C. E., Palmer, D. R., & Schulte, A. C. (2002). Learning and Study Strategies
Inventory (2 ed.). Clearwater, FL: H & H Publishing Company, Inc.

Appendix A
Revised Learning Process Questionnaire (R-LPQ-2F)
J ohn Biggs and David Kember
This questionnaire has a number of questions about your attitudes towards your studies and your
usual way of studying.

There is no right way of studying. It depends on what suits your own style and the course you are
studying. It is accordingly important that you answer each question as honestly as you can. If you
think your answer to a question would depend on the subject being studied, give the answer that
would apply to the subject(s) most important to you.

Please fill in the appropriate circle alongside the question number on the General Purpose
Survey/Answer Sheet. The letters alongside each number stand for the following response.

A this item is never or only rarely true of me
B this item is sometimes true of me
C this item is true of me about half the time
D this item is frequently true of me
E this item is always or almost always true of me

Please choose the one most appropriate response to each question. Fill the oval on the Answer
Sheet that best fits your immediate reaction. Do not spend a long time on each item: your first
reaction is probably the best one. Please answer each item.

Do not worry about projecting a good image. Your answers are CONFIDENTIAL.

Thank you for your cooperation.

(1) I find that at times studying makes me feel really happy and satisfied.
(2) I try to relate what I have learned in one subject to what I learn in other subjects.
(3) I am discouraged by a poor mark on a test and worry about how I will do on the next test.
(4) I see no point in learning material which is not likely to be in the examination.
(5) I feel that nearly any topic can be highly interesting once I get into it.
(6) I like constructing theories to fit odd things together.
(7) Even when I have studied hard for a test, I worry that I may not be able to do well in it.
(8) As long as I feel I am doing enough to pass, I devote as little time to studying as I can. There
are many more interesting things to do.
(9) I work hard at my studies because I find the material interesting.
(10) I try to relate new material, as I am reading it, to what I already know on that topic.
(11) Whether I like it or not, I can see that doing well in school is a good way to get a well-paid
job.
(12) I generally restrict my study to what is specifically set as I think it is unnecessary to do
anything extra.
(13) I spend a lot of my free time finding out more about interesting topics which have been
discussed in different classes.
(14) When I read a textbook, I try to understand what the author means.
(15) I intend to get my A Levels [or equivalent qualification] because I feel that I will then be
able to get a better job.
(16) I find it is not helpful to study topics in depth. You dont really need to know much in order
to get by in most topics.
(17) I come to most classes with questions in mind that I want answering.
(18) I learn some things by rote, going over and over them until I know them by heart even if I
do not understand them.
(19) I find I am continually going over my school work in my mind at times like when I am on
the bus, walking, or lying in bed, and so on.
(20) I find the best way to pass examinations is to try to remember answers to likely questions.
(21) I like to do enough work on a topic so that I can form my own conclusions before I am
satisfied.
(22) I find I can get by in most assessment by memorising key sections rather than trying to
understand them.

Appendix B

Scales in the Revised Learning Process Questionnaire (R-LPQ-2F)

J ohn Biggs and David Kember
The number in parentheses is the item number in the questionnaire.

Deep approach

Deep motive

Intrinsic interest
I find that at times studying makes me feel really happy and satisfied. (1)
I feel that nearly any topic can be highly interesting once I get into it. (5)
I work hard at my studies because I find the material interesting. (9)

Commitment to work
I spend a lot of my free time finding out more about interesting topics which have been discussed
in different classes. (13)
I come to most classes with questions in mind that I want answering. (17)
I find I am continually going over my school work in my mind at times like when I am on the
bus, walking, or lying in bed, and so on. (19)
I like to do enough work on a topic so that I can form my own conclusions before I am satisfied.
(21)

Deep strategy

Relating ideas
I try to relate what I have learned in one subject to what I learn in other subjects. (2)
I like constructing theories to fit odd things together. (6)

Understanding
I try to relate new material, as I am reading it, to what I already know on that topic. (10)
When I read a textbook, I try to understand what the author means. (14)

Surface approach

Surface motive

Fear of failure
I am discouraged by a poor mark on a test and worry about how I will do on the next test. (3)
Even when I have studied hard for a test, I worry that I may not be able to do well in it. (7)


Aim for qualification
Whether I like it or not, I can see that doing well in school is a good way to get a well-paid job.
(11)
I intend to get my A Levels because I feel that I will then be able to get a better job. (15)

Surface strategy

Minimizing scope of study
I see no point in learning material which is not likely to be in the examination. (4)
As long as I feel I am doing enough to pass, I devote as little time to studying as I can. There are
many more interesting things to do. (8)
I generally restrict my study to what is specifically set as I think it is unnecessary to do anything
extra. (12)
I find it is not helpful to study topics in depth. You dont really need to know much in order to
get by in most topics. (16)

Memorisation
I learn some things by rote, going over and over them until I know them by heart. (18)
I find the best way to pass examinations is to try to remember answers to likely questions. (20)
I find I can get by in most assessment by memorising key sections rather than trying to
understand them. (22)

To calculate scores on the scales use the following response scores.

A =1, B =2, C =3, D =4, E =5

Scores for the two main scales, deep approach (DA) and surface approach (SA), can then be
calculated by adding the following item scores:

DA =1 +2 +5 +6 +9 +10 +13 +14 +17 +19 +21
SA =3 +4 +7 +8 +11 +12 +15 +16 +18 +20 +22

Each contains identifiable strategy (DS and SS) and motive (DM and SM) subscales. The
subscale and scale scores can be calculated by adding item scores as follows:

DM =1 +5 +9 +13 +17 +19 +21
DS =2 +6 +10 +14
SM =3 +7 +11 +15
SS =4 +8 +12 +16 +18 +20 +22