Educational Psychology

Vol. 24, No. 5, October 2004

A Note on the Problem Solving Style

Questionnaire: An alternative to
Kolb's Learning Style Inventory?
Angus Duff*

Given the psychometric limitations of existing measures of Kolb's experiential learning model
(ELM), two new scales of learning styles have been developed. The validity of these scales has been
supported in samples of undergraduate and MBA students in the USA. This paper provides
evidence of some psychometric properties of scores yielded by these scales using samples of
undergraduate students in the UK. Only limited support is found for the internal consistency
reliability and construct validity of scores produced by the scales. However, an item attrition
exercise identi®es a two-factor solution providing an acceptable ®t to the data. The scales are
reported as being positively correlated with academic performance and prior academic
achievement. Despite the mixed evidence, we suggest further development of the scales is
warranted to create a psychometrically sound measure of the ELM.

Introduction
Over the past three decades, Kolb's (1985) experiential learning model (ELM) has been
one of the most in¯uential models of learning, particularly in the ®elds of management
development (Reynolds, 1997) andbusiness education (Duff & Duffy, 2002). The ELM
has been widely used by educators for a variety of different purposes. These include
classroom use with higher education students, (Armstrong, 2000; Healey & Jenkins,
2000; Philbin, Meier, Huffman, & Boverie, 1995; Sims & Lindholm, 1993), in
professional education (Rakoczy & Money, 1995; White, 1992), to in¯uence educa-
tional design (Green, Snell, & Parimanath, 1990; Henry, 1989; Wynd & Bozman,
1996), and to identify the relationship between learning styles and educational or
occupational choice (Atkinson, Murrell, & Winters, 1990; Green & Parker, 1989).
Unfortunately, extant measures of the ELM, the Learning Style Inventory (LSI;
Kolb, 1976), the Learning Style Inventory II (LSI-II; Kolb, 1985), and the Learning
Styles Questionnaire (LSQ; Honey & Mumford, 1992), have been shown to possess

*Centre for Research into Learning, University of Paisley, Ayr Campus, Ayr KA8 OSR, Scotland,
UK. Email: [email protected]
ISSN 0144±3410(print)/ISSN 1469±046X (online)/04/050699-11
ã 2004 Taylor & Francis Ltd
DOI: 10.1080/0144341042000262999
700 A. Duff

serious psychometric limitations. Motivated by the poor measurement qualities of the

LSI (see Freedman & Stumpf, 1978, 1980; Geller, 1979; Newstead, 1992; Stout &
Ruble, 1991a; 1991b), LSI-II (see Geiger, Boyle, & Pinto, 1992, 1993; Loo, 1999;
Ruble & Stout, 1993; Willcoxon & Prosser, 1996; Yahra, 1998), and LSQ (see
Allinson & Hayes, 1988 1990; De Ciantis & Kirton, 1996; Duff, 1997, 2001; Duff &
Duffy, 2002; Sims, Veres, & Shake, 1989), Romero, Tepper and Tetrault (1992)
developed a new instrument, the Problem Solving Style Questionnaire (PSSQ). The
PSSQ attempts to capture the content domain of the four-stage, problem solving
hypothesised process associated with the ELM. In the learning process (cycle),
learners ®rst acquire information by concrete experience. Second, a stage of re¯ective
observation on the experience occurs whereby the learner organises the experiential
data from a number of perspectives. Third, a stage of abstract conceptualisation
occurs, whereby the learner develops generalisations from which to assist them
integrate their observations into sound theories or principles. Fourth, through active
conceptualisation, learners use these generalisations as guides to action in new and
more complex situations. The learning cycle explains differences in learning style in
terms of relative abilities (that is, level) at various stages of the learning cycle. An ideal
learner will possess maximum abilities for all four stages.
Since Kolb ®rst developed his concept of a four-stage process, the process has been
developed further as two orthogonal dimensions of learning derived from the LSI
(Kolb, 1976) and its later revision, the LSI-II (Kolb, 1985). These two dimensions
are labelled prehension, that is, grasping information from experience (concrete
experience±abstract conceptualisation); and transformation, that is, the processing of
information grasped (re¯ective observation±active experimentation). This concept
explains differences in terms of two bipolar styles (that is, the manner) by which each
stage in the learning process is approached and operationalised. These bipolar
dimensions are sometimes described as learning types. The four stages and two types
are shown in Figure 1.
The PSSQ is reported as yielding scores with satisfactory internal consistency
reliability, test-retest stability, factorial validity (Romero et al., 1992), and
discriminant and convergent validity (Tepper, Tetrault, Braun, & Romero, 1993).
On the basis of Romero et al.'s (1992) and Tepper et al.'s (1993) analyses, the
psychometric properties of PSSQ scores appear stronger than either the LSI, LSI-II,
or LSQ. Despite the encouraging evidence relating to the PSSQ using samples of
undergraduate students in the USA reported by Romero et al. (1992) and Tepper et
al. (1993), further testing of the PSSQ in another cultural context is warranted
(Tepper et al., 1993). This research extends the work of Romero et al. (1992) and
Tepper et al. (1993) by focusing on the internal consistency reliability and validity of
the PSSQ when administered to undergraduate students in the UK.

Method
Sample and Procedure
The sample consisted of 200 undergraduate students enrolled in two faculties: health
 Problem Solving Style Questionnaire 701

Figure 1. Kolb's ELM, the relationship between four process stages, two orthogonal bipolar
dimensions, and four typesKolb's bipolar dimensions are indicated by the two orthogonal broken
lines labelled ``Prehension'' and ``Transformation''. Honey and Mumford's (1992) LSQ labels for
each of the four learning styles are shown in parentheses.

and social studies (n=128) and business (n=72) at a medium-sized university in

Scotland. The distribution of the sample across undergraduate degree programs was:
accounting (n=62); business (n=12); psychology (n=45); and social studies (n=80).
Of these, 134 respondents were female, 61 were male, and ®ve declined to identify
their gender. The age of the sample ranged from 17 to 52 years (mean=23.14 years).
All respondents completed the PSSQ. The instrument was administered to all
participants in a classroom environment at the start of the second semester of the
academic year. Participation was voluntary. Students were enrolled on programs
requiring attendance on campus on a full-time or part-time basis. As can be seen from
the above description, the sample was demographically diverse enough to generalise
to other Western higher educational settings.
702 A. Duff

Measures
Problem Solving Style Questionnaire. Romero et al.'s (1992) 14-item PSSQ assesses
two learning types: active experimentation±re¯ective observation (AE±RO) and
abstract conceptualisation±concrete experience (AC±CE). Each PSSQ item
consists of two self-descriptive statement anchors, which correspond to the
complementary modes of the dimension being measured, and a six-point response
scale. Two items in each scale are re¯ected, to reduce contamination acquiescence.
The items are shown in the Appendix.

Academic performance. Academic performance was measured as a composite

measure, a grade-point average (GPA) achieved over the course of an academic
year in eight taught modules. GPA scores ranged from 11% to 70%,1
mean=51.1%, SD=10.6%.

Prior academic achievement. Prior academic achievement was measured by a points

score in the Scottish Higher examinations for the best ®ve passes. Higher
examinations are taken by school students in Scotland as a form of school leaving
examination; students will typically undertake ®ve examinations. The Higher points
variable is scored as follows: grade A 6 points, grade B 4 points, and grade C 2
points. The maximum Higher points score is therefore 30 points. Higher points
ranged from 2 to 26, mean=10.9, SD=4.9.

Statistical Analyses
First, the descriptive statistics for the study variables were computed, along with
internal reliability estimates (alpha coef®cients). Second, con®rmatory factor analyses
(CFAs) were conducted with the SPSS version of Amos v4.0 (Arbuckle, 1999). The
sample size (n=200) is satisfactory for the purposes of this study re¯ecting the relative
complexity of the models being examined. Notably, power estimates and precision
increase monotonically with increasing sample size (n) and the number of items per
factor (p/f ratio; Marsh & Hau, 1999) or degrees of freedom (df; MacCallum, Browne,
& Sugawara, 1996). The PSSQ is predicated on a p/f ratio of 7. Marsh and Hau
(1999) report a sample size of only 50 is adequate when p/f is as low as 6.
In evaluating goodness-of-®t, we used a two-index presentation strategy outlined by
Hu and Bentler (1999). This includes the maximum likelihood-based standardised
root mean squared residual (SRMR), supplemented with the Tucker-Lewis Index
(TLI). Hu and Bentler (1999) indicate a TLI of around .95 or greater, along with an
SRMR of around .08 or lower, is indicative of good model ®t to the data.
Third, the respondents were classi®ed according to their program of study ±
accounting, business, psychology, or social studies ± and the group means for each
LSI dimension were plotted on Kolb's learning style grid (Kolb, Rubin, & McIntyre,
1984 p. 33). In the ®nal set of analyses, scores on the two dimensions of the
 Problem Solving Style Questionnaire 703

Table 1. Variable means, standard deviations, intercorrelations, and alpha coef®cients

Variable M SD 1 2 3

1. AE/RO 24.25 14.77 (.487)1

2. AC/CE 23.07 14.80 1.462** (515)
3. GPA 51.16 10.56 .153* .165*
4. PPA 10.93 14.90 .114* 1.287** .344**

High scores on the AE/RO dimension re¯ect an emphasis on active experimentation. High scores
on the AC/CE dimension re¯ect an emphasis on abstract conceptualisation. Scores (in brackets) on
the diagonal are alpha internal-consistency reliability coef®cients. GPA=grade-point average, a
measure of academic achievement. PPA=prior academic achievement.

instrument were correlated with prior academic achievement (as assessed by school
results) and academic performance (measured by a grade-point average, GPA).

Results
Descriptive Statistics
Table 1 shows the means, standard deviations, intercorrelations, and alpha internal-
consistency coef®cients for the PSSQ scales. Means, standard deviations, and
intercorrelations are reported for the academic performance and prior academic
achievement variables. The alpha coef®cients for the concreteness/abstractness and
re¯ection/action scales were .487 and .515 respectively, substantially below the
commonly accepted cut-off value of .7 for measures suitable for use in applied settings
using a wide range of subjects (Nunnally & Bernstein, 1994). An item attrition
exercise raised the concreteness/abstractness scale to .657 (including items 5, 7, 9,
and 13), and the re¯ection/action scale to .612 (including items 2, 4, 8, and 12).
These results are in contrast to Romero et al. (1992); alpha ranging from .73 to .86)
and Tepper et al. (1993); alpha ranging from .74 to .76) who reported satisfactory
internal consistency reliability for scores produced by the two scales.
The correlation between the two PSSQ scales (r=.462) was higher than that of
Tepper et al. (1993) who found r=.22. The correlation between concreteness/
abstractness and re¯ection/action is unexpected as theoretically these two scales are
orthogonal, as indicated in Figure 1.

Con®rmatory Factor Analysis Results

The results of the con®rmatory factor analyses are shown in Table 2, which reveals
that the hypothesised two-factor model is preferable to the rival one-factor model.
However, the goodness-of-®t indices difference tests ®nd little support for the
superiority of the two-factor model over the one-factor model (c2=11.659, df=1, c2/
df=.112, TLI=.002, SRMR=.0009).
704 A. Duff

Table 2. Comparison of con®rmatory factor analysis results

Model c2 df c2/df TLI SRMR

One-factor 237.891 77 3.089 .964 .0971

Two-factor 226.232 76 2.977 .966 .0962
Two-factor (revised) 27.802 13 2.139 .990 .0524

Table 3. Two-factor con®rmatory factor analysis results

Item content I II b2

1 AC v CE .337 .114
3 AC v CE .126 .016
5 AC v CE .633 .401
7 AC v CE .583 .340
9 AC v CE .469 .220
11 AC v CE .097 .009
13 AC v CE .617 .380
2 AE v RO .586 .343
4 AE v RO .639 .409
6 AE v RO .069 .005
8 AE v RO .337 .114
10 AE v RO .053 .003
12 AE v RO .525 .275
14 AE v RO .337 .114

b2 signi®es communality. The items are listed in the appendix.

The factor loadings and communalities for the two-factor model are shown in
Table 3. An analysis of the factor loadings indicates seven are unaccepted low (that is,
below .40, see Ford, MacCallum, & Tait, 1986). A more parsimonious model
excluding those items with factor loading less than .40 was tested, and the results are
shown in Table 2. The revised two-factor model, after the item attrition exercise, is
superior to the 14-item two-factor model of Romero et al. (1992) and indicative of
satisfactory model ®t. However, the disattenutated intercorrelations among the two
latent factors was .775, which does not support the use of these items as distinct
scales.

Validity Evidence
Figure 2 depicts the plot of the average scores on each LSI dimension by
undergraduate major. The mean scores in the present study and that of Romero et
al. (1992) are shown in Table 4. The present study ®nds the four groups are clustered
 Problem Solving Style Questionnaire 705

Figure 2. Plot of average LSI scores by undergraduate major, comparison of results of present study
with means reported by Romero et al. (1992)

around the intersection of the two orthogonal bipolar dimensions, indicating that the
scales provide relatively little discrimination between the four groups. Business,
sociology, and psychology majors are classi®ed as divergers, and accounting majors as
convergers. This is in contrast to the ®nding of Romero et al. (1992), and Wolfe and
Kolb (1994) ± see Table 5. In summary, the ability of the scales to differentiate
between different undergraduate majors is questionable.
Finally, the scales are correlated with two measures of performance: ®rst, academic
performance, and second, prior academic achievement. The results are shown in
Table 1. Prior academic achievement is positively related to both scales, and strongly
related to concreteness/abstractness (r=.287). Academic performance is positively
related to concreteness/abstractness (r=.165) and action/re¯ection (r=.153). These
results indicate that the most effective learners are likely to emphasise abstract
conceptualisation and active experimentation, as conceived by the PSSQ scales.
706 A. Duff

Table 4. Mean PSSQ scores by undergraduate major compared with results of Romero et al.
(1992)

Sociology Psychology Accounting Business

Present Romero Present Romero Present Romero Present Romero

study et al. study et al. study et al. study et al.
(n=80) (n=46) (n=45) (n=59) (n=62) (n=0) (n=12) (n=163)

AC±CE 23.70 19.8 23.93 24.6 25.55 ± 23.08 19.6

AE±RO 21.69 24.9 23.67 20.6 24.60 ± 22.17 29.1

Table 5. LSI types by undergraduate major: results of present investigation versus Romero et al.
(1992) and Wolfe and Kolb (1984)

Undergraduate major Present study Romero et al. (1992) Wolfe and Kolb (1984)

Accounting Converger
Business Diverger Accomodator Accomodator
Psychology Diverger Diverger Assimilator
Sociology Diverger Assimilator Diverger

Discussion
Empirical testing of the ELM has long been limited by the poor measurement
qualities of the three extant measures of the two bipolar style dimensions of Kolb's
model. Previous investigations by Romero et al. (1992) and Tepper et al. (1993)
suggest that the PSSQ may be a suitable instrument to measure the two learning style
dimensions which are the foundation of the ELM, at least when administered to
undergraduate and MBA students in the USA. To establish whether the scales
represent psychometrically sound measures of concreteness/abstractness and re¯ec-
tion/action in other cultural contexts, this study administered the instrument to
samples of UK undergraduate students from four undergraduate majors. The results
of the present study, however, are not particularly encouraging. The current research
provides little support for the internal consistency reliability and factor structure of the
original seven-item scales. An item attrition exercise reducing concreteness/
abstractness to four items, and re¯ection/action to three items, improved the internal
consistency reliability, close to acceptable levels, and provided satisfactory model ®t to
the data. However, the relatively high correlation between the two latent factors in the
con®rmatory factor analytic model indicates the two dimensions are not suf®ciently
distinct. Considering the validity of scores yielded by the PSSQ, when average LSI
scores are plotted the scales fail to categorise student groups in a similar fashion to
prior investigations. However, correlating scale scores with measures of academic
performance and prior educational achievement provides some evidence of validity.
Despite these mixed results, we see potential for future development work
 Problem Solving Style Questionnaire 707

considering the PSSQ. The process of devising a scale with statement anchors
removes the ipsative problem with the LSI and its later revision. Furthermore, seven
(of 14) items appear to be useful as a measure of learning style.

Note
1 As such, academic performance as a behavioural measure suffers a restriction of range.

References
Allinson, C. W., & Hayes, J. (1988). The Learning Styles Questionnaire: an alternative to Kolb's
Inventory? Journal of Management Studies, 25, 269±281.
Allinson, C. W., & Hayes, J. (1990). Validity of the learning styles questionnaire. Psychological
Reports, 67, 859±866.
Arbuckle, J. L. (1999). Amos users' guide version 4.0. Chicago, IL: SPSS Inc.
Armstrong, S. J. (2000). The in¯uence of individual cognitive style on performance on
management education. Educational Psychology, 20, 323±.
Atkinson, Jr., G., Murrell, P. H. & Winters, M. R. (1990). Career personality types and learning
styles. Psychological Reports, 66, 160±162.
De Ciantis, S. M., & Kirton, M. J. (1996). A psychometric reexamination of Kolb's experiential
learning cycle construct: a separation of level, style and process. Educational and Psychological
Measurement, 56, 809±820.
Duff, A. (1997). The reliability and predictive validity of the Learning Styles Questionnaire and the
Inventory of Learning Processes. Accounting Education: An International Journal, 6, 263±272.
Duff, A. (2001). A note on the reliability and validity of the Learning Styles Questionnaire (LSQ).
Accounting Education: An International Journal, 10, 185±197.
Duff, A., & Duffy, T. (2002) Psychometric properties of Honey and Mumford's Learning Style
Questionnaire. Personality and Individual Differences, 33, 147±163.
Ford, J., MacCallum, R., & Tait, M. (1986). The application of factor analysis in applied
psychology: a critical review and analysis. Personnel Psychology, 39, 291±314.
Freedman, R. D., & Stumpf, S. A. (1978). What can one learn from the Learning Styles Inventory?
Academy of Management Journal, 21, 275±282.
Freedman, R. D., & Stumpf, S. A. (1980). Learning Styles Inventory: less than meets the eye.
Academy of Management Review, 5, 445±447.
Geiger, M. A., Boyle, E. J., & Pinto, J. (1992). A factor analysis of Kolb's Revised Learning Styles
Inventory. Educational and Psychological Measurement, 52, 753±759.
Geiger, M. A., Boyle, E. J., & Pinto, J. (1993). An examination of ipsative and normative versions
of Kolb's revised Learning Style Inventory. Educational and Psychological Measurement, 53,
717±726.
Geller, L. (1979). Reliability of the Learning Style Inventory. Psychology Reports, 44, 555±561.
Green, D. W., & Parker, R. M. (1989). Vocational and academic attributes of students with
different learning styles. Journal of College and Student Development, 30, 395±400.
Green, D. W., Snell, J. C., & Parimanath, A. R. (1990). Learning styles in assessment of students.
Perceptual and Motor Skills, 70, 363±369.
Healey, M., & Jenkins, A. (2000). Kolb's experiential learning theory and its application in
geography in higher education. Journal of Geography, 99, 185±195.
Henry, J. (1989). Meaning and practice in experiential learning. In S. Warner-Weil & I. McGill
(Eds.), Making sense of experiential learning: diversity in theory and practice (pp. 25±37). Milton
Keynes, UK Society for Research in Higher Education and Open University Press.
Honey, P., & Mumford, A. (1992). The manual of learning styles. Maidenhead, UK: Peter Honey.
708 A. Duff

Hu, L., & Bentler, P. M. (1999). Cutoff criteria for ®t indexes in covariance structure analysis:
conventional criteria versus new alternatives. Structural Equation Modeling, 6, 1±55.
Kolb, D. A. (1976). Learning Style Inventory: technical manual. Boston, MA: McBer and Company.
Kolb, D. A., Rubin, I. M., & McIntyre, J. M. (1984). Organisational Psychology: An experimental
approach to organizational behavior, Englewood Cliffs, NJ: Prentice±Hall.
Kolb, D. A. (1985). Learning Style Inventory: self-scoring inventory and interpretation booklet. Boston,
MA: McBer and Co.
Loo, R. (1999). Con®rmatory factor analyses of Kolb's Learning Style Inventory (LSI-1985).
British Journal of Educational Psychology, 69, 213±219.
Maccallum, R. C., Browne, M. W., & Sugawara, H. M. (1996). Power analysis and determination
of sample size for covariance structure modelling. Psychological Methods, 1, 130±149.
Marsh, H. W., & Hau, K. T. (1999). Con®rmatory factor analysis: strategies for small sample
research. In R. H. Hoyle (Ed.), Statistical strategies for small sample research. Thousand Oaks,
CA: Sage, pp. 252±284.
Newstead, S. E. (1992). A study of two ``quick and easy'' methods of assessing individual
differences in student learning. British Journal of Educational Psychology, 62, 299±312.
Nunnally, J. C., & Bernstein, I. H. (1994). Psychometric theory (3rd ed.). New York: McGraw-Hill.
Philbin, M., Meier, E., Huffman, S., & Boverie, P. (1995). A survey of gender and learning styles.
Sex Roles, 32, 485±494.
Rakoczy, M., & Money, S. (1995). Learning styles of nursing students: a 3-year cohort longitudinal
study. Journal of Professional Nursing, 11, 170±174.
Reynolds, M. (1997). Learning styles: a critique. Management Learning, 28, 115±133.
Romero, J. E., Tepper, B. J., & Tetrault, L. A. (1992). Development and validation of new scales
to measure Kolb's (1985) learning style dimensions. Educational and Psychological
Measurement, 52, 171±180.
Ruble, T. L., & Stout, D. E. (1993). Comments on the use of the LSI in research on student
performance in accounting education. Accounting Educators' Journal, 5, 35±45.
Sims, R. R., & Lindholm, J. (1993). Kolb's experiential learning model: a ®rst step in learning how
to learn from experience. Journal of Management Education, 17, 95±98.
Sims, R. R., Veres, G. G., & Shake, I. G. (1989). An exploratory examination of the convergence
between the Learning Style Questionnaire and the Learning Style Inventory II. Educational
and Psychological Measurement, 49, 227±233.
Stout, D. E., & Ruble, T. L. (1991a). The LSI and accounting education research: a cautionary
view and suggestions for future research. Issues in Accounting Education, 6, 41±52.
Stout, D. E., & Ruble, T. L. (1991b). A reexamination of accounting students' learning styles.
Journal of Accounting Education, 9, 341±354.
Tepper, B. J., Tetrault, L. A., Braun, C. K., & Romero, J. E. (1993). Discriminant and convergent
validity of the Problem Solving Style Questionnaire. Educational and Psychological
Measurement, 53, 437±444.
White, J. A. (1992). Applying an experiential learning styles framework to management and
professional development. Journal of Management Development, 11 (5), 55±64.
Willcoxon, L., & Prosser, M. (1996). Kolb's Learning Style Inventory (1985): review and further
study of validity and reliability. British Journal of Educational Psychology, 66, 247±257.
Wynd, W. R., & Bozman, C. S. (1996). Student learning style: a segment strategy for higher
education. Journal of Education for Business, 71, 232±235.
Yahya, I. (1998). Willcoxon and Prosser's factor analyses on Kolb's (1985) LSI data: re¯ections
and re-analyses. British Journal of Educational Psychology, 68, 281±286.
 Problem Solving Style Questionnaire 709

Appendix :

Concreteness versus abstractness

1 I would describe myself as 1 2 3 4 5 6 I would describe myself as

impartial (open-minded) explicit (de®nite)
3 I like to be speci®c 1 2 3 4 5 6 I like to remain ¯exible
5 I like things to be varied and 1 2 3 4 5 6 I like things to be exact and
colourful precise
7 I take a creative and 1 2 3 4 5 6 I take a precise and calculated
imaginative approach to approach to solving problems
solving problems
9 I like to stay ¯exible (not get 1 2 3 4 5 6 I like to get as focused as
too focused) possible
11 I would describe myself as 1 2 3 4 5 6 I would describe myself as
evaluative and logical receptive and accepting
13 I strive for versatility 1 2 3 4 5 6 I strive for accuracy
Re¯ection versus action
2 I would describe myself as 1 2 3 4 5 6 I would describe myself as
re¯ective action-oriented
4 I value patience 1 2 3 4 5 6 I value getting things done
6 I would describe myself as a 1 2 3 4 5 6 I would describe myself as an
doer observer
8 I feel good when I understand 1 2 3 4 5 6 I feel good when I have an
things impact on things
10 I am good at getting things 1 2 3 4 5 6 I am good as seeing things
accomplished from many perspectives
12 I like to watch what is going on 1 2 3 4 5 6 I like to see the results of my
actions
14 I am reserved 1 2 3 4 5 6 I am prepared

Note: items 3, 6, 10 and 11 are re¯ected