This article was downloaded by: [Lakehead University]

On: 13 March 2015, At: 09:53

Publisher: Routledge
Informa Ltd Registered in England and Wales Registered Number: 1072954 Registered office: Mortimer House,
37-41 Mortimer Street, London W1T 3JH, UK

Change: The Magazine of Higher Learning

Publication details, including instructions for authors and subscription information:
http://www.tandfonline.com/loi/vchn20

Soft Skills for the Workplace

a
Patrick C. Kyllonen
a
Educational Testing Service
Published online: 20 Nov 2013.

To cite this article: Patrick C. Kyllonen (2013) Soft Skills for the Workplace, Change: The Magazine of Higher Learning, 45:6,
16-23, DOI: 10.1080/00091383.2013.841516

To link to this article: http://dx.doi.org/10.1080/00091383.2013.841516

PLEASE SCROLL DOWN FOR ARTICLE

Taylor & Francis makes every effort to ensure the accuracy of all the information (the “Content”) contained
in the publications on our platform. However, Taylor & Francis, our agents, and our licensors make no
representations or warranties whatsoever as to the accuracy, completeness, or suitability for any purpose of
the Content. Any opinions and views expressed in this publication are the opinions and views of the authors,
and are not the views of or endorsed by Taylor & Francis. The accuracy of the Content should not be relied
upon and should be independently verified with primary sources of information. Taylor and Francis shall not be
liable for any losses, actions, claims, proceedings, demands, costs, expenses, damages, and other liabilities
whatsoever or howsoever caused arising directly or indirectly in connection with, in relation to or arising out of
the use of the Content.

This article may be used for research, teaching, and private study purposes. Any substantial or systematic
reproduction, redistribution, reselling, loan, sub-licensing, systematic supply, or distribution in any
form to anyone is expressly forbidden. Terms & Conditions of access and use can be found at http://
www.tandfonline.com/page/terms-and-conditions
Downloaded by [Lakehead University] at 09:53 13 March 2015

Patrick C. Kyllonen ([email protected]) is the senior research director of the Center for
Academic and Workforce Readiness and Success at the Educational Testing Service. For over
ten years, the Center has been conducting research to identify important 21st-century skills
and creating approaches for developing and measuring them.

16 Change • November/December 2013

 and former owner of The Atlantic and Fast Money—identi-
fies determination as the quality that best predicts success in
the workforce. It was not that long ago that many manage-
ment consultants, economists, industrial-organizational psy-
chologists, and laypeople believed that cognitive skill was
the single most important predictor.
What happened to change that?

The Importance of Soft Skills

Until quite recently, the predominant belief at the policy
level, in education at all levels, and in workforce settings
was that cognitive abilities were the ones that most mat-
tered. This led to the deployment of large-scale efforts to
assess those skills. The National Assessment of Educational
Progress (NAEP, NCES), administered every year in all
50 states and several other jurisdictions in the US, and the
Program for International Student Assessment
(PISA, OECD), administered every three years
in over 70 countries, were initiated primarily to
Downloaded by [Lakehead University] at 09:53 13 March 2015

compare states and countries on the cognitive

abilities of their schoolchildren (e.g., in reading,
mathematics, science, and problem solving).
By Patrick C. Kyllonen The centerpiece of the No Child Left Behind Act of 2001
was “accountability for results,” which meant that “student
progress and achievement will be measured according to
tests that will be given to every child, every year.” By tests,
the Act was referring to cognitive tests.

T
wo years ago, in the middle of the darkest days of the At the postsecondary level college-placement testing,
recession, ABC’s This Week hosted a panel of recent which determines whether students are ready for credit-
college graduates and a pair of industry leaders. After bearing college mathematics and English courses, was and
hearing from the graduates about the difficulties of to a large extent still is exclusively based on performance on
finding a job in a tight, competitive market, host Christiane cognitive tests in those subjects. Students who get passing
Amanpour addressed the employers: scores on the mathematics placement test go into college-
level, credit-bearing mathematics courses; the rest go into
Amanpour: Let me turn to you both. You’ve now lis- developmental courses.
tened to [the graduates]. You see what they’ve studied. Arguments about the validity and fairness of standardized
You’ve heard their prospects. Mort, as the owner of a cognitive admissions tests such as the SAT, ACT, and GRE
real estate company, as the publisher of newspapers have dominated discussions about higher education admis-
and magazines, what do you think they need to do? sions policies. In academic circles and on editorial pages,
And are they hirable, [given] what you’ve just heard there have been national debates about the importance of
right now? cognitive ability, its heritability, race/ethnic differences, the
Zuckerman: Well, I don’t know enough about their “bell curve,” and the cognitive-skills shortage.
individual skills and capacities, but this is the worst In the workplace, companies and the military have histori-
atmosphere for employment that we’ve had in 50 or 60 cally focused selection testing almost exclusively on cogni-
years. I mean, just think of the fact, in the ‘70s, ‘80s, tive abilities, in part because a literature in industrial-
and ‘90s, the United States created over 20 million jobs organizational psychology attested to its preeminent impor-
in each one of those decades. In the first decade of this tance in identifying workers most likely to succeed in training
century, we created zero jobs. and on the job (Schmidt & Hunter, 1998). A generation was
If I were hiring today, ... the one thing that I [would] taught that other variables, such as personality, were unre-
look for more than anything else is some evidence lated to workforce outcomes or to just about anything else.
of determination, which to me is the most important There had been occasional nods to the importance of per-
quality in terms of how people will do in their career. sonal qualities in education and the workplace (Willingham
(ABC News, 2011, p. 13) & Breland, 1982), but these were rare. The situation only
began changing in the 1990s, when psychology began
In this exchange, Mort Zuckerman—co-founder, chair- to coalesce around a five-factor model of personality
man, and CEO of Boston Properties; owner and publisher of (Goldberg, 1990). This led to rapid acceptance and expan-
the New York Daily News and of US News & World Report; sion of the notion that personality mattered, and studies

www.changemag.org 17
 began contributing to an accumulating knowledge base
about its importance.
By the middle of the millennium decade, researchers were
able to link wide areas of human endeavor and outcomes to
personality (Roberts, Kuncel, Shiner, Caspi, & Goldberg,
2007): Personality measures were shown to predict mortality,
divorce, occupational attainment, health behaviors, drug use,
alcoholism, managerial success, leadership effectiveness,
procrastination, creativity, job performance, absenteeism,
team performance, and job satisfaction—to name just a few.
The meta-analytic-based list of predictive relationships and
their magnitude rivaled and in some cases exceeded similar
analyses made a decade earlier for predictions based on cog-
nitive ability (e.g., Gottfredson, 1997).
Particularly relevant were such studies showing that the
big-five personality factors—most often conscientiousness
(i.e., the trait of striving, being organized, and working hard)
—predicted both workplace (Ones, Dilchert, Viswesvaran, &
Judge, 2007) and academic success (Poropat, 2009). Other
Downloaded by [Lakehead University] at 09:53 13 March 2015

meta-analyses suggested additional non-cognitive predictors

of school performance (grades and retention), such as having
academic goals, institutional commitment, social support and
involvement, academic self-efficacy and self-concept, consci-
entiousness, a tendency to procrastinate, a need for cognition,
grade goals, time management skills, and persistence/effort
regulation (Richardson, Abraham, & Bond, 2012).
Meanwhile, new conceptions about human-capital theory
began appearing in the economics literature. Human capital
is a worker’s set of skills, broadly defined, that enhance pro-
ductivity. They can be cognitive skills, abilities, knowledge,
dispositions, attitudes, interests, etc. These attributes come
from innate ability, education (e.g., years in school, quality
of schooling), training, medical care, and/or parenting, or in
other ways.
What was new was an increased awareness of the impor-
tance of non-cognitive skills. First, Heckman and Rubinstein
(2001) showed that students who dropped out of high school
but received GEDs possessed cognitive skills equal to those
of high school graduates (based on standardized cognitive
test scores) but had poorer performance in the workforce
(e.g., lower wages, higher absenteeism, more unemploy-
ment, more legal troubles). The researchers attributed their
relative lack of success to their lower non-cognitive skills, as
reflected in their failure to persist to high school graduation.
The economists Samuel Bowles, Herbert Gintis, and
Melissa Osborne showed that cognitive skills accounted
for only 20 percent of the educational-attainment effects
on labor-market outcomes (i.e., more schooling leads to
higher employment and wages). Here the interpretation was
that schooling develops non-cognitive as well as cognitive
skills, and these non-cognitive skills drive workplace success
(Levin, 2012).
A couple of recent studies make a similar point. In one,
Segal (2012) showed that, controlling for cognitive test
scores and family characteristics, teachers’ ratings of 8th-
grade male students on a non-cognitive five-item checklist—

18 Change • November/December 2013

 is the student “frequently tardy?” “frequently absent?” “con- Can Soft Skills Be Taught?
sistently inattentive?” “rarely completes his homework?” The finding that cognitive test scores accounted for only
“frequently disruptive?” (National Educational Longitudinal 20 percent of educational attainment’s effect on earnings
Study, or NELS:88) —predicted educational attainment. and employment suggests that one possibly unintended
The ratings also predicted workplace earnings 12 years accomplishment of schooling is to teach non-cognitive skills.
later, over and above cognitive test scores. This was true Additional support for this idea comes from studies of early-
regardless of educational attainment, whereas cognitive test childhood intervention programs that, although targeting
scores predicted earnings only for students with postsecond- cognitive skill development, have even greater effects on
ary degrees. non-cognitive skills.
Lindqvist and Vestman (2011) had a similar finding: They For example, Barnett (2011) examined several preschool
tracked 14,000 Swedish 18–19 year-old military enlistees programs for disadvantaged students: the High/Scope Perry
who had been given both a two-hour cognitive test and a Preschool program, the Abecedarian program, and the
25-minute non-cognitive interview. The interview had led to Chicago-Child Parent Centers program. All involved a ran-
a rating (on a nine-point scale) of their willingness to assume dom assignment of students to the program at an early age
responsibility, independence, outgoing character, persis- and followed them through their 20s and later. Barnett found
tence, emotional stability, initiative, social skills, and lack of mixed results for the programs’ success in boosting the cog-
personality disorders. nitive test scores (IQs) of participants but clear results with
The researchers found that both the cognitive and non- regard to many other outcomes, such as educational attain-
cognitive measures predicted employment outcomes (earn- ment, avoiding arrests and legal trouble, avoiding welfare,
Downloaded by [Lakehead University] at 09:53 13 March 2015

ings and unemployment) of those same men, now 32 to 41 owning a home, and maintaining good health habits.
years old. However, the non-cognitive measures predicted If preschool and regular schooling enhance a student’s
outcomes at all levels of educational attainment, while the non-cognitive skills as a byproduct, can targeted interven-
cognitive score predicted them only for those who were tions provide an even greater boost? This has been the theme
above the median. The cognitive measure was a stronger pre- of work by the Collaborative for Academic, Social, and
dictor of educational attainment, but, controlling for educa- Emotional Learning (CASEL), an organization with a mis-
tional attainment, the non-cognitive measure was a stronger sion “to establish social and emotional learning as an essen-
predictor of earnings and employment. tial part of education” (http://casel.org/about-us/mission-
This and other research started to raise the national con- vision/).
sciousness in this decade. For example, Paul Tough’s 2012 It does this by advancing the science of social-emotional
best seller, How Children Succeed, contrasted the “cogni- learning, expanding evidence-based practice, and strengthen-
tive hypothesis” that “success today depends primarily on ing the field. CASEL defines the core social-emotional com-
cognitive skills—the kind of intelligence that gets measured petencies as self-awareness, self-management, social aware-
on I.Q. tests” with a new view that success has more to do ness, relationship skills, and responsible decision-making
with character skills such as perseverance, grit, curiosity, (i.e., considering the social norms involved in and the ethics,
optimism, and self-control. According to Tough, these also safety, and consequences of making decisions).
happen to be more malleable than IQ. A plea to include CASEL performed a trio of meta-analyses: one an exami-
non-cognitive as well as cognitive skills in the educational nation of 69 after-school programs, another of 213 regular
conversation could also be seen in a 2012 report by the school-based studies, and a third of 80 studies of children
National Research Council entitled Education for Life and selected for signs of social-emotional problems (Durlak,
Work: Developing Transferable Knowledge and Skills in the Weissberg, Dymnicki, Taylor, & Schellinger, 2011). The
21st Century. targeted programs were effective in enhancing the students’
This cultural shift is apparent not only in the quote from academic and social-emotional skills.
Mort Zuckerman during This Week but in the results of sev- The meta-analyses also suggested that particular program
eral large-scale surveys that asked employers which were the features could be considered best practices because they
most important skills for workforce success. A 2012 study were associated with program success. One of these was pro-
by Millenial Branding found “communication skills,” a gram implementation adequacy, which seems obvious but is
“positive attitude,” being “adaptable to change,” and “team- sometimes overlooked in reviews such as these. The others
work skills” to be the four most important traits employers are captured by the acronym SAFE: Effective programs were
were looking for when they hired. Another major employer
survey sponsored in 2006 by the Conference Board and • s equenced, involving a planned set of activities to be
others entitled “Are they really ready to work?” identified executed step by step;
“professionalism/ work ethic,” “teamwork/collaboration,” • active, requiring active learning activities such as role
and “oral communications” as the top three “very important” plays and behavioral rehearsals;
skills for job success for new workforce entrants at all three • focused, devoting sufficient time for developing social
education levels: high school graduates, two-year college and emotional skills; and
graduates, and four-year college graduates. • explicit, targeting specific social and emotional skills.

www.changemag.org 19
 A different take on the trainability of soft skills was pro- uses. Companies such as SHL, an industry leader, market
vided in a recent meta-analysis by Brent Roberts and col- tests, simulations, and interview tools to capture biodata
leagues, who had shown in earlier research that personality and measure personality and behavior, situational judgment,
tended to change over a lifespan. Self-confidence, warmth, motivation, dependability, and safety.
self-control, and emotional stability all tend to increase with According to a 2001 survey by the American Management
age, particularly in young adulthood but continuing through Association, 13 percent of employers used a personality test,
middle and old age (Roberts, Walton, & Viechtbauer, 2006). and almost all Fortune 500 companies did so. It is likely
Particular events in one’s life seem to be associated with that those numbers are higher today. In 2009 the United
significant changes in personality: Successful careers are States Department of Defense began to use a personality test
associated with increases in emotional stability and consci- called the Tailored Adaptive Personality Assessment System
entiousness, remarriage is associated with a reduction in (TAPAS) for screening military recruits; over half a million
neuroticism, and engaging in negative workplace behaviors have been tested so far.
is associated with decreases in conscientiousness and emo- Colleges have also recently begun to use personality
tional stability (Roberts & Mroczek, 2008). assessments to help in some admissions decisions. A few
Can personality enhancements be induced through years ago, the Educational Testing Service began adminis-
specific interventions, such as psychotherapy? In a paper tering the Personal Potential Index (PPI) to supplement the
presented at the 2013 meeting of the Association for GRE for graduate school admissions (Kyllonen, 2008). The
Psychological Science, Brent Roberts identified 144 stud- PPI measures six factors: knowledge and creativity, commu-
ies (with 15,047 participants) that included personality nication skills, teamwork, resilience, planning and organiza-
Downloaded by [Lakehead University] at 09:53 13 March 2015

measures as pretests and posttests for both clinical (e.g., tion, and ethics and integrity. Current users range from Notre
for depression) and nonclinical (e.g., for eating disorders) Dame Business School to the American Dental Education
samples, with an average intervention duration of 28 weeks. Association. A major multi-institutional validity study is
On average he found that interventions changed personality currently underway, with results expected in 2014.
factors on average by about a half a standard deviation (e.g., A study of law-school students, graduates, and practicing
from the 50th to the 67th percentile) and that the change did lawyers by Berkeley Law School professor Marjorie Shultz
not fade over as much as five years. and psychology professor Sheldon Zedeck led to the devel-
While the interventions Roberts examined affected many opment of a soft-skills assessment for law-school admis-
personality traits, emotional stability was the most altered, sions—designed to measure, among other factors, commu-
perhaps because of the way the interventions were chosen nications, influencing and advocating, strategic planning,
(e.g., therapy for anxiety or depression is likely to target self-management, conflict resolution and negotiation skills,
emotional stability). Interventions of a different nature networking skills, community involvement, integrity, stress
(e.g., executive or life-skills coaching) might be expected to management, passion, diligence, and self-development.
address different personality traits, such as conscientious- Several experiments with soft-skills assessment for higher
ness, drive, and organization. education admissions have been conducted in the past few
Finally, corporate training, a $50-billion-dollar industry, years. In a series of College Board studies, SAT-taking stu-
is concerned to a considerable extent with soft skills. In their dents were given situational judgment and biodata items
meta-analysis of the effects of corporate training, Arthur, designed to measure a variety of soft skills—including mul-
Bennett, Edens, and Bell (2003) identified 123 training ticultural tolerance, leadership, interpersonal skills, social
programs targeting interpersonal skills that had an effect responsibility, adaptability, perseverance, and ethics—that
size even larger than the intervention effect size reported were shown to have unique predictive validity for certain
by Roberts. The programs used a wide variety of formats college student outcomes (Schmitt, 2012).
(lecture, audiovisual, discussion) and focused on various One exciting development is the use of soft-skills assess-
outcomes (learning the interpersonal skill, transferring it to ment in college-placement testing. Traditionally, placement
the job, and seeing improvement in workplace performance testing has been strictly cognitive—students take a math-
as a result). ematics test to determine their readiness for college-level
Together these studies show that effective programs that mathematics coursework. Those who do not make the cutoff
are already in place, from preschool to the workplace, can are assigned to take a non-credit-bearing developmental or
develop and increase soft skills. Given the newfound rec- remedial course prior to being eligible for the credit-bearing
ognition of the importance of such skills, it is likely that course.
this education and training will prove to be an active area But perhaps non-cognitive skills, such as motivation and
of development in both education and the workplace in the determination, can compensate to some extent for deficient
coming decade. mathematics skills. A determined student is likely to do what
it takes to pass an entry-level course, whether that involves
What Can We Do with Soft Skills Assessments? doing extra homework, studying nights and weekends, or
Soft-skills assessments are commonly included in working with a tutor.
employee recruiting, prescreening, and selection, and there For this reason, ETS is currently evaluating whether
is a major human-resources consulting industry around these non-cognitive information could supplement cognitive

20 Change • November/December 2013

 test results for placement testing. The assessment, ETS’s preference soft-skills assessment. Their conclusion is in
SuccessNavigator, measures academic skills and soft skills agreement with a meta-analysis that compared a single-state-
in the areas of commitment, self-management, and social ment personality measure with several forced-choice ones
support (Markle, Olivera-Aguilar, Jackson, Noeth, & (Salgado & Tauriz, 2012).
Robbins, 2013). The forced-choice measures that were studied by Salgado
In a recent issue of Change, Alexander McCormick and and Tauriz were ipsative (all statements are paired with
colleagues discussed how the National Survey of Student all other statements), quasi-ipsative (some statements are
Engagement (NSSE) is being used to develop student typolo- paired with others, but not all), and normative (only state-
gies. “Disengaged” students, they found, “had lower first-year ments measuring the same dimension are paired with each
GPAs, perceived learning gains, and persistence to the second other, such as “I work hard” vs. “I work too hard”). This is a
year” than “maximizers”—“their most-engaged peers.” technical distinction, but the important finding was that one
of these formats, the quasi-ipsative format, was found to be
Assessment Methods superior to the others.
The overwhelming method of choice for soft-skills assess- Another approach that has been growing in popularity is
ment in both scientific research and in practice has been the the situational judgment test (SJT). In this format, respon-
simple self-rating scale. A student or employment applicant dents are given a situation such as “You have been assigned
might be asked to “indicate your level of agreement with a team project, but one of the team members, call him ‘Joe,’
the following statement: ‘I meet my deadlines’ (a) strongly has a bad attitude and seems determined to thwart the team’s
agree, (b) agree, (c) disagree, or (d) strongly disagree.’” efforts.” Then they are given a series of possible responses,
Downloaded by [Lakehead University] at 09:53 13 March 2015

For many low-stakes applications, there is no incentive with instructions to select the one that would be most appro-
for respondents to lie, so responses are probably reasonably priate. These might include
valid indicators of at least what they believe about them-
selves. However, if scores are used for something like school (a) tell other team members to ignore Joe, (b) confront
admissions or employment screening, there is a strong incen- Joe and threaten to tell the boss about his behavior, (c)
tive for self-reporters to make themselves look good, and speak to Joe in front of the team and encourage him to
scores are much less trustworthy. contribute positively to the effort, or (d) talk to Joe pri-
For this reason, research has focused on several alterna- vately and encourage him to participate by telling him
tives to simple rating scales. One is a rating by another per- how the team will only be successful if he is involved.
son. A meta-analysis by Connelly and Ones (2010) showed
that compared to self-ratings, ratings by others were more SJT items such as these were used in the Schmitt (2012)
accurate, less biased, and more predictive of future out- and law school studies referred to previously. Their popular-
comes. ity might be attributed to their combining authenticity with
Letters of recommendation, which are widely used in reduced susceptibility to faking. SJTs also are amenable
higher education admissions and employee selection, are to video presentation of situations and alternative response
essentially ratings by others, albeit in a non-standardized formats, such as speaking, although the vast majority of
format. ETS’s PPI is essentially a standardized letter of rec- those available today are written and require multiple-choice
ommendation. It asks an evaluator to rate a graduate-student responses.
applicant on six dimensions and for open-text comments on Interviews, a form of assessment, have been used in edu-
each of the dimensions to support the ratings. cational admissions and employment selection for a very
One of the drawbacks of ratings by others is that finding long time. An attractive feature of interviews is that both
someone to do a rating is not always straightforward, the candidates and employers like them. The latter often feel that
recommendations are often inflated, and there is sometimes they cannot get a real sense of the person without an inter-
considerable disagreement between raters. So there is still view, preferably face to face. But interviews are expensive
interest in assessments done by the person who is the target and typically neither standardized nor reliable.
of the assessment. This brings us back to self-assessments. Behavioral interviews are a way to mitigate the lack
A form of self-assessment that might not be as suscep- of standardization. They are based on the soft skills an
tible to faking as the simple rating scale is the pairwise- employer is interested in evaluating, such as drive, enthusi-
preference format. This involves presenting two, three, or asm, or customer orientation. The interview consists of ques-
four statements to examinees and asking them to indicate tions designed to elicit evidence that the applicant possesses
which is the most true. For example, examinees might be those skills.
asked to choose between “I meet my deadlines,” and “I work For example, the applicant might be asked to “describe a
well with others in teams.” In this case, neither statement situation in which you had to meet a deadline but had com-
seems to be clearly superior to the other in terms of what an peting commitments,” or “a time when you felt extremely
employer might be looking for. excited about an event at work.” Often such interviews are
This format is the basis for the TAPAS assessment and is accompanied by scoring rubrics that require the interviewer
the reason why the US Department of Defense, after half a to evaluate the candidate on several scales pertaining to the
century of research, opted to go operational with a pairwise- responses expected.

www.changemag.org 21
 But behavioral interviews are as expensive as any other. Future Developments
Employers are increasingly attempting to reduce those costs In the latter part of the 20th century, behavioral scientists
with technology, using remote interviewing methods (e.g., and society more generally adopted a cultural belief that
teleconferences or video-conference calls). In the near cognitive ability was the most significant determinant of edu-
future, asynchronous interviewing—e.g., where a candidate cational and workforce outcomes. This led to efforts to raise
uploads a video-recorded response to a question presented students test scores, the promotion of teachers who were
through a website—may become the norm. Such systems are successful in doing so, and heavy if not exclusive reliance on
Downloaded by [Lakehead University] at 09:53 13 March 2015

already in wide use. test scores for admissions and employment screening.
The next steps will be to score such interviews with expert But behavioral science research in psychology and eco-
raters, in much the same way essay tests from standardized nomics suggests that non-cognitive factors—soft skills such
exams such as the SAT or GRE are scored today. Automated as motivation, work ethic, teamwork, organization, cultural
scoring is likely to follow soon thereafter. awareness, and effective communication—play a role that is
There is considerable interest in the idea of a standard- as important or even more important in determining success
ized soft-skills assessment that avoids the problems of in school and in the workplace.
ratings. Standardized tests of soft skills, such the Mayer- So the 21st century is becoming the era in which we rec-
Salovey-Caruso Emotional Intelligence Test (MSCEIT), ask ognize the importance of soft skills, the role education plays
examinees to identify the emotions (e.g., happiness, fear, in developing those skills, and the way they evolve through-
sadness) expressed in a picture of a face; the emotion one out the life cycle. And we are developing new education,
might feel if given additional work; or to recognize how training, and intervention methods and new assessments in
much a particular action (e.g., making a list) might affect recognition of this importance. C
one’s mood. Such assessments are beginning to be used in
industry. It is likely that efforts to develop such measures
will continue.

Bring your team of faculty, administrators and

student support staff to develop a plan with the
help of experts at “the best boot camp for learning
communities in the country.”

∙Create Campus partnerships ∙Design integrative learning

∙Develop assessment strategies ∙explore possibilities

evergreen.edu/washingtoncenter
22 Change • November/December 2013
 Resources
n  Arthur, W., Jr., Bennett, W., Jr., Edens, P.S., & Bell, S.T. (2003). Effectiveness of training in organizations: A meta-
analysis of design and evaluation features. Journal of Applied Psychology, 88, 234–245.
n  Barnett, W. (2011). Effectiveness of early educational intervention. Science, 333, 975.
n  Connelly, B. S., & Ones, D. S. (2010). Another perspective on personality: Meta-analytic integration of observers’
accuracy and predictive validity. Psychological Bulletin, 136, 1092–1122.
n  Durlak, J. A., Weissberg, R. P., Dymnicki, A. B., Taylor, R. D., & Schellinger, K. B. (2011). The impact of
enhancing students’ social and emotional learning: a meta-analysis of school-based universal interventions. Child
Development, 82, 405–432.
n  Goldberg, L. R. (1990). An alternative “description of personality”: The big-five factor structure. Journal of
Personality and Social Psychology, 59, 1216–1229. Doi:10.1037/0022-3514.59.6.1216.
n  Gottfredson, L. (1997). Why g matters: the complexity of everyday life. Intelligence, 24, 79–132.
n  Heckman, J., & Rubinstein, Y. (2001). The importance of non-cognitive skills: Lessons from the GED testing pro-
gram. American Economic Review, 91, Papers and Proceedings, 145–149.
Downloaded by [Lakehead University] at 09:53 13 March 2015

n  Kyllonen, P. C. (2008).The research behind the ETS Personal Potential Index (PPI). Princeton, NJ: Educational
Testing Service.
n  Levin, H. M. (2012). More than just test scores. Prospects: Quarterly Review of Comparative Education, 42, 269–
284.
n  Lindqvist, E., & Vestman. R. (2011). The labor market returns to cognitive and non-cognitive ability: Evidence
from the Swedish enlistment. American Economic Journal: Applied Economics, 3, 101–128.
n  Markle, R., Olivera-Aguilar, M., Jackson, T., Noeth, R., & Robbins, S. (2013). Examining evidence of reliabil-
ity, validity, and fairness for the SuccessNavigator assessment. Research Report ETS RR-13-12. Princeton, NJ:
Educational Testing Service.
n  Ones, D. S., Dilchert, S., Viswesvaran, C., & Judge, T. A. (2007). In support of personality assessment in organiza-
tional settings. Personnel Psychology, 60, 995–1027.
n  Poropat, A.E. (2009). A meta-analysis of the five-factor model of personality and academic performance.
Psychological Bulletin, 135, 322–338.
n  Richardson, M., Abraham, C., & Bond, R. (2012). Psychological correlates of university students’ academic
performance: A systematic review and meta-analysis. Psychological Bulletin, 138, 353–387.
n  Roberts, B. W., Kuncel, N. R., Shiner, R., Caspi, A., & Goldberg, L. R. (2007). The power of personality: The
comparative validity of personality traits, socioeconomic status, and cognitive ability for predicting important life
outcomes. Perspectives on Psychological Science, 2, 313–345.
n  Roberts, B. W. & Mroczek, D. K. (2008). Personality trait stability and change. Current Directions in Psychological
Science, 17, 31–35.
n  Roberts B. W., Walton K., & Viechtbauer W. (2006). Patterns of mean-level change in personality traits across the
life course: A meta-analysis of longitudinal studies. Psychological Bulletin, 132, 1–25.
n  Salgado, J. F. & Táuriz, G. (2012). The Five-Factor Model, forced-choice personality inventories and performance:
A comprehensive meta-analysis of academic and occupational validity studies. European Journal of Work and
Organizational Psychology. Retrieved from http://dx.doi.org/10.1080/1359432X.2012.716198
n  Schmidt, F. L., & Hunter, J. E. (1998). The validity and utility of selection methods in personnel psychology:
Practical and theoretical implications of 85 years of research findings. Psychological Bulletin, 124, 262–274.
n  Schmitt, N. (2012). Development of rationale and measures of non-cognitive college student potential. Educational
Psychologist, 47, 18–29.
n  Segal, C. (2012). Misbehavior, education, and labor market outcomes. Journal of the European Economic
Association, 11, 743–779.
n  Willingham, W.W., & Breland, H. M. (1982). Personal qualities and college admissions. New York, NY: College
Entrance Examination Board.

www.changemag.org 23