See discussions, stats, and author profiles for this publication at: https://www.researchgate.

net/publication/26862890

Understanding design fluency: Motor and executive contributions

Article  in  Journal of the International Neuropsychological Society · October 2009

DOI: 10.1017/S1355617709990804 · Source: PubMed

CITATIONS READS
42 3,517

3 authors:

Yana Suchy Matthew L Kraybill

University of Utah Ross University, School of Veterinary Medicine
159 PUBLICATIONS   2,191 CITATIONS    25 PUBLICATIONS   352 CITATIONS   

SEE PROFILE SEE PROFILE

Jennifer C Gidley Larson

University of Michigan
24 PUBLICATIONS   1,404 CITATIONS   

SEE PROFILE

All content following this page was uploaded by Yana Suchy on 20 May 2014.

The user has requested enhancement of the downloaded file.

Journal of the International Neuropsychological Society (2010), 16, 26–37.
Copyright © INS. Published by Cambridge University Press, 2009.
doi:10.1017/S1355617709990804

Understanding design fluency: Motor and executive

contributions

YANA SUCHY, MATTHEW L. KRAYBILL, and JENNIFER C. GIDLEY LARSON

Department of Psychology, University of Utah, Salt Lake City, Utah
(Received December 5, 2008; Final Revision July 19, 2009; Accepted July 28, 2009)

Abstract

Design Fluency (DF) is typically assumed to assess planning, cognitive flexibility, and fluency in generation of visual
patterns, above and beyond contributions from motor speed (Delis, Kaplan, & Kramer, 2001; Ruff, 1998). The present
study examined these assumptions, as little construct validation research has been done in the past. Sixty one community-
dwelling elderly participants were administered the DF, Trail Making, and Letter Fluency tests from the Delis-Kaplan
Executive Function System (D-KEFS), as well as electronically administered measures of motor planning and motor
sequence fluency. Hierarchical regressions were used to parse out unique variance contributions to DF performance.
The results showed that generation of novel designs (i.e., the first two trials on the D-KEFS DF) relied primarily on
motor planning, the ability to generate novel motor actions, and, to a lesser extent, speed of drawing with a writing
implement. In contrast, generation of unique designs while switching (i.e., the third trial on the D-KEFS DF) relied
primarily on visual scanning and perhaps visual-attentional resources. These findings highlight the wisdom of interpreting the
switching trial of the D-KEFS DF separately. Interestingly, cognitive flexibility did not contribute to performance on
any of the three D-KEFS DF trials. (JINS, 2010, 16, 26–37.)
Keywords: Design fluency, Figural fluency, Motor planning, Cognitive flexibility, Executive function, Electronic tests,
Motor action, Motor speed, Visual scanning, Construct validation

INTRODUCTION Chelune, 2003; Tucha, Smely, & Lange, 1999), with the
assumption that sensitivity to frontal lobe pathology implies
Tests of Design Fluency (DF), also known as “figural” or
sensitivity to executive dysfunction.
“nonverbal” fluency, represent a method of assessment of
A handful of studies that did examine the neurocognitive
executive functioning, commonly used in research and clini-
constructs that underpin DF performance have been summa-
cal settings. Examinees draw as many different designs as
rized in the RFFT: Ruff Figural Fluency Test Professional
possible in one minute, while avoiding repeating prior de-
Manual (Ruff, 1996). They suggest that, based on factor-
signs. There are several versions of DF tests, most of which
analytic findings (Baser & Ruff, 1987), DF is a measure of
require that designs be drawn by connecting dots in a series
“initiation, planning, and divergent reasoning” (p. 15), and
of five-dot matrices (see Table 1 for a review of common DF
that poor DF performance cannot be explained by language,
tests). Given the general trend in clinical neuropsychology
memory, or motor deficits (Ruff, Evans, & Marshall 1986).
toward interpreting test results in terms of cognitive constructs,
Beyond these empirical findings, the Delis-Kaplan Execu-
rather than lesion locations, understanding the cognitive
tive Function System (D-KEFS) version of DF (Delis et al.,
underpinnings of DF performance is important. However,
2001) has been described as “fluency in generating visual
construct validation studies of DF have almost entirely relied
patterns” (p. 88), likely based on the self-evident fact that
on examining the neuroanatomic substrates of DF perfor-
“visual patterns” are the output of DF performance.
mance (Baldo, Shimamura, Delis, Kramer, & Kaplan, 2001;
To examine the assertions that DF is a measure of (a)
Butler, Rorsman, Hill, & Tuma, 1993; Elfgren & Risberg,
planning/initiation, (b) cognitive flexibility/divergent think-
1998; Fama et al., 2000; Kramer et al., 2007; Suchy, Sands, &
ing, and (c) fluency in generating visual patterns, we recently
conducted a study (Kraybill & Suchy, 2008) in which we
used the Ruff Figural Fluency Test (RFFT; Ruff, 1998) as the
Correspondence and reprint requests to: Yana Suchy, Ph.D., Department
of Psychology, University of Utah, 380 S. 1530 E., Rm. 502, Salt Lake City, dependent variable, and several motor and executive mea-
UT 84112-0251. E-mail: [email protected] sures as predictors. Two of the motor variables used in that
26
Understanding design fluency 27

Table 1. Design Fluency tests overview

Design Fluency Number of Time per Background Number of

Measure Trials Trial Matrix Lines Distractors Notes
Design Fluency Two 4 min None Any (Trial 1) None n/a
Exactly four (Trial 2)
Five-Point Test One 5 min Five dots Any None n/a
Ruff Figural Fluency Test Five 1 min Five dots Any Trials 2 and 3 Trials 4 and 5 have
asymmetrically
positioned dots.
D-KEFS Design Fluency Three 1 min Five dots Exactly four Trials 2 and 3 Third trial requires
switching between
filled and empty circles

Note. Design Fluency (Jones-Gotman & Milner, 1977); Five-Point Test (Regard, Strauss, & Knapp, 1982); Ruff Figural fluency Test (Ruff, 1998), and
D-KEFS Design Fluency (Delis, Kaplan, & Kramer, 2001).

study proved to be particularly useful in examining the console (Figure 1) and the same hand movements as the
construct of DF. These were “motor planning” and “motor Push-Turn-Taptap task used for assessment of motor planning,
fluency.” and is described in more detail in Figure 2b.
In a series of hierarchical regressions designed to parse
Motor planning refers to the internal strategy that pre- out unique and shared variances among the variables, the
cedes an intended movement (Keele, 1981), and presumably DF number of unique designs was predicted by (a) Trail
contains both general information about the intended goal Making Test-Part B (TMT-B), presumably reflecting cogni-
and specific information about the neuromuscular control tive flexibility, and (b) the two motor variables described
that will be required (Keele, 1981). It was operationalized as earlier, motor planning and motor fluency. Together, these
the latencies prior to initiation of correctly executed sequences findings supported the prevailing conceptualization of DF as
of specified hand movements using the Push-Turn-Taptap a test of cognitive flexibility and planning (even if just planning
task from the Behavioral Dyscontrol Scale–Electronic Version of motor actions) (Delis et al., 2001; Ruff, 1998). Additionally,
battery (BDS-EV; Suchy, Derbidge, & Cope, 2005). In par- these findings introduced the notion that DF may rely on
ticular, participants learned four different sequences (or per- fluency in generating motor actions, in addition to “fluency
mutations) of three specified hand movements performed in generating visual patterns” (Delis, 2001; p. 88). Finally,
on a specialized response console (Figure 1). The task is de- consistent with prior findings (Ruff et al., 1986), motor
scribed in more detail in Figure 2a. speed, assessed via finger tapping, did not contribute to
performance.
Motor fluency refers to the ability to generate novel However, several questions remained. First, it was not
sequences of motor actions. It was operationalized as the clear whether the association between DF and TMT-B was
total number of unique permutations of three specified hand a reflection of shared cognitive flexibility, or a reflection of
movements produced on the BDS-EV Motor Sequence Fluency shared component skills (i.e., visual scanning, motor speed,
test. The Motor Sequence Fluency test uses the same response sequencing). The latter explanation would be consistent with
the assertion of Delis and colleagues (2001, p. 89) that the
Visual Scanning and Motor Speed conditions of the D-KEFS
version of trail making can be used to assess component
skills in DF performance.
Second, although we replicated previous findings that fin-
ger tapping speed was not related to DF performance (Ruff
et al., 1986), it was still possible that graphomotor skills
(i.e., the successful wielding of a writing implement) may
contribute to the number of generated designs, as suggested
by Delis and colleagues (2001, p. 89).
Third, while our prior study found that performance on
the Motor Sequence Fluency test uniquely and significantly
contributed to DF performance, it was not clear whether this
Fig. 1. The figure depicts the Behavioral Dyscontrol Scale– relationship was due to some general fluency ability (i.e.,
Electronic Version (BDS-EV) response console. It was used in one that would be shared by all fluency measures, both ver-
the present study to assess Motor Planning and Motor Sequence bal and nonverbal), or whether it was specific to fluency in
Fluency. the motor domain.
28 Y. Suchy et al.

Fig. 2a. The BDS-EV Push-Turn-Taptap task requires that participants learn different sequences (or permutations) of
three specified hand movements, using a specialized response console (Figure 1). The three hand movements are “Push” –
pushing the joystick forward; “Turn” – turning the joystick clockwise; and “Taptap” – double-tapping on the white dome
of the response console. The task begins by presenting a two-movement sequence on the computer screen, until three
correct trials are completed. Following these three learning trials, participants continue to perform the sequence from
memory, until accomplishing five additional correct trials. After completing the five correct trials, a new sequence is
presented on the screen, and the just described process is repeated. There is a total of four different progressively longer
sequences that participants learn in the course of this task (only the first two sequences are presented in this figure).
Mistakes are followed by an audible tone, along with a presentation of the correct sequence on the screen. Motor Planning
(M-PLN) latencies are indicated in the figure by the thick black vertical arrows, and reflect the preparation time before
initiation of each trial. Only latencies preceeding correctly executed trials are included.

The purpose of the present study was to address these is- Institutional Review Board-approved informed consent pro-
sues by replicating and extending our prior research. To that cedures and the APA ethical guidelines were followed. See
end, we administered a battery of cognitive and motor tasks Table 2 for participant characteristics.
to a sample of community-dwelling elderly. To tease apart
cognitive flexibility and component skills, the present study
Instruments
employed the D-KEFS version of trail making (Delis et al.,
2001), which includes not only alpha-numeric sequencing as Delis-Kaplan Executive Function System (D-KEFS; Delis
a measure of cognitive flexibility, but also carefully designed et al., 2001). For all tests, standard administration and
control tasks that assess visual scanning, sequencing, and scoring procedures were followed unless otherwise noted.
graphomotor speed. Additionally, to address the question Untransformed raw scores were used as variables in
of whether DF is related to general fluency abilities (vs. analyses. The following three tasks from this battery were
a specific fluency in generating motor sequences, as assessed used:
via the Motor Sequence Fluency test), we included the D-KEFS The D-KEFS Design Fluency (DF) test consists of three
Letter Fluency test in this study. Lastly, given that adequate trials in which participants create novel designs by connect-
construct validation requires a multi-method approach, the ing dots in a series of five-dot matrices. The three conditions
present study employed the D-KEFS version of DF, as opposed are referred to as Filled Dots (connecting filled dots), Empty
to the RFFT used in our prior study (Ruff, 1998). Dots (connecting empty dots while filled dots function
as distractors), and Switch (switching between connecting
filled and empty dots). The first two conditions are similar
METHOD
to the Ruff version of DF in that they assess DF both with
and without visual distractors, whereas the third condition
Participants
includes switching, which is not part of the Ruff version of
Participants were 61 right-handed, independently function- the test. In the present study, the first two conditions and the
ing community-dwelling elderly (62 % female). They were third condition were examined separately, resulting in two
recruited from the community via advertisements and re- DF variables: (a) The “Non-switch DF” score, comprised of
ceived $10.00 an hour for their time. University of Utah the sum of correct unique designs generated during the first
Understanding design fluency 29

Fig. 2b. This figure depicts a hypothetical performance of a participant on the first two blocks of the Motor Sequence
Fluency test. Participants are asked to generate as many unique sequences of three previously learned hand movements
as they can. The movements are push, turn, and tap-tap, performed on the BDS-EV response console depicted in
Figure 1. The test is organized into three blocks of progressively longer sequences, with the first block requiring that
exactly three hand movements be strung together to create unique sequences, and the subsequent two blocks requiring
that exactly four and five hand movements be strung together into sequences. Repetition of the same movements
within a given sequence is permitted. However, repetition of the exact same sequence within a block constitutes an
error. Only correct (i.e., unique) sequences are counted to generate the final score. The allotted time is individually
determined based on participants’ performance speed on the Push-Turn-Taptap task administered just prior to the
Motor Sequence Fluency test.

two conditions; and (b) the “Switch DF” score, comprised of respectively. Number-Letter Switching requires connecting
the total number of correct unique designs generated during circles in an alternating alphanumeric sequence. Motor Speed
the “switch” condition. consists of circles that are already connected by a line, and
The D-KEFS Trail Making Test consists of five conditions: participants use a pencil to trace the line as fast as they can.
Visual Scanning, Number Sequencing, Letter Sequencing, The D-KEFS Trail Making Test includes these five condi-
Number-Letter Switching, and Motor Speed. All five condi- tions so as to allow assessment of the speed of cognitive
tions were administered. For all conditions, participants are switching after visual, motor, and sequencing speeds have
asked to work as quickly as they can, and time to completion been accounted for.
(in seconds) represents their raw score. The first four condi- For the purpose of this study, to isolate cognitive flexibil-
tions present the examinee with a page of pseudo-randomly ity, we computed D-KEFS Number-Letter Switching residu-
arranged circles containing letters, numbers, or both. Visual als, after controlling for D-KEFS Visual Scanning, Motor
Scanning requires crossing-out circles that contain a particu- Speed, and both Sequencing Speeds. Additionally, D-KEFS
lar number. Number and Letter Sequencing require connect- Visual Scanning and Motor Speed were examined in analy-
ing circles in the correct numerical or alphabetical order, ses as potential component processes of DF, as was previ-
ously recommended by Delis and colleagues (2001). Note
that hereafter, the D-KEFS Motor Speed will be referred to
Table 2. Characteristics of the sample as the D-KEFS Graphomotor Speed, so as to differentiate
it from motor speed assessed in our prior study via finger
Mean Standard Deviation Range tapping.
Age 70.41 6.79 60–87 The D-KEFS Letter Fluency Test requires that participants
Education 14.51 2.72 10–22 generate as many words as they can that begin with a specific
M-DRS raw scores 137.92 5.34 117–144 letter. The sum of the number of correct responses generated
M-DRS scaled scores 10.93 2.47 5–15 for three different letters (one minute each) represented the
raw score used in analyses.
Note. N = 61. M-DRS = Mattis Dementia Rating Scale. Two participants
(3.3% of the sample) had M-DRS scaled scores of 5 (moderately impaired Behavioral Dyscontrol Scale-Electronic Version (BDS-
range), and two had scores of 6 and 7 (mildly impaired). EV; Suchy et al., 2005). The BDS-EV is an electronically
30 Y. Suchy et al.

administered battery of motor and executive tasks with Statistical Analyses

excellent reliability (Suchy et al., 2005) and promising
All principal analyses in this study employed hierarchical
construct validity (Kraybill et al., 2009; Suchy et al., 2005;
regressions, using DF as the criterion variable. As described
Suchy & Kraybill, 2007). Two tasks from the battery (i.e.,
in the method section, the “Non-switch” and the “Switch”
the Push-Turn-Taptap task and the Motor Sequence Fluency
DF conditions were examined separately.
test) were used in this study and are described next.
The Push-Turn-Taptap task generates variables that reflect
Predictor variables were selected and entered based on
various components of motor programming. However, for
(1) the hierarchical model of cognition, according to which
the purpose of this study, only the BDS-EV Motor Planning
lower-order component processes (i.e., perceptual and motor
(M-PLN) variable was used, as it has been previously found
functions) are governed by, and contribute to, higher-order
to share unique variance with the number of unique designs
control processes (i.e., executive functions) (Stuss, Alexander,
generated during DF performance (Kraybill & Suchy, 2008).
Benson, Trimble, & Cummings, 1997), and (2) the assertions
The M-PLN variable has been shown to have a strong relation-
by Delis and colleagues regarding the nature of component
ship with executive functions (Kraybill, Suchy, & Franchow,
processes presumed to contribute to DF performance (Delis
2009; Suchy & Kraybill, 2007). As seen in Figure 2a, the
et al., 2001; pp. 88–89).
Push-Turn-Taptap task generates the M-PLN variable by
Following this rationale, we first parsed out variance
measuring the amount of time (measured in milliseconds)
contributions from discrete component skills (i.e., D-KEFS
that elapses between the completion of one repetition
Visual Scanning and Graphomotor Speed). Next, we examined
(or trial) of a given sequence and the initiation of the next
contributions from executive processes not deemed specific
correct repetition of that same sequence. Because there are
to generative fluency (i.e., D-KEFS Number-Letter Switch-
four different sequences, and each sequence must be exe-
ing residuals and the M-PLN variable from the BDS-EV).
cuted correctly eight times (i.e., three with computer prompts
Lastly, we examined executive processes deemed specific
and five independently; see Figure 2a) before moving on to
to fluency performance (i.e., the BDS-EV Motor Sequence
the next sequence, the M-PLN variable is based on a total of
Fluency and the D-KEFS Letter Fluency). All analyses were
32 planning latency observations.
conducted in sets of two, reversing the order of variable en-
The BDS-EV Motor Sequence Fluency test was adminis-
tries at consecutive steps, so as to allow partialing of unique
tered immediately following the Push-Turn-Taptap task,
versus shared variance contributions.
using the same response console and the same three hand
movements. Participants were asked to produce as many
unique sequences (i.e., permutations) of the three hand RESULTS
movements as they could within an allotted amount of time.
A hypothetical performance is depicted in Figure 2b. Preliminary Analyses
Because the DF test gives all participants the same amount Descriptive statistics and zero-order correlations for all de-
of time (1 minute per condition), whereas the Motor Sequence pendent and independent variables are presented in Tables 3
Fluency test automatically adjusts for participants’ perfor- and 4, respectively.
mance speed based on their performance speed on the
Push-Turn-Taptap task, the two tasks needed to be placed on Cognitive structure of the Non-switch Design Fluency
a comparable time allotment metric. This was done by com-
(DF) condition
puting an unstandardized residual score of the total number
of generated unique motor sequences, after controlling for Contribution of D-KEFS Graphomotor Speed and Visual
Push-Turn-Taptap performance speed. This score was used Scanning. D-KEFS Graphomotor Speed and Visual Scan-
in analyses. ning were entered on Steps 1 and 2, respectively, as well as

Table 3. Descriptive statistics for dependent and independent variables

Variable Type Variable Mean Median SD Range

DV Non-switch DF 19.23 19.00 5.54 8–33
Switch DF 6.51 7.00 2.69 1–13
IV D-KEFS Motor Speed 29.69 28.00 11.95 15–76
D-KEFS Visual Scanning 27.44 25.00 10.64 16–90
D-KEFS NLS residuals 112.21 95.00 52.02 40–240
D-KEFS Letter Fluency 80.08 79.00 20.32 39–142
BDS-EV Motor Planning 1044.19 1063.00 302.52 486.25–2170.5
BDS-EV MSF 19.97 18.00 9.27 6–63

Note. N = 61. DV = Dependent variable; IV = Independent variable; DF = Design Fluency; D-KEFS = Delis-Kaplan Executive Function
System; BDS-EV = Behavioral Dyscontrol Scale−Electronic Version; NLS residuals = Number-Letter Switching residuals; MSF =
Motor Sequence Fluency.
Understanding design fluency 31

Table 4. Zero order correlations among dependent and motor Speed on Step 1, and D-KEFS Number-Letter Switch-
independent variables ing residuals and the BDS-EV M-PLN variable on Steps 2
and 3, respectively, as well as in reversed order (i.e., M-PLN
Dependent Variables Non-Switch DF Switch DF
and Number-Letter Switching residuals on Steps 2 and 3,
Age −.295* −.289* respectively). See Table 6 for summary of analyses (unique
Education .061 .128 variance is italicized). Taken together, the results showed
Sex .303* .212 that the M-PLN variable uniquely contributed 11.9% of vari-
D-KEFS Motor Speed −.319* −.259* ance to Non-switch DF, with approximately an additional
D-KEFS Visual Scanning −.340** −.439** 4.2% of variance shared between M-PLN and D-KEFS
D-KEFS Number-Letter −.245 −.234
Number-Letter Switching residuals. Unique contribution
Switching residuals
BDS-EV Motor Planning −.547** −.416**
from D-KEFS Number-Letter Switching residuals was small
D-KEFS Letter Fluency .423** .230 (1.7%) and did not reach statistical significance. For that rea-
BDS-EV Motor Sequence .254* .030 son, M-PLN was the only variable added to the model in the
Fluency next set of analyses.

Note. N = 61. * p < .05; ** p < .01; Sex: 1 = female, 2 = male; DF = Design
Fluency; D-KEFS = Delis-Kaplan Executive Function System; BDS-EV =
Contribution of BDS-EV Motor Sequence Fluency and
Behavioral Dyscontrol Scale−Electronic Version. D-KEFS Letter Fluency. We first entered the three sig-
nificant predictors from previous analyses (i.e., Grapho-
motor Speed, Visual Scanning, and M-PLN) on Step 1.
in the reversed order (i.e., Visual Scanning and Grapho- Next, to determine whether the ability to generate novel
motor Speed on Steps 1 and 2, respectively). Summaries designs depended on the ability to generate novel motor
of analyses are presented in Table 5, with the italicized sequences versus the ability to simply be generative in any
numbers in the R2 Δ column reflecting unique variance domain (even verbal), we entered Motor Sequence Flu-
contributed by each variable. The results showed that Visual ency and Letter Fluency on Steps 2 and 3, respectively, as
Scanning and Graphomotor Speed together yielded a sta- well as in the reversed order (i.e., Letter Fluency on Step 2
tistically significant model, F (2, 58) = 5.18, p = .008, that and Motor Sequence Fluency on Step 3). See Table 7. Taken
accounted for 15.2% of variance in Non-switch DF (see together, the above analyses revealed that Motor Sequence
column R2 in Table 5, last Step in either analysis) with Fluency and Letter Fluency each uniquely contributed 7.0%
approximately 6.6% of Non-switch DF variance shared be- and 2.8% of variance, respectively, with virtually no overlap
tween Graphomotor Speed and Visual Scanning (i.e., total in variance (i.e., 1.2%). Only the contribution from Motor
variance minus unique variance, or .152 minus the sum of Sequence Fluency reached statistical significance, suggest-
the italicized numbers in the R2 Δ column). However, Visual ing that it was a better predictor of Non-switch DF than was
Scanning and Graphomotor Speed contributed insignificant Letter Fluency.
amounts of unique variance (see p values in Steps 2 in the The final model then included D-KEFS Visual Scanning
Table). Because of the significant contribution of the two and Graphomotor Speed, and BDS-EV M-PLN and Motor
variables together, both were entered on Step 1 of subse- Sequence Fluency, which together accounted for 39.5% of
quent analyses. variance, F (4, 56) = 9.15, p < .001.

Contribution of D-KEFS Number-Letter Switching residu- Cognitive structure of the Switch Design Fluency (DF)
als and BDS-EV Motor Planning (M-PLN). To determine
condition
whether the D-KEFS Number-Letter Switching residuals
and/or the BDS-EV M-PLN contributed to performance The statistical approach employed with the Non-switch DF
above and beyond the D-KEFS Visual Scanning and Graph- (i.e., the order of variable entry, computation of unique vs.
omotor Speed, we conducted another set of two hierarchical shared variance, etc.) was also employed when using the
regressions, entering D-KEFS Visual Scanning and Grapho- Switch DF as the criterion variable.

Table 5. Summary of hierarchical regressions reflecting variance contributions of Motor Speed and Visual Scanning to
the number of Unique Designs generated in the first two trials of D-KEFS Design Fluency test

Step Predictors R2 Adjusted R2 R2 Δ FΔ df p value

1 D-KEFS Motor Speed .102 .087 — 6.70 1,59 .012
2 D-KEFS Visual Scanning .152 .122 .050 3.39 1,58 .071
1 D-KEFS Visual Scanning .116 .101 — 7.73 1,59 .007
2 D-KEFS Motor Speed .152 .122 .036 2.44 1,58 .123

Note. Italicized numbers in the R2 Δ column reflect unique variance contributions. df = degrees of freedom. The Step 1 values in the F Δ
column in reality represent F, not F Δ, as they are the values for the base model. D-KEFS = Delis-Kaplan Executive Function System.
32 Y. Suchy et al.

Table 6. Summary of hierarchical regression reflecting variance contributions of Visual Scanning, Motor Speed,
Cognitive Flexibility, and Motor Planningto the number of Unique Designs generated in the first two trials of the
D-KEFS Design Fluency test.

Step Predictors R2 Adjusted R2 R2 Δ FΔ df p value

1 D-KEFS Visual Scanning .152 .122 — 5.18 1,58 .008
D-KEFS Motor Speed
2 D-KEFS NLS residuals .212 .170 .060 4.35 1,57 .042
3 BDS-EV M-PLN .330 .283 .119 9.93 1,56 .003
2 BDS-EV M-PLN .313 .277 .162 13.44 1,57 .001
3 D-KEFS NLS residuals .330 .283 .017 1.42 1,56 .238

Note. Italicized numbers in the R2 Δ column reflect unique variance contributions. df = degrees of freedom. The Step 1 values in the F Δ
column in reality represent F, not F Δ, as they are the values for the base model. D-KEFS = Delis-Kaplan Executive Function System;
BDS-EV = Behavioral Dyscontrol Scale−Electronic Version; M-PLN = Motor Planning; NLS residuals = Number-Letter Switching
residuals.

Contribution of D-KEFS Graphomotor Speed and Visual and thus were both added to the model in the next set of
Scanning. See Table 8 for a summary of the results. As can analyses.
be seen, in contrast to Non-switch DF, Switch DF relied
more on Visual Scanning, which contributed 13.1% of unique Contribution of BDS-EV Motor Sequence Fluency and
variance and an additional 6.7% of variance that was shared D-KEFS Letter Fluency. The next set of analyses was
with Graphomotor Speed. This finding likely reflected the conducted with D-KEFS Visual Scanning, BDS-EV M-PLN,
need to alternate between searching for filled and empty and D-KEFS Number-Letter Switching residuals entered
dots. Graphomotor Speed did not contribute unique variance together on Step 1, and BDS-EV Motor Sequence Fluency
to the model. Based on these results, we entered the D-KEFS and D-KEFS Letter Fluency entered on Steps 2 and 3, and
Visual Scanning as the only predictor on Step 1 in the subse- vice versa (Table 10). The results showed that, in contrast to
quent set of analyses. Non-switch DF, neither Motor Sequence Fluency nor Letter
Fluency contributed further to the model. The final model
Contribution of D-KEFS Number-Letter Switching residu- then included D-KEFS Visual Scanning, D-KEFS Number-
als and the BDS-EV Motor Planning (M-PLN). The next Letter Switching residuals, and BDS-EV M-PLN, which
set of analyses was conducted with D-KEFS Visual Scan- together accounted for 28.3% of variance (Table 10, Step 1),
ning entered on Step 1, and D-KEFS Number-Letter Switch- suggesting that the Switch DF relied less on fluency than the
ing residuals and BDS-EV M-PLN entered on Steps 2 and 3 Non-switch DF did.
and vice versa (Table 9). The results showed that Number-
Letter Switching residuals and M-PLN uniquely contributed
Effects of demographic variables
approximately 2.9% and 3.6% of variance, with approxi-
mately an additional 2.5% of variance shared between them, As was done in our previous study, we examined whether
for a total of 9.0% of variance above and beyond D-KEFS demographic variables could account for the findings. Thus,
Visual Scanning. Although neither variable contributed we ran the two final models (one each for Non-switch DF
significantly alone, they contributed significantly together, and Switch DF), entering age, education, and sex on Step 1,

Table 7. Summary of hierarchical regression reflecting variance contributions of Visual Scanning, Motor Speed,
Cognitive Flexibility, Letter Fluency, and Motor Sequence Fluency to the number of Unique Designs generated in the
first two trials of the D-KEFS Design Fluency test

Step Predictors R2 Adjusted R2 R2 Δ FΔ df p value

1 D-KEFS Motor Speed .313 .277 — 8.68 3,57 <.001
D-KEFS Visual Scanning
BDS-EV M-PLN
2 D-KEFS Letter Fluency .354 .307 .040 3.47 1,56 .068
3 BDS-EV MSF .423 .371 .070 6.65 1,55 .013
2 BDS-EV MSF .395 .352 .082 7.57 1,56 .008
3 D-KEFS Letter Fluency .423 .371 .028 2.66 1,55 .108

Note. Italicized numbers in the R2 Δ column reflect unique variance contributions. df = degrees of freedom. The Step 1 values in the F Δ
column in reality represent F, not F Δ, as they are the values for the base model. D-KEFS = Delis-Kaplan Executive Function System;
BDS-EV = Behavioral Dyscontrol Scale−Electronic Version; M-PLN = Motor Planning; MSF = Motor Sequence Fluency.
Understanding design fluency 33

Table 8. Summary of hierarchical regressions reflecting variance contributions of Motor Speed and Visual Scanning to
the number of Unique Designs generated in the Switch condition of D-KEFS Design Fluency test

Step Predictors R2 Adjusted R2 R2 Δ FΔ df p value

1 D-KEFS Motor Speed .067 .051 — 4.25 1,59 .044
2 D-KEFS Visual Scanning .198 .170 .131 9.46 1,58 .003
1 D-KEFS Visual Scanning .193 .179 — 14.07 1,59 <.001
2 D-KEFS Motor Speed .198 .170 .005 .40 1,58 .531

Note. Italicized numbers in the R2 Δ column reflect unique variance contributions. df = degrees of freedom. The Step 1 values in the F Δ
column in reality represent F, not F Δ, as they are the values for the base model. D-KEFS = Delis-Kaplan Executive Function System.

and all the significant predictors identified in previous analy- and Trial 2 (Filled and Empty dots) separately. The results
ses on Step 2. The results showed that the previously identi- showed that, as suspected, Trial 2 relied more heavily than
fied predictors together accounted for 25.4% of variance in Trial 1 on D-KEFS Visual Scanning. In particular, Visual
Non-switch DF above and beyond demographics, F change Scanning contributed 5.4% of unique variance to Trial 2, as
(4, 53) = 5.83, p = .001, and 18.4% of variance in Switch DF compared to 3.4% of variance to Trial 1. Graphomotor Speed
above and beyond demographics, F change (3, 54) = 4.73, contributed equally to Trials 1 and 2 (3.1% and 3.0%, re-
p = .005. A summary of these models appears in Table 11. spectively). These findings support the interpretation that the
Finally, while hierarchical regressions allow parsing out need to separate filled and empty dots may in part explain
of unique variance contributions, they do not allow determi- the increased Visual Scanning demands in the Switch DF.
nation of the most parsimonious model (i.e., variables that
are the best overall predictors). To provide that information,
DISCUSSION
we present simple linear regressions in Table 12. As can be
seen, BDS-EV Motor Sequence Fluency and M-PLN repre- Measures of Design Fluency (DF) represent a common com-
sented the two most prominent predictors for Non-switch ponent of neuropsychological batteries, both in clinical and
DF, as was the case in our prior study. In contrast, D-KEFS in research settings. Although they are generally assumed
Visual Scanning represented the single best predictor for the to assess executive functions (primarily planning, cognitive
Switch DF condition. flexibility, and fluency in generating visual patterns; Delis
et al., 2001; Ruff, 1998), above and beyond motor speed
(Ruff et al., 1986), relatively little empirical research has
Supplementary analyses
tested these assumptions.
Our findings demonstrated that the Switch DF relied much In a recent construct validation study we examined these
more heavily on Visual Scanning than Non-switch DF did, assumptions, finding that DF was related to: (a) BDS-EV
presumably because of the need to visually separate empty Motor Planning (M-PLN), (b) TMT-B, presumed to measure
and filled dots. However, the Non-switch DF itself contained cognitive flexibility, and (c) BDS-EV Motor Sequence
data from two trials, one with filled dots only, and one with Fluency test assessing the ability to generate novel motor
filled and empty dots (filled dots serving as distracters). To sequence without generation of visual patterns. Additionally,
see whether this apparently slight difference reliably affected DF was unrelated to finger-tapping speed. However, ques-
the degree to which Visual Scanning contributed to perfor- tions remained.
mance, we examined the contributions of D-KEFS Visual First, while motor speed assessed via finger-tapping
Scanning and Graphomotor Speed to Trial 1 (Filled dots) appeared unrelated to DF performance, Delis and colleagues

Table 9. Summary of hierarchical regression reflecting contributions of Visual Scanning, Cognitive Flexibility,
and Motor Planning to the number of Unique Designs generated in the Switch condition of the D-KEFS Design
Fluency test

Step Predictors R2 Adjusted R2 R2 Δ FΔ df p value

1 D-KEFS Visual Scanning .193 .179 — 14.72 1,59 <.001
2 D-KEFS NLS residuals .247 .221 .055 .21 1,58 .045
3 BDS-EV M-PLN .283 .245 .036 2.85 1,57 .097
2 BDS-EV M-PLN .254 .228 .061 4.779 1,58 .033
3 D-KEFS NLS residuals .283 .245 .029 2.30 1,57 .135

Note. Italicized numbers in the R2 Δ column reflect unique variance contributions. df = degrees of freedom. The Step 1 values in the F Δ
column in reality represent F, not F Δ, as they are the values for the base model. D-KEFS = Delis-Kaplan Executive Function System;
BDS-EV = Behavioral Dyscontrol Scale−Electronic Version; M-PLN = Motor Planning; NLS residuals = Number-Letter Switching
residuals.
34 Y. Suchy et al.

Table 10. Summary of hierarchical regression reflecting variance contributions of Visual Scanning, Cognitive
Flexibility, Letter Fluency, and Motor Sequence Fluency to the number of Unique Designs generated in the Switch
condition of the D-KEFS Design Fluency test

Step Predictors R2 Adjusted R2 R2 Δ FΔ df p value

1 D-KEFS Visual Scanning .283 .245 — 7.50 3,57 <.001
BDS-EV M-PLN
D-KEFS NLS residuals
2 D-KEFS Letter Fluency .286 .235 .003 .220 1,56 .641
3 BDS-EV MSF .286 .221 .000 .000 1,55 .995
2 BDS-EV MSF .283 .232 .000 .002 1,56 .969
3 D-KEFS Letter Fluency .286 .221 .003 .215 1,55 .645

Note. Italicized numbers in the R2 Δ column reflect unique variance contributions. df = degrees of freedom. The Step 1 values in the F Δ
column in reality represent F, not F Δ, as they are the values for the base model. D-KEFS = Delis-Kaplan Executive Function System;
BDS-EV = Behavioral Dyscontrol Scale−Electronic Version; M-PLN = Motor Planning; MSF = Motor Sequence Fluency; NLS
residuals = Number-Letter Switching residuals.

(2001) nevertheless suggested that performance can be none of the three D-KEFS DF trials were substantially related
affected by motor speed “in drawing” (p. 89); thus, they to cognitive flexibility, as assessed with D-KEFS Number-
recommended that D-KEFS Graphomotor Speed task (which Letter Switching residuals. D-KEFS Visual Scanning ap-
assesses motor speed in drawing) be examined as a potential peared related primarily to increases in the visual-attentional
component skill of DF. Second, it was not clear from our complexity of the task (i.e., addition of distractors), rather
prior study whether the association between DF and TMT-B than generation of unique designs. Interestingly, Visual Scan-
reflected a common variance in cognitive flexibility, grapho- ning represented the only significant predictor of D-KEFS
motor speed, or visual scanning. And third, although we had DF Switch condition.
found that BDS-EV Motor Sequence Fluency test contributed
to DF performance, it was not clear whether this reflected
Component Skills: Graphomotor and Visual
motor fluency in particular, or fluency in general. The present
study sought to address these questions.
Scanning Speeds
The key findings of the present study were that generation As suggested by Delis and colleagues (2001), D-KEFS
of unique designs (i.e., the first two trials of D-KEFS DF) Graphomotor Speed and Visual Scanning tasks both ap-
relied in part on: (a) D-KEFS Graphomotor Speed (in con- peared to tap into component skills required for performance
trast to motor speed assessed via finger-tapping), (b) BDS-EV of Non-switch DF tests. Because our prior study failed to
Motor Sequence Fluency test (as opposed to fluency in general), find a relationship between DF and motor speed (assessed via
and (c) BDS-EV M-PLN. In contrast to previous assumptions, finger-tapping), it appears likely that the speed of wielding a

Table 11. Summary of final hierarchical regression models for the number of unique designs in the D-KEFS Design
Fluency test

Criterion Variable Step Predictors Adjusted R2 R2 Δ FΔ p value

Non-Switch DF 1 Sex .097 .142 3.15 .032
Education
Age
2 D-KEFS Motor Speed .142 .071 2.50 .091
D-KEFS Visual Scanning
3 BDS-EV M-PLN .334 .120 9.71 .003
4 BDS-EV MSF .396 .063 5.50 .023
Switch DF 1 Sex .066 .113 2.43 .075
Education
Age
2 D-KEFS Visual Scanning .194 .134 9.98 .003
3 D-KEFS NLS residuals .220 .050 1.93 .155
BDS-EV M-PLN

Note. N = 61. M-PLN = Motor Planning; MSF = Motor Sequence Fluency. The Step 1 values in the F Δ column in reality represent
F, not F Δ, as they are the values for the base model. D-KEFS = Delis-Kaplan Executive Function System; BDS-EV = Behavioral
Dyscontrol Scale−Electronic Version; M-PLN = Motor Planning; MSF = Motor Sequence Fluency; NLS residuals = Number-Letter
Switching residuals.
Understanding design fluency 35

Table 12. Summary of final linear regression models for the Switching (after controlling for D-KEFS Graphomotor,
number of unique designs in the D-KEFS Design Fluency test Visual Scanning, and Sequencing Speeds), failed to account
for a meaningful amount of variance in the first two D-KEFS
Criterion Variable Predictor Beta t p value
DF conditions (i.e., less than 2%, see Table 6). This finding
Non-Switch DF Constant 2.20 .032 clarified that the association between cognitive flexibility
Sex 1.205 .86 .393 and DF seen in our prior study was simply a function of
Education .087 .36 .718 the association between DF and component skills. Although
Age .048 .45 .653 contributions of flexibility increased somewhat when the
D-KEFS Visual Scanning −.054 .77 .443 switching demands were added to the task, the unique vari-
D-KEFS Motor Speed −.013 .21 .833
ance accounted for was still quite minimal (about 3%, Table 9).
BDS-EV M-PLN −.009 3.65 .001
BDS-EV MSF .163 2.35 .023
Together, these findings suggest that planning of a motor
Switch DF Constant 2.693 .009 response (assessed via BDS-EV M-PLN), but not cognitive
Sex −.120 .192 .849 flexibility, may represent one of the key executive processes
Education .062 .366 .716 assessed by DF.
Age −.046 .872 .387 “Motor planning,” the type of planning assessed by the
D-KEFS Visual Scanning −.275 2.019 .016 BDS-EV M-PLN variable, is a covert aspect of complex
D-KEFS NLS residuals −.292 2.181 .177 motor output that can be thought of as the process of prepara-
BDS-EV M-PLN −.001 1.057 .295 tion for a movement prior to the movement initiation (Keele,
1981). Thus, it reflects planning at the level of immediate
Note. N = 61. D-KEFS = Delis-Kaplan Executive Function System; BDS-
EV = Behavioral Dyscontrol Scale−Electronic Version; M-PLN = Motor motor action. Although M-PLN latencies have been shown
Planning; MSF = Motor Sequence Fluency; NLS residuals = Number-Letter to be related to executive abilities (Suchy & Kraybill, 2007),
Switching residuals. it is not clear whether they are related to the more purely
cognitive planning skills that do not involve immediate
motor output.
writing implement is what contributed to DF performance.
However, the contribution of D-KEFS Graphomotor Speed
Fluency: Specific versus General Construct
to performance was relatively small, with less than 5% of
unique variance accounted for. Thus, graphomotor abilities The present findings replicated our prior results (Kraybill &
likely have minimal impact on performance, except among Suchy, 2008), again demonstrating that the Motor Sequence
individuals with considerable graphomotor deficits. Fluency test was a unique predictor (Table 7) of the number of
In addition to Graphomotor Speed, D-KEFS Visual Scanning generated designs (i.e., the first two conditions of D-KEFS DF),
also contributed to test performance, and this contribution even after demographics and other aspects of motor perfor-
appeared to be a function of increases in the complexity of mance (i.e., M-PLN, Graphomotor Speed) have been accounted
the visual and attentional demands of different DF conditions. for (Table 11). Additionally, together with BDS-EV M-PLN,
In particular, Visual Scanning accounted for 3.4% of unique the BDS-EV Motor Sequence Fluency test represented one
variance on Trial 1 (i.e., filled dots only), 5.4% on Trial 2 of the two strongest predictors of the Non-switch DF overall
(i.e., filled dots functioning as distracters), and 13% on Trial 3 (Table 12). Incidentally, these same two variables emerged
(i.e., switching between empty and filled dots). Moreover, as as the two strongest predictors of DF in our prior study as
can be seen in Table 12, Visual Scanning represented the best well (Kraybill & Suchy, 2008).
single predictor of the Switch DF condition, accounting for Moreover, the results confirmed that specifically motor
nearly 20% of variance overall (Table 9). These findings fluency (i.e., the ability to generate novel motor actions),
suggest that the D-KEFS Visual Scanning contribution taps not fluency in general, predicted DF performance. This was
into the skills needed for separation of relevant and irrele- demonstrated by the lack of association between Letter
vant visual stimuli: That is, the filled and empty dots on the Fluency and DF. In particular, if the ability to generate novel
D-KEFS version of DF (used in this study), and, likely, dots designs relied on some general fluency ability (a construct
and distractors in the latter trials of the RFFT (Ruff, 1996) that is sometimes invoked as a componential aspect of
version of DF. executive abilities), then this general ability would have to
share variance with all three fluency tasks. However, that
was not the case.
Executive Functions: Planning and Flexibility
As was the case in our prior study, BDS-EV M-PLN again
Switch DF Condition
emerged as a powerful predictor of the ability to generate
novel designs (i.e., the first two conditions of D-KEFS DF), The results showed that by adding the Switch condition, the
accounting for 12% of unique variance above and beyond cognitive structure of the D-KEFS DF task changed dramat-
component skills (Table 5), and represented the strongest ically. As the Switch demand was introduced, performance
single predictor (Table 12). In contrast, flexibility, at least as appeared unrelated to BDS-EV Motor Sequence Fluency
assessed by the residual score from D-KEFS Number-Letter and M-PLN, both potent predictors of performance in the
36 Y. Suchy et al.

Non-switch DF trials. These findings also suggest that switch- provided by the principle author’s University of Utah faculty start-up
ing may represent a construct that is: (a) separate from gen- account.
erative fluency, and (b) perhaps more heavily relying on
attentional resources. These results provide support for REFERENCES
considering the third (i.e., Switch) trial of D-KEFS DF as a
separate construct when interpreting test results. Baldo, J.V., Shimamura, A.P., Delis, D.C., Kramer, J., & Kaplan, E.
(2001). Verbal and design fluency in patients with frontal lobe
lesions. Journal of the International Neuropsychological Society,
Limitations 7(5), 586–596.
Baser, C., & Ruff, R.M. (1987). Construct validity of the Sand
Although our prior study, which yielded very similar results, Diego Neuropsychological Test Battery. Archives of Clinical Neuro-
was conducted with a sample that included the full adult life psychology, 2, 13–32.
span (ages 18 to 68 years), it is not clear whether similar Butler, R.W., Rorsman, I., Hill, J.M., & Tuma, R. (1993). The
results would also be yielded by a sample of young healthy effects of frontal brain impairment on fluency: Simple and
adults only, or by clinical samples, such as dementia patients, complex paradigms. Neuropsychology, 7(4), 519–529.
epilepsy patients, and so forth. In fact, different factor struc- Delis, D., Jacobson, M., Bondi, M.W., Hamilton, J.M., & Salmon,
tures are known to emerge in different populations, particularly D.P. (2003). The myth of testing construct validity using factor
when it comes to timed (Psychological Corporation, 1997) analysis or correlations with normal or mixed clinical populations:
Lessons from memory assessment. Journal of the International
and memory tests (Delis, Jacobson, Bondi, Hamiltoon, &
Neuropsychological Society, 9, 936–946.
Salmon, 2003). For example, timed visual-spatial tests gen-
Delis, D., Kaplan, E., & Kramer, J. (2001). Delis-Kaplan Executive
erally load on a different factor than processing-speed tests, Function System: Examiner’s manual. San Antonio, TX: The
except among individuals in their eighties, for whom timed Psychological Corporation.
visual-spatial and processing-speed performances load on a Elfgren, C.I., & Risberg, J. (1998). Lateralized frontal blood flow
single factor (Psychological Corporation, 1997). Similarly, increases during fluency tasks: Influence of cognitive strategy.
while immediate and delayed recall on memory measures Neuropsychologia, 36(6), 505–512.
load on a single factor in most populations, they load on sepa- Fama, R., Sullivan, E.V., Shear, P.K., Cahn-Weiner, D.A., Marsh,
rate factors among individuals with mesial temporal dys- L., Lim, K.O., et al. (2000). Structural brain correlates of verbal
function (Delis et al., 2003). Thus, it is possible that among and nonverbal fluency measures in Alzheimer’s disease. Neuro-
a sample of patients with focal frontal lesions, cognitive flex- psychology, 14(1), 29–41.
Jones-Gotman, M., & Milner, B. (1977). Design fluency: The
ibility would become dissociated from motor planning and
invention of nonsense drawings after focal cortical lesions.
would uniquely account for substantial variance in DF. These
Neuropsychologia, 15, 653–674.
examples highlight the limitation of correlational methodology Keele, S. (1981). Behavioral analysis of movement. In V.B.
in construct validation research (Delis et al., 2003), and point Brooks (Ed.), Handbook of physiology, Vol. 2. Motor control
to the importance of replications with other, preferably clinical, (pp. 1391–1414). Bethesda, MD: American Psychological
populations. Society.
Additionally, as is usually the case, the present study Kramer, J.H., Quitania, L., Dean, D., Neuhaus, J., Rosen, H.J.,
yielded models that accounted for approximately between Halabi, C., et al. (2007). Magnetic resonance imaging correlates
30% and 40% of variance. This means that more than 60 % of set shifting. Journal of the International Neuropsychological
of variance remained unexplained. While some of the un- Society, 13(3), 386–392.
explained variance undoubtedly reflects random error, other Kraybill, M.L., & Suchy, Y. (2008). Evaluating the role of motor
regulation in figural fluency: Partialing variance in the Ruff
systematic sources of variance could likely be identified. In par-
Figural Fluency Test. Journal of Clinical and Experimental
ticular, additional specific cognitive processes, such as work-
Neuropsychology, 30(8), 903–912.
ing memory, attentional vigilance, and visual-constructional Kraybill, M.L., Suchy, Y., & Franchow, E. (2009, February). Longi-
skills, may play a role. Similarly, differences in intellectual tudinal prediction of functional independence and cognition:
capacity could likely explain some variance, as Full Scale IQ The utility of a brief motor programming task. Poster session
is known to be differentially related to different cognitive presented at the annual meeting of the International Neuropsy-
tests (Psychological Corporation, 1997). Lastly, individual chological Society, Atlanta, GA.
differences in temperament and personality, including moti- Psychological Corporation (1997). WAIS-III & WMS-III technical
vation, autonomic arousal, state and trait anxiety, and interest manual. San Antonio, TX: The Psychological Corporation.
and curiosity in new experiences all likely contribute differ- Regard, M., Strauss, E., & Knapp, P. (1982). Children’s production
entially to performance (Williams, Suchy, & Rau, 2009). on verbal and non-verbal fluency tasks. Perceptual and motor
skills, 55(3), 839–844.
Including such a large number of variables would, of course,
Ruff, R. (1998). RFFT: Ruff Figural Fluency Test: Professional
require a much larger sample.
manual. Odessa, FL.: Psychological Assessment Resources.
Ruff, R., Evans, R., & Marshall, L.F. (1986). Impaired verbal and
ACKNOWLEDGMENTS figural fluency after head injury. Archives of Clinical Neuropsy-
chology, 1(2), 87–101.
The authors had no conflict of interest when conducting this Stuss, D.T., Alexander, M.P., Benson, D.F., Trimble, M.R., &
research or reporting the results. Funding for this research was Cummings, J.L. (1997). Frontal lobe functions. In Contemporary
 Understanding design fluency 37

behavioral neurology (pp. 169–187). Woburn, MA: Butterworth- Suchy, Y., Sands, K., & Chelune, G.J. (2003). Verbal and nonverbal
Heinemann. fluency performance before and after seizure surgery. Journal of
Suchy, Y., Derbidge, C., & Cope, C. (2005). Behavioral Dyscon- Clinical and Experimental Neuropsychology, 25(2), 190–200.
trol Scale–Electronic Version: First examination of reliability, Tucha, O., Smely, C., & Lange, K.W. (1999). Verbal and figural
validity, and incremental utility. Clinical Neuropsychologist, fluency in patients with mass lesions of the left or right frontal
19(1), 4–26. lobes. Journal of Clinical and Experimental Neuropsychology,
Suchy, Y., & Kraybill, M.L. (2007). The relationship between motor 21(2), 229–236.
programming and executive abilities: Constructs measured by Williams, P.G., Suchy, Y., & Rau, H. (2009). Individual differences
the Push-Turn-Taptap task from the BDS-EV. Journal of Clinical in executive functioning: Implications for stress regulation.
and Experimental Neuropsychology, 29(6), 648–659. Annals of Behavioral Medicine, 37, 126–140.

View publication stats