Behavioural Processes 152 (2018) 18–25

Contents lists available at ScienceDirect

Behavioural Processes
journal homepage: www.elsevier.com/locate/behavproc

Algorithmic analysis of relational learning processes in instructional T

technology: Some implications for basic, translational, and applied research
⁎
William J. McIlvane , Joanne B. Kledaras, Christophe J. Gerard, Lorin Wilde, David Smelson
University of Massachusetts Medical School, Worcester, MA, 01655, United States

A R T I C LE I N FO A B S T R A C T

Keywords: A few noteworthy exceptions notwithstanding, quantitative analyses of relational learning are most often simple
Stimulus control descriptive measures of study outcomes. For example, studies of stimulus equivalence have made much progress
Learning by exclusion using measures such as percentage consistent with equivalence relations, discrimination ratio, and response
Stimulus equivalence latency. Although procedures may have ad hoc variations, they remain fairly similar across studies. Comparison
Computerized algorithmic learning supports
studies of training variables that lead to different outcomes are few. Yet to be developed are tools designed
specifically for dynamic and/or parametric analyses of relational learning processes. This paper will focus on
recent studies to develop (1) quality computer-based programmed instruction for supporting relational learning
in children with autism spectrum disorders and intellectual disabilities and (2) formal algorithms that permit
ongoing, dynamic assessment of learner performance and procedure changes to optimize instructional efficacy
and efficiency. Because these algorithms have a strong basis in evidence and in theories of stimulus control, they
may have utility also for basic and translational research. We present an overview of the research program,
details of algorithm features, and summary results that illustrate their possible benefits. It also presents argu-
ments that such algorithm development may encourage parametric research, help in integrating new research
findings, and support in-depth quantitative analyses of stimulus control processes in relational learning. Such
algorithms may also serve to model control of basic behavioral processes that is important to the design of
effective programmed instruction for human learners with and without functional disabilities.

1. Introduction The primary inspiration for the overall research program at the
Shriver Center came from the foundational work of B. F. Skinner and
This article discusses recent developments in the area of automated Fred Keller (Skinner, 1968; Keller, 1968) as summarized in widely
instruction. It focuses specifically on new technology that has potential known works such as The Technology of Teaching (1968) and Goodbye
for integrating stimulus control research with research on quantitative Teacher (1968). The goal of such work was to develop and apply be-
analyses of behavior that concern the Society for Quantitative Analysis of havior analytic principles to create systematic instructional technolo-
Behavior (SQAB). In making this contribution, we acknowledge that gies that could be applied in applications with a wide range of learners.
none of the authors has much expertise in advanced quantitative ana- In that vein, Murray Sidman and his group at the Massachusetts General
lysis. That acknowledged, some of us have worked on certain problems Hospital (MGH) initiated our program when they conducted among the
in this area, for example, studies of choice processes and behavioral earliest instances of translational behavior analysis (McIlvane, 2009). In
momentum (e.g. Dube and McIlvane, 2006). We were drawn to such their work, Sidman and his colleagues showed that functionally non-
work by findings of our stimulus control research and indicated direc- verbal individuals and other people with profound neurological dis-
tions of our long-term program of research on relational learning (cf. orders could be effectively taught and evaluated using stimulus control
McIlvane et al., 2011). Our quantitative studies have been fruitful en- shaping procedures deriving from studies of pigeons (Sidman and
ough for us to incorporate certain quantitative concepts in our theory Stoddard, 1966 and Terrace, 1963a,b, respectively).
building efforts. For example, the McIlvane and Dube (2003) paper on One direction of the Sidman MGH research program and its suc-
stimulus control topography coherence theory acknowledged its con- cessors at the Shriver Center was to develop automated methods for
ceptual debt to the work of prominent members of SQAB, most notably providing extensive support to people who needed an unusual level of
that represented in the Davison and Nevin (1999) analysis. help to learn. That work yielded impressive results in many areas, most

⁎
Corresponding author.
E-mail address: [email protected] (W.J. McIlvane).

https://doi.org/10.1016/j.beproc.2018.03.001
Received 10 January 2018; Received in revised form 1 March 2018; Accepted 1 March 2018
Available online 12 March 2018
0376-6357/ © 2018 Elsevier B.V. All rights reserved.
W.J. McIlvane et al. Behavioural Processes 152 (2018) 18–25

notably in teaching simple discrimination (Sidman and Stoddard, 1966) by contrasting such relations with those learned previously, (2) is more
and conditional discrimination (Stoddard, 1982) performances to effective than mere differential reinforcement (e.g., Ferrari et al., 1993)
people with severe intellectual disabilities who had been deemed pre- and (3) does not rely on graduated stimulus changes as in stimulus
viously to be unable to learn them. More broadly, the vast majority control shaping procedures. Developed by McIlvane and Stoddard
foundational studies in behavior analytic stimulus control research on (1981), it is based on stimulus control analyses of the underlying be-
relational learning have been conducted using automated apparatus havioral processes (Dixon, 1981), analyses of “fast mapping” in early
(e.g., Sidman, 1971; McIlvane and Stoddard, 1981). Notably, grant child word learning (Carey and Bartlett, 1978), and cross-cultural ob-
proposals of that era often mentioned potential applications of that servations by Kagan (1981) of near-universality in toddler-aged chil-
work to development of more effective teaching machines. dren.
Given the many successes of the work just described, there was LBE involves (1) matching-to-sample procedures combining defined
great optimism that automated technologies and even completely au- (i.e. already learned) matching relations with undefined (not yet
tomated instructional environments would inevitably emerge. In ret- learned) matching relations (e.g., sample:comparison pairs
rospect, however, that work went against trends in instructional design “Spoon”:SPOON [defined] vs. “Spatula”:SPATULA [undefined] for a
that criticized then-available teaching machines specifically and care- toddler-aged child), (2) providing various contextual supports for
fully programmed instructional procedures generally as inappropriate learning the latter relations, (3) assessing quality of learning during and
methodologies for teaching broad classes of learners. As Benjamin after training, and (4) managing introduction of new relations, while
(1988) has recounted, the successes of classroom teaching machines in (5) minimizing and often virtually eliminating errors during training for
the Skinnerian tradition proved short-lived and eventually failed, thus most children. These features maintain attention to relevant aspects of
undercutting arguments that automated instruction was an area of fu- stimulus relations to be learned and motivate the child via the positive
ture promise. We think it likely that such failure was due in part to the feedback associated with ongoing success.
hardware used in automated instruction prior to the 1980s. Most such Table 1 presents an illustrative subset of matching-to-sample (MTS)
hardware was aesthetically unattractive, limited and inflexible in its trials from an LBE session designed to teach a new relation involving an
programming capabilities, and typically too expensive to replicate on a as-yet undefined dictated word from Set A and an as-yet undefined
large scale. picture that corresponds to it from Set B in the context of an MTS
Beginning in the mid-1980s, powerful graphics-oriented computers procedure. In this illustration, the contingencies of reinforcement have
began to offer alternative hardware options that promised to render already defined three comparisons stimuli (B1–B3) in relation to cor-
automated instruction increasingly attractive along many dimensions, responding sample stimuli (A1–A3) (Table 1, Lines 1–3; positive and
including cost. The Shriver group was a very early leader in use of such negative comparisons are indicated by + and −, respectively). Un-
technology to advance its stimulus control research program. That defined comparison stimulus B4 is introduced in a display with two
program focused specifically on developing the potential of Apple defined comparison stimuli and sample that corresponds to one of the
Macintosh hardware and software (the early stages of that apparatus latter (Table 1, Line 4). On this trial, selection of the positive compar-
development were summarized by Dube and McIlvane, 1989). One ison controls for indiscriminate selections of undefined stimuli. The key
product was a highly programmable authoring system (MTS, Dube and trial type of an LBE procedure is shown in Table 1, Line 6; the sample is
Hiris, 1999) for implementing a variety of simple and conditional dis- A4 and B4 is the positive comparison, to be selected by exclusion of the
crimination procedures supported by a number of stimulus control previously defined comparison stimuli.
shaping procedures. The authoring system permitted assessment of The LBE procedure has two types of tests for learning outcomes
learner performance and highly flexible program branching via a mini- (Table 1, Lines 9 and 11). On the first type, the sample is A4 and the
command language developed for that specific purpose. While clearly comparison display includes now defined B4 and an as-yet undefined
an advance in the field of automated instructional programming, MTS comparison, B5. This trial type controls for indiscriminate selection of
exists today only as laboratory software. It requires training and ex- recently introduced comparison stimuli. The second test trial type
perience to use it effectively, and dissemination outside the behavioral presents an undefined sample stimulus (A5) with the same comparison
research laboratory has not been practically achievable. display. It assesses whether the learner can now exclude the recently
A major purpose of this paper is to describe some next-generation defined comparison (i.e., B4). If the learner reliably selects the positive
software research and development in the Shriver program that has comparison stimuli on these two trial types, such performances indicate
been aimed specifically at development of more powerful, more user- that the specific A4:B4 relation has been acquired.
friendly procedures for accomplishing many of the functional outcomes
that the MTS program was designed to achieve. We will describe gen- 3. LBE Computerized Algorithmic Steering Logic (CASL)
eral procedures of this program and illustrative applications that il-
lustrate the capabilities of the software. With this context established, Recent Shriver research has sought to optimize the LBE method,
we will discuss how this approach might be further enriched by in- specifically incorporating an intelligent computerized algorithmic steering
corporating methods and perspectives that have been developed by logic for implementing the procedure. The main purpose of the CASL
members of SQAB and others pursuing quantitative analyses of beha-
vior. Table 1
Of note, while the matching-to-sample procedure embeds within it Subset of trials from a typical LBE teaching session.
concurrent schedules of reinforcement (cf. Jones and Elliffe, 2013),
Trial Type Sample Comparison Stimuli
potential schedule interactions of the type done in matching law re-
search are only rarely considered in stimulus control analyses. Beha- 1 Baseline A1 B1+ B2- B3-
vioral momentum principles are similarly understudied in stimulus 2 Baseline A2 B3- B1- B2+
3 Baseline A3 B2- B3+ B1-
control research. We hope to call attention to these and other possibi-
4 “Control” A2 B1- B4- B2+
lities for research to address an obvious gap in the literature. 5 Baseline A3 B3+ B2- B1-
6 “Exclusion” A4 B2- B1- B4+
2. Learning by exclusion 7 Baseline A1 B2- B3- B1+
8 Baseline A3 B1- B3+ B2-
9 “Ex. Test1” A4 B4+ B2- B5-
A primary target of our development effort has been refinement of
10 Baseline A2 B2+ B1- B3-
the learning by exclusion (LBE) method, a potentially errorless teaching 11 “Ex. Test2” A5 B4- B5+ B3-
method that (1) promotes learning of new stimulus-stimulus relations

19
W.J. McIlvane et al. Behavioural Processes 152 (2018) 18–25

has been to systematize implementation of the various components of 4.1. Application 1: teaching prerequisites for picture-aided communication
the LBE procedure and eliminate one of its drawbacks–the requirement
that the user make a large number of initial and ongoing informed 4.1.1. Three-dimensional (3D): two-dimensional (2D) stimulus-stimulus
decisions about the structuring of trial types within a session. As a relations
practical matter, such decisions require considering a great amount of During typical development, children come very early to recognize
procedural detail, sometimes within a very short time window. that 2D images (photos, pictures, drawings, movies, etc.) represent their
Moreover, managing those decisions while considering differences in 3D counterparts and are functionally equivalent in certain respects
learning outcomes of individuals adds further complexity (i.e., mere (e.g., both may be named with the same label, pointing to a picture of a
template methods may prove inefficient). desired item may be functionally equivalent to pointing to the item
Critical decisions include: (1) the number of exclusion trials and itself in a request, etc.). Most children exhibit this capability by the
control trials programmed; (2) the number and nature of baseline MTS second year of life (Daehler et al., 1976). When a child has a neuro-
trials that provide procedural context for LBE teaching; (3) the number developmental disorder (e.g., autism spectrum disorder, Down Syn-
and nature of trials that test learning outcomes; (4) the rate of in- drome, etc.), however, development of 3D2D equivalence may be de-
troducing new relations via LBE; (5) the proportion of relations recently layed and may even fail to develop (Carr and Felce, 2008). When
acquired via LBE that are maintained in the baseline; (6) the pro- development of this pivotal capability does not occur, this is a major
grammed schedules of reinforcement for each of the trial types; (7) the obstacle for individualized intervention by behavior therapists, speech-
method for handling any persistent errors on learning outcome trials of language pathologists, and special educators. Indeed, most approaches
individual relations, and (8) the quality of balancing of the many type in the augmentative/alternative communication (AAC) field (e.g., the
of trials presented (cf. Gerard et al., 2014). Picture Exchange Communication System [PECS; Bondy and Frost,
More subtle decisions in the design of LBE steering logics include (9) 1994) depend ultimately on establishing or established 3D2D feature/
dynamic variations in the proportion of LBE trials relative to baseline perceptual classes and functional equivalence.
trials: increases result in more practice with LBE trial types and coun- In a recently concluded project, our group developed a broadly
terintuitive decreases in LBE trials render them relatively more novel, a applicable LBE CASL to teach 3D2D visual symbol matching as a pre-
feature that may help in acquisition (Zeaman, 1976). Another decision, requisite for AAC communication by nonverbal children with severe
also perhaps counterintuitive, derives from analyses of variables that IDD (McIlvane et al., 2016). Fig. 1 illustrates the matching task,
promote equivalence relations (Lionello-DeNolf, 2009), specifically showing that we began with food items and corresponding pictures.
(10) the inclusion of interspersed identity matching trials that may During the initial teaching, thirteen nonverbal/minimally verbal
encourage development of sample-positive comparison controlling re- participants learned a three-sample, three-comparison matching-to-
lations (cf. McIlvane, 2013). Further details of and rationale for the sample. The procedure introduced a new object:picture relation on
structure of the LBE CASL can be found in McIlvane et al. (2016). control and exclusion trials (18 each in a 72-trial session). Overall, the
We wish that we were able to report that the extant literature first LBE session performance was virtually errorless for most partici-
contains sufficient data to guide researchers and teachers in selecting pants (i.e. including all baseline, exclusion, and control trials). Next, the
values for each of the parameters just listed. Regrettably, we know of no new relation was added to the baseline and reviewed in three sub-
such parametric studies. Much of the work by the Shriver group has sequent baseline sessions in which all sample:comparison relations
focused on longer-term studies of minimally verbal children with neu- were presented in equal proportions. This same approach was re-
rodevelopmental disorders–a population whose behavioral deficits plicated twice more, adding two new object:picture relations to the
virtually preclude parametric studies for reasons of logistics and cost. baseline (see McIlvane et al., 2016 for additional details of procedure
Opportunistically, we have explored variations within parameters 1–3, and illustrative individual data).
5–6, and 9 that seemed to have good effect with some learners, but Summarizing our findings, the 3D2D LBE CASL yielded LBE learning
controlled studies are lacking. There are good logical arguments and/or outcomes better than the best published 3D2D data thus far reported
supporting data for the remaining parameters, but none within the LBE (Carr and Felce, 2008). Fig. 2 is adapted from McIlvane et al. (2016). It
framework specifically. shows that the magnitude of difference in accuracy on exclusion (left)
In pointing to this current gap in the literature, we should empha- and outcome (right) trials with the LBE CASL and the Carr and Felce
size that a number of these parameters are not specific to LBE proce- data (re-analyzed to account for differences in performance attributable
dures but apply also other widely studied procedures (e.g., basic dis- to “chance” on 3- vs. 2-choice MTS tasks). Note that CASL above-chance
crimination learning, stimulus equivalence). We believe that the virtual accuracy was significantly greater and inter-subject variability (black
lack of parametric studies of these variables is due in part to some lack range bars) was much less than those reported in the 2008 paper.
of recognition of their possible importance and perhaps more to the lack
of tools that render such studies practical. Early in the course of de- 4.1.2. Teaching the first instances of reliable spoken word: picture matching
veloping our LBE CASLs, we recognized that adaptations of them could In these auditory-visual MTS studies, we combined the LBE CASL
potentially bypass many logistical challenges with other stimulus con- just described with a progressive structuring procedure that used au-
trol procedures. ditory sample stimuli that were progressively more similar acoustically.
Briefly, we developed a stimulus series that began with maximally
4. Exemplary applications of LBE CASLs discriminable auditory stimuli and progressed to minimally discrimin-
able ones. The stimulus series was scaled by detailed analyses of the
Shriver researchers have sought to develop programs for teaching distinctive acoustical features of the stimuli within the series.
stimulus-stimulus relations in which the stimuli are not physically For program validation purposes, extended case studies were de-
identical. Two types are needed: (1) programs for establishing so-called signed to teach the first instances of reliable auditory-visual conditional
feature (McIlvane et al., 1993) or perceptual (Fields, 2009) equivalence discrimination to two nonverbal children with autism spectrum dis-
classes and (2) arbitrary equivalence classes (Sidman, 1994). The former orders. The results of this study were extremely promising. In training
type involves relations between/among stimuli that share defining extending over about two months, both children acquired 12 auditory-
physical features (i.e., stimuli that resemble each other in some way). visual relations with maintained accuracy that exceeded 99% in both
The latter classes do not share defining features (e.g., pictures and cases. Results with one child were startling indeed. This child acquired
corresponding printed words) and are often termed symbolic MTS. What the 12 new relations while making only one error in the many hundreds
follows will summarize validation studies that evaluated efficacy of LBE of trials that comprised the overall teaching program. The other child
CASLs in teaching children with intellectual disabilities (ID). made only six errors during that program.

20
W.J. McIlvane et al. Behavioural Processes 152 (2018) 18–25

Fig. 1. Illustrative display one a matching-to-sample trial presented in the LBE CASL to teach 3D (object) to 2D (picture) functional equivalence.

controlled comparison studies with standard-of-care procedures for

teaching relations of this nature.

4.2. Application 2: teaching Reading readiness skills to learners with ID

Benefitting from nearly 40 years of research since Sidman (1977),

our group was well-positioned to develop an evidence-based reading
readiness curriculum that could teach more children with ID more skills
more efficiently than ever before. We recently completed a feasibility
study aimed at developing and validating a CASL to systematize LBE
implementation with this subject matter; details of its procedures and
preliminary results were published (Kledaras et al., 2016).
Earlier research had shown that the LBE procedure has produced
highly efficient learning of reading prerequisites in children with long
histories of school failure (e.g., de Rose et al., 1996; de Souza et al.,
2009). The present work focused on preschool children with ID and/or
other developmental disorders associated with deficits with this subject
matter. The aim was to give such children their best chances to (1)
acquire prerequisite reading readiness skills at or near the typical age
and (2) develop functional reading skills in subsequent special educa-
tion. Fig. 3 exemplifies the LBE method for teaching relations involving
upper case (UC) and corresponding lower-case (LC) letters of the al-
phabet, a critical behavioral prerequisite for skilled reading. It pre-
supposes that the learner has mastered with identity matching to
sample (IDMTS) with UC and LC letters, thus showing that the letters
Fig. 2. Simplified presentation of data reported originally by McIlvane et al.
are discriminated visually from one another.
(2016), (Table 2). The histograms compare mean accuracy and accuracy
The upper left panel shows two trials from the IDMTS baseline with
variability on exclusion trials (left histogram) and learning outcome trials (right
histogram) of children’s performance given training with our LBE CASL (left UC letters. The panel below it shows two trials from an UC:LC MTS task
histogram pair) and data reanalyzed from Carr and Felce (2008) (right histo- in which the LC letters differ in size but not form (i.e., members of
gram pair). feature/perceptual classes). LBE procedures are shown in the right panels
in which physically dissimilar sample and comparison stimuli are in-
troduced in combinations with familiar ones. The first trial in the upper
As the time course and achieved accuracy may suggest, this teaching
right panel shows a task that requires the learner to match LC-a with
was conducted extremely carefully due to the very low levels of func-
UC-A without prior training to do so. S/he can do that by excluding LC-
tioning of the children studied. We do not know–but strongly sus-
x, previously correct only in relation to UC-X. The trial beside it is a
pect–that these excellent results were due to (1) the exceptionally
control trial. After a history of reinforced LBE teaching trials, tasks in
careful selection of auditory stimuli via distinctive feature analysis to
the lower panel assess whether the new UC:LC matching relation has
maximize discriminability and (2) the high level of teaching support
been learned. These tasks require the learner to select yet another as-yet
incorporated in the LBE CASL. We consider these to be very impressive
undefined comparison (LC-g in this case) in relation to another as-yet
preliminary findings that establish the rationale for conducting
undefined sample (UC-G) while maintaining accurate matching of the

21
W.J. McIlvane et al. Behavioural Processes 152 (2018) 18–25

Fig. 3. Illustrative trial displays from LBE procedures used to teach children with neurodevelopmental disabilities to relate upper case with lower case letters of the
Latin alphabet.

recently defined UC:LC relation (A:a in this example). slower than that of other children. With added direct training on au-
Work thus far has systematically replicated procedures from Sidman ditory-visual relations, however, he met criterion rapidly. Notably both
et al. (1974). We sought to teach children to relate (1) UC letters with this child and one other who had no prior exposure to relational tasks
LC letters and (2) both letter sets with their corresponding dictated with letters of the alphabet met criterion on all relations.
names. Letter selection was guided by literature on letter similarity and Panel C shows that Participant S was similar to Participant G
discriminability (e.g., Gilmore et al. 1979) to establish a set of UC:LC through Step 4, and we treated him similarly. Unlike Participant G,
pairs that ranged from highly similar to dissimilar. Participants were 12 direct auditory-visual training did not immediately bring in criterion
children aged three to five years. Teaching was implemented in the 7- performance with Participant S and one other child who showed this
step sequence shown in Table 2 (see Kledaras et al., 2016 for additional acquisition pattern before we had to conclude data collection. That
details of procedure and their rationale). acknowledged, both children had made substantial progress and
Training time available was up to 5–6 hours per child. In that time, achieved near-criterion scores in their final sessions. We are confident
ten children learned all targeted UC:LC relations to criterion (> 90% that both would have met criterion had we been able to conduct a few
correct selections) and nine completed the entire program, meeting more sessions.
criterion on all relations tested including equivalence relations. Trial
accuracy scores were typically high during all seven steps of the 5. Concluding comments on contributions and potential
teaching sequence and inter-participant variability was typically low implications
except during the first step of the sequence. During Steps 6 and 7, ac-
curacy on teaching and test trials was uniformly high, inter-participant We think that the three application studies summarized here make a
variability was low, and equivalence class formation was robust. Thus, good case for further research and development of algorithmic teaching
the procedures were comparably effective with possibly familiar and with LBE and stimulus equivalence procedures. We note also that CASL
clearly unfamiliar letters. development for other stimulus control teaching programs is underway
Fig. 4 shows data for selected individual children. Participant N in (e.g., for remediating so-called overselective attention in individuals
Panel A represents the outcome for most children. Error rate was low with neurodevelopmental disabilities (Farber et al., 2017)) and has had
and children met criterion on all relations. comparably impressive successes. These studies together lead us to
Panel B shows results with Participant G, our youngest child. His suggest that automated algorithmic supports will prove to be increas-
Step 4 accuracy was 81% – well above chance but his progress was a bit ingly important tools for both computer-implemented teaching apps

Table 2
Summary of procedures that taught Upper Case (UC) and Lower Case (LC) matching relations and tested for equivalence among them in Kledaras et al. (2016).
Program Step Procedures Purpose

1 Two-comparison identity matching to sample (IDMTS): colors, pictures & UC letters C, P, Verify reliable IDMTS
X
2 Three-comparison IDMTS: UC C, P, X, V, S, X; feature matching: C:c, P:p, X:x, V:v, S:s, X:x Verify IDMTS with six letters; verify feature matching with similar
UC and LC stimuli
3 Three-comparison IDMTS: UC C, P, X, V, S, X Verify IDMTS with six letters to be used in teaching arbitrary UC:LC
relations
4 Three-comparison IDMTS and arbitrary matching (Arb-MTS): G:g, A:a, R:r, D:d, E:e, and LBE teaching of arbitrary visual-visual symbolic relations
H:h, introduced successively
5 Three-comparison auditory-visual MTS with UC and LC letters used in Step 4 and LBE teaching of arbitrary auditory-visual symbolic relations
corresponding dictated letter names
6 Three-comparison auditory-visual MTS with script letters (G, D, a, r) and selected UC and Control procedure to assess possible extra-experimental learning of
LC letters: LC-g, LC-d, UC-A, and UC-R UC and/or LC letter names
7 Three-comparison auditory-visual MTS with script letters (G, D, a, r) and selected UC and Tests for emergent relations based on stimulus equivalence
LC letters: UC-D, UC-D, LC-a, and LC-r

22
W.J. McIlvane et al. Behavioural Processes 152 (2018) 18–25

Fig. 4. Individual data for three children representing the small performance variability summarized in Fig. 4. The upper graphs represent children who were
reported to have very little knowledge of the alphabet (AK), those who had no prior exposure to the alphabet, and those who had made little or no progress in their
preschool programs despite exposure to procedures that had successfully taught other children. The lower graphs show individual analyses of cumulative training
time for each of the seven steps of the teaching program.

and in computer-managed face-to-face instruction (as exemplified in stimulus equivalence and other relational discriminations (e.g., gen-
our 3D2D CASL study) for persons with severe neurodevelopmental eralized identity and oddity) in humans with or without developmental
disabilities. If so, then the ambitious objectives of the early Sidman limitations and in comparative cognition research with nonhumans. If
MGH research initiative may ultimately be achieved. the values of such parameters are left to vary across studies, then one
Mindful as we are of the history of “overselling” and associated would expect variability in outcomes–and such is the case in the present
failures in the area of automated programmed instruction, however, we literature (e.g., Lionello-DeNolf, 2009). We think that creating CASLs
need to emphasize that the work here represents a preliminary de- that require attention to setting and reporting the values of these
monstration of what may be possible rather than a mature, general- parameters may help to minimize variability in across relational
izable behavior analytic instructional technology ready for widespread learning studies and failures to replicate findings with research parti-
dissemination. Even within the narrow applications we described and cipants and procedures that seem superficially comparable. Also, CASLs
considering our successes in teaching children with severe learning that allow one to conveniently set values of parameters of interest
challenges, there remains a need for fairly challenging randomized would encourage parametric research to define the contribution of
controlled trials comparing our CASLs with other methods (e.g. verbal those parameter across their range.
instruction with point prompts, stimulus control shaping methodologies Regarding parameters of interest to members of SQAB, we note that
(McIlvane and Dube, 1992), etc.) in order to conclude that the CASLs our present CASLs include only the most basic attention to the inter-
represent superior instructional technology. actions between/among concurrent schedules of reinforcement that are
inherent in conditional discriminations. This limitation represents
common practice in much other relational learning research in which
Potential contributions to development of behavior analytic sci-
interactions among programmed schedules are virtually ignored (e.g.,
ence
as when potentially emergent equivalence relations are tested in ex-
One objective in developing the LBE CASL was to develop a tool that
tinction). Input from advances in quantitative analyses of behavior may
could be used to conduct parametric studies of learning by exclusion
not only help to further augment the effectiveness and efficiency of the
and other stimulus control procedures such as those used in stimulus
CASLs but also advance the integration of principles and procedures of
equivalence research. In an earlier section, we listed 11 parameters that
quantitative analyses of choice, behavioral momentum, and other SQAB
could be considered in designing a fairly simple LBE teaching proce-
foci with those of research on relational stimulus control, thus building
dure. We emphasize these parameters based on our history in devel-
upon the initial efforts of Davison and Nevin (1999) and others pur-
oping and disseminating the LBE method. Surveying the LBE literature
suing a similarly comprehensive analytical scope.
as a whole, however, one sees very little attention to the range of
One small example from our own work can be found in a paper by
parameters that may affect learning outcomes. We think even the cur-
Dube and McIlvane (2002a). Briefly, we found that discrimination re-
rent LBE CASL could be helpful in addressing this gap in the literature.
versal learning taught by a stimulus control shaping procedure was
We note also that most of these parameters and a number of others
more efficient if the original discrimination was taught with inter-
are potentially important in designing procedures to assess and promote
mittent vs. continuous reinforcement–a finding that is predictable from

23
W.J. McIlvane et al. Behavioural Processes 152 (2018) 18–25

behavioral momentum analyses. As part of that same program, Dube (SQAB) for the invitation to contribute to this special section of
and McIlvane (2002b) found differential sensitivity to reinforce rate Behavioral Processes. The research program described in this article has
and magnitude in individuals with neurodevelopmental disabilities that had long-term support from the U. S. National Institutes of Health, most
had clear implications for behavior analyses of learning outcomes in recently in Grants HD04147, HD25995, DC10365, and MH90272.
this population. Our own quantitative program has barely scratched the Manuscript preparation was supported by Grant AA026751 and an
surface, however, of possible applications of SQAB perspectives in sti- Intergovernmental Personnel Agreement with the United States
mulus control research. Veterans Administration. All of these funding sources were investigator-
Real possibilities for developing research programs this area have initiated, and none of the funding agencies were involved in formula-
been recognized also by prominent SQAB members. As examples, tion, design, or execution of the supported projects. Contact Dr.
Randy Grace opened the 2003 SQAB meeting in part by articulating McIlvane for additional information concerning the EAHBB Research in
what he viewed as fundamentally important objective: to “build a Progress article cited in the Application 2 section (william.mcilvane@
bridge between behavioral research with nonhumans and human re- umassmed.edu).
search in cognition and decision making.” Dianne McCarthy’s (1991)
chapter on behavioral detection theory expressed similar thoughts. References
Future versions of CASLs developed by our group or others could
explore quantitative variables that are rarely if ever studied in stimulus Benjamin, L.T., 1988. A history of teaching machines. Am. Psychol. 43, 703–712.
control work. One obvious example derived from our 2002 study would Bondy, A., Frost, L., 1994. The picture exchange communication system. Focus Autism 9,
1–19.
be to study the influence of intermittent reinforcement schedules on the Carey, S., Bartlett, E., 1978. Acquiring a single new word. Papers and Reports on Child
baseline trials that provide context for exclusion and control trials that Language Development 15. pp. 17–29.
teach new relations in the LBE procedure: Would intermittent re- Carr, D., Felce, J., 2008. Teaching picture-to-object relations in picture-based requesting
by children with autism: a comparison between error prevention and error correction
inforcement on the former enhance accuracy on LBE learning outcome teaching procedures. J. Intell. Disabil. Res. 52, 309–317.
tests and perhaps subsequent retention tests for the subset of children Daehler, M.W., Permutter, M., Myers, N.A., 1976. Equivalence of pictures and objects for
who make errors on learning outcomes tests? Other research might very young children. Child Dev. 47, 96–102.
Davison, P., Nevin, J.A., 1999. Stimuli, reinforcers, and behavior: an integration. J. Exp.
address more subtle interactions among the concurrent schedules
Anal. Behav. 71, 439–482.
within matching-to-sample procedures, for example, perhaps devising de Rose, J.C.C., Souza, D.G., Hanna, E.S., 1996. Teaching reading and spelling: stimulus
optimal schedule of reinforcement for promoting discrimination equivalence and exclusion. J. Appl. Behav. Anal. 29, 451–469.
de Souza, D.G., de Rose, J.C., Faleiros, T.C., Bortoloti, R., Hanna, E.S., McIlvane, W.J.,
learning of individual research participants or coming to understand
2009. Teaching generative reading via recombination of minimal textual units: a
persistent conditional discrimination learning failures in seemingly legacy of verbal behavior to children in Brazil. Int. J. Psychol. Psychol. Ther. 9,
well-structured applied behavior analytic work. 19–44.
To conclude, we believe that work of the type reported here has Dixon, L.S., 1981. A functional analysis of photo-object matching skills of severely re-
tarded adolescents. J. Appl. Behav. Anal. 14, 465–478.
potential for integrating major lines of research that have tended to go Dube, W.V., Hiris, E.J., 1999. MTS v 11.6.7 [Computer Software Documentation]. E. K.
forward mostly in isolation and perhaps miss opportunities to develop Shriver Center, Waltham, MA.
more comprehensive analyses of behavior. One can certainly take ad- Dube, W.V., McIlvane, W.J., 1989. Adapting a microcomputer for behavioral evaluation
of mentally retarded individuals. In: In: Mulick, J., Antonak, R. (Eds.), Transitions in
ditional steps in that direction through traditional academic activities Mental Retardation, vol. 4 Ablex, Norwood, NJ pp. 104–127.
(i.e., review articles, integrative chapter and books, etc.). However, we Dube, W.V., McIlvane, W.J., 2002a. Reinforcer rate and stimulus control in discrimination
believe that an even more tangible contribution would be creating reversal learning. Psychol. Rec. 52, 405–416.
Dube, W.V., McIlvane, W.J., 2002b. Quantitative assessments of sensitivity to reinforce-
useful tools that incorporate empirical research across sub-disciplines of ment contingencies in mental retardation. Am. J. Ment. Retard. 107, 136–145.
behavior analysis. To the extent that such tools can also have clinical Dube, W.V., McIlvane, W.J., 2006. Behavior-analytic experimental strategies and moti-
and education applications that render them useful to other disciplines, vational processes in persons with mental retardation. In: Switzky, H., Hickson, L.,
Schalock, R. (Eds.), International Review of Research in Mental Retardation, vol. 31:
that contribution should also lead to both additional recognition of and Mental Retardation, Personality, and Motivational Systems. Academic Press, San
resources for behavior analytic science. In addition, training programs Diego, CA pp. 261–288.
that incorporate such a focus would require interdisciplinary colla- Farber, R.S., Dickson, C.A., Dube, W.V., 2017. Reducing overselective stimulus control
with differential observing responses. J. Appl. Behav. Anal. 50, 87–105.
borations that would likely prepare graduates for attractive careers
Ferrari, C., de Rose, J.C., McIlvane, W.J., 1993. Exclusion vs. selection training of audi-
outside traditional academic environments. tory-visual conditional relations. J. Exp. Child Psychol. 56, 49–63.
As to how such activity might go forward, we think that there are Fields, L., 2009. The synthesis of complex categories from perceptual and equivalence
good reasons to implement evolving CASLs in open source formats (cf. classes: effects of training and testing parameters. Eur. J. Behav. Anal. 10, 119–142.
Gerard, C.J., Mackay, H.A., Thompson, B., McIlvane, W.J., 2014. Rapid generation of
https://en.wikipedia.org/wiki/Open-source_software) that would allow balanced trial distributions for matching-to-sample procedures: a technical note. J.
(1) open examinations of specific details of implementation and (2) Exp. Anal. Behav. 110, 171–178.
field-wide contributions that could be open to peer review and re- Gilmore, G.C., Hersh, H., Caramazza, A., Griffin, J., 1979. Multidimensional letter simi-
larity derived from recognition errors. Percept. Psychophys. 25, 425–431.
cognition as scholarly contributions. Some process of this nature seems Jones, B.M., Elliffe, D.M., 2013. Matching-to-sample performance is better analyzed in
necessary because it could help to distribute the considerable costs of terms of a four-term contingency than in terms of a three-term contingency. J. Exp.
software development that are difficult practically to offset within Anal. Behav. 100, 5–26.
Kagan, J., 1981. The Second Year: The Emergence of Self Awareness. Harvard University
current research grants and contract structures in the U. S. and else- Press, USA.
where. While we understand the major challenges entailed in our Keller, F.S., 1968. Goodbye, teacher. J. Appl. Behav. Anal. 1, 78–89.
proposal, we think that potential benefits are worth the needed in- Kledaras, J.B., Gerard, C.J., Twyman, J.S., McIlvane, W.J., 2016. Teaching some basic
prerequisites for reading in 2015: algorithmic learning by exclusion. Exp. Anal. Hum.
vestment of time, energy, and money. Behav. Bull. 30, 23–27.
Lionello-DeNolf, K.M., 2009. The search for symmetry: 25 years in review. Learn. Behav.
Disclosures and Conflicts of Interest 37, 188–203.
McCarthy, D.C., 1991. Behavioral detection theory: Some implications for applied human
research. In: Commons, M.L., Nevin, J.A., Davison, M.C. (Eds.), Signal Detection:
None of the authors have financial interests beyond past and/or Mechanisms, Models and Applications. Erlbaum, Hillsdale, NJ pp. 239–255.
current grant supports listed above. There are no conflicts of interest to McIlvane, W.J., 2009. Translational behavior analysis: from laboratory science in sti-
report. mulus control to intervention with persons with neurodevelopmental disabilities.
Behav. Anal. 32, 273–280.
McIlvane, W.J., 2013. Simple and complex discrimination learning. In: Madden, G.J.
Acknowledgements (Ed.), APA Handbook of Behavior Analysis. American Psychological Association,
Washington, DC pp. 129–163.
McIlvane, W.J., Dube, W.V., 1992. Stimulus control shaping and stimulus control
We are grateful to the Society for Quantitative Analysis of Behavior

24
W.J. McIlvane et al. Behavioural Processes 152 (2018) 18–25

topographies. Behav. Anal. 15, 89–94. Research to Practice in Mental Retardation, vol. II. University Park Press, Baltimore,
McIlvane, W.J., Dube, W.V., 2003. Stimulus control topography coherence theory: MD pp. 353-360.
foundations and extensions. Behav. Anal. 26, 195–213. Sidman, M., 1994. Equivalence Relations and Behavior: A Research Story. Authors
McIlvane, W.J., Dube, W.V., Green, G., Serna, R.W., 1993. Programming conceptual and Cooperative, Boston.
communication skill development: A methodological stimulus class analysis. In: Sidman, M., Cresson Jr., O., Willson-Morris, M., 1974. Acquisition of matching to sample
Kaiser, A.P., Gray, D.B. (Eds.), Understanding Children’s Language. Brookes, via mediated transfer. J. Exp. Anal. Behav. 22, 261–273.
Baltimore, MD. Sidman, M., Stoddard, L.T., 1966. Programming perception and learning for retarded
McIlvane, W.J., Dube, W.V., Lionello-DeNolf, K.M., Serna, R.W., Barros, R.S., Galvão, children. In: In: Ellis, N.R. (Ed.), International Review of Research in Mental
O.F., 2011. Some current dimensions of translational behavior analysis: From la- Retardation, vol. 2 Academic Press, New York pp. 151–208.
boratory research to intervention for persons with autism spectrum disorders. In: Skinner, B.F., 1968. The Technology of Teaching. Appleton-Century-Crofts, New York.
Mayville, E., Mulick, J. (Eds.), Behavioral Foundations of Effective Autism Treatment. Stoddard, L.T., 1982. An investigation of automated methods for teaching severely re-
Sloan Publishing, Cornwall-on-Hudson, NY pp. 155–181. tarded individuals. In: Ellis, N.R. (Ed.), International Review of Research in Mental
McIlvane, W.J., Gerard, C.J., Kledaras, J.B., Mackay, H.M., Lionello-DeNolf, K.M., 2016. Retardation. Academic Press, New York pp. 163–207.
Teaching stimulus-stimulus relations to nonverbal individuals: reflections on tech- Terrace, H.S., 1963a. Discrimination learning with and without “errors”. J. Exp. Anal.
nology and future directions. Eur. J. Behav. Anal. 17, 49–68. Behav. 6, 1–27.
McIlvane, W.J., Stoddard, L.T., 1981. Acquisition of matching-to-sample performances in Terrace, H.S., 1963b. Errorless transfer of a discrimination across two continua. J. Exp.
severe retardation: learning by exclusion. J. Ment. Defic. Res. 25, 33–48. Anal. Behav. 6, 223–232.
Sidman, M., 1971. Reading and auditory-visual equivalences. J. Speech Hear. Res. 14, Zeaman, D., 1976. The ubiquity of novelty-familiarity (habituation) effects. In: Tighe,
5–13. T.J., Leaton, R.N. (Eds.), Habituation: Perspectives from Child Development, Animal
Sidman, M., 1977. Teaching some basic prerequisites for reading. In: Mittler, P. (Ed.), Behavior, and Neurophysiology. Erlbaum, Hillsdale, NJ pp. 297–320.

25