A ROLE ANALYSIS EXERCISE TO MINIMIZE ROLE AMBIGUITY AND PROMOTE

ROLE CLARITY IN INSTRUCTIONAL DESIGN TEAMS

by

Veena Radhakrishnan

A dissertation submitted to Johns Hopkins University in conformity with the requirements for the

degree of Doctor of Education

Baltimore, Maryland

October 2018

© Copyright by Veena Radhakrishnan 2018

All Rights Reserved
 Abstract

This dissertation reports outcomes from a mixed methods study designed to decrease role

ambiguity and clarify individual roles of members of instructional design teams (IDTs) involved

in online course development in higher education institutions. Based on the empirical evidence

collected and a review of the literature, a role analysis intervention based on Dayal and Thomas’s

(1968) role analysis technique (RAT) was implemented in three instructional design teams at a

large private research university. While the pilot version of this intervention failed to elicit a

statistically significant decrease in role ambiguity as measured by Pareek’s Organizational Role

Stress (ORS) scale, qualitative data revealed several important themes relevant to participants’

perceptions of the value of the role analysis exercise in instructional design contexts. It

highlighted the intervention’s potential to: (a) promote collaboration between faculty and staff,

(b) provide clarification of roles and expectations, (c) reveal different perspectives and

expectations of roles, and (d) promote self-reflection and analysis of one’s own roles. Further, it

gave participants the opportunity to validate the presence of role stress in IDTs and underscored

the ways in which role stressors exist within the online learning contexts. Finally, consistent with

past findings, it confirmed the diverse and constantly evolving roles of individuals involved in

online course development. While limited in sample size and scope, this study revealed

meaningful results that can be applied to IDTs and higher education institutions involved in

online course development.

Keywords: role stressors, role ambiguity, role clarification, role analysis technique,

instructional design teams, online education

ii
Signatures

iii
 Dedication

This thesis is dedicated to my husband, Prabhu, who cooked, cleaned, cared for our daughter,

wiped my tears, and gave me endless support throughout my graduate studies.

iv
 Acknowledgments

To my adviser, Dr. Wendy Drexler, I would like to thank you for your guidance and

support. I can never truly express my gratitude for your dedication in guiding me to bring rigor

and clarity to my ideas and research. Your positive words kept me going when I thought all was

lost. It is due to your kindness, patience, and encouragement that this thesis was completed.

To Dr. Carey Borkoski, thank you so much for your brilliance and for making me feel

excited about my work. I am forever grateful for your thoughtful comments and insights that

have given me much to contemplate. You are an amazing teacher, and I am so lucky to have

been your student.

To Dr. Diana Zilberman, I have no words to describe my profound appreciation and

gratitude to you. Thank you for believing in me and standing by me through all the years.

To Dr. Rick Milter, thank you for playing a pivotal role in the implementation of my

intervention and offering wisdom and perspective throughout my doctoral journey.

To Dr. Peter Ariev, thank you for offering extraordinary feedback and help with my

methods and research process.

To Patrick Dempsey, you have been an incredible friend, support, and guide to me.

Thank you for staying by me through trying times and always knowing the perfect thing to say.

To Jie Zhang, thank you for listening, brainstorming, and sustaining me with good food,

humor, and friendship.

Finally, to my husband Prabhu, thank you for being a single parent on the countless

weekends that I spent away from home while working on this thesis. I could not have made it

this far without your sacrifices, unconditional love, and support. I love you with all my heart. I

dedicate this thesis to you.

v
 Table of Contents

Abstract ............................................................................................................................... ii

Signatures........................................................................................................................... iii

Dedication .......................................................................................................................... iv

Acknowledgments............................................................................................................... v

Table of Contents ............................................................................................................... vi

List of Tables ..................................................................................................................... xii

List of Figures .................................................................................................................. xiii

Chapter 1: Introduction of the Problem of Practice ............................................................ 1

Overview ......................................................................................................................... 1

Quality in Online Education ........................................................................................... 3

Theoretical Foundations of Quality Online Education ............................................... 5

Defining Quality ....................................................................................................... 11

Instructional Design ...................................................................................................... 12

Instructional Design Process ..................................................................................... 12

Instructional Design Team Roles and Responsibilities ............................................. 14

Role Stressors in Instructional Design .......................................................................... 21

Role Theory .............................................................................................................. 21

Stressors in Instructional Design Teams ................................................................... 23

Summary of Literature Review ..................................................................................... 23

vi
Chapter 2: Needs Assessment ........................................................................................... 27

Overview ....................................................................................................................... 27

Research Question ........................................................................................................ 28

Hypothesis..................................................................................................................... 28

Needs Assessment Design............................................................................................. 28

Method .......................................................................................................................... 28

Participants................................................................................................................ 28

Measures and Instrumentation .................................................................................. 31

Procedure ...................................................................................................................... 33

Data Collection ......................................................................................................... 33

Data Analysis ............................................................................................................ 34

Reliability tests.......................................................................................................... 34

Assumptions tests...................................................................................................... 35

Results ........................................................................................................................... 37

Descriptive Statistics................................................................................................. 37

Research Question .................................................................................................... 39

Hypothesis................................................................................................................. 39

Results of Hypothesis ............................................................................................... 39

Discussion ..................................................................................................................... 42

Limitations .................................................................................................................... 43

vii
 Conclusion .................................................................................................................... 44

Chapter 3: Intervention Literature Review ....................................................................... 45

Background ................................................................................................................... 45

The Influence of Role Stressors .................................................................................... 46

Role Clarification Literature ......................................................................................... 47

Role Clarification ...................................................................................................... 48

Role Clarification Techniques ................................................................................... 49

Role Analysis Intervention ............................................................................................ 53

Leadership Implications of Role Clarification Intervention ......................................... 57

Conclusion .................................................................................................................... 60

Chapter 4: Intervention Design ......................................................................................... 61

Overview ....................................................................................................................... 61

Intervention Goal and Rationale ................................................................................... 61

Research Questions ....................................................................................................... 62

Intervention Design ....................................................................................................... 63

Outcome Evaluation.................................................................................................. 63

Process Evaluation .................................................................................................... 64

Number of participants. ............................................................................................ 64

Attendance ................................................................................................................ 65

Participant responsiveness. ....................................................................................... 65

viii
 Quality of program delivery. ..................................................................................... 65

Effect Size ................................................................................................................. 67

Method .......................................................................................................................... 69

Participants................................................................................................................ 69

Measures and Instrumentation .................................................................................. 71

Procedure ...................................................................................................................... 73

Intervention ............................................................................................................... 73

Data Collection ......................................................................................................... 77

Data Management ..................................................................................................... 78

Data Analysis ............................................................................................................ 78

Reliability tests.......................................................................................................... 79

Assumptions tests...................................................................................................... 79

Qualitative data. ........................................................................................................ 81

Summary Matrix ....................................................................................................... 82

Conclusion .................................................................................................................... 84

Chapter 5: Findings, Discussion, and Implications for Practice ....................................... 85

Findings......................................................................................................................... 85

Role Ambiguity in Instructional Design Teams ........................................................ 85

Perceptions of the Value of the Role Analysis Intervention ..................................... 87

Collaboration and reflection on roles ........................................................................ 89

ix
 Role stressors and challenges.................................................................................... 98

Intervention Fidelity................................................................................................ 105

Intervention design, implementation, and representation of roles .......................... 108

Discussion ................................................................................................................... 115

Implications for Practice ............................................................................................. 118

Opportunities for Collaboration between Stakeholders .......................................... 119

Processes and Policies Surrounding Online Learning ............................................ 121

Resources to Support Online Course Development................................................ 123

Limitations .................................................................................................................. 125

Conclusion .................................................................................................................. 127

References ....................................................................................................................... 129

Appendices...................................................................................................................... 160

Appendix A ................................................................................................................. 160

Appendix B ................................................................................................................. 164

Appendix C ................................................................................................................. 166

Appendix D ................................................................................................................. 167

Appendix E ................................................................................................................. 169

Appendix F.................................................................................................................. 170

Appendix G ................................................................................................................. 171

Appendix H ................................................................................................................. 172

x
 Appendix I .................................................................................................................. 175

Appendix J .................................................................................................................. 176

Appendix K ................................................................................................................. 177

Appendix L ................................................................................................................. 178

Appendix M ................................................................................................................ 179

Appendix N ................................................................................................................. 180

Curriculum Vitae ............................................................................................................. 181

xi
 List of Tables

Table 1: Needs Assessment Participants’ Demographic Information ……………………………32

Table 2: Needs Assessment Reliability Statistics …………………………………………….......37

Table 3: Needs Assessment Descriptive Statistics …………………………………………….....39

Table 4: Summary of Roles Assumed by Participants …………………………………………..40

Table 5: Correlation Matrix ………...……………………………………………………………42

Table 6: Data Collection Matrix Assessing Fidelity ...……………………………………………71

Table 7: Intervention Participants’ Demographic Information …………………………………...75

Table 8: Intervention Timeline ……………...……………………………………………………82

Table 9: Intervention Reliability Statistics ……………………………………………………….84

Table 10: Evaluation Summary Matrix …………..………………………………………………87

Table 11: Intervention Descriptive Statistics ………………….……………………………….....92

Table 12: RQ2 Hierarchical Analysis Results ...…………….………………………………........94

Table 13: RQ3 Hierarchical Analysis Results ...…………….………………………………......114

xii
 List of Figures

Figure 1: Needs Assessment Scatter Plots ……………………………………………………….38

Figure 2: Pathways to Successful Team Relationships ...………………………………………....59

Figure 3: Power Analysis ………………………………………………………………...............73

Figure 4: Intervention Box and Whisker Plots ……………………………………………….......85

Figure 5: Intervention Q-Q Plots ……………….…………………………………………….......86

Figure 6: Related Sample Sign Test Results ……………….……………………………………..93

xiii
 Chapter 1: Introduction of the Problem of Practice

Overview

With the rapid growth of information and communication technologies, eLearning has

become increasingly prevalent in higher education (Simonson, Smaldino, Albright, & Zvacek,

2012). Over the past decade, there has been a concerted shift toward eLearning in both

traditional, brick-and-mortar institutions, and institutions focused primarily on the distribution of

online learning and other distance education products (Allen & Seaman, 2016). Increasingly,

institutions of higher education are augmenting or replacing traditional face-to-face courses with

blended and fully online courses (Allen & Seaman, 2016). Between 2002 and 2011 the number

of students taking at least one online course increased by nearly 350%, from 1.6 million to 6.7

million (Allen & Seaman, 2011), and 28% of students currently enrolled in higher education take

at least one distance education course (Allen & Seaman, 2016). In response to these shifts,

almost two thirds of chief academic leaders stated that online learning is critical to the long-term

strategy of their institutions (Allen & Seaman, 2016).

As institutions continue to adapt courses to online formats, the quality of those courses

becomes a critical driver in the creation of effective and marketable education programs (Clark

& Mayer, 2011). Course quality has the potential to significantly influence student outcomes

(Gore, Bond, & Steven, 2000; Xu & Jaggars, 2013), student satisfaction (Bolliger & Martindale,

2004; Lee, 2010), student retention (Dietz-Uhler, Fisher, & Han, 2007; Herbert, 2006), and

overall effectiveness of an organization (Koehler & Mishra, 2005; Newton, 2003; Strawser,

Buckner, & Kaufmann, 2015). Therefore, institutions of higher education are challenged to

create and promote genuine and sustainable learner engagement and deliver effective, high

quality online learning experiences for students (Bart, 2012).

1
 Effective online learning requires the use of a systematic process when building

instruction to enable faculty and students to confidently navigate and harness the power of this

fluid medium. However, creating successful online learning experiences that are grounded in and

informed by research principles and best practices remains a challenge for higher education

institutions. Online courses often lack design considerations and may focus only on the delivery

of instruction rather than active student learning (Carr-Chellman & Duchastel, 2000; Chuang,

2010). Online courses that are designed for the mere distribution of information and lack

effective online instructional design practices can result in student dissatisfaction and poor

educational outcomes (Jaggars & Xu, 2016). Therefore, online learning should be designed to

provide pedagogical, technological, and social affordances for meaningful teaching and learning.

Just as the quality of online courses informs student outcomes, online course design

significantly informs the quality of online courses in higher education (Stevens, 2013).

Ultimately, both the design process and roles of instructional design team (IDT) members are

critical to the success of online course design and so the quality of online courses (Kearsley,

2000). The online course design process involves several strategic steps, including: identifying

measurable learning objectives; aligning course assessments, instructional materials, and learner

activities associated with the learning objectives; and selecting appropriate course technologies

that support the learning objectives and promote student engagement (Discenza, Howard, &

Schenk, 2002). The online course design process will also typically involve several personnel

(such as instructional designers, instructional technologists, project managers, executive team

members, and others involved in the instructional design process, such as faculty), with each

member of the instructional design team (IDT) playing one or more crucial roles in the design,

development, and eventual deployment of an online course. Together, members of the IDT,

2
working through the course design process, ensures the timely and satisfactory completion of the

online course according to the quality standards established by the IDT and the institution more

generally.

With the dynamic nature of the online course design process and the various role

expectations and responsibilities of IDT members, role stressors within IDTs have the potential

to affect the quality of the online courses (Boles, Wood, & Johnson, 2003; Idris, 2011; Kemery,

2006; Onyemah, 2008; Strawser, Buckner, & Kaufmann, 2015). Katz and Kahn (1978) identified

three separate yet related constructs of role stress: role conflict, role ambiguity, and role

overload. Role conflict results from the simultaneous occurrence of incompatible role demands

from two or more role senders (e.g., faculty, instructional design manager, director). Role

ambiguity occurs due to the lack of clear expectations and responsibilities (e.g., ambiguity

related to faculty’s role in the online course design process). Role overload occurs when there is

lack of time and resources needed to meet one’s role expectations (e.g., instructional designers

expected to design and develop online courses in a condensed timeframe). Overall, role stress,

“the pressure experienced by an individual as a result of organizational and job-specific factors

in the form of demands and constraints that have been placed on them” (Kahn et al., 1964. p.

237), has the potential to negatively influence IDT members’ performance and the quality of

online course design process, and consequently, the quality of online courses delivered by higher

education institutions.

Quality in Online Education

The rapid growth of eLearning makes the quality of online education difficult to define

and control. As Greenberg (2010) noted, “quality as a descriptive term in higher education is

notoriously hard to define.” What is more, ensuring quality in online learning is complex and

3
“depends on a range of factors arising from the student, the curriculum, the instructional design,

technology used, and faculty characteristics” (Meyer, 2012, p. 101). In turn, while many

educators view online education as a medium that offers great opportunities for the present and

the future, others have questioned the quality of instruction online education provides (Allen &

Seaman, 2016). Allen and Seaman (2016) found that only approximately 30% of higher

education administrators reported that their faculty recognize the value and legitimacy of online

learning. The great potential of online learning is found in its flexible and asynchronous nature;

however, this flexibility is precisely what has led many to doubt the quality of online courses

(Allen & Seaman, 2016; Greenberg, 2010). Thus, a major challenge for online education is

defining and ensuring a quality standard amidst the ever-changing nature of the online learning

landscape and rapidly changing technologies.

Despite uncertainty over quality and quality standards, and the protean nature of

eLearning, characteristics of quality online education will necessarily resemble the

characteristics of quality education in general. While online education calls for different

instructional strategies, its outcomes and instructional objectives are parallel to traditional

notions of education (Anderson, 2008). Garrison (2017) remarked similarly, suggesting that “e-

learning is not a radical new innovation but a return to traditional values [of education]” (p. 2). In

this view, theories that have historically emerged from and been applied to traditional education

can inform the creation of quality online educational experiences. As Amdrade (2015) noted,

“Theories provide a foundation for training and guide instructors in establishing a quality online

teaching and learning experience” (p. 1). Further, theories can offer insight into the pedagogical

basis for student learning and enable faculty and instructional designers to make sound

instructional strategies for the online environment (Ertmer & Newby, 2013). Therefore,

4
traditional and emerging learning theories can serve as the cornerstone for the design and

delivery of quality online learning experiences, as they guide educators in their thinking about

how students learn and process information (Ertmer & Newby, 2013).

Theoretical Foundations of Quality Online Education

Historical and emerging learning theories have implications for defining and

operationalizing quality in online education. Three historical learning theories that have

influenced education and guided instructional practice since the 1800s—and that now inform the

design of quality online learning experiences—are behaviorism, cognitivism, and constructivism.

Behaviorism focuses on the external factors that influence learning and “equates learning with

changes in either the form or frequency of observable performance” (Ertmer & Newby, 2013, p.

48). The implications of behaviorism for online learning include providing students with explicit

learning competencies, testing students to determine whether they have achieved the desired

competencies, appropriate sequencing of instructional materials, and instructor feedback to help

students monitor their progress. Cognitivism focuses on the importance of the acquisition of

knowledge and the internal mental structures involved in how individuals acquire, process, store,

and retrieve information (Ertmer & Newby, 2013). The implications of cognitivism for online

learning include helping students retrieve existing information to make sense of new

information, incorporating activities and assessments that allow for deep learning and transfer of

knowledge to long-term memory, providing information in manageable chunks to prevent

information overload, and utilizing the appropriate mode for the delivery of information (e.g.,

visual, audio, text). Constructivism draws attention to how individuals create meaning from their

experiences. Learners are viewed as actively engaged in the interpretation and processing of

information and the creation of knowledge (Ertmer & Newby, 2013). The implications of

5
constructivism for online learning include actively engaging students in the learning process to

encourage personalized learning, providing students with opportunities to reflect on their

learning, establishing real-life connections to the course content, and providing opportunities for

collaborative and cooperative learning experiences in the online environment.

Another theory often used to support quality online and blended learning is activity

theory, which takes into consideration the cultural contexts and historical aspects of learning

(Carter, 2013). Activity theory explains learning as mediated by cultural tools and students’

contributions and activities within culturally defined tools, including language. It goes beyond

the learners, the interface, and the content, to take into account the social and cultural

components that affect learning. According to activity theory, “activity cannot be understood

or analyzed outside the context in which it occurs” (Jonassen & Rohrer-Murphy, 1999, p. 62).

Therefore, a key implication of activity theory for online learning is taking into consideration

learners’ context and presenting “instructional materials through situated activities with the

concepts represented in those materials” (Jonassen, Tessmer, & Hannum, 1998, p. 164).

Similar to activity theory, situated learning theory emphasizes the importance of activity,

context, and culture to learning. Situated learning theory proposes that real-world activities and

authentic contexts are essential for learning to occur (Anderson, 2008). Social interaction is also

a key component of situated learning where students engage in a community of practice. Situated

learning theory has many key implications for online learning, including creating activities that

have real-world applications (rather than mere memory checks or recall exercises), creating

problem-based learning and problem-centered activities, and providing students with

opportunities for social interaction and transactions with their peers.

6
 Along the lines of creating authentic learning experiences, pragmatism highlights the

importance of experience-based learning and hands-on application (Siemens, 2005). One of the

key principles of pragmatism is practical learning. Pragmatism proposes that students should

acquire the knowledge and skills needed to solve everyday problems and situations. When

designing online instruction, pragmatism calls for incorporating problem-solving methods where

students learn by doing. According to the pragmatic approach, students should engage in

activities that are real, purposeful, and reflect their day-to-day lives.

The complex and intricate nuances of online learning have resulted in other theories and

models developed to help educators understand and adapt to these unique challenges and

opportunities. Behaviorism, cognitivism, and constructivism were developed “in a time when

learning was not impacted through technology . . . [and therefore] do not address learning that

occurs outside of people (i.e., learning that is stored and manipulated by technology)” (Siemens,

2014, p. 15). Connectivism emerged as a learning theory for the digital age, positioning learning

as occurring through connections within networks. Connectivism views students “as a network

phenomenon influenced by technology and socialization” (Goldie, 2016, p. 1). Furthermore,

given the abundance of information available to students in the digital age, connectivism

underscores the importance of helping students develop metaskills—the ability to search, find,

retrieve, and critically evaluate relevant information from various networks.

Garrison, Anderson, and Archer (2000) developed the Community of Inquiry (CoI) as a

framework for creating engaging and immersive online, face-to-face, and blended learning

experiences with the ultimate goal of helping students realize these metaskills. The CoI

framework facilitates meaningful learning through three interdependent elements: social,

teaching, and cognitive presence (Garrison et al., 2000). Social presence includes the affective

7
climate and relationships within the community. Teaching presence refers to the design,

facilitation, and direction of the course. Cognitive presence involves the progressive phases of

practical inquiry leading to resolution of a cognitive challenges. Cognitive presence—activated

in the pursuit of the course academic objectives—involves learners constructing and confirming

meaning through reflection and discourse in a shared online environment (Anderson, 2008). The

core thesis of the CoI framework is that in an “environment that is supportive intellectually and

socially, and with the guidance of a knowledgeable instructor, students will engage in

meaningful discourse and develop personal and lasting understanding of course topics” (Rourke

& Kanuka, 2009, p. 21). The CoI survey instrument has been widely used to measure and guide

the quality of online course design (Garrison, 2017; Moore & Shelton, 2013 Swan, Day, Bogle,

& Matthews, 2014).

Another example of the developmental approach to evaluating the quality of online

courses is Quality Matters (QM), “a faculty-centered, peer review process that is designed to

certify the quality of online and blended courses” (“Quality Matters,” 2016). Developed by

MarylandOnline, QM offers both a process and rubric for evaluating online and hybrid courses.

“The process is formal peer review of online courses by a trained group of reviewers.”

(Greenberg, 2010, p. 2). The QM rubric is based on research-supported national standards of best

practices in online course design and contains a set of standards that evaluates online courses on

the following criteria: course overview and introduction, learning objectives (competencies),

assessment and measurement, instructional materials, course activities and learner interaction,

course technology, learner support, and accessibility and usability.

Several studies have been conducted to measure the validity of the QM rubric and its

impact on student learning. Legon (2006) compared the QM rubric standards to the “Best

8
Practices for Electronically Offered Degree and Certificate Programs” endorsed by the Council

for Higher Education Accreditation. The QM rubric is “fully consistent with published

accreditation standards for online education” and “the implementation of QM reviews in an

institution or program can serve as a major element of the quality assurance process for online

education that accreditation requires” (Legon, 2006, p. 9). Hoffman (2012) reported that the QM

review process can help faculty create well-designed online courses and ensure that learning

objectives are aligned with instructional materials, learning activities, course technologies, and

assessments.

Researchers have also explored using the QM rubric to evaluate the quality of online

courses and found a positive correlation between QM adoption and increases in student retention

and satisfaction (Altman, Schwegler, & Bunkowski, 2014; Ralston-Berg & Nath, 2008; Swan et

al., 2012). Ralston-Berg and Nath (2008) found that “students who claim high satisfaction in

online courses also significantly value all QM features more than students who claim low

satisfaction generally in online courses” (p. 3), revealing a positive relationship between QM

items and students’ satisfaction in online courses. Likewise, Hixon, Buckenmeyer, and Barzcyk

(2015) extended the work of Ralston-Berg (2014) and found that students value the QM criteria

regardless of whether the course is offered in a traditional, blended, or online format.

Several frameworks and models have been developed based on principles of quality

design as defined by students’ mastery of desired learning outcomes (Reigelut, 2013). At the

core of these instructional design models is understanding and operationalizing how learning

takes place and how sound instructional design practices can yield the most efficient learning

among students. These instructional design theories seek to make connections between learners,

their inherent ways of learning, and the intended outcomes of learning.

9
 Bloom’s (1956) taxonomy has been widely used in online learning for describing

learning objectives. Learning objectives can be used to help students recognize what is expected

of them, focus on key learning activities, and organize personal study efforts. In an online

environment, where students are disconnected from the physical teaching environment, well-

written learning objectives can provide direction to the learning process and serve as a basis for

evaluating student progress (Brown & Green, 2016). Mager (1962) proposed that a learning

objective is a “description of a performance you want learners to be able to exhibit before you

consider them competent” and should contain three identifiable parts: action, condition, and

criterion. Dick, Carey, and Carey (2009), and Smaldino, Lowther, and Russell (2012) have

proposed similar approaches to learning objectives, focusing on actions, conditions, and criteria.

A commonly used framework for establishing effective online instruction comes from

Merrill (2013), who proposed that learning is promoted when learners are engaged in solving

real-world problems; existing knowledge is activated as a foundation for new knowledge; and

new knowledge is demonstrated to the learner, is applied by the learner, and is integrated into the

learner’s world. Rather than focusing on creating objectives early in the instructional design

process, Merrill proposed that the first step should be creating “an instance that represents the

whole problem that learners will be able to solve following the instruction” (Merrill, 2013, p.

254).

Gagné, Briggs, and Wager (1992) theorized nine events of learning to provide a

systematic approach to course design, namely: (a) gain the learners’ attention; (b) inform learners

of the objective; (c) stimulate recall of prior learning; (d) present the stimulus; (e) provide

guidance for the learners; (f) elicit learner performance; (g) provide feedback; (h) assess learner

performance; and (i) enhance retention and transfer. Gagné’s nine events of instruction have

10
been widely used in online education to design engaging and meaningful instruction for learners

(Gagné, Briggs, & Wager, 1992).

Taking a different approach to instructional design, Wiggins and McTighe (2011)

proposed the backward design framework in which “one starts with the end—the desired results

(goals or standards)—and then derives the curriculum from the evidence of learning

(performances) called for by the standard and the teaching needed to equip students to perform”

(p. 8). This framework organizes the design process in three phases: (a) identify desired results,

(b) determine acceptable evidence, and (c) plan learning experiences and instruction (Wiggins &

McTighe, 2011, p. 9).

Defining Quality

While several researchers and practitioners have proposed theories as frameworks for

designing quality online education, it is unlikely that a single theory can fully explain or account

for how learning takes place in a technology-driven environment (Ertmer & Newby, 2013).

However, a combination of various theories, approaches, and frameworks can potentially provide

an adequate description of and definition for what characterizes quality in online education. In

this way, irrespective of a specific theory, there are several recurring themes that inform the

general characteristics of a quality online experience, such as authentic learning experiences and

opportunities for interactivity and collaboration (Rossett, 2002). Creating such meaningful

learning experiences requires intentional design of not only learning outcomes but also the

strategies and selection of supporting content to achieve those outcomes. Thus, while certain

theoretical foundations support the definition of quality in online education, it is the design

process that supports the actualization of this quality.

11
 Instructional Design

As Ertmer and Newby (2013) pointed out, “the critical question . . . is not ‘which is the

best theory?’ but ‘which theory is the most effective in fostering mastery of specific tasks by

specific learners?’” (p. 61). Thus, quality course design—that is, instructional design—can be

construed in terms of a series of decisions regarding course objectives and the most effective

methods of making sure that students achieve those objectives (Discenza, Howard, & Schenk,

2002). Instructional design in general requires that instructors and instructional designers create

course content that is functional, navigable, and accessible. Online course design requires that

instructors and instructional designers create courses that are also effective in a technology-

driven environment (Discenza, Howard, & Schenk, 2002).

Instructional Design Process

Over the years, the definition of instructional design (often used interchangeably with

course design) has evolved significantly. Reigeluth (1983) saw instructional design functionally,

as the act “of deciding what methods of instruction are best for bringing about desired changes in

student knowledge and skills” (p. 7). More recently, instructional design has come to be viewed

as “the systematic and reflective process of translating principles of learning and instruction into

plans for instructional materials, activities, information resources, and evaluation” (Smith &

Ragan, 2005, p. 4), all based on “learning theories, information technology, systematic analysis,

educational research, and management methods” (Morrison, Ross, & Kemp, 2007, p. 6). The

view of course design as a process, while an important corrective to the functionalist perspective,

can result in the oversight of the theories, research, and practices that undergird the quality of

that process.

12
 In this way, the application of a robust instructional systems design process is a key

predictor of a quality course (Stevens, 2013; Zheng & Smaldino, 2003). The primary goal of the

course design process is to create an online course that meets the desired learning objectives and

complies with national standards of best practices to promote student engagement, retention, and

satisfaction (“Quality Matters,” 2014). Piskurich (2000) discussed the significance of

instructional design:

The instructional design will help you create good, clear objectives for your program that

can be understood and mastered by your trainees. It will help you develop evaluations

that truly test for the knowledge and skills that our objectives are based on. It will help

you or whoever instructs that course to facilitate the participants’ learning effectively and

efficiently and, most important, it will help you make sure that what is in your program is

what your trainees need to learn. This reduces wasted time, wasted money, and wasted

opportunities for helping to develop more effective employees who, through their

knowledge and skills, increase corporate profitability. (p. 3)

Piskurich’s (2000) insights reveal the significance of instructional design as a process to

ensure the quality of online courses. Dijkstra et al. (1997) illustrated the relationship between

process and product with respect to instructional design. They described instructional design as

“a certain mode of producing or developing instruction as well as a product that defines an

educational setting” (p. 27). In this view, instructional design is a process that results in the

creation of a product—the course—in an educational setting.

The online course development process is complex and multifaceted, involving not only

specialized knowledge in the content area but also the allocation of time and consideration to

incorporating sound pedagogical practices in a technology-driven environment (Goodyear, 2005;

13
Redmond, 2011). Other factors that add to this complexity include expertise in implementing

educational technologies, considerations for user experiences, and the faculty’s comfort with

using technology to develop materials and interact with students (Stevens, 2013). The increasing

complexity of the online course development process, requiring new techniques and skills with

the development of new technologies, has led many higher education institutions to adopt a

team-based approach to instructional design, often guided by standards established by

instructional design teams (IDTs) (Hixon, 2008).

Instructional Design Team Roles and Responsibilities

Instructional design teams (IDTs) can include instructional designers, instructional

technologists, project managers, executive team members, and others involved in the

instructional design process, such as faculty. Thach and Murphy (1995) identified 11 roles for

distance education professionals, including: “instructor, instructional designer, technology

expert, technician, administrator, site facilitator, support staff, editor, librarian, evaluation

specialist, and graphic designer” (p. 64). Each of these individuals could be involved in the

instructional design process, assuming specific roles with the potential for overlap between the

roles performed by them. Thus, an online course design process can involve each member of a

team playing one or more key roles to ensure the timely and satisfactory completion of the online

course design and development tasks, resulting in a course that is compliant with the quality

standards established by the IDT.

Instructional designers. Dooley et al. (2007) pointed out the complex and multifaceted

roles that instructional designers play in the course development process. They highlighted the

core competencies for instructional designers, including adult learning, instructional design and

course development, instructional technologies, accessibility, evaluation, and training. Ritzhaupt

14
and Kumar (2015) conducted an exploratory study to gain insight into the skills and knowledge

essential to instructional designers within higher education. Semi-structured, online interviews

were conducted individually among eight IDs in the United States. The results of the study

showed that the role of instructional designers in higher education is not limited to supporting

online faculty but is often extended to include faculty who teach face-to-face and hybrid courses.

Further, instructional designers were found to take on roles outside instructional design, serving

as system administrators, performing website maintenance, and providing program assessment.

According to the study, the skills and knowledge essential to instructional designers include:

knowledge of instructional design and learning theory, soft skills, teaching experience,

willingness to learn, communication skills, ability to juggle multiple projects, time management,

and project management (Ritzhaupt & Kumar, 2015).

Given the complexity of roles and responsibilities within IDTs, Pan and Thompson

(2009) conducted an exploratory, ethnographic case study to gain insight into the individual and

team traits of IDTs. Three particular elements were identified as critical contributors to

instructional designer’s job performance and successful collaboration among IDT members:

team member expertise, motivation (both external and internal) to complete an instructional

design project, and a positive team culture. Successful functioning of the IDT was found to have

a direct correlation to individual expertise, work motivation, and positive team culture.

With respect to instructional designers’ roles and responsibilities in higher education,

several themes emerge from the studies conducted by Pan and Thompson (2009) and Ritzhaupt

and Kumar (2015). One key finding across the two studies is the need for instructional designers

to possess soft skills to effectively navigate the different needs and changes in higher education.

Ritzhaupt and Kumar (2015) found that soft skills such as diplomacy, persuasive skills,

15
relationship building, and emotional intelligence are essential, considering the service role of

instructional designers. Pan and Thompson (2009) found that instructional designers are often

required to have the ability to handle conflict effectively, especially conflicts that arise between

the team and faculty.

Another theme is the need for instructional designers to be proactive, motivated, and

willing to learn, considering the ever-changing nature of online education. Pan and Thompson

(2009) found that in terms of expertise, following a task mental model, assertiveness, and

proactivity were the key contributing factors to instructional designer job performance. In terms

of work motivation, instructional designers’ growth needs—advancement, achievement,

responsibility, and work itself—were significant contributing factors to their job performance

within IDTs. Instructional designers who participated in Ritzhaupt and Kumar’s (2015) study

also reported that being a self-starter and able to self-learn were crucial to their job

responsibilities.

Dicks and Ives (2008) interviewed eight instructional designers to find answers to the

following two questions: “What constitutes good instructional design?” and “How do

instructional designers create good design?” (p. 1). Echoing the findings of Pan and Thompson

(2009) and Ritzhaupt and Kumar (2015), Dicks and Ives (2008) noted the importance of soft

skills to the success of instructional designers. Additionally, Dicks and Ives identified different

responsibilities that the role of instructional designers call for, such as “establishing credibility,

validating their expertise, letting content experts think they are driving the project, finding

middle ground, identifying the needs of clients, making design decisions, managing

commitments, aligning perspectives, flagging constraints, compromising, showing leadership,

16
building relationships, mentoring clients, teaching, using learner feedback to persuade, and

managing resources” (pp. 7–9).

Given the complexity of responsibilities within the position, Hokanson and Miller (2009)

identified four major roles that IDs assume within IDTs: instructional artist, instructional

architect, instructional engineer, and instructional craftsperson. Tracey, Hutchinson, and Grzebyk

(2014) classified the roles and skills of instructional designers to include “the establishment of

design precedents, reflective thinking skills, and the foundations of professional identity” (p.

315). Campbell, Schwier, and Kenny (2009) contended that IDs often serve as agents of social

change within institutions. Their interviews with 20 instructional designers at 6 Canadian tertiary

educational institutions revealed that IDs’ social-change agency roles can be classified into four

categories: interpersonal, professional, institutional, and societal.

The theme of IDs serving multiple roles was also emphasized by Moskal (2012), who

found five major themes with respect to the roles of instructional designers’ skills and traits,

including: flexibility, moral purpose, relationship building, time and project management, and

ongoing professional development. Moskal’s (2012) study highlighted the fact that IDs’ roles

often extend beyond the design of courses, and that IDs often act as information leaders.

Similarly, Liu, Gibby, Quiros, and Demps (2002) interviewed 11 instructional designers to

determine what their responsibilities and challenges were, and what skills were important for

their effective job performance. They found that balancing multiple roles is one of the major

challenges that instructional designers face; instructional designers often perform multiple roles

when working with clients. Furthermore, the participants pointed out that the instructional

designers were often asked to function as project managers, spotlighting the complex nature of

ID work.

17
 Faculty. Two common approaches to online course design are the team-based approach

and the faculty-based approach. The team-based approach is “typified by the large course teams

represented by broad-scale distance educational institutions” (p. 433), whereas the faculty-driven

approach “puts the instructor in the roles of the subject matter expert, course designer, manager,

and implementer” (Hawkes and Coldeway, 2002, p. 434). Even in institutions that follow the

team-based approach, faculty play crucial roles in the design process as subject matter experts

(Schwartzman, 2006). These roles include, but are not limited to: reviewer of work, test student,

coach to other faculty, project manager, graphical consultant, evaluator, and help-desk consultant

(Hawkes & Coldeway, 2002). Reilly, Vandenhouten, Gallagher-Lepak, and Ralston-Berg (2012)

also confirmed the complex roles faculty play in eLearning, which include serving as the content

facilitator, course designer, collaborator, and technologist. They observed that the role of faculty

requires them to work collaboratively with instructional designers during the instructional design

process to create learning activities, design assessments, and develop course materials. They

pointed out that ensuring faculty know “how to interact with support staff is a best practice for e-

learning development” (p. 101).

Akdemir (2008) conducted semi-structured interviews with four faculty members to

investigate their experiences in online courses. Akdemir found that the faculty’s interest in using

technology and their skill level had a positive correlation to their role in online course design—

“faculty members with more technical skills stated that they had designed and developed their

own online courses” (p. 100). On the other hand, “faculty members with less technical skills

declared that they designed their own online courses but they received technical help from

assigned graduate assistants to develop their courses on the online course management systems”

(p. 100).

18
 Other studies conducted by Gerlich (2005), Strawser, Buckner, and Kaufmann (2015),

and Koehler and Mishra (2005) also highlighted the key roles that faculty play in the design of

online courses. Several factors affected the roles that faculty found themselves in during the

course design process, including faculty members’ interest in technology, time they can devote,

the availability of resources, and the presence of a course development process (Akdemir, 2008;

Oblinger & Hawkins, 2006). Furthermore, Zheng and Smaldino (2006) suggested that additional

research should investigate how instructors “look at their role as instructional designers and how

they apply instructional design elements in designing distance courses. There is a need to

examine how distance instructors understand and apply instructional design elements” (p. 35).

Conceicao (2006) suggested that increasing awareness of instructors’ roles in the online course

design process and bringing to light the need for better faculty training and support among

administrators and other executive decision makers are essential to improving the quality of

online education. Conceicao (2006) wrote, “experience is a valid source of knowledge and that

one way to understand how faculty members experience online teaching is by studying situations

using faculty members’ reconstructed experiences and elaborating on the meaning they assign to

those experiences” (p. 27). This approach can lead to insights into faculty’s experience and

perceptions of their roles in designing and developing online courses.

Instructional technologists. The increased adoption and use of technology in academia

has resulted in the development of and appreciation for the specialized role of instructional

technologists. Shell, Crawford, and Harris (2013) defined the role of an instructional technologist

as being “responsible for understanding the goals of the course and recommending and

implementing appropriate technologies for meeting these goals” (p. 150). Some of the most

commonly used metaphors for the role of an instructional technologist include: consultant,

19
computer-aided instruction developer, trainer, lab manager, distance learning expert, and

technician (Surry, 1996). While the primary focus of instructional technologists appears to be the

implementation and use of technology, the diversity of information communication teams in

higher education institutions often require them to adopt additional roles such as conducting

training, performing special projects (such as designing a computer lab), developing instructional

materials, and even providing pedagogical support and guidance in the design and development

of online courses (Sugar, 2005).

Under the broad umbrella of supporting technology adoption and implementation,

instructional technologists can function as technology coaches (Sugar, 2005). Although not all

instructors may need a technology coach, those who are reluctant or skeptical about new

technologies may benefit from instructional technologists assuming a mentoring role (Sugar,

2005). “They need the extra confidence boost and cajoling from their technology coach to feel

confident to start using the particular technology. They are not ready to learn the necessary skills;

they need to have empathetic patience from the particular technology coach to proceed” (Sugar,

2005, p. 564). Furthermore, in assuming the role of a technology coach, instructional

technologists may move beyond the technical considerations and interpret technology needs to

suit instructor’s needs and context (Sugar, 2005).

To examine the various characteristics of instructional technologists, Oliver (2002)

conducted in-depth interviews. Oliver highlighted major activities of instructional technologists’

work: (a) collaborative curriculum development; (b) administrative, technical, research or

management functions; (c) engagement in broader educational issues; (d) learning as well as

teaching during collaboration; and (e) being responsible but without authority. Hixon (2008) also

noted the collaborative nature of instructional technologists’ job function, helping faculty

20
determine possible technology options to support learning objectives. Even though instructional

technologists support faculty and other stakeholders in key technology choices, their authority in

the decision-making process appears to be limited (Hixon, 2008; Oliver, 2002). While

instructional technologists may bring their technical expertise to the course design process, the

ultimate decisions on the tools and technology-related practices to be implemented often rests

with the subject matter expert (i.e., faculty) or the instructional designer.

Considering the various roles that instructional technologists play, İzmirli and Kurt

(2009) conducted a descriptive research study to examine the key competencies of an

instructional technologist. In addition to noting the traditional roles and technology competencies

(hardware, software, and virtual environments), they highlighted key social competencies such as

cooperative working, communication, and planning, as well as educational competencies such as

knowledge in adult education, instructional design, consulting, and technology integration. These

findings are in accordance with existing literature on the diverse roles and responsibilities of

instructional technologists (Campbell, 2008; Palmieri, Semich, & Graham, 2010; Shell,

Crawford, & Harris, 2013; Sugar, Hoard, Brown, & Daniels, 2012). Overlap in the roles and

responsibilities of instructional designers and instructional technologists is evident, and the scope

and complexity of the online course development process may require that the roles be

interchangeable.

Role Stressors in Instructional Design

Role Theory

Considering the diverse, multifaceted, and complex nature of the instructional design

process and IDT member roles, it is useful to explore how the process and associated roles affect

the design and, therefore, the quality of online courses. A role “is a set of behaviors pertaining to

21
a particular task or social function” (Collins, 1982, p. 109). According to role theory, a role is an

individual’s behavior in relation to his or her social environment (Katz & Kahn, 1978). Katz and

Kahn (1966) stated that the “role that a person takes is the central fact for understanding the

behavior of the individual” (p. 45). Role behaviors, which are “the recurring actions of an

individual, appropriately interrelated with the repetitive activities of others so as to yield a

predictable outcome” (Katz & Kahn, 1978, p. 189), often result from organizational, social, and

personal demands. Katz and Kahn (1966) also stated that individuals may take on multiple roles

and activities within an organization, and may change roles (Katz & Kahn, 1978). When

individuals assume one or more roles, they are expected to perform certain activities associated

with those roles, even though the specific functions may not be explicitly known by the

individuals. This lack of clarity can lead to frustration both individually and corporately,

resulting in a failure to fully meet objectives.

Kahn et al. (1964) defined role stressor as “the pressure experienced by an individual as a

result of organizational and job-specific factors in the form of demands and constraints that have

been placed on them” (p. 237). According to role stress theory, there are three major types of role

stressors: role conflict, role ambiguity, and role overload (Katz & Kahn, 1978; Rizzo, House, &

Lirtzman, 1970). Role conflict is the simultaneous occurrence of two or more incompatible role

demands where compliance with one makes it more difficult to comply with the other (Katz &

Kahn, 1978). Role ambiguity is the lack of a clear set of role expectations required for the

adequate performance of one’s role (Katz & Kahn, 1978). Role overload is the lack of adequate

time and resources needed to meet one’s role expectations (Spector & Jex, 1998). Conceptually,

all three components of role stress—role conflict, role ambiguity, and role overload—are

referred to as role stressors (Boles et al., 2003; Kemery, 2006; Onyemah, 2008), with each

22
individual factor having the potential to significantly affect the quality of the design process and

ultimately the quality of the course.

Stressors in Instructional Design Teams

Considering the complex nature of instructional design teams (IDTs) and the roles that

instructional designers, instructional technologists, and faculty play within them in the online

course design process, understanding the dynamics of these teams and the potential challenges

that could arise due to role stressors can potentially improve the quality of the course design

process and the course itself. Research on IDT members reveals common themes related to

collaboration and team dynamics, including balancing multiple roles (Larson, 2005), the

significance of team culture (Macpherson & Smith, 1998; Yusoff & Salim, 2012), individual and

team traits of IDTs (Pan & Thompson, 2009), and social and intellectual skills (Dicks & Ives,

2008). Briggs (2005) noted the, “potential challenges with role overload, role ambiguity, and role

balance (e.g., time spent on each role)” (p. 264) within IDTs. Clear roles and responsibilities are

essential to ensure successful job performance as well as efficiency in IDTs, and the quality of

the courses produced by these teams (Briggs, 2005). While other studies have explored the

effects of role ambiguity, role conflict, and role overload during online course delivery

(Arbaugh, 2004; Bork & Rucks-Ahidiana, 2013; Heuer & King, 2004), fewer studies have

examined the effect of these role stressors among IDTs during the online course design process.

Summary of Literature Review

The problem of a lack of effective design practices in the development of online courses

is supported in the literature and can result from a complicated interplay of several factors.

Research sheds light on the complexity of instructional design for online courses and the

multifaceted and diverse roles assumed by individuals within instructional design teams (IDTs)

23
(Dooley et al., 2007; Hawkes & Coldeway, 2002; Hixon, 2008; Oliver, 2002; Pan & Thompson,

2009; Ritzhaupt & Kumar, 2015; Sugar, 2005; Thach & Murphy, 1995). Essential roles of

instructional designers and faculty are highlighted as well as traits that contribute to successful

job performance. Specific to the roles and responsibilities, future researchers could focus on the

effect and practical use of group motivation in IDTs. Pan and Thompson (2009) suggested that,

at both a team and organizational level, more research is needed to address group motivation.

They proposed groupthink as a potential concern to team decision making. The effect of work

empowerment on instructional designers’ job performance. Pan and Thompson (2009) pointed

out that “though employee’s work empowerment has a positive effect on job performance,

whether they are capable and whether they are willing to be empowered may be two legitimate

concerns” (p. 49).

Ritzhaupt and Kumar (2015) proposed that “the location where an instructional designer

is housed has implications for his or her job role within the organization” (p. 64). Some higher

education institutions hire instructional designers within the units they support while others have

centralized instructional design staff in a shared center. Also, there may be variability in the roles

that instructional designers serve among the various types of higher education institutions—

private colleges, public universities, community colleges, etc. While some participants of

Ritzhaupt and Kumar’s (2015) study indicated that their academic degrees had significantly

prepared them for the instructional design career, others noted that these academic programs

could be improved to better prepare graduates “for the unique conditions found in the context of

higher education” (p. 65). Future research could focus on ways that academic programs could

address the growing needs and changing demands that instructional designers face in higher

education.

24
 Role stressors in IDTs can contribute to online courses not reflecting effective

instructional design practices. One key gap in the literature is how the three major role

stressors—role conflict, role overload, and role ambiguity—impact the quality of the online

course design process, and consequently, the quality of online courses delivered by higher

education institutions. While Briggs’s (2005) study addressed role stressors broadly among

academics in online learning, existing literature does not examine how role stressors impact the

effectiveness of IDT members, specifically. Furthermore, much of the existing research literature

has focused on instructional designers and faculty, and overlooked other key players within IDTs,

such as instructional technologists, project managers, support staff, executive team members, and

teaching assistants. Given the rapid growth of online education and the increasing need to ensure

student retention and satisfaction, future research could examine the effects of role stressors on

the quality of the online course design process with the purpose of enhancing the design process,

and thereby improving online course quality.

Despite advancements in synchronous and asynchronous technologies, online learning

presents a unique set of challenges, including varying technical skill among teachers and

students, their preparedness for teaching and learning online, and the availability of institutional

resources such as student services and development programs to support diverse populations. A

key factor that influences learning experiences in online courses is course design (Stevens,

2013). While several frameworks and models have been developed to create effective online

instruction (Dick, Carey, & Carey, 2009; Gagné, Briggs, & Wager, 1992; Garrison, Anderson, &

Archer, 2000; Mager, 1962; Merrill, 2013; Wiggins & McTighe, 2011), designing meaningful

student learning experience requires “instructional design needs to be sufficiently flexible and to

25
ensure that learning activities and tasks are designed to take learners’ needs and perspectives into

account” (McLoughlin & Oliver, 2000, p. 62).

A critical aspect of instructional design is the process by which IDT members create

online courses and the roles that they play in the course development process (Dooley et al.,

2007; Koehler & Mishra, 2005; Pan & Thompson, 2009). Existing literature highlights the

multiple roles and responsibilities assumed by individuals involved in the instructional design

process as well as the potential for overlap between roles. The challenge for institutions is

creating processes and systems that support individuals who are required to design, develop, and

deliver quality education. Given the complexity of IDT members’ roles, future research should

examine the influence of role stressors in IDT members’ practice and performance (Arbaugh,

2004; Briggs, 2005; Bork & Rucks-Ahidiana, 2013; Heuer) and, specifically, the potential of role

stressors to influence the online course development process and the quality of courses.

26
 Chapter 2: Needs Assessment

Overview

At the core of instructional design is understanding and operationalizing how learning

takes place and how instructional design practices can contribute to the most effective learning

among students. Individuals involved in the instructional design process assume several roles

and responsibilities. Some of the major challenges experienced by instructional design team

(IDT) members are: working with subject matter experts and helping them realize the need for

and value of instructional design; collaborating with other key stakeholders throughout the

process to assess and ensure quality standards; and in all this, managing instructional design

across multiple projects and keeping track of progress (Dicks & Ives, 2008; Hawkes &

Coldeway, 2002; Pan & Thompson, 2009; Ritzhaupt & Kumar, 2015; Shell, Crawford, & Harris,

2013; Sugar, 2005).

Considering the complex nature of the online course design process and the various role

expectations and responsibilities of the IDT members, role stressors within IDTs have the

potential to affect the quality of online courses (Boles, Wood, & Johnson, 2003; Idris, 2011;

Kemery, 2006; Onyemah, 2008; Strawser, Buckner, & Kaufmann, 2015). A growing body of

research highlights the negative effects of role stressors on job performance and satisfaction

(Idris, 2011; Karimi et al., 2014; Keim et al., 2014; Schmidt et al., 2014; Yuryur & Sarikaya,

2012). The purpose of the needs assessment was to examine the relationship between role

stressors—role conflict, role overload, and role ambiguity—in IDTs and the quality of the online

course design process, with the objective of enhancing the design process and thereby improving

online course quality. The needs assessment may be beneficial to instructional designers, faculty,

and other staff involved in the online course design and development process by bringing

27
clarification to the roles of IDT members. Members of IDTs from institutions of higher education

served as participants for the study. Participants provided demographic information and

completed an online questionnaire that measured role conflict, role overload, role ambiguity, and

their overall perceptions of the quality of the online course design process. Data were analyzed

using SPSS.

Research Question

The research question for the needs assessment is:

RQ1: Is there a statistically significant relationship between role stressors in instructional

design teams and the quality of the online course design process?

Hypothesis

The hypothesis for the needs assessment is:

H1: There is a statistically significant relationship between role stressors in instructional

design teams and the quality of the online course design process.

Needs Assessment Design

A cross-sectional survey design, with the data collected at one point in time, was chosen

for the needs assessment. This allowed for the rapid turnaround in data collection in an

inexpensive way and the ability of “identifying attributes of a large population from a small

group of individuals” (Creswell, 2003, p.154). Data were collected using a self-administered

online questionnaire, which made it easily accessible to the potential respondents.

Method

Participants

The participants for the needs assessment were drawn from a convenience sample of

instructional design team (IDT) members working in five types of higher education institutions

28
(public, four-year institutions; private, four-year institutions; public, two-year colleges; private,

two-year colleges; and for-profit institutions) and who have played a key role in the online

course design process within their institutions. IDT members included faculty, instructional

designers, instructional technologists, project managers, executive team members, and others

significantly involved in the instructional design process (such as instructional teaching

specialists and other support staff). This convenience sample was easily accessible by the

researcher and represented various key roles in the online course design process. The sampling

design for the population was single stage, where the researcher had access to names in the

population and was able to sample the participants directly (Creswell, 2003).

The online questionnaire was distributed to a sample of 95 IDT members across the

United States. A total of 55 responses were received, out of which 44 (46.3%) were usable for the

data analysis. Participants consisted of 12 faculty members, 11 instructional designers, 4

instructional technologists, 5 executive team members, 2 fulfilling other roles, and 10 identified

as cross-functional and categorized as multiple for the purpose of data analysis. See Table 1 for

demographic data.

29
Table 1

Participants’ Demographic Information

Category Percentage of Respondents
Ethnicity
White (non-Hispanic) 65.9%
Asian 13.6%
African American 11.4%
Other 6.8%
Did not disclose 2.3%

Gender
Female 59.1%
Male 38.6%
Did not disclose 2.3%

Age
18–30 6.8%
31–40 31.8%
41–50 25.0%
51 and above 36.4%

Roles
Faculty 27.3%
Instructional designer 25.0%
Instructional technologist 9.1%
Executive team member 11.4%
Other 4.5%
Multiple 22.7%

Type of Institution
Private, four-year institution 36.4%
Public, four-year institution 31.8%
Public, two-year college 27.3%
Private, two-year college 4.5%

Education Level
Advanced degree (Master’s or Other) 93.2%
Four-year college degree (Bachelor’s) 6.8%

Years of Professional Experience

1–2 years 6.8%
2–6 years 9.1%
6 or more years 84.1%

30
Measures and Instrumentation

The online questionnaire included components from Rizzo et al.’s (1970) scales to

measure role conflict and ambiguity, and Spector and Jex’s (1998) Quantitative Workload

Inventory (QWI) to measure role overload. An instrument was designed based on the Quality

Matters (QM) rubric to measure the quality of the online course design process. The 30-item

questionnaire included four sub-scales—role conflict, role ambiguity, role overload, and quality

of the online course design process. All of the questionnaire items were measured either on a

five-point, Likert-style scale based on strength of agreement (1 = strongly disagree, 2 = disagree,

3 = neutral, 4 = agree, 5 = strongly agree), or on a five-point frequency of occurrence scale (1 =

less than once per month or never, 2 = once or twice per month, 3 = once or twice per week, 4 =

once or twice per day, and 5 = several times per day) (see Appendix A for the survey

instruments). Following the Likert scales, the questionnaire included an open-ended question:

“Do you have any additional comments or recommendations that will improve the effectiveness

of the online course design process?” This question was specifically designed for participants to

comment on how to improve the quality of the online course design process. The questionnaire

also included demographic questions, including: age, sex, ethnicity, highest level of education,

years of professional experience, the type of institution at which participants worked, and

participant role(s) within an instructional design team (IDT). The instrument was self-

administered through an online survey platform and took approximately 10 minutes for each

participant to complete.

The role conflict subscale consisted of eight items measuring the perception of resource

adequacy, conflicting requests, group interdependence, and different working styles experienced

by IDT members. Scores range from 8 to 40—a score of 8 representing the absence of role

31
conflict and a score of 40 representing maximum role conflict. The role ambiguity subscale

consisted of six items measuring the level of IDT members’ perceived ambiguity about their

roles’ authority and responsibility, their work objectives, necessary information about the job,

and the expectations others have of them. Scores range from 6 to 30—a score of 6 representing

the presence of maximum role ambiguity and a score of 30 representing the absence of role

ambiguity. The role overload subscale consisted of five items representing the elements of

quantity of work, amount of workload, and time pressure experienced by individuals. Scores

range from 5 to 25—a score of 5 representing the absence of role overload and a score of 25

representing maximum role overload.

The role conflict, role ambiguity, and role overload instruments were chosen based on use

in previous research studies (Idris, 2011; Tang & Chang, 2010; Trayambak, Kumar, & Jha,

2012). Ganster, Fusilier, and Mayes (1986), House, Schuler, and Levanoni (1983), and Jackson,

Schwab, and Schuler (1986) have confirmed the construct validity of Rizzo et al.’s (1970) scales

for measuring role conflict and role ambiguity. More recently, Moss (2015) reconfirmed the

reliability and validity of the instrument: “extensive reviews of the psychometric validity of the

Role Conflict and Role Ambiguity Scale concluded that the factor structure of the items is

consistent with the two scales, that it has adequate concurrent and predictive validity, and good

reliability” (p. 50). Spector and Jex’s (1998) QWI was also demonstrated to be reliable with a

Cronbach alpha of > 0.80 (Wyk, 2015).

An 11-item instrument, based on the QM rubric, was designed to measure the quality of

the online course design process within IDTs. Scores range from 11 to 55—a score of 11

representing maximum negative perceptions toward the online course design process and a score

of 55 representing maximum positive perceptions toward the online course design process. The

32
items within the questionnaire were based on the QM rubric, a designed and research-based

inventory for the creation of effective online courses (Adair & Shattuck, 2015). Legon and

Runyon (2007) and Swan, Matthews, Bogle, Boles, and Day (2011) found improved student

outcomes in courses designed on QM principles. Dietz-Uhler, Fisher, and Han (2007) showed

that courses built on QM standards result in significantly higher completion rates. Altman,

Schwegler, and Bunkowski (2014), Hoffman, (2012), Legon, (2006), and Ralston-Berg and Nath

(2008) further demonstrated the positive relationship between the incorporation of QM principles

and student outcomes.

Procedure

Data Collection

Data collection began in April 2016 and ended in May 2016. An email invitation to

participate in the study was sent to 95 individuals who were part of instructional design teams

(IDTs) in higher education institutions. The email also asked potential participants to forward the

invitation to other IDT members in their respective institutions who satisfied the criteria

established for participation in the needs assessment. This allowed for snowball sampling where

the “researcher makes initial contact with a small group of people who are relevant to the

research topic and then uses these to establish contacts with others” (Bryman, 2015, p. 188). The

email included a summary of the needs assessment and its importance, as well as a link to

complete the survey through Qualtrics, an online survey platform. The Qualtrics link reiterated:

the purpose of the study, procedures, risks/discomforts, and anticipated timeframe

(approximately 10 minutes); that measures would be followed to maintain confidentiality; and

personal right to decline participation from the study. IDT members were asked to provide

informed consent through the survey platform (see Appendix B for the form). Participants who

33
did not provide consent were unable to access the questionnaire. All participants completed the

same questionnaire and all responses to the questionnaire were anonymous.

Data Analysis

Data were compiled in an Excel spreadsheet and uploaded to SPSS (Statistical Package

for Social Sciences, Version 24) for analysis. A descriptive analysis of all independent and

dependent variables in the needs assessment was conducted. Assumptions testing was conducted

to check for normality and to determine whether there is a monotonic relationship between the

independent and dependent variable. Spearman’s rho correlation coefficients were used to

examine the relationship between role stressors—role conflict, role ambiguity, role overload—

and quality of the online course design process. “Spearman’s rho measures the strength of an

increasing or decreasing relationship between variables,” (Elliott & Woodward, 2007, p. 192),

including paired-observation ordinal variables, and therefore is suitable for studying the

proposed needs assessment research question.

Reliability tests. Cronbach’s alpha coefficient (α) is a commonly used measure for

determining internal consistency or reliability when using a Likert-scale survey or questionnaire

(Warner, 2013). To establish the reliability of the role conflict (RC), role ambiguity (RA), role

overload (RO), and quality of the online course design process (DPQ) scales, Cronbach’s alphas

were calculated. For the internal consistency of data, 0.65 to 0.80 (or higher) value of Cronbach’s

alpha is recommended (Warner, 2013). As shown in Table 2, the four scales in the questionnaire

have Cronbach’s alpha greater than 0.80. Further, the Cronbach’s alpha coefficient for the total

role stress (TRS) variable—which was created by combining RC, RA, and RO—was 0.875.

34
Table 2

Reliability Statistics

Cronbach’s Alpha Cronbach’s Alpha N of Items

Based on
Standardized Items
RC 0.826 0.829 8
RA 0.868 0.870 6
RO 0.884 0.885 5
TRS 0.875 0.879 19
DPQ 0.895 0.897 11

Assumptions tests.

Data screening. Data screening was performed for independent and dependent variables

using box and whisker plots to identify outliers. The box and whisker plots showed one outlier

for RA and DPQ. Visual inspection as well as records associated with outliers were inspected to

ensure there were no entry errors in the data. Furthermore, to ensure that the two values did not

alter the results significantly, data analysis was conducted with and without the outliers to

compare the outcomes. Since the results remained the same, the outliers did not have a

significant influence in the distribution of the variable, and therefore the outliers were included

in the analysis. Further, the Spearman’s correlation used to test the hypothesis is robust to

outliers (Warner, 2013).

Normality tests. The assumption of normality was tested using the Shapiro-Wilk Test for

normality (n < 50) alpha of .05. Role conflict (p = .436), role overload (p = .227), total role stress

(p = .212), and quality of the design process (p = .139) were not significant, while role ambiguity

(p = .04) was significant. Since the data violated the parametric assumption of normally

distributed data (Field, 2009), a non-parametric test was needed. Further, since the data were in

rank-order, they did not meet the assumptions of the parametric, Pearson’s correlation

35
coefficient. Hence, the non-parametric, Spearman’s rank-order correlation coefficient was used.

Spearman’s rho allows for correlations even within non-normal distribution shapes (Warner,

2013).

Assumptions for Spearman’s Correlation. To use Spearman’s correlation, a monotonic

relationship must exist between variables. The assumption of monotonic relationships was tested

using scatterplots. Lines of best fit indicate monotonic relationships; therefore, the assumption is

met. See Figure 1 for scatterplots.

Figure 1. Scatterplot of dependent variable (DPQ) and independent variables (RC, RA, RO, and
TRS), showing no extreme outliers, linearity, and normal distribution.

36
 Results

Descriptive Statistics

The sample included 44 participants. Data were obtained for the role stressor variables—

role conflict (RC), role ambiguity (RA), and role overload (RO)—and quality of the online

course design process (DPQ). Following previous research (Childs & Stoeber, 2012; Coetzer &

Richmond, 2009; Doraiswamy & Deshmukh, 2015; Zorlu, 2012), the score for total role stress

(TRS) for each participant was obtained by adding the participant’s score for the three role

stressors—role conflict, role overload, role ambiguity. The composite score for each role stressor

variable, including TRS, was analyzed at the interval measurement scale of 1 to 5. Mean,

median, and standard deviation for each variable were calculated. Descriptive statistics were

used to provide a general characterization of the sample (see Table 3).

Table 3

Descriptive Statistics
RC RA RO TRSa DPQ
Mean 3.04 2.26 2.80 2.73 3.50
Median 3.06 2.33 2.60 2.74 3.55
Standard Deviation 0.79 0.74 0.99 0.62 0.71
a
TRS for each participant was obtained by adding RC, RA and RO scores.

The mean scores of the role stress measures were categorized into the following levels:

 Low level: Mean score of 2.4 or below

 Moderate level: Mean score of between 2.5 and 3.5

 High level: Mean score of 3.6 or above

Role conflict had a mean of 3.04 with a standard deviation of 0.79, indicating that the

average respondent reported experiencing moderate levels of role conflict. Similarly, role

overload had a mean of 2.80 with a standard deviation of 0.99, indicating that the average

respondent reported experiencing moderate levels of role overload. On the other hand, role

37
ambiguity had a mean of 2.26 with a standard deviation of 0.74, indicating that the average

respondent reported experiencing low levels of role ambiguity. The total role stress had a mean

of 2.73 with a standard deviation of 0.62, indicating that the average respondent reported

experiencing moderate levels of total role stress. The quality of the online course design process

(DPQ) had a mean of 3.50 with a standard deviation of 0.71, indicating that the average

respondent felt slightly positive about the online course design process.

Data were analyzed to examine if participants reported performing multiple roles within

IDTs. Among the 44 respondents, 10 (22.7%) reported performing more than one role within

their teams. This finding is consistent with existing research literature on the roles and

responsibilities of IDT members (Campbell, Schwier, & Kenny, 2009; Hokanson & Miller, 2009;

Liu et al., 2002; Moskal, 2012; Tracey, Hutchinson, & Grzebyk, 2014). Participants who reported

performing multiple roles indicated their primary role to be faculty, instructional designer,

instructional technologist, and/or project manager. Six participants who indicated their primary

role to be faculty indicated that they also assumed the roles of instructional designer,

instructional technologist, and/or project manager (see Table 4). This finding aligns with the

existing literature on the complex roles faculty play in online course design (Koehler & Mishra,

2005; Reilly et al., 2012; Strawser, Buckner, & Kaufmann, 2015).

Table 4

Summary of Roles Assumed by Participants

Roles (Single and Multiple) Number of Number of Percentage of
Roles Respondents Respondents
Single Roles
Faculty 1 12 27.3 Single Role
Instructional Designer 1 11 25.0 = 34
Instructional Technologist 1 4 9.1 (77.3%)
Executive Team Member 1 5 11.4 Respondents
Other 1 2 4.5

38
 Roles (Single and Multiple) Number of Number of Percentage of
Roles Respondents Respondents
Multiple Roles
Faculty, Instructional Designer 2 4 Multiple
Faculty, Instructional Designer, 3 1 Roles = 10
Instructional Technologist (22.7%)
Instructional Designer, Instructional 2 2 Respondents
Technologist
Instructional Technologist, Project 2 1
Manager
Instructional Designer, Project 2 1
Manager
Faculty, Project Manager 2 1
Total 44

Research Question

Is there a statistically significant relationship between role stressors in instructional

design teams and the quality of the online course design process?

Hypothesis

There is a statistically significant relationship between role stressors in instructional

design teams and the quality of the online course design process.

Results of Hypothesis

A Spearman correlation was used to test the hypothesis that there is a statistically

significant relationship between total role stress (TRS), as measured by the total of the three role

stress subscales (RC, RA, RO), and the quality of the online course design process (DPQ). The

hypothesized relationship was supported although the relationship was weak (r = -0.363),

significant at the 0.05 level (see Table 5). Thus, the relationship specified in H1 is supported.

Among the three role stressor variables measured, role conflict (r = -0.249; p = .118) and role

overload (r = -0.082; p = .595) did not show a statistically significant relationship with the

quality of the online course design process. However, the results showed a moderately negative

39
relationship between role ambiguity and the quality of the online course design process (r = -

0.459; p = .002). See Table 5 for the results of the Spearman correlation.

Table 5

Correlation Matrix among Role Stressors and the Quality of the Online Course Design Process
Variables Online Course Design Process
rs p
Role Conflict -0.249 0.118
Role Ambiguity -0.459** 0.002
Role Overload -0.082 0.595
Total Role Stress -0.363* 0.015
** Correlation is significant at the 0.01 level (2-tailed).
* Correlation is significant at the 0.05 level (2-tailed).

The open-ended question asked: “Do you have any additional comments or

recommendations that will improve the effectiveness of the online course design process?” Even

though the open-ended question did not directly ask participants for their opinions about

perceived role stress, the responses provided valuable insights into its potential influence on the

online course design process and the need for clarity and consistency around instructional design

processes.

Role ambiguity was considered a frequent stressor affecting online course quality, as

evidenced in the responses. The lack of clear and consistent expectations while working with

various stakeholders was perceived to have a negative impact on the online course development

process. This was expressed in the following statements: “Lack of a policy as well as lack of

buy-in from key senior faculty administrators greatly diminishes the effectiveness of the online

course design process”; “I have to work with various groups whose aims are different than

simply quality instruction, and my institution as a whole has no clue about creating work flows

and processes.” Workload was also considered to be a stressor. Participants remarked that the

lack of meaningful institutional support diminished the effectiveness of IDTs: “Let me just say

40
that my situation is unique. I am essentially the instructional design shop. I have a manager, but I

do everything”; “Creating a realistic timeline involves administrative buy-in so that an

appropriate budget (time and costs) [is allocated] for each course.” The breadth of responses to

the open-ended question suggest that a lack of clarity around processes and policies (role

ambiguity), inadequate time and resources (role overload), and incompatible role demands (role

conflict) have the potential to influence online course development.

In addition, consistent with existing literature, some of the responses to the open-ended

question revealed the complexities of the instructional design process and the interrelationships

between various individual, team, and institutional factors (Conceicao, 2006; Gerlich, 2005;

Koehler & Mishra, 2005; Strawser, Buckner, & Kaufmann, 2015). For example, two participants

highlighted the importance of professional development to the successful design and

development of online courses: “I believe that the training for individuals should be a multi-year

proposition, with specific incremental goals each year. The training should increase in

complexity with deliverables of the actual courses to be implemented”; “Faculty preparation for

online course design, such as learning how to write measurable objectives and training in the

learning management system would be helpful in the course design process.” Furthermore, the

widespread concern regarding the overall quality of online learning (Allen & Seaman, 2016;

Greenberg, 2010; Meyer, 2012) was echoed in one participant’s remark: “The quality of our

online courses is abysmal. Granted, our on-campus courses aren’t so hot either.” Other responses

emphasized the importance of related factors such as improving retention and student

performance, complexities surrounding vendor partnerships, and the importance of effective,

successful, and engaging course facilitation by the instructor.

41
 Discussion

The needs assessment examined the relationship between role stressors and quality of the

online course design process. The hypothesized relationship between role stress and the

perceived quality of the online course design process was supported, although the relationship

was weak (see Table 5). The small sample size of 44 participants limits the generalizability of

these findings. However, the results are consistent with the theoretical framework of role theory,

which suggests that unclear expectations, constrained resources, and incompatibility among

different roles can have negative implications on performance and productivity (Kahn & Katz,

1966, 1978).

While role conflict and role overload were both reported by the participants, only role

ambiguity was found to have a statistically significant relationship with the perceived quality of

the online course design process. Role ambiguity suggests a lack of clear role expectations,

which are required for the adequate job performance (Katz & Kahn, 1978). The moderate,

negative relationship between role ambiguity and the perceived quality of the online course

design process suggests that a lack of clear expectations for each member of an instructional

design team (IDT) can influence how individuals perceive the overall quality and effectiveness

of the online course development process. The results also highlighted the relevance of the role

stress construct generally among IDTs, especially considering an increasing level of complexity

and diversity in their roles and responsibilities (Pan & Thompson, 2009; Rapanta et al., 2013).

Extant research reveals the complexity of the instructional design process in online

courses, and the multifaceted and diverse roles assumed by individuals within IDTs (Macpherson

& Smith, 1998; Pan & Thompson, 2009; Pan et al., 2003; Rapanta et al., 2013; Stewart &

Waight, 2008; Yusoff & Salim, 2012). In addition, research also highlights the essential roles of

42
instructional designers, instructional technologists, and faculty, defining the factors, skills, and

traits that contribute to successful job performance (Dooley et al., 2007; Hokanson & Miller,

2009; Pan & Thompson, 2009; Ritzhaupt & Kumar, 2015).

The needs assessment adds to the literature, revealing the importance of helping IDTs

recognize the ways in which role stressors, especially role ambiguity, can exist in their teams and

providing them with strategies and tools to alleviate or minimize the stressors. Awareness of

roles stressors and the ability to mitigate them may lead to an improvement in the online course

design process, and therefore, the quality of the courses produced. Leaders and managers of IDTs

should be aware of their role in establishing and maintaining a positive and productive

environment for their team members. Interventions in the form of stress management and role

clarification training have the potential to reduce role stressors among IDTs and establish a

positive and productive environment for the team members (Srivastav, 2011; Sims, Klein, &

Salas, 2006, Rao & Vijayalakshmi, 2000).

Limitations

There were several limitations to the needs assessment. First, since the sample size was

relatively small (n = 44), generalization of the findings should be made with caution. Second,

while the survey results provided a general understanding of the potential influence of role

stressors, quantitative data alone did not provide in-depth information on the experiences and

potential manifestations of role stressors in instructional design teams. So, while it may be useful

to understand that there is a relationship between role stress and the quality of the online course

design process, it may be more helpful to understand how to mitigate role stress or shape it in

such a way that leads to improved processes.

43
 Conclusion

While research on role stressors within instructional design teams is limited, the findings

of the needs assessment are consistent with existing literature on the complex roles of

instructional design team members (Macpherson & Smith, 1998; Pan & Thompson, 2009; Pan et

al., 2003; Rapanta et al., 2013; Stewart & Waight, 2008; Yusoff & Salim, 2012). Briggs (2005),

in a study of business school academics, found that role overload, role ambiguity, and role

balance inform the quality of online courses. Likewise, in an examination of instructor and

student roles in online courses, Bork and Rucks-Ahidiana (2013) found that role ambiguity has

the potential to cause frustration, confusion, and tension, leading to misaligned expectations

among online instructors and students. Given that IDT members may assume multiple roles and

that those roles are often ill-defined, there is potential for role stressors to influence the online

course development process and, ultimately, the quality of courses.

Given the findings in the existing literature and the needs assessment, the next chapter

explores in more detail the influence of role stressors. It also examines advantages of role

clarification techniques to minimize role stress, especially role ambiguity among instructional

design teams. Further, it offers an evidence-based approach for defining and clarifying

expectations in IDTs to decrease role ambiguity and improve the online course design process.

44
 Chapter 3: Intervention Literature Review

Background

With role theory as a guiding framework, the needs assessment examined the relationship

between role stressors—role conflict, role overload, and role ambiguity—in instructional design

teams (IDTs) and the quality of the online course design process. A weak correlation between

role stressors and the quality of the online course design process was identified. Among the three

role stressor variables measured, role ambiguity was found to have a moderate, significant

relationship with the perceived quality of the online course design process. Furthermore,

participants’ responses to an open-ended question related to challenges of the instructional design

process revealed the potential for role ambiguity to negatively impact the online course design

process.

The results of the needs assessment suggest the need to examine further the sources and

effects of role stress in IDTs. Existing research highlights the negative effects of role stressors,

including uncertainty in the workplace (Schmidt et al., 2014), job insecurity (Keim et al., 2014),

and emotional exhaustion and depersonalization (Yuryur & Sarikaya, 2012). Identifying ways to

help IDTs recognize how role stressors, especially role ambiguity, can exist in their teams and

providing them with strategies and tools to alleviate or minimize the stressors may lead to an

improvement in the online course design process and the quality of the courses produced by

those teams. In this way, role clarification interventions have the potential to reduce role

ambiguity (Rao & Vijayalakshmi, 2000; Sims, Klein, & Salas, 2006; Srivastav, 2011) among

IDTs and establish a positive and productive environment for team members.

45
 The Influence of Role Stressors

The needs assessment focused on the relationship between three role stressor variables—

role conflict, role overload, and role ambiguity—and the quality of the online course design

process. While role conflict and role overload did not show a statistically significant relationship

with the quality of the online course design process, role ambiguity showed a moderately

negative relationship with the quality of the online course design process. In Pareek’s (1983)

Organizational Role Stress (ORS) scale, role ambiguity (RA) is identified as a role stressor that

arises due to lack of clarity in role expectations. Even though studies show that the other two

variables—role conflict and role overload—may have a negative impact on job performance and

satisfaction (Karimi et al., 2014; Idris, 2011; Schmidt, 2014), existing literature also sheds light

on the potential of role conflict and role overload to have positive affects (Lepine, Podsakoff, &

Lepine, 2005).

In a meta-analytic study of work stressors’ relationships with strains, motivation, and

performance, Lepine, Podsakoff, and Lepine (2005) distinguished between challenge stressors

and hindrance stressors. They categorized challenge stressors to include measures of job/role

demands, pressure, time urgency, and workload. Hindrance stressors included measures of

constraints, hassles, resource inadequacy, role ambiguity, role and interpersonal conflict, role

interference, role strain, supervisor-related stress, and organizational politics. The study showed

that hindrance stressors had a negative impact on performance while challenge stressors had a

positive impact on performance. The researchers noted that high workload had a positive impact,

given that it was often addressed through an active, problem-solving method of coping—such as

through increased effort. On the other hand, hindrance stressors, such as role ambiguity, were not

addressed by increased effort, rather through withdrawal and cognitive distancing.

46
 Other studies also highlighted the negative impact of role ambiguity and some of the

potential positive impacts of role conflict and role overload on’ job performance and satisfaction.

Ackfeldt and Malhotra (2013) examined the effect that empowerment and professional

development has on role-stress commitment relationships and the impact of role stress on

organizational commitment. While role ambiguity was found to negatively influence continuance

commitment, role conflict was found to positively influence continuation commitment. In a

similar study, Tang and Chang (2010) examined how role ambiguity and role conflict (through

self-efficacy and job satisfaction) impacted creativity, both directly and indirectly. The results

indicated that perceived role ambiguity had a negative, direct impact on creativity. On the other

hand, perception of role conflict was found to have a positive and direct impact on their

creativity. Using role theory as a framework for a quantitative study of 80 faculty members from

a university in Pakistan, Abbas, Roger, and Asadullah (2012) examined the contribution of

various role stressors to stress and burnout. The results of the study indicated that role ambiguity

significantly affected stress and burnout among faculty.

Overall, the findings on the impact of role overload and role conflict on job performance

and satisfaction are mixed and call for further investigation. However, existing research

consistently highlights the negative effects of role ambiguity, including uncertainty in the

workplace (Schmidt et al., 2014), job insecurity (Keim et al., 2014), and emotional exhaustion

and depersonalization (Yuryur & Sarikaya, 2012). Therefore, a role analysis intervention has the

potential to promote role clarification to decrease role ambiguity in instructional design teams.

Role Clarification Literature

Existing literature and the results of the needs assessment confirm the need to address

role ambiguity in instructional design teams (IDTs). Identifying ways to help IDTs recognize

47
how role ambiguity can exist in their teams and providing them with strategies and tools to

alleviate or minimize the ambiguity could lead to an improvement in the online course design

process, and therefore, the quality of the courses produced by these teams. Interventions in the

form of role clarification have the potential to reduce role ambiguity (Rao & Vijayalakshmi,

2000; Sims, Klein, & Salas, 2006; Srivastav, 2011) among IDTs and establish a positive and

productive environment for the team members.

Role Clarification

Role clarity has been studied from different angles, including job performance, job

satisfaction, and turnover. Research has revealed several positive outcomes of role clarification.

Samie, Riahi, and Tabibi (2015) conducted a cross-sectional descriptive-analytic study to

examine the relationship between role clarity and the efficiency. A total of 133 participants from

the Management and Resources Development Department of the Ministry of Health and Medical

Education of Iran completed two questionnaires on role clarity and efficiency. The results

indicated a significant positive relationship between role clarity and job performance alignment

with organizational goals, work pace, the use of equipment and facilities, training, being

committed to the workplace regulations, self-assessment, and efficiency.

Based on role theory and the path-goal theory of leadership, Hassan (2013) undertook a

quantitative study of 2,136 participants working in 65 geographically dispersed offices of a

government agency to measure the impact of role clarification on the work satisfaction and

turnover rates in workgroups. Data were collected during two time periods from personnel

records and a survey regarding perceptions of managerial practices and their work climate. The

results of the study indicated that role clarification, including clarifying work objectives and

48
performance expectations improved participants’ perceived role clarity, and, in turn, increased

work satisfaction and decreased turnover rates.

Using role theory and leader-member exchange (LMX) theory as frameworks, Lawrence

and Kacmar (2012) conducted a quantitative study of 418 employees of a water management

district to investigate the mediating role of job involvement and role conflict on leader-member

exchange and stress. Participants completed anonymous, written surveys that measured LMX,

job involvement, the extent of role conflict, and stress. As predicted in the hypotheses, the results

showed that LMX was negatively related to role conflict and positively related to job

involvement. Similarly, drawing on role theory and the organizational role stress model,

Rajarajeswari (2010) conducted a quantitative study to analyze stress among faculty in aided and

self-financing colleges. The results of the study showed that faculty working in the self-financing

institutions reported experiencing more stress than their counterparts working in government

institutions. Rajarajeswari recommended managerial interventions to control the workload and

clearly evaluate the duties and responsibilities assigned to faculty.

Role Clarification Techniques

Role clarification techniques have been implemented in several occupational settings to

reduce role stressors and “address the void left unfulfilled by classical job descriptions”

(Srivastav, 2011, p. 103). Role clarification is an intervention in which “the supervisor states his

or her expectations to the direct report subordinate, and together the two parties discuss means by

which the report’s role obligations can be managed effectively” (Schaubroeck, Ganster, Sime, &

Ditman, 1993, p. 4). With the help of a trained process facilitator, the supervisor and the

subordinate can develop a mutual understanding regarding the purpose of the subordinate’s

49
position, the various responsibilities that are prescribed, and the specific elements that may result

in effective job performance.

Responsibility charting, an early intervention strategy proposed to counter role ambiguity,

is a graphical tool used for recording and analyzing organizational structures, departmental

relationships, environmental assessments, strategic alternatives, executive job content, functional

responsibilities and authority, and decision-making processes (Korey, 1988). Expanding on

Korey’s (1988) recommendation, Schaubroeck et al. (1993) proposed responsibility charting to

promote role clarification. They described a responsibility chart as a “diagram of roles held by

members of a top management team within the ‘critical result areas’ (CRAs) of an organization

or autonomous business unit” (Schaubroeck et al., 1993, p. 5). After the identification and

clarification of the list of CRAs, responsibility charting involves conducting an individual survey

of perceived roles within the CRAs, which is then followed by a group discussion to achieve

agreement on each manager’s role in each CRA (Schaubroeck et al., 1993). To test responsibility

charting as an intervention to clarify individual roles, an experimental study was conducted on

the business service division of a major university over a period of two years. The study found

that responsibility charting reduced both role ambiguity and dissatisfaction. It should be noted,

however, that further research is needed to understand the impact of responsibility charting on

reducing role stress (Schaubroeck et al., 1993). Furthermore, responsibility charting places

emphasis on supervisory role clarification and limits “reciprocal exchanges between peers to

achieve coordination” (Schaubroeck et al., 1993, p. 22).

Three techniques have been developed for role clarification: job expectation technique,

role negotiation technique, and role analysis technique (Srivastav, 2011). Huse (1980) developed

the job expectation technique (JET) as an intervention for the distribution of authority and roles.

50
JET is a process in which “members take turns listing their perceived job duties and

responsibilities; others comment until a consensus job definition is reached, and so on until the

whole team has developed an understanding and agreement of each member’s prescribed and

discretionary role space” (Lundberg, 1980, p. 260). JET can be particularly useful when new

members are introduced to a team (Srivastav, 2011).

Another technique developed for role clarification is the role negotiation method

(Harrison, 1972), which is based on the assumption that most individuals prefer a fair, negotiated

settlement rather than a state of unsuccessful conflict. The role negotiation technique involves

each party making a list of their role expectations of each other and then engaging in an

interpersonal negotiation session in the presence of a facilitator. They exchange the lists and

engage in a negotiation discussion with each other (“I will, if you will…”) until both sides agree

on changes in role performance. All parties taking part in the negotiation then receive a master

list of agreements. Role negotiation can be particularly useful when role conflicts are pronounced

(Srivastav, 2011).

The role analysis technique (RAT) was developed to define and clarify role expectations

(Dayal and Thomas, 1968). In RAT, the person occupying a focal role initiates an analysis of that

role and its rationale and responsibilities. Specific duties, behaviors, and responsibilities are

added or subtracted until the role incumbent and the group involved in the role analysis are

satisfied with the role description. A role profile is then developed, which is followed by other

roles analyzed and clarified. RAT can “enhance team task functioning, while also alleviating the

stress associated with role ambiguity” (Buch & Aldridge, 1990, p. 36). RAT can be particularly

suitable as an intervention to reduce role ambiguity (Srivastav, 2011).

51
 Building on Dayal and Thomas’s (1968) role analysis technique, Srivastav (2006)

proposed that the analysis and design of roles should take place based on processes served by the

role. Specific to role clarification interventions for addressing role stressors, Srivastav (2006)

proposed that the analysis and design of roles should take place based on business processes

served by the role. The complex and often dynamic nature of the instructional design process

calls for an intervention that not only targets clarifying roles of those involved but does so within

the context of existing processes. Srivastav (2011) highlighted the significance of designing role

clarification interventions within existing business processes:

An organizational role may contribute to more than one business process. Performance of

an organization can, therefore, be improved by enhancing the format of its business

processes and their alignment with each other. This in turn will lead to reinvigorating

effectiveness of the related roles and technologies employed and their alignment with

each other. Maximization of organizational performance, in fact needs comprehensive

organizational connection among all the organizational components . . . to realign roles

with organizational structure and policies so activities and tasks can effectively be carried

out by the role occupants in their new roles. Processes have to be coordinated with

organizational systems and procedures must be linked with processes ensuring the

organizational system is aligned with organizational structure and policies. In short,

processes must be integrated with roles, and procedures must be aligned with activities

and tasks. (p. 4)

To clarify and align roles and improve the effectiveness of roles based on processes,

Srivastav (2006) proposed process-based role analysis and design (PROBRAD) as an

organizational development intervention. PROBRAD involves a detailed analysis of the focal

52
role and its systematic design/redesign to maximize role effectiveness, the effectiveness of

related business processes, and alignment with organizational components (Srivastav, 2010).

Implementing PROBRAD as a role clarification exercise involves five major steps, including:

(1) organizational study, (2) role-set identification, (3) role-set member training, (4) tentative role

element design, and (5) final role design. After completing these steps, teams record the details

of the role set in the PROBRAD format developed by Srivastav (2011). Each of the five steps

involves several sub-steps, and some intermediate actions should be completed between the

steps. While the implementation of PROBRAD requires significant planning, preparation, and

commitment, it has the potential to help IDTs: enhance the effectiveness of roles and the related

instructional design processes; and identify the weaknesses of structures, policies, systems,

processes, and procedures that could negatively affect role effectiveness (Srivastav, 2010).

Role Analysis Intervention

The intervention strategies in existing literature include responsibility charting

(Schaubroeck et al., 1993), job expectation clarification (Huse, 1980), role negotiation (Harrison,

1972), role analysis (Dayal & Thomas, 1968), process-based role analysis and design (Srivastav

2006), participative decision-making (Carbonell & Rodriguez, 2013; Newton & Jimmieson,

2008), and leader-member exchange (Kim & Barak, 2015; Breevart et al., 2015). While research

highlights the potential for role clarification to reduce role stressors in general, role analysis is

particularly relevant for clarifying role expectations and minimizing role ambiguity (Srivastav,

2010).

The two role analysis interventions that focus specifically on clarifying role expectations

and improving role performance are Dayal and Thomas’s (1968) role analysis technique (RAT)

and Srivastav’s (2006) process-based role analysis and design (PROBRAD). While both

53
interventions involve analyzing roles with the goal of establishing role clarification, there are key

differences. PROBRAD involves the analysis of a critical role (the focal role) and its systematic

design/redesign to maximize role effectiveness and related business processes. RAT involves

clarification of role expectations and obligations to minimize role ambiguity and mismatched

role expectations to improve team effectiveness. While PROBRAD focuses on the design and

definition of the focal role, RAT emphasizes clarifying team members’ role expectations and

requirements. The broad focus of PROBRAD allows for its application at the organizational

level and alignment with organizational goals, mission, and vision. On the other hand, the

narrower and more specific focus of RAT makes it suitable for implementation at a team and/or

project level and exploration of potential overlap, misalignment, and/or ambiguity across team

roles.

In light of the results of the needs assessment and existing literature, a role analysis

exercise based on RAT has the potential to decrease role ambiguity in instructional design teams

(IDTs). RAT is a step-by-step process for defining and clarifying who is responsible for what.

RAT is based on the premise that when members of a team do not know what is expected of them

and/or have different expectations about the roles and responsibilities of team members, it can

result in role ambiguity (Dayal & Thomas, 1968). Schermerhorn, Osborn, Uhl-Bien, and Hunt

(2012) described RAT as:

The Role Analysis Technique, or RAT, is a method for improving the effectiveness of a

team or group. RAT helps to clarify role expectations, and all organization members have

responsibilities that translate to expectations. Determination of role requirements, by

consensus—involving all concerned—will ultimately result in more effective and

mutually satisfactory behavior. Participation and collaboration in the definition and

54
 analysis of roles by group members should result in clarification regarding who is to do

what as well as increase the level of commitment to the decisions made. (p. 333)

The five key steps in RAT involve the following: 1) each team member outlines his or her

role as he or she perceives it; 2) each team member outlines his or her perceived expectations of

each of his or her fellow team members; 3) the team discusses each individual’s roles, focusing

on where expectations match and differ; 4) the team reaches consensus regarding the roles and

responsibilities of each team member; and 5) a role profile is created for each role analyzed.

Carter et al. (2005) developed a visual representation to conceptualize the processes underlying a

role analysis exercise (see Figure 2).

Clearer
expectations
Role and enhanced
clarification team capacity
Recognition
of different
Role analysis perceptions

Figure 2. Pathways to successful team relationships. Adapted from “Roles and Responsibilities

of Team Members,” Collaboration: A Training Curriculum to Enhance the Effectiveness of

Criminal Justice Teams (Carter et al., 2005, p. 130).

The implementation of a role analysis exercise may result in role clarification that

reduces and/or alleviates role ambiguity in the IDT and may lead to an improvement in the

online course design process, and therefore, the quality of the courses produced by these teams.

However, the empirical research on the implications of RAT in teams is scant and none is

dedicated to IDTs. However, existing research highlights the potential benefits of RAT in

improving organizational and team performance (Buch & Aldridge, 1990; Singh, 1997;
55
Srivastav, 2010). Buch and Aldridge (1991) recommended RAT as an organization development

(OD) tool for managing the corporate downsizing process and minimizing its negative

consequences. They noted that following RAT can reduce conflict between and within

departments and contribute to reducing role ambiguity that can arise after downsizing and

restructuring.

While the primary function of RAT is to assist groups in clarifying role expectations and

responsibilities in organizations, it has been adapted for use as an “experiential learning activity

in undergraduate and graduate business school courses” to clarify role expectations for the

instructor and students (Lyons, 1993). RAT was implemented as an icebreaker activity during the

initial class session and allowed the instructor to establish the following: “(a) one may expect to

be actively involved in the course; (b) one’s ideas and contributions have value; and (c) the

course represents joint expectations (exchanges) among students and the instructor” (p. 389).

Lyons (1999) implemented RAT with the staff of a large, residential retirement/health care

facility in the mid-Atlantic region. Since the staff reported experiencing problems with

responsibilities and scheduling, RAT was used to create “definitions, interpretations, and

understandings regarding role, functions, task sequencing, and the like” (p. 7). Even though the

staff displayed some initial resistance and reported that the technique was time-consuming, they

recognized that it was helpful in clarifying role responsibilities.

Meier (2001) conducted a descriptive research study of 35 Northern California school

district superintendents to examine issues that create conflict, the types of conflicts that occur,

and strategies used by superintendents to manage conflicts. The study utilized a semi-structured

interview instrument to guide the telephone interviews with the superintendents and found that

56
while communication and feedback were primarily used for managing conflicts, role analysis

technique was the second most used strategy.

More recently, Beech, MacIntosh, and MacLean (2010) examined how “researchers and

practitioners work together in order to develop solutions to problems in the world of practice . . .

[and the reasons why] dialogues between academics and practitioners appear to remain

problematic” (p. 1342). To uncover how academics and practitioners perceived and acted toward

each other, the researchers adapted Dayal and Thomas’s (1968) role analysis technique and

conducted semi-structured interviews with 10 academics and 10 practitioners. “Each of the 20

interviewees was asked what they thought the other group (practitioners or academics) actually

did and what they should be doing. Interviewees were then asked the same questions in relation

to themselves. In this way, mismatches of expectations and perceived action were discerned”

(Beech et al., p. 1346). The role analysis uncovered the underlying assumptions that academics

and practitioners had concerning each other and highlighted mismatched expectations.

While existing research on the effectiveness of RAT is limited and RAT has not been

studied specifically within the context of higher education, it does open the opportunity to

explore a key gap in existing literature—that is, implementing a role analysis intervention that

focuses on promoting role clarification to reduce and/or alleviate role ambiguity. Furthermore,

the nature and effects of role stress have not been extensively studied within higher education

generally and instructional design team contexts more specifically (Dyer & Dyer, 2013; Harvey

& Drolet, 2004; Lyons, 1993; Rahim, 2010).

Leadership Implications of Role Clarification Intervention

The implementation of RAT as a role clarification intervention has leadership

implications at organizational, team, and individual levels (Buch & Aldridge, 1990, 1991;

57
Schermerhorn et al., 2012; Srivastav, 2010). Considering that RAT involves defining and

analyzing roles related to online course development, it has the potential to bring clarity to

instructional design processes. Since RAT involves teams engaging in an exercise that requires

clarifying roles and responsibilities, it has the potential to motivate and enable instructional

design teams (IDTs) to function more effectively and at the same time reduce the stress attributed

to role ambiguity (Carter et al., 2005). Furthermore, the role clarification that RAT affords may

bring clarity to the specific roles of IDT members in contributing to instructional design

processes, and the overall quality of online courses.

The five critical components of the theory of action for leading school turnaround include

awareness of the problems, understanding why the problems exist, planning to provide focus and

direction to guide action, competence to lead staff to address the problems, and commitment to

lead staff members in addressing the problems (Duke, 2014). These components represent a

framework for implementing an intervention to promote role clarification and decrease role

ambiguity in IDTs. Leaders who manage IDTs should: (a) be aware of the existence of and levels

of role stressors in IDTs; (b) understand the impact of role stressors on the online course design

process; (c) prepare and plan for minimizing the role stressors; (d) exhibit competence in leading

a process change; and (e) commit to achieving the group vision established through a

collaborative process.

For role clarification interventions such as RAT to be successfully implemented,

managers and members of IDTs should recognize role ambiguity as a potential stressor and its

potential negative impact on the course design process. For such awareness, a clear vision for the

IDT and the outcomes at the individual, group, and organizational levels is crucial (O’Connell,

Hickerson, & Pillutia, 2011). Individual leader visioning and collaborative group visioning are

58
essential to clearly establish goals and ensuring team effectiveness (O’Connell, Hickerson, &

Pillutia, 2011). Once the vision of an IDT is clearly established and assimilated, IDTs can

connect vision to action. A clear vision can also help teams see the factors (e.g., role stressors)

that could be potential hindrances to achieving the established vision. The role clarity that RAT

affords may “result in a greater capacity to achieve what the team wants to achieve” (Carter et

al., 2005, p. 130). RAT may allow for the IDT to be better equipped to attend to critical details,

allow for greater accountability within the team, minimize chances for duplication of effort, and

reduce confusion and frustration related to roles and responsibilities (Carter et al., 2005).

An intervention involving IDTs engaging in the role analysis exercise may result in

decreased levels of role stress and role ambiguity. However, the potential limitations of RAT as

an intervention to address role ambiguity should be taken into consideration. Existing literature

on how RAT promotes role clarification is limited and predominantly outside higher education

contexts. Therefore, cross-context transferability could be a challenge. The nuances and

subtleties specific to IDTs and higher education may cause difficulty in addressing and/or

alleviating role ambiguity. Further empirical studies are needed to examine the effect of RAT in

minimizing role stress, specifically role ambiguity, both within and outside of instructional

design and higher education contexts.

One key assumption of the intervention is that higher education institutions recognize and

perceive role stress as a significant factor influencing the online course design process and,

therefore, the quality of the courses produced by IDTs. If institutions do not recognize role stress

as a significant problem or do not want to discuss it, the intervention may not be of interest or

relevant. The intervention also assumes that role stressors negatively contribute to the online

course design process and proposes strategies to reduce them. However, there is some evidence

59
that certain elements of role stress can yield positive results (Tang & Chang, 2010). Another key

assumption is that data collected during the role analysis exercise will reflect the actual

conditions related to the existence of role stressors in IDTs. Since role stress can be a sensitive

topic, participants may not accurately report actual conditions, especially in the presence of a

leader or a manager. Furthermore, RAT involves the instructional design team engaging in a role

analysis. A key assumption is that institutions follow a team-based approach to online course

design. Therefore, deliberate thought and consideration should be given to how RAT can be

integrated as part of the IDT’s existing processes (Srivastav, 2006).

Conclusion

While assumptions are made in the intervention design regarding how IDTs function, the

role analysis intervention may reduce role stressors among IDTs and establish a positive and

productive environment for team members. The role analysis exercise may create opportunities

for IDTs to recognize role stress as a significant issue that has the potential to negatively impact

the online course design process. In addition, the intervention may create opportunities for teams

to engage in active collaboration and exchange of information related to their roles and

associated instructional design processes. The team-based nature of the intervention, where

decisions will be made collaboratively after defining and analyzing roles, may result in effective

leader-member exchanges (LMX) (House & Aditya, 1997) as well as participative decision-

making (PDM) (Beehr, 1996; Karasek, 1979; Vroom & Jago, 1988).

60
 Chapter 4: Intervention Design

Overview

This chapter describes the intervention to decrease role ambiguity and promote role

clarification in instructional design teams (IDTs). The intervention involved members of IDTs in

higher education institutions engaging in a role analysis exercise based on Dayal and Thomas’s

(1968) role analysis technique (RAT). RAT is a step-by-step process for defining and clarifying

who is responsible for what. RAT is based on the premise that when team members do not know

what is expected of them and/or have different expectations about roles and responsibilities, they

experience role ambiguity and performance becomes suboptimal. The five essential steps in RAT

are (a) team members outline their respective roles as they perceive them; (b) team members

outline their perceived expectations of the other team members; (c) the team discusses each

individual’s roles, focusing on where expectations match and differ; (d) the team reaches

consensus on the roles and responsibilities of each team member; and (e) a role profile is created

for each role analyzed.

Intervention Goal and Rationale

The intervention was designed to decrease role ambiguity and promote role clarification

among members of instructional design teams (IDTs). Role clarification has been shown to have

the potential to reduce role stressors, particularly role ambiguity, by creating an opportunity for

team members to discuss expectations regarding their own and others’ roles and responsibilities

within team processes (Rao and Vijayalakshmi, 2000; Sims et al., 2006; Srivastav, 2011, 2012).

In light of the results of the needs assessment and existing literature, a role analysis exercise

based on Dayal and Thomas’s (1968) RAT was implemented to decrease role ambiguity in IDTs

and help achieve role clarity in existing instructional design processes. In accordance with the

61
Dayal and Thomas framework, the intervention was designed to allow IDTs to (a) analyze key

roles within existing online course development processes, (b) identify limited perceptions or

misperceptions around roles that could affect role effectiveness, and (c) develop clarity around

specific roles of IDT members in contributing to online course development processes and the

overall quality of online courses.

Limited research exists on the effectiveness of RAT in higher education, and no research

exists on its application to IDTs. Furthermore, the nature and effects of role stress have been only

minimally studied in higher education generally and IDTs more specifically (Dyer & Dyer, 2013;

Harvey & Drolet, 2004; Lyons, 1993; Rahim, 2010). Therefore, this study aimed to fill a

significant gap in the literature by researching whether the implementation of a role analysis

intervention that focuses on promoting role clarification decreases role ambiguity in IDTs in

higher education.

Research Questions

The intervention tested the hypothesis that IDTs that participate in a role analysis exercise

based on Dayal and Thomas’s (1968) role analysis technique (RAT) will experience decreased

role ambiguity and increased role clarity around existing instructional design processes.

Comparisons between pre-intervention and post-intervention results were performed to assess the

efficacy of the intervention. The following intervention research questions were developed:

RQ1: To what extent did the role analysis exercise result in a decrease in role ambiguity

among instructional design team (IDT) members, as measured by the Role Ambiguity (RA)

subscale in Pareek’s (1983) Organizational Role Stress (ORS) scale?

RQ2: What were instructional design team members’ perceptions of the value of a role

analysis exercise and its potential to decrease role ambiguity?

62
 RQ3: To what extent were all elements of the role analysis exercise based on Dayal and

Thomas’s (1968) role analysis technique (RAT) implemented as planned?

Intervention Design

Consistent with a convergent mixed methods approach (Creswell & Plano Clark, 2011),

the intervention captured both qualitative and quantitative data. The convergent parallel design

was used to “implement the qualitative and qualitative strands during the same phase of the

research process, prioritize the methods equally, and keep the strands independent during

analysis and then mix the results during the overall interpretation” (Creswell & Plano Clark,

2011, p. 97). This design allowed for a complete and holistic understanding of role ambiguity in

IDTs and the opportunity to validate and corroborate the quantitative data obtained from the role

ambiguity scale.

The research design logic model (Appendix C) illustrates the flow of participant inputs,

activity and participation outputs, and the anticipated short-, medium-, and long-term outcomes.

Inherent in the second and third research questions is an evaluation of the process of

implementation and implementation fidelity. Therefore, the research questions addressed not

only the outcome evaluation, but also evaluation of the process.

Outcome Evaluation

The intervention addressed role ambiguity in instructional design teams (IDTs) with the

following goals: (a) decrease role ambiguity, (b) maximize role effectiveness, (c) maximize the

effectiveness of related business processes, and (d) maximize role alignment with organizational

components. The logic model (Appendix C) identifies the short-term outcomes intended for the

intervention. The short-term outcome of the intervention was decreased levels of role ambiguity

63
among IDTs. The short-term outcome measures consisted of role ambiguity scores, field notes,

semi-structured interviews, and individual and group reflections.

Process Evaluation

Fidelity of implementation was defined to provide a consistent framework for evaluating

the experiences of IDTs (Dusenbury, Brannigan, Falco, & Hansen, 2003). Fidelity of

implementation was conceptualized for the intervention as follows: the implementation of the

role analysis exercise in accordance with the instructional procedures of Dayal and Thomas’s

(1968) role analysis technique (RAT). The key elements of high fidelity included: (a) a minimum

of three and maximum of 15 IDT members from each team recruited to participate, (b)

participants’ attendance of approximately four hours of face-to-face training delivered as a half-

day workshop, (c) participants’ completion of the role profile for each role analyzed, and (d)

quality of program delivery by the facilitator (Dusenbury et al., 2003). Table 6 provides the data

collection matrix for assessing the fidelity of implementation.

Number of participants. A key fidelity indicator was the number of participants taking

part in the intervention. Individuals are more likely to experience role ambiguity when their roles

and responsibilities overlap and/or cross boundaries and they receive diverse role expectations

from different role senders (Hang-Yue, Foley, & Loi, 2005; Monahan, 1999; Singh & Rhoads,

1991). Given the complexity of the online course design process, which often requires

individuals to assume multiple roles (Hokanson & Miller, 2009; Oliver, 2002; Reilly et al.,

2012), role ambiguity is more likely when responsibilities overlap and interrelate among IDT

members. Therefore, for the intervention to have significance and relevance, a minimum of three

IDT members from each team recruited to participate was required. The researcher maintained

64
attendance records during the intervention to ensure at least three IDT members were

participating in the sessions.

Attendance. The second, related fidelity indicator was participants’ attendance of

approximately four hours of face-to-face training delivered as a half-day workshop. Considering

that the intervention follows a step-by-step sequential approach to role analysis, and the creation

of the role profile is dependent on the successful completion of the previous steps, it was

essential that participants attend the entire workshop to complete the intervention. Participants

were required to sign in at the beginning and sign out at the end of the intervention. The

researcher maintained attendance records during the intervention.

Participant responsiveness. The third fidelity indicator was participants’ completion of

the role profile for each role analyzed. Participant responsiveness is “the extent to which

participants are engaged by and involved in the activities and content of the program”

(Dusenbury et al., 2003, p. 34). At the beginning of the intervention, the facilitator established

the significance of completing all the required steps and the final role profile to the success of

RAT. Participants’ completion of all the activities and the final role profiles were recorded in the

established worksheet formats (Appendices L and M), allowing for the researcher to measure

completion.

Quality of program delivery. The fourth fidelity indicator was the quality of program

delivery by the facilitator. A team development expert was recruited as the facilitator of the

intervention. Dusenbury et al. (2003) defined quality facilitation as being more than merely

performing from a script. Instead, quality facilitation positions the program developer as a coach

who works with all participants to achieve program objectives. The role that the facilitator

played was applicable to the implementation of the role analysis exercise. The facilitator and

65
researcher reviewed the steps involved in Dayal and Thomas’s (1968) role analysis technique and

adapted them for the instructional design context, while still ensuring that the intervention

remained true to the intended outcomes. To obtain data on participants’ experience of the role

analysis exercise and quality of program delivery, participants were asked to respond to the

following question: “What was your experience in taking part in the role analysis exercise?” The

use of an open question methodology allowed participants to document their thoughts about the

overall program quality as well as each step involved in the role analysis exercise (Savin-Baden

& Major, 2010). Following the written reflection, participants engaged in a group debrief

session, allowing them to share their experiences in the role analysis exercise, discuss themes

that emerged about roles within IDTs, and provide feedback for the facilitator (see Appendix K

for debrief guide).

Table 6

Data Collection Matrix Assessing Fidelity

Fidelity Data Source(s) Data Frequency Responsibility
Indicator Collection
Tool
Minimum A minimum of three Attendance Participants were Researcher
number of instructional design records from required to sign in
participants team members were the at the beginning and
taking part in required to intervention sign out at the end
the intervention implement the of the intervention.
intervention

Participant Participants’ Attendance Participants were Researcher

attendance attendance of the records from required to sign in
four hours of face-to- the at the beginning and
face training intervention sign out at the end
provided as half-day of the intervention.
workshop

Participants’ Creation of role Completed Participants Researcher

completion of profile for each team Role Profile completed a role
the role profile member Worksheets profile for each role
analyzed.

66
 Fidelity Data Source(s) Data Frequency Responsibility
Indicator Collection
Tool
for each role
analyzed
Quality of Participants’ Reflection Participants Researcher
program experience in the role exercise completed this
delivery by the analysis exercise where reflection exercise
facilitator participants at the end of the
were asked role analysis
to respond to exercise.
the following
question:
“What was
your
experience in
taking part in
the role
analysis
exercise?”

Group Participants
debrief engaged in a group
session debrief session at
the end of the role
analysis exercise.

Effect Size

Empirical research on the implications of role analysis exercise in teams is scant and

none is dedicated solely to IDTs. Even though some pilot studies and small empirical evaluations

have been published on the potential benefits of role analysis exercises to improving

organizational and team performance (Buch & Aldridge, 1990, 1991; Singh, 1997; Srivastav,

2010), published results often do not report effect sizes for the primary outcomes. While the

effect size for interventions involving role analysis specifically is not available, existing literature

does show that the effect size for organizational stress management programs and interventions

67
is 0.5 (medium effect size) (Baer, 2003; Klink, Blonk, Schene, & Dijk, 2001; Murray, Davidson,

& Schweitzer, 2010; Richardson & Rothstein, 2008).

Given the effect size of 0.5 based on previous research in organizational stress

management interventions, a power analysis using G*Power was conducted to determine

approximately how many participants would be needed for testing the research hypothesis. The a

priori calculation was performed for the intervention design and resulted in an optimal sample

size of n = 27 (see Figure 3). Based on the sample size determined from the power analysis, the

researcher recruited three teams to participate in the intervention, resulting in 29 participants in

total. The intervention and associated activities were conducted independently for each team

recruited to participate.

Figure 3. Power analysis using the G*Power software program.

68
 Method

This section describes the instructional design team (IDT) participants and the procedures

used to conduct the intervention study.

Participants

Participants for this study were members of IDTs involved in online course development.

The teams included instructional designers, instructional technologists, project managers,

multimedia personnel, videographers, executive team members, and others involved in the

design of online courses—such as faculty and subject matter experts. The study participants were

recruited from three professional schools at a large private research university in Maryland. The

Director (or Manager) of the IDT in each school was contacted to request approval for

participation (see Appendix E for the email sent to the Director of the IDT requesting approval).

Upon receipt of the Directors’ approval (see Appendix F for sample approval letter), they were

requested to provide contact information of potential participants for the study (names and email

addresses). Since the Directors had oversight over the IDTs and the roles and responsibilities of

the team members, they were well positioned to recommend potential participants who met the

established criteria for participation. The Director of the IDT had access to the email addresses of

the individuals involved in the online course development process.

The researcher sent potential participants a recruitment email (see Appendix G) that

contained: a description of the study, nature of participation, duration and purpose, possible risks

and benefits, assurances regarding confidentiality, right to withdraw without penalty, and

opportunities for participants to ask questions or raise concerns regarding the research. The

intention of this information was to lessen any pressure the individual may feel to immediately

69
decide on participation. The email also included an informed consent form that participants

completed and returned to the researcher (see Appendix H).

A key requirement for participants was that they must be involved, in some significant

capacity, in the development of online courses. The researcher confirmed the selected

participants’ role in online course development with the Director of the IDT to ensure the

selected participants met this criterion. A total of 29 participants were recruited to participate in

the study (School 1: n = 10, School 2: n = 6, School 3: n = 13). The intervention was conducted

independently for the IDT recruited from each school. This structuring allowed the facilitator to

give personalized attention to the nuances of the instructional design process followed by each

school. Participants were not responsible for any research-related costs and did not receive

payment or incentives for their participation in the study. See Table 7 for demographic data for

participants from each IDT.

Table 7

Participants’ Demographic Information

Category Percentage of Respondents
Ethnicity
White (non-Hispanic) 82.6%
Asian 8.7%
African American 8.7%
Other 0.0%
Did not disclose 0.0%

Gender
Female 56.5%
Male 43.5%
Did not disclose 0.0%

Age
18–30 13.0%
31–40 30.4%
41–50 34.8%
51 and above 21.7%

70
 Category Percentage of Respondents

Roles
Faculty 13.0%
Instructional designer 26.1%
Instructional technologist 21.7%
Executive team member 8.7%
Other 17.4%
Multiple 13.0%

Type of Institution
Private, four-year institution 100.0%
Public, four-year institution 0.0%
Public, two-year college 0.0%
Private, two-year college 0.0%

Education Level
Advanced degree (Master’s or Other) 73.9%
Four-year college degree (Bachelor’s) 21.7%
Two-year college degree (Associate’s) 4.3%

Years of Professional Experience

1–2 years 0.0%
2–6 years 26.1%
6 or more years 73.9%

Measures and Instrumentation

The study followed a quasi-experimental design, one-group pretest-posttest design

without a control group (Shadish, Cook, & Campbell, 2002, p. 108). Participants self-selected to

participate in the study as long as they met the following two criteria: (a) the IDT followed a

team-based approach to online course development, requiring involvement from more than one

person, excluding the subject matter expert (i.e., the team should include a minimum of three

members); and (b) participants were involved, in some significant capacity, in the development

of online courses. A within-participants design was used to evaluate the effects of the

71
intervention (i.e., role analysis exercise) on role ambiguity in IDT members (Shadish et al.,

2002).

Participants completed the role ambiguity subscale of Pareek’s (1983) Organizational

Role Stress (ORS) instrument before and after the intervention to measure role ambiguity. Part

one of the questionnaire consisted of demographic items: (a) role in the IDT, (b) highest level of

education, (c) length of service in the IDT, (d) years of professional experience, (e) gender, (f)

ethnicity, and (g) age. Part two of the questionnaire consisted of the role ambiguity subscale of

the ORS instrument. The ORS is a 5-point Likert scale (0–4) containing five items for measuring

the role ambiguity dimension of role stress. Scores can range from a minimum of 0 to a

maximum of 20—a score of 0 represents the absence of role ambiguity and a score of 20

represents maximum role ambiguity (Srivastav & Pareek, 2008). See Appendix D for the

complete questionnaire. The ORS scale has been commonly utilized for studying role stress in

organizations (Bhattacharya & Basu, 2007; Dasgupta & Kumar, 2009; Srivastav, 2006, 2007,

2010) and has been shown to have high reliability and validity (Aziz, 2004; Pareek, 2004, 2005).

Therefore, the role ambiguity subscale in ORS was relevant for this study.

In addition to the quantitative data collected from the pretest and posttest scores, the

researcher collected qualitative data—through field notes, participant written and verbal

reflections during the intervention, and follow-up interviews after the intervention—to help

explain participants’ reasoning for their responses in the questionnaire and to further explain the

quantitative results. Throughout the role analysis exercise, the researcher took field notes using

an observational protocol (Appendix I) to record contextual notes (observations and reflections).

The field notes allowed for capturing of nonverbal behavior and the context in which these

behaviors took place, as well as the researcher’s own impressions and insights. Following the

72
role analysis exercise, participants were asked to respond to the following question: “What was

your experience taking part in the role analysis exercise?” Participants completed this written

reflection individually after the role analysis exercise. Following the written reflection,

participants engaged in a group discussion session, allowing for verbal reflections. Written and

verbal reflections provided insights into participants’ reactions to the intervention,

implementation process, and extent to which the intervention was meaningful to the participants.

Approximately three weeks after the intervention, 30-minute semi-structured

interviews—either face-to-face or via web-based conferencing software—were conducted with

the study participants. Of the 29 study participants invited to participate, 22 completed the

interviews. The interviews provided qualitative data on the opinions, attitudes, and beliefs of

participants about the outcome of the intervention. Specifically, questions were designed around

the participant’s role in the IDT and the influences of the role analysis exercise on clarifying role

expectations and responsibilities within the team (see Appendix J for the interview protocol used

to guide the interviews).

Procedure

This section provides a description of the role analysis intervention including the

implementation protocols, data collection, and data analyses procedures. An intervention

timeline is shown in Table 8.

Intervention

The instructional design team (IDT) from each school participated in a four-hour role

analysis workshop in April 2018. The intervention took place in a large conference room on the

campus of each participating school. The conference room setting allowed for small group

discussions, and optimal interaction and collaboration among participants. Upon arrival, the team

73
was placed in a round-table format to encourage the free flow of conversation. Before the

beginning of the intervention, participants completed the role ambiguity questionnaire based on

the role ambiguity subscale of Pareek’s (1983) Organizational Role Stress (ORS) scale.

Participants who did not complete the informed consent form were given the opportunity to

complete it at the workshop. The facilitator and participants provided brief introductions,

followed by a concise overview of the purpose of the study, the importance of clear team roles

and responsibilities, and differences between roles and job titles.

The intervention began with an ice-breaker activity based on Goldsmith’s (2010)

“Feedforward” exercise. Participants were instructed to think about their role in the IDT and

identify a challenge they would like to address concerning their role. In turn, each participant

posed the challenge to the person across from them in the room, using this format: “How can I

better address the challenge of _________?” Questioners were then instructed to stay completely

quiet while, for the next 60 seconds, the person opposite him or her brainstormed as many

responses as possible to that question. When the 60-second bell rang, the questioner thanked his

or her respondent, and the team moved counterclockwise one place to create new dialogue

partners. This exercise was repeated several times, with participants rotating the role of

questioner and respondent. The purpose of the Feedforward exercise was to introduce a level of

familiarity and group cohesion between team members as well as provide participants the

opportunity to reflect on their role intentionally.

Following the Feedforward exercise, the team engaged in the role analysis exercise

involving the following activities:

 Activity 1–Individual Roles and Responsibilities: Using the Role Expectations

Worksheet template (Appendix L), team members individually wrote brief statements

74
 describing their perception of their roles and responsibilities as individual members of

the team as well as how they perceived others’ expectations of them. Each participant’s

worksheet was placed on the wall for easy access and continuous feedback during the

workshop. Participants spent approximately 15 minutes on this activity.

 Activity 2–Team Role Expectations: Upon completing the individual roles and

responsibilities activity, participants went around the room and read each team

member’s Role Expectations Worksheet and identified areas where they had questions

and/or needed clarifications. Sticky notes were used to identify these areas where there

were differences in opinions and/or general questions about roles and responsibilities.

During the later portion of this activity, the team was introduced to the theoretical

framework of role theory and the three types of role stressors—namely role conflict, role

ambiguity, and role overload. As participants reviewed each team member’s Role

Expectations Worksheet (including their own worksheet), they were instructed to use

colored stickers to identify areas where role stressors might potentially exist.

Participants spent approximately 20 minutes on this activity.

 Activity 3–Role Clarification: After a 10-minute break, the team had the opportunity to

review the worksheets as a group and discuss any differences in opinion or general

observations. The facilitator guided these conversations to help the team gain clarity

around each person’s role and contribution to the team’s operation. Participants could

ask questions and seek clarification from team members. The team also compared each

member’s ideas about what he or she expected the other team members to contribute and

discuss any differences in opinion or observations. To the extent possible, the team came

to an understanding of each person’s role and contribution to the team’s operation. In

75
 cases where disagreements could not be resolve—and for matters beyond the scope of

the intervention—participants were encouraged to address those areas with the

management and/or individuals in leadership or supervisory roles to gain clarity.

Participants spent approximately 30 minutes on this activity.

 Activity 4–Role Profile: Based on the newly established clarity around roles and

responsibilities, using the Role Profile Worksheet template (Appendix M), participants

created a role profile for each role analyzed. While this was an individual activity for the

most part, if more than one team member represented the same role (for example, more

than one instructional designer), those participants with shared roles collaborated in

creating the role profile for the role. Participants spent approximately 30 minutes on this

activity.

At the completion of all the necessary activities and associated worksheets, participants

took part in individual written reflections followed by a group debrief. Following the reflection

activities, participants were also given the opportunity to ask any questions and engage in open

discussion for approximately 30 minutes. See Appendix N for a sample workshop agenda.

Before concluding the intervention, participants were reminded that they would receive a follow-

up post-intervention questionnaire within two weeks via email (noting that the post-intervention

questionnaire should be completed within 48 hours of receipt). Participants were also reminded

that they would be invited to participate in follow-up interviews within three weeks after the

intervention. Table 8 provides a summary of the intervention timeline.

Table 8

Intervention Timeline

76
 Intervention Activity Intervention Timeline
Participant Recruitment March 2018
Role Analysis Workshop – School 1 April 2018
Role Analysis Workshop – School 2 April 2018
Role Analysis Workshop – School 3 April 2018
Post-Intervention Survey May 2018
Semi-Structured Interviews May 2018

Data Collection

Consistent with the convergent mixed methods design, both qualitative and quantitative

data were collected and used to assess the outcomes of the intervention as well as the experiences

of participants and implementation fidelity (Creswell & Plano Clark, 2011). Data from the pre-

test and post-test questionnaire, based on the role ambiguity subscale of Pareek’s (1983)

Organizational Role Stress (ORS) scale, were collected before the intervention and two weeks

after the intervention. Participant reflections (individual, written responses, and verbal group

debrief) were gathered at the end of the role analysis exercise (see debrief guide in Appendix K).

The researcher recorded observations using the observational protocol (Appendix I), which

included both descriptive field notes and reflective field notes. In addition, approximately three

weeks following the intervention, face-to-face semi-structured interviews were conducted with

the study participants using an interview protocol (Appendix J) to obtain qualitative data about

the outcome of the intervention. The interviews were audio recorded.

The quantitative data obtained from the questionnaire and qualitative data obtained from

observations and interviews allowed for empirical evidence from different sources, meaningful

interpretation of data, and conclusions to be drawn with an understanding of how role ambiguity

influences instructional design teams (IDTs). While the role ambiguity scale served as a useful

instrument to measure the effectiveness of the role analysis exercise, qualitative data obtained

from the field notes and interviews helped explain and elaborate on the quantitative results.

77
Furthermore, qualitative data revealed insights into the intricacies and nuances of higher

education IDTs that did not surface in the questionnaire. This approach provided the opportunity

to interpret to what extent and in what ways the quantitative and qualitative data converge,

diverge, and relate to each other, and lead to a better understanding of the implications of the role

analysis exercise to minimize role ambiguity in IDTs.

Data Management

Data were stored in the researcher’s password-protected computer. Records that

identified participants were available only to the researcher and principal investigator.

Quantitative data obtained from the pre- and post-test questionnaire were compiled in an Excel

spreadsheet and uploaded to SPSS for analysis. Data captured through interviews were audio

recorded and kept on the researcher’s password-protected computer. Each interview was first

typed verbatim and then thoroughly revised to make more comprehensible and readable

transcript. Before analysis, pseudonyms were substituted for the interview participants’ names.

Data Analysis

Consistent with the convergent mixed methods design, both qualitative and quantitative

data were captured through the various instruments. This section describes how the data were

analyzed to address the research questions. Data from the pre-and post-intervention questionnaire

were compiled in an Excel spreadsheet and uploaded to SPSS for analysis. Descriptive statistics

were calculated for the role ambiguity questionnaire. A box and whisker plot was generated to

identify any outliers. Next, assumptions testing was conducted using the Shapiro-Wilk test of

normality and Q-Q Plots to determine whether data were normally distributed. Then, to address

research question one, a sign test was used to determine whether there was a median difference

in participants’ role ambiguity scores before and after the intervention.

78
 Reliability tests. Cronbach’s alpha coefficient (α) is one of the commonly used measures

for determining internal consistency or reliability when using the Likert-scale survey or

questionnaire (Warner, 2013). To establish the reliability of role ambiguity (RA) subscale of

Pareek’s (1983) ORS scale, Cronbach’s alpha was calculated. For the internal consistency of

data, a Cronbach’s alpha of 0.65 to 0.80 (or higher) is recommended (Warner, 2013). As shown

in Table 9, the scale has Cronbach’s alpha greater than 0.70.

Table 9

Reliability Statistics

Cronbach’s Alpha Cronbach’s Alpha N of Items

Based on
Standardized Items
RA 0.779 0.769 5

Assumptions tests. Data were examined for outlines. The assumption of normality was

tested on the differences between the paired values. The difference scores were calculated from

the pre-intervention and post-intervention RA scores.

Data screening. A simple box and whisker plot was generated to determine if there were

any outliers in the data. As shown in Figure 4 there were no outliers in the data, as assessed by

inspection of a box and whisker plot for values greater than 1.5 box-lengths from the edge of the

box.

79
Figure 4. Box and whisker plot for difference scores of the pre-intervention and post-intervention
RA scores, showing no outliers.

Normality tests. The assumption of normality was tested using the Shapiro-Wilk Test for

normality (n < 50) alpha of 0.05. The difference scores for the pre-intervention and post-

intervention RA scores were not normally distributed, as assessed by Shapiro-Wilk Test (p

= .009). Since the data violated the parametric assumption of normal distribution (Field, 2009), a

non-parametric test is needed. A sign test was used as an alternative to the paired-samples t-test

or Wilcoxon signed-rank test, since the distribution of differences between the paired

observations was neither normal nor symmetrical, respectively.

80
Figure 5. Histogram for difference scores of the pre-intervention and post-intervention RA

scores, showing non-symmetrical distribution.

Qualitative data. Hierarchical content analysis was used to analyze the collected

qualitative data. Qualitative data from individual and group reflections after the intervention,

researcher’s field notes kept during the intervention, and participant responses to follow-up

interviews were transcribed verbatim, resulting in approximately 95, single-spaced pages text.

Data from semi-structured interviews and individual and group reflections were used to address

the value of the role analysis exercise and its potential to decrease role ambiguity (RQ2).

Attendance records, worksheets completed by the participants, and participant reflections and

interview responses were used to address the fidelity of implementation (RQ3). The researcher’s

field notes were used to address both value of the role analysis exercise (RQ2) and

implementation of the intervention (RQ3).

81
 Each collection of qualitative data was thoroughly examined to record initial thoughts

and ideas about the text. An inductive analysis was conducted to identify common themes or

patterns of greater generality (Corbin & Strauss, 2008). A hierarchy of responses moving from

specific (e.g., raw data themes) to general levels (e.g., first-order themes, second-order themes,

and general dimensions) was then established. Frequency analysis was used to determine the

number of times a theme was cited within each of the second-order themes. After all general

dimensions were identified, the researcher reviewed the emergent patterns in interpreting and

generalizing the findings of this study and the implications for practice (Corbin & Strauss, 2008).

Summary Matrix

The summary matrix demonstrates the relationship between the research questions,

proximal outcomes, variables, and the data gathering instruments (Table 10). The summary

matrix was informed by the literature, reflected the mixed methods approach, and incorporated

elements of the intervention.

Table 10

Evaluation Summary Matrix

Research Questions Indicators Role of Data Source Data Analysis
Indicator
RQ1: Outcome Role Outcome Pre- and Post- Sign test to
To what extent did Ambiguity Measure Intervention compare the
the role analysis Scores Questionnaire: median
exercise result in a Role ambiguity difference of
role ambiguity subscale of Pareek’s the pre-
among instructional (1983) intervention
design team (IDT) Organizational Role and post-
members, as Stress (ORS) scale intervention
measured by the role ambiguity
Role ambiguity scores
(RA) subscale in
Pareek’s (1983)
Organizational Role
Stress (ORS) scale?

82
Research Questions Indicators Role of Data Source Data Analysis
Indicator
RQ2: Outcome Field notes Outcome Observations
What were Measure recorded by the Inductive
instructional design researcher during the thematic coding
team members’ role analysis exercise
perceptions of the
value of a role
analysis Semi- Outcome One-on-one
exercise and its Structured Measure interview with the Inductive
potential to decrease Interviews study participants thematic coding
role ambiguity? approximately two
weeks following the
intervention

Individual and Outcome Reflection exercise

group Measure sheets completed by Inductive
reflections the participants at thematic coding
the end of the role
analysis exercise.

Group debrief
session at the end of
the role analysis
exercise.

RQ3: Process – Field notes Process Observations Inductive

Fidelity Measure recorded by the thematic coding
To what extent were researcher during the
all elements of the role analysis exercise
role analysis
exercise based on
Dayal and Thomas’s Minimum Process Maintenance of Participants
(1968) role analysis number of Measure attendance records will be required
technique (RAT) participants during the to sign in at the
implemented as taking part in intervention beginning and
planned? the sign out at the
intervention end of the
intervention

Participant Process Maintenance of Participants

attendance Measure attendance records will be required
during the to sign in at the
intervention beginning and
sign out at the

83
 Research Questions Indicators Role of Data Source Data Analysis
Indicator
end of the
intervention.

Participants’ Process Completion of the Participants

completion of Measure Role Profile will create a
the role Worksheet role profile for
profile for each role
each role analyzed
analyzed

Quality of Process Reflection exercise Inductive

program Measure sheets completed by thematic coding
delivery by the participants at
the facilitator the end of the role
analysis exercise.

Group debrief
session at the end of
the role analysis
exercise.

Conclusion

Informed by the literature and needs assessment, a role analysis intervention was

implemented in three instructional design teams (IDTs) to decrease role ambiguity and promote

role clarification. This chapter presented a mixed methods approach to collecting and analyzing

intervention data. The proposed research questions guided decisions regarding data collection

and data analysis. The chapter provided an overview of the role analysis intervention, including

the goals of the intervention, intervention design, participants recruited, measures and

instruments used, procedures, data collection, and data analysis. Participant responses to the pre-

and post-intervention questionnaire, reflection exercises, and semi-structured interviews, as well

as the researcher’s field notes, were analyzed to evaluate the intervention outcome and process of

implementation. Chapter 5 describes key findings from the quantitative and qualitative data

collection and analysis.

84
 Chapter 5: Findings, Discussion, and Implications for Practice

The purpose of this study was to explore the potential of a role analysis intervention to

decrease role ambiguity and promote role clarification among members of instructional design

teams (IDTs) in higher education institutions. Chapter 4 presented the research study design and

the components of the role analysis intervention. This chapter describes the study findings

organized by each research question. The following research questions focused the analyses

within this study.

RQ1: To what extent did the role analysis exercise result in a decrease in role ambiguity

among instructional design team (IDT) members, as measured by the Role Ambiguity (RA)

subscale in Pareek’s (1983) Organizational Role Stress (ORS) scale?

RQ2: What were instructional design team members’ perceptions of the value of a role

analysis exercise and its potential to decrease role ambiguity?

RQ3: To what extent were all elements of the role analysis exercise based on Dayal and

Thomas’s (1968) role analysis technique (RAT) implemented as planned?

Findings

Role Ambiguity in Instructional Design Teams

The first research question focused on the extent to which the role analysis exercise

resulted in a decrease in role ambiguity among instructional design team (IDT) members, as

measured by the Role Ambiguity (RA) subscale in Pareek’s (1983) Organizational Role Stress

(ORS) scale. The ORS is a 5-point Likert scale (0–4), containing five items for measuring the

role ambiguity dimension of role stress. Scores can range from a minimum of 0 to a maximum

of 20—a score of 0 representing the absence of role ambiguity and a score of 20 representing

maximum role ambiguity (Srivastav & Pareek, 2008). Among the 29 participants, 23 completed

85
the role ambiguity scale before and after the intervention. Mean, median, and standard deviation

for pre-intervention scores (RA_Pre_Score) and post-intervention scores (RA_Post_Score) are

shown in Table 11. Descriptive statistics provided a general characterization of the sample. An

examination of the descriptive statistics showed an increase from pre- to post-intervention for

role ambiguity.

Table 11

Descriptive Statistics
RA_Pre_Score RA_Post_Score
Mean 6.39 11.61
Median 7.00 11.00
Standard Deviation 3.652 4.175

A sign test was used to compare the differences in role ambiguity scores before and after

the role analysis intervention. The median role ambiguity score before the intervention was 7.00,

while the post-intervention score was 11.00. The role analysis intervention elicited a statistically

significant median increase in role ambiguity after the intervention (4.00) compared to before the

intervention, p < .0005. It would appear that participants experienced an increase in role

ambiguity after the role analysis intervention. Figure 6 shows the results of the sign test. The

implications of this finding are explored in the discussion section.

86
Figure 6. Results of the related samples sign test showing positive differences.

Perceptions of the Value of the Role Analysis Intervention

To address the second research question, qualitative results of the study provided

comprehensive information about the instructional design team members’ (IDT) perceptions of

the value of the role analysis exercise and its potential to decrease role ambiguity. As described

in Chapter 4, in accordance with hierarchical content analysis, a hierarchy of responses moving

from specific (e.g., raw data themes) to general levels (e.g., first-order themes, second-order

themes, and general dimensions) was established. First-order themes represented the factual and

preliminary data, while second-order themes were attempts taken by the researcher in explaining

“the patterning of the first-order data,” and creating “interpretations of interpretations” (Maanen,

1979, pp. 540–541). Finally, second-order themes were clustered into more abstract, higher-order

themes (i.e., general dimension).

Themes from the data fell into seven second-order themes and three general dimensions.

The three dimensions represented: (a) participants’ perceptions of the value of the role analysis

exercise in providing opportunities for collaboration and reflection on roles, (b) stressors and

challenges faced by the IDT, and (c) the diversity of the IDT roles and responsibilities in the

87
online course development process. Table 12 shows an overall representation of the classification

system of responses obtained regarding RQ2 via written reflections, group debrief, field notes,

and follow-up interviews.

Table 12

Value of the Role Analysis Intervention: Hierarchical Analysis Results

First-Order Themes Second-Order General
Themes Dimension
 Positive, fun, and enjoyable
 Space to have conversations with the team Social
 Informative, insightful, and educational Experience
 Good chance to bond with faculty n = 18
 Opportunity to interact and collaborate
 Clarify roles versus expectations
 Underscore the dynamics of the other roles Clarity Around
 Ask questions and provide direct clarification Roles and
 Clarity in how everyone fits in the overall vision Responsibilities
 Differences between titles versus roles n = 45

 Different perspectives and expectations of roles

 Congruence or lack of congruence in perceptions Collaboration
of roles and Reflection
 Consider ideas from different perspectives Different on Roles
 Gain different perspectives— faculty, Perspectives
instructional designer, instructional technologist, and Perceptions
manager etc. n = 27
 Roles versus expectations—expectations are far
greater than current roles
 Systematically analyze and engage in an in-depth
dive into one’s own roles Introspection
 An introspective experience and Self-
 Allow for self-reflection and community Reflection
awareness n = 22
 Help see how much more IDTs do
 Highlight the issues faced by team members
 Bring perspective to how much IDTs manage and
Presence of
juggle. Role Stressors
Role Stress
 Acknowledge and reinforce areas of role stress and Challenges
n = 12
 Not necessarily eye opening, but comforting
 Highlight team’s common questions and concerns

88
 First-Order Themes Second-Order General
Themes Dimension
 Ambiguity among faculty about IDTs’ roles
 Acceptance that the IDT is overloaded
 Insight into types of role stressors and the Manifestation of
underlying reasons Role Stressors
 Overlap between roles n = 28
 Imbalance or an inequity in workload
 Multiple roles of people involved in online course
development
 Often being asked to “play out of position”
Multiple Roles
 Instructional designer becomes the main point of Diversity of
of the IDT
contact for all support Roles and
Members
 IDT responsible for technical/administrative tasks Responsibilities
n = 40
 IDT viewed as IT help desk
 Overlap in roles and responsibilities

Collaboration and reflection on roles. Four second-order themes on the role analysis

exercise contributing to opportunities for collaboration with the team and reflection on roles

were identified: (1) social experience; (2) clarity around roles and responsibilities; (3) sharing of

different perspectives and perceptions; and (4) opportunities for introspection and self-reflection.

Each of the second-order themes related to this dimension is delineated below.

Social experience. Qualitative data indicated that the participants enjoyed the social

interaction, and that the intervention gave them the opportunity for meaningful conversation with

their team members and others involved in the development of online courses. They described

the interaction being a crucial component of the IDT’s ongoing collaboration. Positive team

culture and the opportunities for relationship building with team members is a benefit that is

consistent in the literature on factors that contribute to successful functioning of the IDT (Pan &

89
Thompson, 2009; Ritzhaupt & Kumar, 2015). Nora1, a faculty participant, explained that the

intervention allowed for interaction between faculty and staff involved in online course

development:

One of the best things about it [intervention] was the interaction between faculty and

staff. I sometimes feel like everyone is in their own little world and they don’t think to

tell people simple things, and it causes confusion and angst. It allowed us to bridge a lot

of gaps in terms of interaction between faculty and staff. You don’t interact with people

every day because if you’re a professor, you work from home or work from different

campuses. So there are a lot of people whose names I see, but I don’t connect it to the

person. I have a much better idea what each person in the team [IDT] does and have a

much better idea of who each person is. And, it was just really nice to interact with them.

There wasn’t something we had to get done—to get from point A to point B. It was

almost a social occasion, but it was still working. (Nora, interview, May 16, 2018)

The idea that the role analysis exercise provided a space for social interaction between

faculty and staff was also echoed in Steven’s comment:

I think it was excellent to have a chance to work with and talk with some of the faculty in

a non-working function. It allowed us to find some common ground. I personally have

found, not just here but in higher education in general, there’s a bit of a divide between

staff and faculty. It’s a tricky one that every institution deals in trying to bridge that. It

was a good opportunity to interact with faculty in a non-formal setting. (Steven,

interview, May 16, 2018)

1
All participant names are pseudonyms.

90
 In addition to providing a space for social interaction, participants expressed that the

intervention provided a forum for open conversation between team members regarding roles and

responsibilities. For Kate, an instructional technologist, the process of going through the role

analysis exercise helped her empathize with the challenges that her team members face:

I think it gave me more empathy. It helped me empathize with some of the issues going

on with other members of the team. I try to be a little bit more communicative with my

team members about projects and that does lead to a better collaboration or at least the

potential for better collaboration. (Kate, interview, May 16, 2018)

Qualitative data obtained from individual written reflections and during group debrief

revealed attitudes about the experience of going through the role analysis exercise. Uniformly,

participants enjoyed these conversations and felt that the open dialogue allowed for relationship

building between IDT members that can extend beyond the development of online courses. One

participant expressed that it provided an avenue for team members to vent their concerns, share

their experiences, and voice their opinions: “The workshop was also helpful because it’s good for

some people to vent. This was very helpful for the team to bring clarity to roles and where we are

going with our growth. It made them feel like they are part of the process” (Nancy, interview,

May 8, 2018).

Clarity around roles and responsibilities. A primary goal of the intervention was to

initiate and promote role clarification among participants. Consistent with the intervention goal,

a theme that emerged was the opportunity to seek and acquire clarity around roles and

responsibilities. Participants found that they were able to clarify roles and responsibilities as well

as focus on the expectations of others. This was enabled through the opportunities to ask team

91
members what their expectations were and provide direct clarification. Betsy, a faculty

participant, shared,

It really helped us know what all the different people did within the team, who was

responsible for what and how the team worked together. With the growth of the team, it’s

just hard to keep track of the new people and how things are getting divvied up

differently. We got a clear understanding of what our role was in designing online classes

and then the roles of people in the online team [IDT]. We were able to find out what their

responsibilities were and where they came in at different points to help and assist faculty

with different things. I also have a better idea now about who to contact about what.

(Betsy, interview, May 17, 2018)

Considering the diverse roles that instructional designers and instructional technologists

serve in online course development (Dooley et al., 2007; Koehler & Mishra, 2005; Pan &

Thompson, 2009), it can be challenging for faculty to have a nuanced sense of how these roles

unfold and interact, and how the IDT as a whole fits together in the online course development

process, more generally. One faculty participant, Chen shared:

We always want to get help sooner, but I don’t think we are all very patient to spend time

to understand each other’s responsibilities and roles. We make assumptions about what

people do and that’s part of the source of delay and frustration. So, the workshop actually

forced me to understand the team’s roles a bit better. (Chen, interview, May 16, 2018).

Faculty participants shared that understanding roles allowed them to better leverage

IDT’s expertise to improve online teaching and learning (that is, “know who to go to”).

Participants in other roles (such as instructional designers, instructional technologists,

92
multimedia personnel, etc.) felt the intervention helped see where their role was in relation to

other individuals’ in the team. Steven explained:

It was really good for having an understanding of what everyone’s roles were. It wasn’t

just good for team building and camaraderie, but it was also good for understanding what

goes into other people’s piece of the puzzle and how I can conduct myself a little better,

how I can revise my workflow to make their life a little easier. (Steven, interview, May

16, 2018)

Besides being able to articulate roles and convey expectations, participants also noted

that the intervention provided an opportunity to share stressors and begin to clarify roles versus

expectations. Participants found the process of thinking about their roles intentionally and

articulating expectations to others beneficial to team collaboration and performance. This was

reflected in Kate’s comments:

It was nice to bounce ideas off of another Instructional Technologist, talk through the

responsibilities of our position and come up with expectations—what other people expect

and what we won’t necessarily want to do in our position. It was helpful to extract more

meaningful examples that we could share with everybody. (Kate, interview, May 16,

2018)

Different perspectives and perceptions. Role ambiguity arises when individuals lack

clear knowledge about how to perform their job (Srivastav, 2007). Recognizing the congruence

or lack of congruence in how individuals perceive their roles versus how others perceive them

can lead to identifying areas where role ambiguity might exist. Participants shared that the

intervention allowed them to make a purposeful effort in examining their work from different

perspectives and perceptions. John explained:

93
 The activity of writing up your roles and responsibilities from different perspectives is

definitely very helpful. Talking to other people and mapping out my perspectives and

thinking about how other people view my role and writing down the job description

helped me look at my role more objectively. (John, interview, May 15, 2018)

The intervention highlighted differences in how participants perceived their roles in

contributing to online learning. One faculty participant expressed that people have an impression

that she was supposed to be a “teacher” or “giver of knowledge” while she saw herself as a

“facilitator” in an online setting. Similarly, participants also discussed the misconceptions that

exist around the roles of technology staff involved in online course development. Michael

shared:

We need to educate faculty on the roles of technology staff. For example, the difference

between a videographer/video editor versus a project coordinator. Editor is specific to a

task while coordinator is more holistic to the project. An editor is told what the end

product should be. Coordinator brainstorms how a video fits in to the project. (Michael,

interview, May 17, 2018)

Different perspectives and perceptions about the role of instructional designers emerged

consistently during the intervention as well as follow-up interviews with the participants. A

common theme was that faculty perceived instructional designers as being responsible for the

managerial and operational aspects of a course rather than as experts who provide pedagogical,

content, and instructional support to improve teaching and learning. Nancy shared:

In our role as instructional designers, a lot of people think we are tech support. A lot of

people also think that the way it works is they just dictate to us the changes to their

course sites and we are supposed to go in and make them. Of course, that’s really not

94
 what our role is. We are supposed to be instructional designers. We are supposed to be

collaborating with faculty to set up their course site in a way that promotes student

learning and engagement. But quite often, we are in crunch time. We have all these

courses that need to be set up and you do start to slip into that, “Fine, I’m just the task

manager. I’m just going to get these things set up and out the door.” Faculty forward

questions that come from students. Student troubleshooting is totally out of our realm—

that is why we have Blackboard help desk. (Nancy, interview, May 8, 2018)

Another theme was that staff were often charged with fulfilling technical and

administrative responsibilities when supporting faculty. Some participants noted that they were

viewed as “faculty secretary”—to provide faculty support in the role of an administrative

assistant or a technician. One instructional technologist noted, “They know I am here, why won’t

they ask? I am the answer lady. I am constantly in the reactionary administrative role as opposed

to being an instructional technologist” (Lisa, interview, May 24, 2018). The support roles

identified by participants ranged from providing general technology support in online courses to

helping faculty with scanning documents, making copies, and supplying stationery materials for

face-to-face classes. The general theme of instructional technologists being viewed as an IT help

desk as opposed to providing instructional technology expertise was also echoed by several

participants.

Participants in leadership roles also experienced differences in perceptions of roles and

responsibilities. One participant noted that the Director of the IDT is often perceived as

responsible for “everything online.” For example, issues related to advising online students as

well as student behavioral issues in online courses are perceived as under the purview of the

Director. Robert shared:

95
 The Director’s role is diverse—includes technical aspects (day-to-day managerial

responsibilities) as well as serving as a leader. The Director is expected to serve as a

Financial Manager (revenue projections) that can be out of their scope of expertise. I am

not a financial manager preparing budget and revenue projections. (Robert, group debrief,

April 17, 2018)

The activities led to organic conversations around how individuals saw their roles versus

how others saw their roles, and the frustrations they felt when people lacked an understanding of

their roles. One participant noted, “I was forced to react to the problems or concerns of other

roles. I was able to pursue a more comprehensive understanding of my colleagues’ understanding

of their roles and conflict between their expectations and the expectations of others” (Michael,

interview, May 17, 2018). Overall, participants found the opportunity to identify their own roles

and responsibilities and compare them with what others expected of them to be helpful.

Introspection and self-reflection. Nearly all participants reported that they valued the

intervention for self-reflection. They noted that it gave them the ability to “think on a deeper

level” about their own roles and responsibilities and allowed for self-awareness and

introspection. Michael explained:

It was introspective because I focused a lot on my role. It is something that I have always

struggled with, but it was a problem I never set down and evaluated to truly understand—

what elements were confusing about the expectations of my role and the actual

responsibilities of my role. It gave me a better understanding of all factors at play. It was

very self-reflective. (Michael, interview, May 17, 2018)

John shared similar thoughts on the opportunity for self-reflection:

96
 I felt like I never had an opportunity to sit down and reflect on my roles and

responsibilities. So, I think that’s a very helpful exercise. It forced me to jump out of my

own role or in my own thinking and think from others’ perspectives. It provided me with

the opportunity to systematically reflect on my roles, what the functions and

responsibilities are and how it is related to other parts of the department. Overall, it was

useful for individual self-growth or revelation. It helped me think about how my role

connects with rest of the team. (John, interview, May 15, 2018)

Furthermore, the hands-on and visual components of the role analysis exercise lent

themselves to cultivating a better understanding of the various dimensions of each role within the

IDT. Lee, who is in a senior leadership role, explained:

It was helpful in terms of making me think explicitly about my own role and the

dimensions of my own role. I very much liked the templates, the structured graphical

organization that you gave us on the flip chart paper. It allowed us to chart and separate

the different components of our role and then to look at some of the related aspects. I

thought everyone charting that out individually was really helpful. In terms of just

naming and giving clarity to my understanding of my own role, the graphical interface

for this workshop was a very helpful tool. (Lee, interview, May 17, 2018)

The four themes discussed above formed the general dimension of the role analysis

exercise contributing to collaboration and reflection on roles. Qualitative data suggested that it

not only gave participants the chance to talk more deeply and share with colleagues, but also to

seek clarity on the different roles involved in creating and facilitating an online course.

Participants also noted that the intervention highlighted the main areas where differences in

perceptions existed. One participant noted, “I found that identifying my roles and responsibilities

97
and comparing them with what others expect of me was very helpful. Time to re-write job

descriptions” (Lauren, interview, May 17, 2018). Furthermore, it sparked self-reflection and

community awareness in terms of the IDT members’ roles, how they relate to each other, and

ways to support each other.

Role stressors and challenges. Two second-order themes on the role analysis exercise

underscoring the role stressors and challenges faced by instructional design teams (IDTs) were

identified: (1) presence of role stress, and (2) manifestation of role stressors. Each of the second-

order themes related to this dimension is delineated below.

Presence of role stress. A common theme that emerged across the three intervention

groups was the need to validate the presence of role stress in IDTs. Participants suggested that

they benefited from seeing that others had similar questions and concerns about their roles in the

organization. They also noted that the exercises brought perspective to how much the team

manages and juggles and “to see the pull from different directions.” Kate shared:

I think that it helped me understand that there is role ambiguity for everyone. When you

experience role ambiguity or any job stressors, you just sort of internalize and say, “Okay,

this is only happening to me.” It allowed me to break out of that and realize that other

people also have stressors—and in some cases those are exactly the same stressors and in

other cases they are completely different. It helps us see stressors and pain points for the

team. So, we can think about how we can reduce the organizational stressors in a

systematic way that will reduce it in the future. (Kate, interview, May 16, 2018)

Echoing Kate’s comments regarding role ambiguity, another participant noted that being

able to see the ambiguity across roles was “not necessarily eye-opening, but it was comforting”

(Nancy, interview, May 8, 2018). Given that the nature and effects of role stress have not been

98
extensively studied within the higher education, and, specifically, IDT contexts (Dyer & Dyer,

2013; Harvey & Drolet, 2004; Lyons, 1993; Rahim, 2010), the intervention provided a platform

for validation of role stress among individuals involved in online course development.

Manifestation of role stressors. In addition to validating the existence of role stress,

participants noted that the intervention provided insights into the various ways in which role

stressors can manifest in online course development contexts. A common theme that emerged

was a general sense of acceptance that IDTs experienced role stressors on a regular basis and had

become accustomed to this milieu of competing demands, ambiguity and overlaps in roles, and

overload in the work environment. Amy shared:

There is a general sense of acceptance that the team is overloaded at certain times and

there is crossover in terms of roles and responsibilities. I think we all know what our

responsibilities are, but we have never really realized what we are doing in addition to

that—what we have put on ourselves. I think that it made us aware of how much we are

putting on ourselves in addition to what our primary role should be. (Amy, interview,

May 17, 2018)

Speaking specifically about role stressors experienced by instructional technologists,

another participant shared:

I’m an Instructional Technologist and technically I’m just supposed to fix issues and train

faculty on how to use different technologies. But I’m managing several other projects

outside those areas. It’s just assumed that these other projects will be done and there’s no

clear definition of what the supervisor will handle. The distribution of work is not equal.

(Lauren, interview, May 17, 2018)

Another participant shared challenges associated with the role of instructional designers:

99
 There’s an imbalance or an inequity in workload. I’m not saying we need more

instructional designers. I think we just need people on the team with a different skill set.

So right now the expectation is on the instructional designer to go and build media

elements. And, some of us have the skill set, some of us don’t. We don’t have the time to

learn. We have the desire to learn, but we don’t have the time. (Tina, interview, May 11,

2018)

Participants shared factors that contributed to role stressors: “We are all striving to give

the best customer support to our faculty. Hence, we are overloaded and have multiple role

conflicts” (Emily, interview, May 9, 2018); “I have a lot of projects that I am managing . . . it’s

because of convergence of multiple projects” (Amy, interview, May 9, 2018); “Faculty’s ability

to meet deadlines presents a huge challenge to the expectation that we have to bring innovation,

creativity, and student engagement” (Tina, interview, May 11, 2018); and “There is role

ambiguity, particularly with regard to leadership hierarchy. So, I have three people in leadership

telling me different things and giving me a somewhat conflicting directives” (Steven, interview,

May 16, 2018).

Another theme that emerged was that participants’ work commitment, motivation

(personal and professional), and self-regulating mechanisms influenced how they managed role

stressors. For example, in discussing role overload in IDTs, one participant noted, “I am doing

this to myself because I care about my job.” Similarly, another participant shared, “When

someone asks for help, I don’t want to push them off.” Tina explained further:

I feel like we are all overloaded at the same time very dedicated to provide the services

that faculty and students want. Sure, you can define all these roles, but ultimately it is

about what is best for our faculty and students. (Tina, group debrief, April 6, 2018)

100
Amy echoed similar thoughts:

It was really good to see how much in addition that we have made our jobs and that’s

because we love what we do. We want to do as much as we can and we want to be rock

stars. Well, I want to be a rock star. (Amy, interview, May 9, 2018)

Some participants also noted that they are often willing to take on tasks that fall outside

the scope of their job, owing to long-term career goals. For example, Michael shared, “It is how I

execute my role that is going to be indicative of where I want to go. So, if I want to be in a

different place, I am going to take on more roles and responsibilities for myself” (Michael,

interview, May 17, 2018). This seemed to indicate that IDT members may be overachieving or

taking on tasks that are outside of their role if those additional responsibilities align with future

career goals. This finding is consistent with Lepine, Podsakoff, and Lepine’s (2005) research on

the relationship of work stressors and strains, motivation, and performance.

While participants provided valuable information about their experiences of role stressors

within the online learning context, they recognized that the intervention prompted them to think

about some of the underlying areas of tension or the dimensions that need to be kept in balance

for effective team functioning. It also gave them a glimpse of what challenges others encounter

and areas where they experience similar stressors. One participant expressed, “Very helpful to

hear about the multiple roles of various stakeholders. This detailed look at our own roles and

exposure to others’ provided a lot of insight into types of role stressors and the reasons for those”

(Lee, written reflection, May 17, 2018).

Diversity of roles and responsibilities. Consistent with existing literature, the

intervention revealed the diverse nature of instructional design practice (Dooley et al., 2007;

Hokanson & Miller, 2009; Pan & Thompson, 2009; Ritzhaupt & Kumar, 2015). Participants

101
identified several competencies relevant to instructional designers’ work, including project

management, course design, course administration, training, and technology support. Further,

even though an instructional designer’s work typically involved providing pedagogical support,

one participant shared that instructional designers are often expected to train faculty on using

software on an as-needed basis—that is, provide in context technology training. In many cases,

the instructional designer became the main point of contact, providing faculty support in all

facets of online course development and management. One faculty participant described the role

of an instructional designer as follows:

Instructional designers’ role includes helping “ease nerves” and letting faculty vent. They

need to be there and be available. They need to guide faculty in presenting content. They

need to draw from me what I did in the onsite [face-to-face] offering and transfer that to

an online learning environment. They perform content curation, provide technology

guidance, and share experiences in what works and what doesn’t work. (Nora, interview,

May 16, 2018)

In discussing the core competencies of instructional designers, one instructional designer

shared the importance of project management:

For me, the project management piece is probably the part that I’m really going to need

to keep up with. As an instructional designer, project management is critical. It doesn’t

matter if you are creative or not. If the courses aren’t ready, nobody cares if you are

creative. But if the courses are not running, then they care. If they’re not ready, then they

[faculty] get upset. So that’s the piece I’m going to need to keep tweaking and making

sure that system is still working for me as my course load begins to grow. (Amy,

interview, May 9, 2018)

102
 The wide variance in the interpretation of faculty’s role in online course development and

facilitation was also discussed during the intervention. This included serving as a course

designer, providing a roadmap for teaching, revising and gathering feedback, collaborating with

instructional designers and other faculty, as well as “moving the course forward.” Several

participants shared concerns involving crossover in roles and responsibilities and often being

asked to “play out of position.” For example, Kate shared that “instructional technologists are

sometimes perceived as serving the same role as the multimedia technician. Instructional

technologists provide course support, not student support. Student issues should go to the

technical helpline, not instructional technologists.” She added,

I think one of the biggest issues as far as instructional technologists goes is that the role

can be very ambiguous—what is instructional technology, where does it stop, what level

of support do we provide in terms of course delivery? For example, when an instructor

has an issue with their live course, do they go to the instructional technologist,

instructional designer, support center, or do they email everybody? So defining roles is an

issue. (Kate, interview, May 16, 2018)

The overlap in roles and responsibilities of instructional designers, instructional

technologists, and multimedia technicians was a recurring theme. Participants suggested that one

way to delineate the role of instructional designer and instructional technologist is the

instructional designer provides content (pedagogical) guidance while the instructional

technologist provides guidance on how to use the tool (technical). Similarly, a multimedia

technician provides “how to” while an instructional designer provides support in terms of

instructional strategies for implementation.

103
 Ultimately, participants acknowledged that the evolving field of online education does

not lend itself to rigid role definitions; organizational structures need to become more fluid to

accommodate greater flexibility, collaboration, and optimal use of people’s skills and expertise.

This was reflected in statements such as the following:

I think our roles are already evolving pretty often and I think it’s just going to accelerate.

It is difficult to have very defined roles because of external factors. So, you need to be

flexible to adapt to social changes and evolution of technology. Also, when new people

join the team or existing members’ roles change, roles need to be redefined. I think we

need to not have a rigid definition of a role because we need to be able to adapt. (Ronald,

interview, May 9, 2018)

Another participant shared:

I don’t think it is possible to say you can just do this one role. I don’t think it will ever be

this “box.” It is always going to go outside the box. If we have a clear definition of what

our roles and our responsibilities are, it will help us in figuring out what we are going to

contribute outside of that area. If we have a clear definition, we can decide what we can

contribute outside the area. As much as you try to make systems, you can never make it

cookie cutter because we are people and there is gray area. Ambiguity exists, gray areas

exist. People often strive for black and white because it is easier for the individuals.

However, this can sacrifice what the team can accomplish. (Michael, interview, May 17,

2018)

While participants recognized the diverse roles of individuals involved in online course

development, they also concurred that it could be challenging to have rigid role definitions given

the evolving nature of online education. Participants shared that the role analysis exercise was

104
helpful in understanding the dimensions and nuances of the roles fulfilled by team members. As

one participant noted,

Before you write it all down, it’s really hard to grasp the multitude of things that our team

does. There are many different hats that are worn by members of the team and I think that

having an activity we did to map it all out was valuable. It was beneficial browsing each

other’s notes and then making comments. It was a vivid thing to point out like, “Oh, how

did they do all of these things.” So I think that was revealing. (Lee, interview, May 17,

2018)

Overall, qualitative data obtained via written reflections, group debrief, field notes, and

interviews provided several insights in addressing RQ2 that focused on participants’

perceptions of the value of a role analysis exercise and its potential to decrease role ambiguity.

While participants did not report that the role analysis exercise had a direct influence on

decreasing role ambiguity, it led to uncovering common themes on the roles of the IDT. These

themes included: (a) the significance of opportunities for interaction between faculty and staff in

formal and informal settings, (b) clarification of the roles of team members, (c) understanding of

different perspectives and perceptions of roles, (d) examination of one’s own roles and reflection

on associated responsibilities and tasks, (e) validation of issues related to role stress, (f) insights

into how role stressors manifest in the context of online course development, and (g) the multiple

roles of individuals involved in online course development.

Intervention Fidelity

Results from several measures informed the response to the third research question,

which explored the fidelity of intervention implementation. According to Rossi, Lipsey, and

Freeman (2004), process evaluation measures “whether or not it [program] is delivered as

105
intended to the target recipients” (p. 171). In this study, process evaluation involved the extent to

which all elements of the role analysis exercise based on Dayal and Thomas’s (1968) role

analysis technique (RAT) were implemented as planned. The intervention fidelity was assessed

using: (a) a minimum of three and maximum of 15 instructional design team (IDT) members

from each team recruited to participate, (b) participants’ attendance of approximately four hours

of face-to-face training delivered as a half-day workshop, (c) participants’ completion of the role

profile for each role analyzed, and (d) quality of program delivery by the facilitator (Dusenbury

et al., 2003).

The first measure of fidelity was the number of participants taking part in the

intervention. Considering that complexity of the online course design process often requires

individuals to assume multiple roles (Hokanson & Miller, 2009; Oliver, 2002; Reilly et al., 2012)

and role ambiguity is more likely when responsibilities overlap and interrelate among IDT

members, a minimum of three and maximum of 15 participants from each team was required for

high fidelity. In accordance with this requirement, a total of 29 individuals participated in the

study (School 1: n = 10, School 2: n = 6, School 3: n = 13).

The second fidelity measure was participants’ attendance of approximately four hours of

face-to-face training delivered as a half-day workshop. Since the intervention followed a

sequential approach to role analysis and the creation of the role profile was dependent on the

successful completion of the previous steps, participants were required to attend the entire

workshop to complete the intervention. The attendance records maintained by the researcher

during the intervention confirmed full attendance of all participants throughout the role analysis

exercise.

106
 The third fidelity measure was participants’ completion of the role profile for each role

analyzed. During data analysis, all role profile sheets were thoroughly analyzed and it was

determined that participants completed all the required steps and the final role profiles were

recorded in the established worksheet formats.

The fourth fidelity measure, quality of program delivery, was measured with tools

designed for participant feedback after the intervention. Qualitative data obtained from

interviews, written reflections, and group debrief sessions resulted in five second-order themes

and one general dimension. Table 13 shows an overall representation of the classification system

of responses obtained regarding the quality of program delivery via written reflections, group

debriefs, and follow-up interviews.

Table 13

Quality of Program Delivery: Hierarchical Analysis Results

First-Order Themes Second-Order General
Themes Dimension
 Broken down concisely to facilitate analysis
 Role definition at the end was beneficial Analysis of
 Enable thinking of all of the aspects of the role in Roles and
a quick but thorough way Responsibilities
 Clearly defined and explained n = 14
 Concisely structured for analysis of roles
 Provide timely feedback for challenges Intervention
 Clarify terminology when explaining activities Instructions and Design,
 Offer pointed instructions and tags Guidance Implementation,
 Clarify instructions for the different activities n = 15 and
 Provide more structure and rules for engagement Representation
of Roles
 Extend the workshop for a more in-depth
conversation and collaboration with the team Opportunity for
 Offer follow up materials to work on as a team Preparation, In-
 Provide a follow-up workshop at a later date Depth
 Offer resources to accompany the agenda Discussion and
 Provide preparatory materials and opportunity to Follow-up
do pre-work to formulate ideas more coherently n = 33

107
 First-Order Themes Second-Order General
Themes Dimension
 Faculty representation is crucial
 Involve faculty more to minimize ambiguity
 Less impactful if only person represents a role
 Ensure each role has more than one Representation
representative of Roles
 Ensure diverse roles are represented n = 20
 Consider the advantages and disadvantages of
having faculty and leadership representation

Intervention design, implementation, and representation of roles. Four second-order

themes regarding the intervention design and implementation provided insights into the quality

of program delivery: (a) analysis of roles and responsibilities; (b) instructions and guidance; (c)

opportunity for preparation, in-depth discussion and follow-up; and (d) representation of roles.

Each of the second-order themes related to this dimension is delineated below.

Analysis of roles and responsibilities. High fidelity was evident in participants’

responses that the intervention design and implementation processes provided opportunities to

engage in a structured analysis of roles and responsibilities. Participants noted that the exercises

were broken down concisely to facilitate analysis of roles and the use of role stressors as a

framework was useful in understanding the specific challenges faced by team members. Roger

explained:

I thought it was structured very well. I liked how it built up throughout the workshop. I

especially appreciate it because we didn’t think we had a ton of foundation for it before

we walked in. But it became apparent very quickly to me that it was well designed and

scaffolded properly so that, by the time we got through the end we were doing something

relatively complex. (Roger, group debrief, April 17, 2018)

108
 A faculty participant commented on the structure and attributes of the exercise

contributing to a thorough analysis of roles and responsibilities:

I thought it was a great balance of opportunities to get up, walk around, and interact with

everyone at different points. You were asking fundamental questions about people’s roles

and everybody had to answer from their perspective, and in doing that we all learned

about what each other does. It was very simple, straightforward, direct, meaningful

questions from the perspective of each person’s job. Everyone was giving their answer

from the perspective of what they did. By doing that, we all learned about what each

other did. (Nora, interview, May 16, 2018)

Participants liked the hands-on nature of the activities as it gave them the opportunity to

“get up and move around.” Amy shared, “It was a good variety of things so I definitely was not

bored and I was engaged the whole time” (Amy, written reflection, April 6, 2018). Participants

also remarked on the facilitator’s management of discussions, keeping them on task and

establishing the objectives of each activity. One participant said the facilitator created awareness

of roles and responsibilities and identified areas for collaboration. Another participant noted that

the intervention was non-threatening, positive, and allowed for teambuilding. In addition, the

facilitator’s approach to “keeping the discussion going” and guiding conversations was echoed in

Michael’s comments:

I liked how concise the workshop was structured. Often, there is just so much extra that

gets put into conversations, and then conversations get longer and more complex, and

bigger than they need to be. I liked that we were able to focus on what’s the one thing that

we cared about and were able to dig into that in concise ways. It forced people to

prioritize. In a more extended workshop, where you have more time, people allow

109
 themselves not to do that because they have more time, more flexibility, or more control

over engagement. (Michael, interview, May 17, 2018)

Michael’s remarks demonstrated how the intervention design and implementation

techniques promoted dialog and progress toward role analysis and clarification.

Instructions and guidance. Participants mentioned that more detailed instructions and

rules of engagement would have been useful in completing the activities. Responses suggested

that succinctly articulating ideas and differences between tasks would have been helpful in

completing the role analysis exercise. Roger elaborated on the need for more explicit instructions

and guidance during the intervention:

I think the only thing that I would tweak is that it was confusing at times when we got

through defining the roles on paper. I don’t know if it was us or the description of it, but

it took us a little while to figure out what we were supposed to do on the paper. (Roger,

written reflection, April 17, 2018)

Participant responses indicated the need for more clarity in instructions and terminology

when explaining activities. Lee described the need for more guidance to allow participants to see

the connections between the role analysis and role profile:

The facilitator can guide the discussion to help people look at the broad landscape of the

role and make connections. Help the group analyze and identify themes that emerged

from the role profiles. For example, when you identify role stressors, encourage

participants to discuss them more openly in the group and make meaning of those

theories in the context of real practice. Give people more of an inductive chance to make

their own connections between themselves and the stressors as they identified them. (Lee,

interview, May 17, 2018)

110
 Opportunity for preparation, in-depth discussion, and follow-up. An important practical

lesson emerged regarding the length of the intervention. Qualitative feedback suggested that the

design would have benefited from more time for the participants to process and reflect, as well as

revisit topics discussed during the intervention. Due to scheduling challenges, the intervention

was limited to a four-hour face-to-face workshop. Participants shared that the intervention would

have benefited from more time before, during, and after the workshop. This would have

permitted participants to engage in deeper and more meaningful discourse. Lee explained:

It would be helpful to have a second step to the workshop, one that would allow for

deeper, meaningful analysis. It’s almost like a second step and it might be good to have a

gap in between the experiences because of the additional processing and reflection that

might happen in between the two workshops. I think that having an individual reflective

component and then making sense of that in terms of sharing with everyone else and

coming up with some group identified themes can be very helpful. Since there are teams

that are already working together, it can also lead to action steps in some cases. There

might be some obvious remedies that can be taken based on what was revealed during

that second discussion. (Lee, interview, May 17, 2018)

He added:

While I found that [intervention] helpful, one thing that is worth exploring more and

could be developed into a fuller discussion, if there were time for that, would be to

examine as a group more of the connection between roles as we see them. The connection

between our roles to be able to identify, analyze and devise strategies to deal with actual

instances of where there might be real ambiguity or any of the role stressors that we

111
 talked about in theory. A more extensive discussion around those and bringing out

examples of it could be even more helpful as a follow-up. (Lee, interview, May 17, 2018)

Karen echoed Lee’s comments:

I think it would have been good to have it as a day, then there’s more time for organic

conversations. You could also have an intentional reconvene with a moderator two or

three weeks later. If it was close enough that you still remember all of what you did but

far enough apart that you had time to reflect, it could be helpful. I walked away going,

okay, where’s the follow-up material that I can take away with me? And we all said, yeah,

we’re going to talk about this again, but we’re so busy that it didn’t happen. So having an

intentional planned session would be helpful. (Karen, interview, May 11, 2018)

Other participants also suggested either extending the four-hour workshop to a full-day

session or scheduling a follow-up workshop to allow more in-depth discussion and analysis

around broader issues of roles and connections between roles. A follow-up session would allow

for the focus to extend beyond merely defining and clarifying roles, to a discussion around the

application of new knowledge to existing team and organizational processes. Lee explained,

“Additional conversations could help contextualize and help everyone make meaning of those

roles and where they overlap. The connectivity between all the pieces could be better woven

together” (Lee, interview, May 17, 2018).

Further, participants reported the need for a list of concrete action items after the

workshop that would help create momentum and guide the IDT in addressing potential role

stressors and fully realizing effective collaborations within the team. Amy explained:

112
 I feel like we didn’t know, where do we go from here? We got our little rolled up papers

and then, what are we going to do with it now? I feel like that would have been good

because I don’t think we came up with any action items. (Amy, interview, May 9, 2018)

Another theme that emerged regarding the intervention design was the need for greater

preparation time to allow participants to reflect on their roles and perceptions before coming to

the session, allowing them to formulate them more coherently. Some participants shared that

they would have benefited from preparatory materials, such as articles and resources to

accompany the agenda. Nora, a faculty participant, also shared, “When we wrote that first

question, I think some people were saying, what are we going to do with this? So I think maybe a

little bit more information beforehand would be helpful” (Nora, interview, May 16, 2018).

Representation of roles. Role representation was not consistent across the three schools

where the intervention was implemented. While the teams from School 1 and School 2 were

primarily comprised of instructional designers, instructional technologists, and managers (or

executive team members), School 3 included faculty participants (n = 4). Participant responses

indicated the need for more diverse role representation, both within the IDT as well as with

external stakeholders. Specific to faculty representation, Emily shared:

I think it will be helpful to have maybe one or two faculty present so they can get an idea

of our perspective and maybe we can get their perspective of what they’re feeling and

what their frustrations are and why things aren’t done. We can then come up with a plan

to work together. Involving faculty or a program lead from each department in the

workshop can help bridge ideas and come up with action items to make the process

better. (Emily, interview, May 9, 2018)

113
 At the same time, some participants also noted the potential downside to having faculty

present during the role analysis exercise. Ronald shared, “I actually think it was a really good

idea that there were no faculty. From the perspective of the instructional designers, to get really

genuine feedback but not be afraid of ruffling feathers, I think that was a really good idea”

(Ronald, interview, May 9, 2018). Participants also expressed that an approach for future

programs could be implementing a follow-up session with faculty. Ronald explained:

When you don’t involve faculty, you do miss genuine feedback. You miss the interaction

and getting some of the back and forth between the instructional designers and faculty.

But, on the positive side, not involving faculty allows for more genuine feedback from

the instructional designers without the feeling of being judged. But in the future, having

an open discussion between instructional designers and faculty with a third party of

arbitrating could be a good thing. For me, it might not be a bad idea for these kinds of

workshops to be a two parter—one with and one without faculty. (Ronald, interview, May

9, 2018)

Similar to faculty representation, Roger shared reservations around having leadership

present during the intervention:

Having leadership present is always a double-edged sword in the sense that you can both

talk about what kind of help you need as far as role clarity and support goes, but it can

also feel like there is some kind of an eye in the sky watching you and feedback is being

evaluated. So it’s a double-edged sword in that way. It can encourage people to provide

feedback but also stifle and make people not be frank about how they feel about their role

and position. (Ronald, interview, May 9, 2018)

114
 Another theme was that the interactive and collaborative process seemed more effective

when there was more than one participant representing the same role—preferably at least three.

For example, the group of four faculty seemed engaged with one another and took the most time

to complete the activities. Participants who completed the activities individually finished in the

shortest time. Michael, who was the only person representing his role, reflected:

One thing I would make sure is that there is more than one person for each role. But,

when you have two people, it’s too easy for them to have a very similar view and agree

with each other’s views. I think having at least three people for each role would be

helpful to make sure there is enough diversity between the individuals and then when it

comes to synthesis, they can gain multiple perspectives, synthesize different perspectives

and see how they fit into that role. Having more than one and preferably three people

representing each role would help people better understand that dichotomy of the

individual versus the role. (Michael, interview, May 17, 2018)

Future program design should thoroughly evaluate the advantages and disadvantages of

having faculty and leadership representation during the role analysis exercise. Perhaps, as one

participant suggested, the exercise can be offered independently for the IDT and faculty or

individuals in leadership roles.

Discussion

Given the complexity of instructional design for online courses and the diverse roles

assumed by individuals involved, role ambiguity in instructional design teams (IDTs) can

contribute to ineffective design practices in instructional design. Thus, a role analysis

intervention was designed to decrease role ambiguity and promote role clarification among

members of IDTs in higher education. The research questions focused on the extent to which the

115
role analysis exercise resulted in a decrease in role ambiguity (outcome evaluation), participants’

perceptions of the value of the role analysis exercise and its potential to decrease role ambiguity

(outcome evaluation), and the fidelity with which it was implemented (process evaluation).

Although many types of role clarification interventions have been implemented in

organizational settings, this is the first study that has explored the potential for a role analysis

exercise to decrease role ambiguity among IDTs involved in online course development. The

findings suggested that the role analysis intervention did not result in a decrease in role

ambiguity as measured by the Role Ambiguity (RA) subscale in Pareek’s Organizational Role

Stress (ORS) scale. In contrast, the intervention elicited a statistically significant median increase

in role ambiguity after the intervention compared to before the intervention, indicating that

participants reported experiencing an increase in role ambiguity after the role analysis

intervention. On closer investigation, several factors could have contributed to this finding.

First, considering that the topics of roles, role stressors, and role analysis have not been

previously explored in a formal setting among the participating IDTs, it is possible that the

intervention increased participants’ awareness of the existence of role stress and uncovered areas

of uncertainty. This new knowledge, coupled with the intervention lacking in concrete action

items to address the identified issues, could have resulted in an unintended effect of increased

role ambiguity after the intervention. Second, considering that participants completed the post-

test two weeks following the intervention, the interim happenings between the pre- and the post-

test could have influenced the results. Third, the role analysis intervention consisted of a four-

hour workshop. It is possible that the outcomes would have been more beneficial to decreasing

role ambiguity if more sessions (or an extended session) were provided. Finally, pre-test and

post-test instruments measured only role ambiguity, while other related concepts, such as role

116
conflict and role overload, were not studied. While role ambiguity scores increased, a decrease in

role conflict or role overload might have been encouraged through the intervention.

While the quantitative measure did not indicate a decrease in role ambiguity following

the intervention, qualitative data obtained through written reflections, group debrief, field notes,

and follow-up interviews revealed several important themes relevant to participants’

perceptions of the value of the role analysis exercise and its potential to decrease role ambiguity.

Participant responses demonstrated the intervention’s potential to: (a) promote collaboration

between faculty and staff, (b) provide clarification of roles and expectations, (c) reveal different

perspectives and expectations of roles, and (d) promote self-reflection and analysis of one’s own

roles. Further, it gave participants the opportunity to validate the presence of role stress in IDTs

and underscored the ways in which role stressors exist within online learning contexts. Finally,

consistent with past findings (Dooley et al., 2007; Hokanson & Miller, 2009; Pan & Thompson,

2009; Ritzhaupt & Kumar, 2015), it confirmed the diverse and constantly evolving roles of

individuals involved in online course development.

Taken together, quantitative and qualitative data can provide a clearer and more

comprehensive picture of the value of the role analysis intervention. Although the quantitative

results did not yield a decrease in role ambiguity, qualitative analyses were illuminating. As

noted above, the increase in role ambiguity reported by participants after the intervention could

be an unintended consequence of participating in the role analysis exercise. The process of

intentionally reflecting on one’s own roles as well as areas where gaps, overlaps, and

uncertainties exist could lead to participants’ perceptions of heightened levels of role ambiguity.

This is consistent with the qualitative data that indicated that the role analysis intervention

afforded participants the opportunity to engage in self-reflection as well as recognize the

117
existence of different perspectives on IDTs’ roles and responsibilities. Another reason for the

increase could be that the quantitative measure did not adequately capture the complexity of how

role stressors, especially role ambiguity, manifest in IDTs. Qualitative measures provided a richer

understanding of the value of the role analysis exercise in the IDT context and online education,

in general.

An examination of fidelity measures indicated that the intervention design yielded high

fidelity in terms of the number of participants recruited to participate, attendance of a four-hour,

face-to-face workshop, and participant responsiveness (participation in the activities of the role

analysis exercise). The quality of program delivery was measured using data obtained from

interviews, written reflections, and group debrief sessions. While participants reported that the

intervention helped facilitate the analysis of roles and responsibilities, several practical lessons

emerged to enhance the intervention design as well as the quality of facilitation and moderation

for future implementation. These included providing: (a) detailed instructions and rules of

engagement for participants; (b) opportunities for pre-work and preparatory materials; (c)

extending the workshop from four hours to one day to allow more in-depth, meaningful

discussions; and (d) a follow-up session that focuses on the application of new knowledge to

existing team and organizational processes. Further, ensuring diverse roles are represented and

each role has more than one representative to allow for meaningful dialogue and discussions

could be beneficial. At the same time, genuine conversations and honest feedback among IDTs

might be compromised in the presence of leadership.

Implications for Practice

While limited in sample size and scope, this study revealed results that can be applied to

instructional design teams (IDTs) and higher education institutions involved in online course

118
development. Although a statistically significant decrease was not observed in post-intervention

role ambiguity scores, participants reported an increased understanding of roles and expectations.

Research indicates the potential benefits of role analysis technique (RAT) to improving

organizational and team performance (Buch & Aldridge, 1990; Singh, 1997; Srivastav, 2010). By

implementing the role analysis intervention on an ongoing basis, IDTs may realize its potential

benefits over time. Knowledge about how role stressors manifest in online education contexts

could provide stakeholders with critical information that can lead to improved team

communication, motivation, and performance. More work is needed to implement the role

analysis intervention appropriately for its use in instructional design contexts. However, this

study suggests that a role analysis exercise has the potential to improve the functioning of IDTs

and the quality of courses produced by them. Therefore, this study invites future researchers to

continue to pursue the study of role analysis intervention to bring clarity to the roles of

individuals involved in online course development.

Opportunities for Collaboration between Stakeholders

The study revealed the need for opportunities to promote faculty and staff collaboration

in formal and non-formal settings. Considering that role ambiguity can stem from faculty

misconceptions around roles and responsibilities, opportunities for interaction between faculty

and staff could result in improved communication and expectation setting. Lisa, an instructional

technologist, noted:

Not having a clear channel to communicate with them [faculty] or not knowing what

channels to use, there’s a feeling of faculty being a protected group and that we can’t

contact them in certain ways. We need to learn the culture of that, where it originated and

how to break it. (Lisa, interview, May 24, 2018)

119
 Another participant, Nancy, explained the importance of clarifying roles and expectations

with faculty:

Role ambiguity could stem from faculty’s expectations of teams. They dial the number

where they can reach us. Sometimes they don’t know the difference between design and

tech support. They think Blackboard support is something instructional designers should

do. They always look for immediate assistance, like hitting the “0” on customer calls.

(Nancy, interview, May 8, 2018)

Similarly, creating opportunities for IDTs to collaborate and engage in meaningful

conversations has the potential to result in improved clarity around roles and responsibilities.

Ronald shared:

It would be helpful to have more team exercises to explore dynamics within the group.

Also, I would recommend more opportunities for reflections about roles within the

group—for example, try to put people in each other’s shoes to see things from the other

person’s perspectives. (Ronald, interview, May 9, 2018)

The need for opportunities to collaborate with team members was also reflected in

Kristi’s comments:

It would be helpful to meet regularly and share what we are doing for the week. I would

love to hear a two minute debrief of how everyone’s doing. That would be a half an hour

meeting and then I think we might be a little more on the same page. Perhaps talking

about what our career goals are during a staff retreat. This will give us opportunity to talk

about how we can help each other grow. (Kristi, interview, May 11, 2018)

120
 The findings suggest the need for institutions to provide opportunities for faculty and

staff to get together in non-threatening and collegial environments. Such opportunities can enable

mutual knowledge sharing, collaborative efforts, and clarity in each other’s roles.

Processes and Policies Surrounding Online Learning

Within instructional design teams (IDTs), several rules, often enforced as policies and

procedures, exist to establish certain boundaries around course development processes.

Institutional policies can serve as a written course of action to facilitate the seamless functioning

of all the processes associated with online learning. Policies can help bring clarity and

consistency to the processes associated with online course development. However, for

institutional policies to be effective and of value, they should “cascade down from the top

organizational level and articulate into faculty goals and rules” (Stacey, 2009, p. 235).

Furthermore, institutional policies should be written in clear and concise language, leaving little

room for ambiguity and misinterpretation, especially given that some of the issues have the

possibility of having serious legal ramifications.

Participants shared the need for clear policies surrounding online course development,

including communication of roles and expectations to relevant stakeholders. An instructional

designer shared:

We have policies and processes in place but faculty are not held accountable. In terms of

their work with us, they can still get paid their x amount of dollars for a new development

or revision and we are the ones who suffer because we are the ones who get it [course

content] last minute expected to get it online and expected to work the evenings and

weekends. We need to start to build in language [in policies] where you have timelines

121
 and you have deliverables along the way. If you don’t meet those milestones, you may

not get paid. (Tina, interview, May 11, 2018)

Follow-up interviews with participants also elicited the need for leadership support in

ensuring policies are followed. Nancy expressed:

We are supposed to be collaborating with faculty to make their courses interactive. We

are supposed to be innovative and help facilitate student learning in the online

environment. But we do get pushback when we resist being task managers or doing

things that don’t really seem relevant. We end up accommodating these ad hoc requests.

It is also messaging expectations to faculty. This is really not my job and we need to set

those boundaries. So having that clarity and knowing that, well, if I do push back or I do

tell somebody what my responsibilities are and aren’t and if that person comes back and

is upset about that, somebody from leadership would explain and support us. (Nancy,

interview, May 8, 2018)

The communication of expectations to stakeholders—especially faculty—was a recurring

theme in participant responses. One participant shared,

Maybe it’s clarifying for faculty what our roles are and ensuring that they understand

what their roles are. They don’t know where to go. So they come to us and then we tell

them that’s not really what we’re here for and then they feel frustrated because they don’t

know where to go. (Nancy, interview, May 8, 2018)

Participant responses indicated that there is a need for carefully crafted communication

that clearly outlines the roles of individuals involved in online course development. On the other

hand, from a faculty perspective, Nora argued that a barrier to communication between faculty

and staff is the technical jargon used to describe the roles of IDT members:

122
 It would be helpful if the team communicated roles and responsibilities without using

your own “private language” that only the team understands. Often, the titles are

ambiguous to faculty. I use the term Instructional Designer now because I know what an

instructional designer does. But your team often uses a private language and they don’t

realize that they’re isolating themselves from faculty who don’t understand what they’re

talking about. You’re making it harder for people to know how to ask for help. Some of

the language that is used makes what you do a little more opaque than you intended.

(Nora, interview, May 16, 2018)

Resources to Support Online Course Development

The need for adequate resources to support online course development is an ongoing

issue that should be addressed. Further, a related challenge is the IDT’s ability to respond to all

institutional needs, not just those specifically for online; rather, the whole gamut of teaching,

learning, and instructional technology needs. Several participants shared the need for adequate

administrative staffing to handle the operational aspects of online course development. For

example, an instructional technologist shared:

Faculty do not have any administrative staff. I think if the faculty actually were supported

the way that they need to be supported, they would then be able to see our [IDT] roles as

they are meant. There’s no one to give them messages or put a note on their door when

they are going to be late for class. Those are their first level needs. If their first level

needs aren’t being met, they’ll never be able to take advantage of their second level

needs—such as brainstorming with us for a few hours on how we can integrate

technology into their classes. (Lisa, interview, May 24, 2018)

123
 Instructional designers also shared the need for administrative staffing to support the

team’s operations and perform clerical tasks. Amy noted:

An administrative person can help me with communicating to all the stakeholders,

sending documents to the editor, uploading closed captioning, and doing quality

assurance. That kind of stuff would be helpful for me so I could focus on the higher level

functions. Someone to handle some basic administrative duties, not to say anything is

beneath me, but it would help free up some time that we spend course tagging, filling out

the spreadsheet, and those little [administrative] things. (Amy, interview, May 9, 2018)

The physical space where the IDT is located and its proximity to faculty is another

resource issue that several participants reported. Organizationally, IDTs can be located in

different places in higher education institutions. Ritzhaupt and Kumar (2015) noted that this

could range from departments, schools, and colleges, to centers for teaching and learning. At the

same time, “the location where an instructional designer is housed has implications for his or her

job role within the organization” (Ritzhaupt & Kumar, 2015, p. 64). Proximity to faculty, the

location of offices, and prior knowledge (or misinformation) about roles have the potential to

influence how the IDT members are perceived. For example, Lisa stated, “Location of the team

plays a role in how faculty/others perceive our roles and responsibilities. Closest person is often

contacted (staff proximity). Spatial convenience could lead to misaligned role expectations”

(Lisa, interview, May 24, 2018). Considering these findings, a practical implication is that

organizations should evaluate the physical space of IDTs and take steps to ensure that it is

optimal for realizing the team’s full potential.

124
 Limitations

There are several limitations to this study including a small sample size, consolidated

intervention length, short follow-up period, and the absence of a comparison group. The study

sample included 29 instructional design team (IDT) members, recruited from three schools

within the same university. The small sample size may be limiting in scope regarding the

generalizability of the findings. A larger sample of IDT members from different types of

institutions (public, two-year, for-profit, etc.) might reveal more information about the usefulness

of the role analysis exercise in decreasing role ambiguity and promoting role clarity. Since all of

the participants knew each other prior to participating in the intervention, it is possible that some

participants simply went along with what others said to avoid conflict of opinion. Further, the

participation of faculty was limited (n = 4), and therefore the study did not capture faculty

perspectives adequately to address the full scope of the online course development process.

The study participants were all volunteers and may have already been interested in the

topic of study. Therefore, they may have been biased toward more positive outcomes and

differed in important and unmeasured ways from nonparticipants. Additionally, the intervention

involved four hours of face-to-face training delivered as a half-day workshop. This may not be

enough time to measure a change in role ambiguity and role clarity. Further, considering that

participants completed the post-test two weeks following the intervention, the interim

happenings between the pre- and the post-test could have influenced the results. The absence of a

comparison group limits the ability to draw any firm conclusions or generalize the results to

other IDT contexts in higher education.

Further, the sensitive nature of the intervention topic, as well as the presence of a

supervisor, may have inhibited participants from expressing their honest opinions during the

125
intervention. Participants expressed concerns about sharing genuine feelings during and after the

intervention. This came as no surprise given the sensitive nature of the issues discussed during

the role analysis exercise. One participant noted that having subordinates in the same room as the

supervisor was a hindrance to having honest conversations and sharing “the true state of the

team” (Ivan, interview, May 5, 2018). Some participants also worried about offending others and

stirring up negative feelings. Kristi shared, “I couldn’t help feeling a little bit of the awkwardness

during the workshop because of the nature of the topics we were discussing” (Kristi, interview,

May 11, 2018). She added, “I was worried that people are going to think I hated my job or that I

hate how people look at my role. But I didn’t have any of that and I just truly wanted to explore

it.” Roger who is in a leadership role expressed:

I was a little bit concerned with how candid the feedback would be from them [team]. I

do wonder what kind of feedback you would have gotten or what would have happened

had you done some of that stuff without us [leadership] in the room. In the sense that how

much more candid they might have been about some of their frustrations. I worry in

situations like that. I fully understand the approach of stepping back and letting things

happen because you don’t want to overpower the conversation or influence them. I’m

always concerned about that. (Roger, interview, May 16, 2018)

Even though the facilitator took measures to establish that the intervention environment

was a safe space and the goal was to engage in constructive conversations, the sensitive nature of

the topic and presence of a supervisor could have inevitably inhibited candid responses from

participants. As one participant explained, in the presence of a supervisor, “you will be little

more cautious about what you say and do, and if there’s that sense of tension already, that would

126
change the dynamic” (Karen, interview, May 11, 2018). Therefore, representation of roles during

the role analysis exercise should be carefully evaluated.

Conclusion

Literature shows that role overload, role ambiguity, and role balance (e.g., time spent on

each role) are potential challenges concerning online education (Briggs, 2005). This study

contributed to taking the first step toward recognizing and understanding the potential for role

stressors to negatively influence instructional design team (IDT) performance and the quality of

online course design process, and consequently, the quality of online courses delivered by higher

education institutions. Thus, the role analysis intervention was an important first step in the

development of opportunities for IDTs to (a) analyze key roles within the context of existing

online course development processes; (b) identify limited perceptions or misperceptions around

the roles that could affect role effectiveness; and (c) develop clarity around the specific roles of

IDT members in contributing to online course development processes and the overall quality of

online courses. Additional research is needed to assess the value of the role analysis

exercise and its potential to decrease role ambiguity in IDTs and to explore additional measures

of this complex concept.

Additionally, this study revealed implications for practitioners and policymakers in online

education contexts. These included the importance of ongoing collaboration between faculty and

staff in formal and non-formal settings and the need for consistent policies and processes that

clearly outline the roles and responsibilities for individuals involved in the online course

development process—such as instructional designers, instructional technologists, project

managers, multimedia personnel, videographers, executive team members, faculty and/or subject

127
matter experts. Finally, it also brought to the forefront the need for resources to support the work

of faculty and staff responsible for designing, developing, and facilitating online learning.

Further research needs to be conducted to determine how role analysis interventions can

support IDTs in minimizing role ambiguity and promoting role clarity. Ultimately, interventions

such as the role analysis technique (RAT) should be integrated into the IDT’s existing processes

(Srivastav, 2006). Duke (2014) pointed out that designing a program or intervention should be

iterative. IDTs should evaluate and reevaluate the quality of instructional design processes, and

the roles and responsibilities of the members involved in those processes. Deming’s (1986)

continuous improvement model recommended a plan-do-study-act cycle where teams can engage

in continuous improvement. By implementing the role analysis exercise on a routine basis, the

IDT can expect continuous improvement through planned changes—that is, clarification of roles

and decrease in role ambiguity.

128
 References

Abbas, S. G., Roger, A., & Asadullah, M. A. (2012). Impact of organizational role stressors on

faculty stress & burnout: An exploratory analysis of a public sector university of

Pakistan. ISEOR-AOM, 1. Retrieved from http://halshs.archives-

ouvertes.fr/docs/00/69/88/06/PDF/STRESS_BURNOUT-ISEOR_AOM.pdf

Abdous, M., & He, W. (2009). Streamlining the online course development process by using

project management tools. In Orellana, A., Hudgins, T., & M. Simonson (Eds.), The

Perfect Online Course: Best Practices for Designing and Teaching (pp. 389–399).

Charlotte, NC: IAP-Information Age Publishing.

Ackfeldt, A. L., & Malhotra, N. (2013). Revisiting the role stress-commitment relationship: Can

managerial interventions help? European Journal of Marketing, 47(3/4), 353–374.

Adair, D., & Shattuck, K. (2015). Quality Matters: An educational input in an ongoing design-

based research project. American Journal of Distance Education, 29(3), 159–165.

doi:10.1080/08923647.2015.1057094

Akdemir, O. (2008). Teaching in online courses: Experiences of instructional technology faculty

members. Turkish Online Journal of Distance Education, 9(2), 97–108. Retrieved from

http://eric.ed.gov/?id=ED501108

Allen, I. E., & Seaman, J. (2008). Staying the course: Online education in the United States,

2008. Newburyport, MA: Babson Survey Research Group and The Sloan Consortium.

Allen, I. E., & Seaman, J. (2011). Changing course: Ten years of tracking online education in the

United States. Newburyport, MA: Babson Survey Research Group and The Sloan

Consortium.

129
Allen, I. E., & Seaman, J. (2010). Class differences: Online education in the United States, 2010.

Newburyport, MA: Babson Survey Research Group and The Sloan Consortium.

Allen, I. E., & Seaman, J. (2016). Online report card: Tracking online education in the United

States, 2015. Newburyport, MA: Babson Survey Research Group and The Sloan

Consortium.

Alley, L., & Jansak, K. (2001). The ten keys to quality assurance and assessment in online

learning. Journal of Interactive Instruction Development, 13(3), 3–18.

Ally, M. (2004). Foundations of educational theory for online learning. In T. Anderson & F.

Elloumi (Eds.), Theory and practice of online learning (pp.15–44). Athabasca, Canada:

Creative Commons: Athabasca University.

Altman, B. W., Schwegler, A. F., & Bunkowski, L. M. (2014). Beliefs regarding faculty

participation in peer reviews of online courses. Internet Learning, 3(1). Retrieved from

http://www.ipsonet.org/images/Westphalia_Press/Internet_Learning_Journal_2-2/3-

1/8.%20Schwegler%20ILJ%203-1.pdf

Anderson, T. (Ed.). (2008). The theory and practice of online learning (2nd ed.). Athabasca,

Canada: AU Press.

Andrade, M. S. (2015). Teaching online: A theory-based approach to student success. Journal of

Education and Training Studies, 3(5), 1–9.

Appana, S. (2008). A review of benefits and limitations of online learning in the context of the

student, the instructor, and the tenured faculty. International Journal on ELearning, 7(1),

5–22.

Arbaugh, J. B. (2004). Learning to learn online: A study of perceptual changes between multiple

online course experiences. The Internet and Higher Education, 7, 169–182.

130
Aziz, M. (2004). Role stress among women in the Indian information technology sector. Women

in Management Review, 19(7), 356–363.

Bacsich, P. & Ash, C. (1999). The hidden costs of networked learning: The impact of a costing

framework on educational practice. Retrieved from

http://www.ascilite.org.au/conferences/brisbane99/papers/bacsichash.pdf

Baer, R. A. (2003). Mindfulness training as a clinical intervention: A conceptual and empirical

review. Clinical Psychology: Science and Practice, 10(2), 125–143.

Bart, M. (2012). Online student engagement tools and strategies. Faculty Focus Special Report.

Wisconsin: Magna Publications.

Bartolic, S., & Bates, A. (1999). Investing in online learning: potential benefits and

limitations. Canadian Journal of Communications, 24, 349–366.

Bates, A. W. (2000). Managing technological change: Strategies for college and university

leaders. San Francisco, CA: Jossey-Bass Publishers.

Beech, N., MacIntosh, R., & MacLean, D. (2010). Dialogues between academics and

practitioners: The role of generative dialogic encounters. Organization Studies, 31(9),

1341–1367.

Beehr, T. A. (1996). Basic organizational psychology. Needham Heights, MA: Allyn & Bacon.

Berge, Z. L., & Muilenburg, L. Y. (2001). Obstacles faced at various stages of capability

regarding distance education in institutions of higher learning. Tech Trends 46(4), 40–45.

Bhattacharya, S., & Basu, J. (2007). Distress, wellness and organizational role stress among IT

professionals: Role of life events and coping resources. Journal of the Indian Academy of

Applied Psychology, 33(2), 169–178.

131
Bird, L. (2007). The 3’C design model for networked collaborative e-learning: A tool for novice

designers. Innovations in Education and Teaching International, 44(2), 153–168.

Boles, J. S., Wood, J., & Johnson, J. (2003). Interrelationships of role conflict, role ambiguity,

and work-family conflict with different facets of job satisfaction and the moderating

effects of gender. Journal of Personal Selling & Sales Management, 23(2), 99–113.

Bolliger, D. U., & Martindale, T. (2004). Key factors for determining student satisfaction in

online courses. International Journal on E-learning, 3(1), 61–68.

Bork, R. H., & Rucks-Ahidiana, Z. (2013). Role ambiguity in online courses: An analysis of

student and instructor expectations. Retrieved from

http://ccrc.tc.columbia.edu/media/k2/attachments/role-ambiguity-in-online-courses.pdf

Breevart, K., Bakker, A. B., Demerouti, E., & van den Heuvel, M. (2015). Leader-member

exchange, work engagement, and job performance. Journal of Managerial

Psychology, 30(7), 754–770.

Briggs, S. (2005). Changing roles and competencies of academics. Active Learning in Higher

Education, 6(3), 256–268.

Brown, A. H., & Green, T. D. (2016). The essentials of instructional design: Connecting

fundamental principles with process and practice (3rd ed.). New York, NY: Routledge.

Bryman, A. (2015). Social research methods (5th ed.). Oxford, UK: Oxford University Press.

Buch, K., & Aldridge, J. (1990). Downsizing challenges and OD interventions: A matching

strategy. Journal of Managerial Psychology, 5(4), 32–37.

Buch, K., & Aldridge, J. (1991). OD under conditions of organization decline. Organization

Development Journal, 9(1), 1–5.

132
Campbell, T. (2008). The capacity of instructional technologists to provide systemic support for

science education reform. Teacher Development, 12(1), 67–83.

Campbell, K., Schwier, R., & Kenny, R. (2009). The critical, relational practice of instructional

design in higher education: An emerging model of change agency. Educational

Technology Research and Development, 57(5), 645–664.

Carbonell, P., & Rodriguez-Escudero, A. I. (2013). Management control, role expectations and

job satisfaction of new product development teams: The moderating effect of

participative decision-making. Industrial Marketing Management, 42(2), 248–259.

Carr, S. (2000). As distance education comes of age, the challenge is keeping the students. The

Chronicle of Higher Education, 46(23), 39–41.

Carr-Chellman, A., & Duchastel, P. (2000). The ideal online course. British Journal of

Educational Technology, 31(3), 229–241.

Carter, S. (2013). Intervening in informal learning: Activity theory as teaching tool. McGill

Journal of Education 48, 491–510. doi:10.7202/1021916ar

Carter, M. M., Bumby, K., Gavin, F., Stroker, R., & Woodward, W. (2005). Collaboration: A

training curriculum to enhance the effectiveness of criminal justice teams. State Justice

Institute. Retrieved from

http://www.collaborativejustice.org/docs/2005%20Collaboration%20Curriculum.pdf

Chao, T., Saj, T., & Tessier, F. (2006). Establishing a quality review for online

courses. EDUCAUSE Quarterly, 29(3). Retrieved from

https://net.educause.edu/ir/library/pdf/EQM0635.pdf

133
Childs, J. H., & Stoeber, J. (2012). Do you want me to be perfect? Two longitudinal studies on

socially prescribed perfectionism, stress and burnout in the workplace. Work &

Stress, 26(4), 347–364.

Chou, C., & Tsai, C. C. (2002). Developing web-based curricula: issues and challenges. Journal

of Curriculum Studies, 34(6), 623–636.

Chuang, H. H. (2010). Weblog-based electronic portfolios for student teachers in Taiwan.

Educational Technology Research and Development, 58(2), 211–227.

Clark, R. C., & Mayer, R. E. (2011). E-learning and the science of instruction: Proven guidelines

for consumers and designers of multimedia learning. San Francisco, CA: John Wiley &

Sons.

Coetzer, G. H., & Richmond, L. (2009). An empirical examination of the relationships between

adult attention deficit, personal task management systems and role stress. Journal of

Behavioral and Applied Management, 10(2), 206.

Collins, J. L. (1982). Self-efficacy and ability in achievement behavior. Paper presented at the

meeting of the American Educational Research Association, New York.

Conceicao, S. C. O. (2006). Faculty lived experiences in the online environment. Adult

Education Quarterly, 57(1), 26–45.

Corbin, J., & Strauss, A. (2008). Basics of qualitative research: Techniques and procedures for

developing grounded theory (3rd ed.). Thousand Oaks, CA: Sage Publications.

Creswell, J. W. (2003). Research design: Qualitative, quantitative, and mixed methods

approaches (2nd ed.). Thousand Oaks, CA: Sage.

Creswell, J. W., & Poth, C. N. (2017). Qualitative inquiry and research design: Choosing among

five approaches (4th ed.). Thousand Oaks, CA: Sage publications.

134
Creswell, J. W., & Plano Clark, V. L. (2011). Designing and conducting mixed methods research

(2nd ed.). Thousand Oaks, CA: Sage.

Crumpacker, N. (2001). Faculty pedagogical approach, skill, and motivation in today’s distance

education milieu. The Online Journal of Distance Learning Administration, 4(4).

Cukusic, M., Alfirevic, N., Granic, A., & Garaca, Ž. (2010). e-Learning process management and

the e-learning performance: Results of a European empirical study. Computers &

Education, 55(2), 554–565.

Dasgupta, H., & Kumar, S. (2009). Role stress among doctors working in a government hospital

in Shimla. European Journal of Social Sciences, 9(3), 356–370.

Dayal, I., & Thomas, J. (1968). Operation KPE: Developing a new organization. Journal of

Applied Behavioral Science, 4(4), 473–506.

Deming, W. E. (1986). Out of crisis. Cambridge, MA: MIT Center for Advanced Engineering.

Dicks, D., & Ives, C. (2008). Instructional designers at work: A study of how designers

design. Canadian Journal of Learning and Technology, 34(2).

Dick, W., Carey, L., & Carey, J. O. (2009). The systematic design of instruction (7th ed.). Upper

Saddle River, NJ: Pearson.

Dietz-Uhler, B., Fisher, A., & Han, A. (2007). Designing online courses to promote student

retention. Journal of Educational Technology Systems, 36(1), 105–112.

doi:10.2190/ET.36.1.g.

Dijkstra, S., Schott, F., Seel, M., & Tennyson, R. D. (1997). Instructional design: International

Perspectives, Vol. I: Theory and research. Mahwah, NJ: Erlbaum.

Discenza, R., Howard, C., & Schenk, K. (2002). The design and management of effective

distance learning programs. Hershey, PA: IGI Global.

135
Dooley, K. E., Lindner, J. R., Moore, L., Telg, R. W., Tracy, I., & Lundy, L. (2007). Roadmap to

measuring distance education instructional design competencies. Quarterly Review of

Distance Education, 8, 151–159.

Doraiswamy, I. R., & Deshmukh, M. (2015). Meaningful work and role stress. International

Journal of Management & Organizational Studies, 4(4), 174–177.

Duke, D. L. (2014). A bold approach to developing leaders for low-performing

schools. Management in Education, 28(3), 80–85. doi:10.1177/0892020614537665

Dunegan, K. J., Uhl-Bien, M., & Duchon, D. (2002). LMX and subordinate performance: The

moderating effects of task characteristics. Journal of Business and Psychology, 17(2),

275–285.

Dusenbury, L., Brannigan, R., Falco, M., & Hansen, W. B. (2003). A review of research on

fidelity of implementation: Implications for drug abuse prevention in school

settings. Health Education Research, 18, 237–256. doi:10.1093/her/18.2.237

Dyer, W. G., & Dyer, J. H. (2013). Team building: Proven strategies for improving team

performance. Somerset, NJ: John Wiley & Sons.

Ertmer, P. A., & Newby, T. J. (2013). Behaviorism, cognitivism, constructivism: Comparing

critical features from an instructional design perspective. Performance Improvement

Quarterly, 26(2), 43–71.

Field, A. (2009). Discovering statistics using SPSS (3rd ed.). Thousand Oaks, CA: Sage.

Filipczak, B. (1995). Putting the learning into distance learning. Training, 32(10), 111-112.

Fish, W. W., & Wickersham, L. E. (2009). Best practices for online instructors. The Quarterly

Review of Distance Education, 10(3), 279–284.

Frankola, K. (2001). Why online learners dropout. Workforce, 10, 53–63.

136
Gagné, R. M., Briggs, L. J., & Wager, W. W. (1992). Principles of instructional design (4th ed.).

Fort Worth, TX: Harcourt Brace Jovanovich College Publishers.

Ganster, D. E., Fusilier, M. R, & Mayes, B. T. (1986). Role of social support in the experience of

stress at work. Journal of Applied Psychology, 71(1), 102–110.

Garrison, D. R. (2000). Theoretical challenges for distance education in the 21st century: A shift

from structural to transactional issues. International Review of Research in Open and

Distance Learning, 1(1), 1–17.

Garrison, D. R. (2017). E-learning in the 21st century: A community of inquiry framework for

research and practice. New York, NY: Routledge.

Garrison, D. R. (2016). Thinking collaboratively: Learning in a community of inquiry. London:

Routledge.

Garrison, D. R., Anderson, T., & Archer, W. (2000). Critical inquiry in a text-based environment:

Computer conferencing in higher education. The Internet and Higher Education, 2(2–3),

87–105.

Garrison, D. R., Cleveland-Innes, M., & Fung, T. (2004). Student role adjustment in online

communities of inquiry: Model and instrument validation. Journal of Asynchronous

Learning Networks, 8(2), 61–74.

Gerlich, R. N. (2005). Faculty perception of distance learning. Distance Education Report, 9(17),

8.

Gillespie, N. A., Walsh, M., Winefield, A. H., Dua, J., & Stough, C. (2001). Occupational stress

in universities: Staff perceptions of the causes, consequences and moderators of

stress. Work & Stress, 15, 53–72.

137
Greenberg, G. (2010). Conceptions of quality in course design for web-supported education.

Proceedings of the 26th Annual Conference on Distance Teaching & Learning. Madison,

WI. Retrieved from

http://www.uwex.edu/disted/conference/Resource_library/proceedings/28667_10.pdf

Goldie, J. G. S. (2016). Connectivism: A knowledge learning theory for the digital age? Medical

teacher, 1–6.

Goldsmith, M. (2010). What got you here won’t get you there: How successful people become

even more successful. New York, NY: Hyperion.

Goodyear, P. (2005). Educational design and networked learning: Patterns, pattern languages and

design practice. Australasian Journal of Educational Technology, 21(1), 82–101.

Goodyear, P., Salmon, G., Spector, J. M., Steeples, C., & Tickner, S. (2001). Competences for

online teaching: A special report. Educational Technology Research and Development,

49(1), 65–72.

Gordon, J. (2004). Pfeiffer classic inventories, questionnaires, and surveys for training and

development. San Francisco, CA: John Wiley & Sons.

Gore, C., Bond, C., & Steven, V. (2000). Organisational self-assessment: measuring educational

quality in two paradigms. Quality Assurance in Education, 8(2), 76–84.

Greenberg, G. (2010). Conceptions of quality in course design for web-supported education.

Proceedings of the 26th Annual Conference on Distance Teaching & Learning. Madison,

WI. Retrieved from

http://www.uwex.edu/disted/conference/Resource_library/proceedings/28667_10.pdf

138
Grissom, J. A. (2012). Revisiting the impact of participative decision making on public employee

retention the moderating influence of effective managers. The 133 American Review of

Public Administration, 42, 400–418. doi:10.1177/0275074011404209

Hang-Yue, N., Foley, S., & Loi, R. (2005). Work role stressors and turnover intentions: a study of

professional clergy in Hong Kong. The International Journal of Human Resource

Management, 16(11), 2133–2146.

Harasim, L. (2012). Learning theory and online technology: How new technologies are

transforming learning opportunities. New York, NY: Routledge Press.

Harber, J., & Mills, M. (2008). Perceptions of barriers concerning effective online teaching and

policies: Florida Community College Faculty. Community College Journal of Research

and Practice, 3, 266–283.

Harrison, R. (1972). Role negotiation: A tough minded approach to team development. In W. W.

Burke & H. A. Hornstein (Eds.). The social technology of organizational development

(pp. 84–96). La Jolla, CA; University Associates.

Harvey, T. R., & Drolet, B. (2004). Building teams, building people: Expanding the fifth

resource. Lanham, MD: Scarecrow Education.

Hassan, S. (2013). The importance of role clarification in workgroups: Effects on perceived role

clarity, work satisfaction, and turnover rates. Public Administration Review, 73(5), 716–

725.

Hawkes, M., & Coldeway, D. O. (2002). An analysis of team vs. faculty-based online course

development: Implications for instructional design. The Quarterly Review of Distance

Education, 3(4), 431–441.

139
Herbert, M. (2006). Staying the course: A study in online student satisfaction and

retention. Online Journal of Distance Learning Administration, 9(4), 300–317.

Herman, J. H. (2013). Faculty incentives for online course design, delivery, and professional

development. Innovative Higher Education, 38(5), 397–410.

Heuer, B. P., & King, K. P. (2004). Leading the band: The role of the instructor in online learning

for educators. The Journal of Interactive Online Learning, 3(1), 1–11.

Hixon, E. (2008). Team-based online course development: A case study of collaboration

models. Online Journal of Distance Learning Administration, 11(4), 1–8.

Hixon, E., Buckenmeyer, J. A., & Barczyk, C. (2015). Closing the feedback loop: Hearing the

student voice in course quality. Quality Approaches in Higher Education, 6(1), 26–36.

Hoffman, G. L. (2012). Using the Quality Matters rubric to improve online cataloging courses.

Cataloging and Classification Quarterly, 50(2–3), 158–171.

Hokanson, B., & Miller, C. (2009). Role-based design: A contemporary framework for

innovation and creativity in instructional design. Educational Technology, 49(2), 21–28.

House, R. J., & Aditya, R. N. (1997). The social scientific study of leadership: Quo vadis?

Journal of Management, 23(3), 409–473.

House, R. J., Schuler, R. S., & Levanoni, E. (1983). Role conflict and ambiguity scales: Reality

or artifacts? Journal of Applied Psychology, 68, 334–337.

Hulsmann, T. (2000). The costs of open learning: A handbook. Oldenburg: BIS, University of

Oldenbur.

Huse, E. F (1980). Organization development and change (2nd ed.). St. Paul, MN: West

Publishing.

140
Idris, M. (2011). Over time effects of role stress on psychological strain among Malaysian public

university academics. International Journal of Business and Social Science, 2(9), 154–

161.

İzmirli, Ö. Ş., & Kurt, A. A. (2009). Basic competencies of instructional technologists. Procedia-

Social and Behavioral Sciences, 1(1), 998–1002.

Jackson, S., & Schuler, R. (1985). A meta-analysis and conceptual critique of research on role

ambiguity and role conflict in work settings. Organizational Behavior and Human

Decision Processes, 36, 16–78.

Jackson, S. E., Schwab, R. L., & Schuler, R. S. (1986). Toward an understanding of the burnout

phenomenon. Journal of Applied Psychology, 71(4), 630–640.

Jaggars, S. S., & Xu, D. (2016). How do online course design features influence student

performance? Computers & Education, 95, 270–284.

Jonassen, D., Tessmer, M., & Hannum, W. (1999). Task analysis methods for instructional

design. Mahwah, New Jersey/London: Lawrence Erlbaum Associates, Publishers.

Jonassen, D. H., & Rohrer-Murphy, L. (1999). Activity theory as a framework for designing

constructivist learning environments. Educational Technology Research and

Development, 47(1), 61–79.

Kahn, R., Wolfe, D., Quinn, R., Snoek, J., & Rosenthal, R. (1964). Organizational stress: Studies

in role conflict and ambiguity. New York: Wiley.

Karasek, R. A. (1979). Job demands, job decision latitude, and mental strain: implications for job

redesign. Administrative Science Quarterly, 24, 285–308.

141
Karimi, R., Omar, Z. B., Alipour, F., & Karimi, Z. (2014). The influence of role overload, role

conflict, and role ambiguity on occupational stress among nurses in selected Iranian

hospitals. International Journal of Asian Social Science, 4(1), 34–40.

Katz, D., & Kahn, R. L. (1966). The social psychology of organizations. New York: Wiley.

Katz, D., & Kahn, R. L. (1978). The social psychology of organizations (2nd ed.). New York:

Wiley.

Kahn, R. L., Wolfe, D. M., Quinn, R. P., & Snoek, J. D. (1964). Organizational stress: Studies in

role conflict and ambiguity. New York, NY: Wiley.

Kearsley, G. (2000). Online education: Learning and teaching in cyberspace. Belmont, CA:

Wadsworth Thomson Learning.

Keegan, D. (1996). Foundations of distance education (3rd ed.). New York,

NY: Routledge Press.

Keim, A., Landis, R., Pierce, C., & Earnest, D. (2014). Why do employees worry about their

jobs? A meta-analytic review of predictors of job insecurity. Journal of Occupational

Health Psychology, 19(3), 269–290.

Kemery, E. R. (2006). Clergy role stress and satisfaction: Role ambiguity isn’t always bad.

Pastoral Psychology, 54(6), 561–570.

Keppell, M. (2001). Optimizing instructional designer—subject matter expert communication in

the design and development of multimedia projects. Journal of Interactive Learning

Research, 12(2/3), 205–223.

Kidney, G., Cummings, L., & Boehm, A. (2007). Toward a quality assurance approach to e-

learning courses. International Journal of E-Learning, 6(1), 17–30.

142
Kim, A., & Barak, M. E. M. (2015). The mediating roles of leader-member exchange and

perceived organizational support in the role stress-turnover intention relationship among

child welfare workers: A longitudinal analysis. Children and Youth Services Review, 52,

135–143.

Kirschner, P., Carr, C., van Merriënboer, J., & Sloep, P. (2002). How expert designers design?

Performance Improvement Quarterly, 15(4), 86–104.

Klink, J. J., Blonk, R. W., Schene, A. H., & Van Dijk, F. J. (2001). The benefits of interventions

for work-related stress. American Journal of Public Health, 91(2), 270.

Koehler, M. J., & Mishra, P. (2005). What happens when teachers design educational

technology? The development of Technological Pedagogical Content Knowledge.

Journal of Educational Computing Research, 32(2), 131–152.

Koehler, M. J., Mishra, P., Hershey, K., & Peruski, L. (2004). With a little help from your

students: A new model for faculty development and online course design. Journal of

Technology and Teacher Education, 12(1), 25–55.

Korey, G. (1988). Linear responsibility charting: A technique for strategic management.

Management Decision, 26, 11–21.

Larson, M. B. (2005). Instructional design career environments: Survey of the alignment of

preparation and practice. TechTrends, 49(6), 22–32.

Lawrence, E. R., & Kacmar, K. M. (2012). Leader-member exchange and stress: The mediating

role of job involvement and role conflict. Journal of Behavioral and Applied

Management, 14(1), 39.

Lee, J. W. (2010). Online support service quality, online learning acceptance, and student

satisfaction. The Internet and Higher Education, 13(4), 277–283.

143
Lee, J., & Jang, S. (2014). A methodological framework for instructional design model

development: Critical dimensions and synthesized procedures. Educational Technology

Research and Development, 62(6), 743–765.

Legon, R. (2006). Comparison of the Quality Matters rubric to accreditation standards for

distance learning. Quality Matters. Retrieved from

https://confluence.delhi.edu/download/attachments/74055682/Comparison+of+the+Quali

ty+Matters+Rubric+-+Summary.pdf

Legon, R., & Runyon, J. (2007). Research on the impact of the quality matters course review

process. In 23rd Annual Conference on Distance Teaching & Learning (pp. 8–10).

Madison, WI: University of Wisconsin.

Lehman, W. E., Greener, J. M., & Simpson, D. D. (2002). Assessing organizational readiness for

change. Journal of Substance Abuse Treatment, 22(4), 197–209.

LePine, J. A., Podsakoff, N. P., & LePine, M. A. (2005). A meta-analytic test of the challenge

stressor—hindrance stressor framework: An explanation for inconsistent relationships

among stressors and performance. Academy of Management Journal, 48, 764–775.

Levy, S. (2003). Six factors to consider when planning online distance learning programs in

higher education. Online Journal of Distance Learning Administration, 6(1).

Liu, M., Gibby, S., Quiros, O., & Demps, E. (2002). Challenges of being an instructional

designer for new media development: A view from the practitioners. Journal of

Educational Multimedia and Hypermedia, 11(3), 195–219.

Lundberg, C. (1980). On organizational development interventions: A general systems-

cybernetic perspective. In T. G. Cummings (Ed.). Systems theory for organizational

development. New York, NY: Wiley.

144
Lowenthal, P., Wilson, B. G., & Dunlap, J. C. (2010). An analysis of what instructional designers

need to know and be able to do to get a job. Presented at the annual meeting of the

Association for Educational Communications and Technology, Anaheim, CA.

Lyons, P. (1999). Assessment techniques to enhance organizational learning. (ED 426 678)

Retrieved from https://eric.ed.gov/?id=ED426678

Lyons, P. (1993). Developing expectations with the role analysis technique (RAT). Journal of

Management Education, 17(3), 386–389.

Macpherson, C., & Smith, A. (1998). Academic authors’ perceptions of the instructional design

and development process for distance education: A case study. Distance Education, 19(1),

124–141.

Mager, R. (1962). Preparing instructional objectives. Palo Alto, CA: Fearon.

Major, D. L., & Major, H. (2011). A systems approach to improving community college

courses. The Community College Enterprise, 17(1), 51.

McQuiggan, C. A. (2007). The role of faculty development in online teaching’s potential to

question teaching beliefs and assumptions. Online Journal of Distance Learning

Administration, 10(3). Retrieved from

http://www.westga.edu/~distance/ojdla/fall103/mcquiggan103.htm

Meier, S. E. (2001). Northern California school superintendents’ perceptions regarding conflicts

with board members in the area of human resource administration (Doctoral dissertation,

University of La Verne). Retrieved from https://elibrary.ru/item.asp?id=5345019

Merrill, M. D. (2002). First principles of instruction. Educational Technology Research and

Development, 50(3), 43–59.

145
Merrill, M. D. (2013). First principles of instruction: Identifying and designing effective,

efficient, and engaging instruction. San Francisco, CA: Pfeiffer.

Meyer, K. A. (2002). Quality in distance education: Focus on on-line learning. San Francisco,

CA: Jossey-Bass.

Monahan S. C. (1999). Role ambiguity among Protestant clergy: Consequences of the activated

laity. Review of Religious Research, 41, 79–94.

Moore, J. C., & Shelton, K. (2013). Social and student engagement and support: The Sloan-C

quality scorecard for the administration of online programs. Journal of Asynchronous

Learning Networks, 17(1), 53–72.

Morrison, G. L., Ross, S. M., & Kemp, J. E. (2007). Designing effective instruction (5th

ed.). Hoboken, NJ: John Wiley & Sons.

Morton-Cooper, A., & Palmer, A. (2000). Mentoring, preceptorship and clinical

supervision. Nurse Education Today, 20, 418–421.

Moskal, T. M. (2012). Instructional designers in higher education (Doctoral dissertation,

University of Nebraska-Lincoln). Retrieved from

http://digitalcommons.unl.edu/cgi/viewcontent.cgi?article=1125&context=cehsedaddiss

Moss, C. (2015). Role conflict and role ambiguity as predictors of burnout in special and general

education co-teachers (Order No. 3646995). Available from ProQuest Dissertations &

Theses Global. (1637727036). Retrieved from

http://search.proquest.com/docview/1637727036?accountid=11752

Murray, K. E., Davidson, G. R., & Schweitzer, R. D. (2010). Review of refugee mental health

interventions following resettlement: best practices and recommendations. American

Journal of Orthopsychiatry, 80(4), 576–585.

146
Nazneen, A., Bhalla, P., & Zafar, S. (2014). A comparative study of organizational role stress

(ORS), stress tolerance level and its management among the top executives of Indian

public and private enterprises. International Journal of Business Management &

Research, 4(3), 85–94.

Nelson, M. C., Cordray, D. S., Hulleman, C. S., Darrow, C. L., & Sommer, E. C. (2012). A

procedure for assessing intervention fidelity in experiments testing educational and

behavioral interventions. The Journal of Behavioral Health Services & Research, 39,

374–396. doi:10.1007/s11414-012-9295-x

Newton, C. J., & Jimmieson, N. L. (2008). Role stressors, participative control, and subjective fit

with organizational values: Main and moderating effects on employee outcomes. Journal

of Management & Organization, 14(1), 20–39.

Newman, F., Couturier, L., & Scurry, J. (2010). The future of higher education: Rhetoric, reality,

and the risks of the market. San Francisco, CA: Jossey-Bass.

Newton, R. (2003). Staff attitudes to the development and delivery of e-learning. New Library

World, 104(10), 412–425.

Oblinger, D. G., & Hawkins, B. L. (2006). The myth about online course development.

EDUCAUSE Review, 41(1), 14–15.

O’Brien, B. S. (2002). Online student retention: Can it be done? In P. Barker & S. Rebelsky

(Eds.), Proceedings of world conference on educational multimedia, hypermedia and

telecommunications 2002 (pp. 1479–1483). Chesapeake, VA: AACE.

O’Connell, D., Hickerson, K., & Pillutla, A. (2011). Organizational visioning: An integrative

review. Group & Organization Management, 36(1), 103–125.

147
Oliver, M. (2002). What do learning technologists do? Innovations in Education and Teaching

International, 39(4), 245–252.

Onyemah, V. (2008). Role ambiguity, role conflict, and performance: Empirical evidence of an

inverted-U relationship. Journal of Personal Selling & Sales Management, 28(3), 299–

313.

O’Rourke, M., Hammond, S., O’Flynn, S., & Boylan, G. (2010). The medical student stress

profile: A tool for stress audit in medical training. Med Teach, 44, 1027–1037.

Ormond, W. E., Keown‐Gerrard, J. L., & Kline, T. (2003). Stress audits as a precursor to stress

management workshops: An evaluation of the process. Human Resource Development

Quarterly, 14(1), 111–116.

Pacheco, G., & Webber, D. (2016). Job satisfaction: how crucial is participative decision

making? Personnel Review, 45(1), 183–200.

Palmieri, L., Semich, G., & Graham, J. (2010). The role of technology leaders in schools.

In EdMedia: World Conference on Educational Media and Technology (pp. 2240–2247).

Association for the Advancement of Computing in Education (AACE).

Pan, C., Deets, J., Phillips, W., & Cornell, R. (2003). Pulling tigers’ teeth without getting bitten:

Instructional designers and faculty. The Quarterly Review of Distance Education, 4(3),

289–302.

Pan, C., & Thompson, K. (2009). Exploring dynamics between instructional designers and

higher education faculty: An ethnographic case study. Journal of Educational Technology

Development & Exchange, 2(1). doi:10.1080/07377363.2013.758554

Pareek, U. (1993). Making organizational roles effective. New Delhi: Tata McGraw-Hill.

148
Pareek, U. (2004). Organizational role stress. In J. Gordon (Ed.), Pfeiffer’s Classic Inventories,

Questionnaires, and Surveys for Training and Development (pp. 319–330). London, UK:

John Wiley.

Pareek, U. (2005). The role in training instruments in human resource development and

organisational development. In U. Pareek (Ed.). Training Instruments in Human Resource

Development and Organisational Development (pp. 475–567). New Delhi: Tata

McGraw-Hill.

Pareek, U. N. (1983). Organizational roles stress. In L. D. Goodstein & J. W. Pfeiffer (Eds.). The

1983 annual (pp. 115–123). San Diego, CA: University Associates.

Pareek, U. N. (1997). Training instruments for human resource development. New Delhi: Tata

McGraw-Hill.

Parrot, S. (1995). Future learning: Distance education in community colleges. ERIC Digest.

Retrieved from http://www.ericdigests.org/1996-1/future.htm

Pestonjee, D. M. (1987). Executive stress: Should it always be avoided? Vikalpa, 12, 23–30.

Pestonjee, D. M. (1999). Stress and coping: The Indian experience (2nd ed.). New Delhi: Sage

Publications.

Piskurich, G. M. (2000). Rapid instructional design: Learning ID fast and right. San Francisco,

CA: Jossey-Bass.

Quality Matters. (2014). Higher education rubric (5th ed.). MarylandOnline. Retrieved from

https://www.qmprogram.org/myqm/

Quality Matters. (2016). The Quality Matters (QM) program. MarylandOnline. Retrieved from

https://www.qualitymatters.org/about

Rahim, M.A. (2010) Managing conflict in organizations. New Brunswick, NJ: Transaction.

149
Rajarajeswari, S. (2010). Role stress among the aided and self-financing college teachers: A

discriminant analysis. Global Management Review, 4(4), 73–86.

Ralston-Berg, P. (2014). Surveying student perspectives of quality: Value of QM rubric items.

Internet Learning, 3(1), 117–126.

Ralston-Berg, P. & Nath, L. (2008). What makes a quality online course? The student

perspective. 24th Annual Conference on Distance Teaching & Learning. Retrieved from

http://www.uwex.edu/disted/conference/Resource_library/proceedings/08_12876.pdf

Rao, T. V., & Vijayalakshmi, M. (2000). Organization development in India. Organization

Development Journal, 18, 51–64.

Rapanta, C., Maina, M., Lotz, N., & Bacchelli, A. (2013). Team design communication patterns

in e-learning design and development. Educational Technology Research and

Development, 61(4), 581–605.

Redmond, P. (2011). From face-to-face teaching to online teaching: Pedagogical transitions. In

G. Williams, P. Statham, N. Brown & B. Cleland (Eds.), Changing Demands, Changing

Directions: Proceedings ascilite Hobart 2011 (pp. 1050–1060). Hobart: ASCILITE.

Retrieved from

http://www.ascilite.org/conferences/hobart11/downloads/ProceedingsV3.pdf

Reid, H. M. (2014). Introduction to statistics: Fundamental concepts and procedures for data

analysis. Los Angeles, CA: Sage.

Reigeluth, C. M. (1983). Instructional design: What is it and why is it. In C. M. Reigeluth (Ed.),

Instructional-design theories and models: An overview of their current status (pp. 4–36).

Hillsdale, NJ: Lawrence Erlbaum Associations.

150
Reigeluth, C. M. (2013). Instructional design theories and models: An overview of their current

status. Hoboken, NJ: Taylor and Francis.

Reilly, J., Vandenhouten, C., Gallagher-Lepak, S., & Ralston-Berg, P. (2012). Faculty

development for e-learning: A multi-campus community of practice (COP) approach.

Journal of Asynchronous Learning Networks, 16(2), 99–110.

Richardson, K. M., & Rothstein, H. R. (2008). Effects of occupational stress management

intervention programs: A meta-analysis. Journal of Occupational Health Psychology, 13,

69–93.

Rickles, N. M., Wertheimer, A. I., & Smith, M. C. (2010). Social and behavioral aspects of

pharmaceutical care. Sudbury, MA: Jones & Bartlett Learning.

Ritzhaupt, A. D., & Kumar, S. (2015). Knowledge and skills needed by instructional designers in

higher education. Performance Improvement Quarterly, 28(3), 51–69.

doi:10.1002/piq.21196

Rizzo, J. R., House, R. J., & Lirtzman, S. I. (1970). Role conflict and ambiguity in complex

organizations. Administrative Science Quarterly, 15, 150–163.

Rossett, A. (2002). Waking in the night and thinking about e-learning. In A. Rossett

(Ed.), The ASTD E-Learning Handbook (pp. 3–18). New York, NY: McGraw-Hill.

Rourke, L., & Kanuka, H. (2009). Learning in communities of inquiry: A review of the

literature. Journal of Distance Education, 23(1), 19–48.

Rumble, G. (1997). The costs and economics of open and distance learning. London, UK: Kogan

Page.

Rowland, G. (1992). What do instructional designers actually do? An initial investigation of

expert practice. Performance Improvement Quarterly, 5(2), 65–86.

151
Samie, F., Riahi, L., & Tabibi, S. J. (2015). The relationship between role clarity and efficiency

of employees in management & resource development department of ministry of health

and medical education of IR Iran, 2014. Biosciences Biotechnology Research Asia, 12(3),

2803–2812. doi:http://dx.doi.org/10.13005/bbra/1964

Savin-Baden, M., & Major, C. (2010). An introduction to qualitative research synthesis:

Managing the information explosion in social science research. London, UK: Routledge.

Schaubroeck, J., Ganster, D. C., Sime, W., & Ditman, D. (1993). A field experiment testing

supervisory role clarification. Personnel Psychology, 46, 1–25.

Schermerhorn, J. R., Osborn, R. N., Uhl-Bien, M., & Hunt, J. G. (2012). Organizational

behavior (12th ed.). Danvers, MA: John Wiley & Sons.

Schmidt, H. G., Loyens, S. M. M., van Gog, T., & Paas, F. (2007). Problem-based learning is

compatible with human cognitive architecture: Commentary on Kirschner, Sweller, and

Clark (2006). Educational Psychologist, 42, 91–97. doi:10.1080/00461520701263350

Schmidt, S., Roesler, U., Kusserow, T., & Rau, R. (2014). Uncertainty in the workplace:

Examining role ambiguity and role conflict, and their link to depression—A

metaanalysis. European Journal of Work and Organizational Psychology, 23, 91–106.

Schwartzman, R. (2006). Virtual group problem solving in the basic communication course:

Lessons for online learning. Journal of Instructional Psychology, 33, 3–14.

Seaman, J. (2009). Online learning as a strategic asset. Volume II: The paradox of faculty

voices: Views and experiences with online learning. Washington, DC: Association of

Public and Land-grant Universities and Babson Survey Research Group. Retrieved from

http://www.aplu.org/document.doc?id=1879

152
Seethamraju, R. (2010). Business process management: A missing link in business education.

Proceedings of the sixteenth Americas conference on information systems. Lima, Peru.

Retrieved from http://aisel.aisnet.org/amcis2010/243

Seethamraju, R., & Marjanovic, O. (2009). Role of process knowledge in business process

improvement methodology: A case study. Business Process Management Journal, 15(6),

920–936.

Sen, A., & Srivastava, M. (2012). Regression analysis: Theory, methods, and applications. New

York, NY: Springer.

Shadish, W., Cook, T., & Campbell, D. (2002). Experimental and quasi-experimental designs for

generalized causal inference. Boston, MA: Houghton Mifflin.

Shell, L., Crawford, S., & Harris, P. (2013). Aided and embedded: The team approach to

instructional design. Journal of Library & Information Services in Distance

Learning, 7(1–2), 143–155.

Siemens, G. (2005). Connectivism: A learning theory for the digital age. International Journal of

Instructional Technology and Distance Education, 2(1), 3–9. Retrieved

from http://www.itdl.org

Siemens, G. (2014). Connectivism: A learning theory for the digital age. Retrieved from

http://er.dut.ac.za/handle/123456789/69

Simonson, M., Smaldino, S., Albright, M., & Zvacek, S. (2012). Teaching and learning at a

distance (5th ed.). Boston, MA: Pearson.

Sims, D. E., Klein, C., & Salas, E. (2006). Team-building. In W. Karwowsky (Ed.). International

encyclopedia of ergonomics and human factors (2nd ed.). Boca Raton, FL: CRC Press.

153
Singh, R. (1997). Developing personal effectiveness: Some guidelines for managers. Indian

Journal of Industrial Relations, 33(1), 101–112.

Singh J., & Rhoads G. K. (1991). Boundary role ambiguity in marketing-oriented positions: A

multidimensional, multifaceted operationalization. Journal of Marketing Research,

28, 328–338.

Smaldino, S. E., Lowther, D. L., & Russell, J. D. (2012). Instructional technology and media for

learning (10th ed.). Boston, MA: Pearson.

Smith, P. L., & Ragan, T. J. (1993). Instructional design. New York, NY: Macmillan Publishing

Company.

Smith, P. L., & Ragan, T. J. (2005). Instructional design (3rd ed.). Hoboken, NJ: John Wiley &

Sons.

Snelbecker, G.E. (1989). Contrasting and complementary approaches to instructional design. In

C. M. Reigeluth (Ed.), Instructional theories in action (pp. 321–337). Hillsdale, NJ:

Lawrence Erlbaum Associates.

Spector, P. E., & Jex, S. M. (1998). Development of four self-report measures of job stressors

and strain: Interpersonal conflict at work scale, organizational constraints scale,

quantitative workload inventory, and physical symptoms inventory. Journal of

Occupational Health Psychology, 3, 356–367.

Srivastav, A. K. (2011). Harnessing the potential of organization development. In E. Biech

(Ed.). The 2011 Pfeiffer annual (pp. 241–260). San Francisco, CA: Pfeiffer.

Srivastav, A. K. (2006). Organizational climate as a dependent variable: Relationship with role

stress, coping strategy and personal variables. Journal of Management Research, 6(3),

125.

154
Srivastav, A. K. (2012). Process based role analysis and design. In E. Biech (Ed.). The 2012

Pfeiffer annual: Consulting (pp. 233–250). San Francisco, CA: John Wiley & Sons, Inc.

Srivastav, A. K. (2010). Process based role analysis and design: Application in information

technology industry. DIAS Technology Review, 7(1).

Srivastav, A. K. (2007). Stress in organizational roles: Individual and organizational

implications. Icfaian Journal of Management Research, 6(12), 64–74.

Srivastav, A. K., & Pareek, U. (2008). Measurement of stress in organizational roles:

Revalidating the framework. In E. Biech (Ed.). The 2008 Pfeiffer annual: Training (pp.

187–208). San Francisco, CA: Pfeiffer.

Stacey, E. (Ed.). (2009). Effective blended learning practices: Evidence-based perspectives in

ICT-facilitated education. London, UK: IGI Global.

Stevens, K. B. (2013). Contributing factors to a successful online course development

process. The Journal of Continuing Higher Education, 61(1), 2–11.

Stewart, B., & Waight, C. (2008). E-learning teams and their adult learning efforts in corporate

settings: A cross analysis of four case studies. International Journal of E-Learning, 7(2),

293–309.

Strawser, M. G., Buckner, M. M., & Kaufmann, R. (2015). Design and delivery: Embracing

instructor responsibility in the online communication course. Florida Journal of

Communication, 43(2), 119–125.

Sugar, W. (2005). Instructional technologist as a coach: Impact of a situated professional

development program on teachers’ technology use. Journal of Technology and Teacher

Education, 13(4), 547.

155
Sugar, W., Hoard, B., Brown, A., & Daniels, L. (2012). Identifying multimedia production

competencies and skills of instructional design and technology professionals: An analysis

of recent job postings. Journal of Educational Technology Systems, 40(3), 227–249.

Surry, D. W. (1996). Defining the role of the instructional technologist in higher education. In

Proceedings of the 1996 Mid-South Instructional Technology Conference, Murfreesboro,

TN: Middle Tennessee State University.

Swan, K., Day, S., Bogle, L., & Matthews, D. (2014). A collaborative, design-based approach to

improving an online program. The Internet and Higher Education, 21, 74–81.

Swan, K., Matthews, D., Bogle, L., Boles, E. & Day, S. (2011). Linking online course design and

implementation to learning outcomes: A design experiment. The Internet and Higher

Education, 15, 81–88.

Tang, Y., & Chang, C. (2010). Impact of role ambiguity and role conflict on employee creativity.

African Journal of Business Management, 4(6), 869–881.

Tavakol, M., & Dennick, R. (2011). Making sense of Cronbach’s alpha. International Journal of

Medical Education, 2, 53.

Thach, E. C., & Murphy, K. L. (1995). Competencies for distance education

professionals. Educational Technology Research and Development, 43(1), 57–79.

Retrieved from http://www.jstor.org/stable/30220112

Tracey, M. W., Hutchinson, A., & Grzebyk, T. Q. (2014). Instructional designers as reflective

practitioners: Developing professional identity through reflection. Educational

Technology Research and Development, 62(3), 315–334.

156
Trayambak, S., Kumar, P., & Jha, A. N. (2012). A conceptual study on role stressors, their impact

and strategies to manage role stressors. IOSR Journal of Business and Management, 4(1),

44–48.

Volmer, J., Niessen, C., Spurk, D., Linz, A., & Abele, A. E. (2011). Reciprocal relationships

between leader–member exchange (LMX) and job satisfaction: A cross‐lagged

analysis. Applied Psychology, 60(4), 522–545.

Vroom, V. H., & Jago, A. G. (1988). The new leadership: Managing participation in

organizations. Englewood Cliffs, NJ: Prentice Hall.

Wakefield, J., Warren, S., & Mills, L. (2012). Traits, skills, & competencies aligned with

workplace demands: What today’s instructional designers need to master. In P. Resta

(Ed.), Proceedings of Society for Information Technology & Teacher Education

International Conference 2012 (pp. 3126–3132).

Warner, R. M. (2013). Applied statistics from bivariate through multivariate techniques.

Thousand Oaks, CA: Sage.

Wegner, S. B., Holloway, K., & Crader, A. (1997). Utilizing a problem-based approach on the

World Wide Web. Southwest Missouri State University. Retrieved from

http://files.eric.ed.gov/fulltext/ED414262.pdf

Weinberg, A., Sutherland, V. J., & Cooper, C. (2010). Organizational stress management: A

strategic approach. New York, NY: Palgrave Macmillan.

Wiesenmayer, R., Kupczynski, L., & Ice, P. (2008). The role of technical support and

pedagogical guidance provided to faculty in online programs: Considerations for higher

education administrators. Online Journal of Distance Learning Administration, 11(4).

157
Wiggins, G., & McTighe, J. (2011). Understanding by design guide to advanced concepts in

creating and reviewing units. Alexandria, VA: ASCD.

Wyk, S. V. (2015). The validation of a workplace boredom scale within the South African

context. North-West University. Retrieved from

http://dspace.nwu.ac.za/bitstream/handle/10394/15729/Van_Wyk_SM_2015.pdf?sequenc

e=1

Xu, D., & Jaggars, S. S. (2013). The impact of online learning on students’ course outcomes:

Evidence from a large community and technical college system. Economics of Education

Review, 37, 46–57.

Yasmeen, H., & Supriya, M. V. (2008). Organizational role stress: Confirmatory factor analysis

approach. Asia Pacific Business Review, 4(2), 29–33.

Yeung, D. (2001). Toward an effective quality assurance model of web-based learning: The

perspectives of academic staff. Online Journal of Distance Learning Administration,

4(4). Retrieved from http://www.westga.edu/~distance/ojdla/winter44/yeung44.html

Yuryur, S., & Sarikaya, M. (2012). The effects of workload, role ambiguity, and social support on

burnout among social workers in Turkey. Administration in Social Work, 36, 457–478

doi:10.1080/03643107.2011.613365

Yusoff, N., & Salim, S. S. (2012). Expert skills in e-learning storyboards using a cognitive task

analysis technique. Computers & Education, 58, 652–665.

Zhang, R. P., Tsingan, L., & Zhang, L. P. (2013). Role stressors and job attitudes: A mediated

model of leader-member exchange. The Journal of social psychology, 153(5), 560–576.

Zheng, L., & Smaldino, S. (2003). Key instructional design elements for distance education. The

Quarterly Review of Distance Education, 4, 153–166.

158
Zheng, L., & Smaldino, S. E. (2006). Teacher’s perceptions of the application of design

elements. Quarterly Review of Distance Education, 7, 35–47.

Ziegenfuss, D. H., & Lawler, P. A. (2008). Collaborative course design: Changing the process,

acknowledging the context, and implications for academic development. International

Journal for Academic Development, 13(3), 151–160.

Zorlu, K. (2012). The perception of self-esteem and self-efficacy as transforming factors in the

sources of role stress and job satisfaction relationship of employees: A trial of a staged

model based on the artificial neural network method. African Journal of Business

Management, 6(8), 3014.

159
 Appendices

Appendix A

Needs Assessment Data Collection Instrument

Research Survey: Examining the Impact of Role Stressors on the Quality of the Online
Course Design Process

Demographic Questionnaire:
The following questions ask about your background and your demographic characteristics. The
purpose of this information is only to describe the group of all respondents for the purpose of
comparison with other research studies. Individual responses will not be disclosed or shared with
any person working in your institution. Your answers will be kept strictly confidential and will
not be used to identify you or any of your responses in this study. You have the right not to
answer any questions, should you feel uncomfortable.

1. Choose your role from the following (check all that apply):
o Faculty
o Instructional Designer
o Instructional Technologist
o Project Manager
o Executive Team Member (Director etc.)
o Other (please specify): ______________________________

2. At what type of institution do you work?

o Public, four-year institution
o Private, four-year institution
o Public, two-year college
o Private, two-year college
o For-profit institution

3. Please select your appropriate age range:

o 18-30
o 31-40
o 41-50
o 51 and above

4. What is your gender?

o Male
o Female

5. What is your ethnicity? (Choose all that apply)

o White (non-Hispanic)
o Hispanic/Latino
o African American

160
 o Asian
o Native American
o Native Alaskan or Pacific Islander
o Other (please specify): ______________________________

6. What is the highest level of education you have completed?

o High school/GED
o Some college
o 2-year college degree (Associate’s)
o 4-year college degree (Bachelor’s)
o Advanced degree (Master’s or other)
o Other (please specify): ______________________________

7. How many years of professional experience do you have in your field?

o 1-2 years of professional experience
o 2 to 4 years of professional experience
o 4 to 5 years of professional experience
o 5 to 6 years of professional experience
o 6 or more years of professional experience

Directions:
Please read the statements and questions carefully. Your options for answers will change
throughout the survey. Most questions have a number associated with the answer option you
agree with the most. Please select the number that corresponds with the option you agree with
the most. Please note that there is no right or wrong answer. All that is important is that you
indicate your personal feeling.
Disagree
Disagree
Strongly

Strongly
Neutral
To what extent do you agree with the following
Agree

Agree
statements with respect to your role in an instructional
design team?
1. I have to do things that should be done differently 1 2 3 4 5
under different conditions
2. I receive an assignment without the manpower to 1 2 3 4 5
complete
3. I don’t have a clear rule or policy that I need in 1 2 3 4 5
order to carry out an assignment
4. I work with two or more groups who operate quite 1 2 3 4 5
differently
5. I receive incompatible requests from two or more 1 2 3 4 5
people
6. I do things that are apt to be accepted by one person 1 2 3 4 5
and not by others
7. I receive an assignment without adequate resources 1 2 3 4 5
and materials to execute it
8. I work on unnecessary things 1 2 3 4 5

161
 Disagree
Disagree
Strongly

Strongly
To what extent do you agree with the following

Neutral

Agree

Agree
statements with respect to your role in an instructional
design team?
1. I have clear, planned goals and objectives for my 1 2 3 4 5
job
2. I know that I have divided my time properly 1 2 3 4 5
3. I know what my responsibilities are 1 2 3 4 5
4. I know exactly what is expected of me 1 2 3 4 5
5. I feel certain about how much authority I have on 1 2 3 4 5
the job
6. Explanation is clear of what has to be done 1 2 3 4 5

Once or twice per day

Several times per day

Less than once per

Once or twice per

Once or twice per

month or never

month

week
How often have you experienced the following with
respect to your role in an instructional design team?
1. How often does your job require you to work very 1 2 3 4 5
fast?
2. How often does your job require you to work on 1 2 3 4 5
very complex projects?
3. How often does your job leave you with little time 1 2 3 4 5
to get things done?
4. How often is there a great deal to be done? 1 2 3 4 5
5. How often do you have to do more work than you 1 2 3 4 5
can do well?

162
 Disagree
Disagree
Strongly

Strongly
To what extent do you agree with the following

Neutral

Agree

Agree
statements regarding the quality of the online course
design process at your institution?
1. Online course design projects are completed as per 1 2 3 4 5
the set timeline.
2. The online course design process consistently 1 2 3 4 5
resulted in creating learning activities that actively
engage students with the content.
3. The online course design process consistently 1 2 3 4 5
resulted in the clear alignment between student
learning outcomes and major course assessments.
4. The online course design process consistently 1 2 3 4 5
resulted in designing activities that promote
student-student, student-instructor, and student-
content interaction.
5. The online course design process consistently 1 2 3 4 5
resulted in the appropriate alignment of course
objectives with assessments, instructional materials,
and course technology.
6. The online course design process resulted in a 1 2 3 4 5
complete and error-free product for students.
7. The instructional design team made timely progress 1 2 3 4 5
during the online course design process.
8. The instructional design team made exemplary 1 2 3 4 5
progress during the online course design process.
9. Overall, I am satisfied with the collaboration of the 1 2 3 4 5
instructional design team.
10. I would recommend participation in the online 1 2 3 4 5
course development process to my colleagues.
11. Overall, I am satisfied with the online course 1 2 3 4 5
design and development process.

Additional Comments
Do you have any additional comments or recommendations that will improve the effectiveness of
the online course design process?

163
 Appendix B

Needs Assessment Letter of Consent for Form

Title of Research: Examining the Impact of Role Stressors on the Quality of the Online Course
Design Process – Baltimore, MD
Principal Investigator: Veena Radhakrishnan, Johns Hopkins School of Education

PURPOSE OF THE RESEARCH STUDY:

The purpose of this research study is to determine whether role stressors, namely role conflict,
role ambiguity, and role overload affect the quality of the online course design process in higher
education instructional design teams. The research findings will be beneficial to instructional
designers, faculty, and other staff involved in the online course design process.

PROCEDURES:
This Qualtrics survey has four sections and takes approximately 10-15 minutes to complete.
Please complete this survey by 05/01/2016.

RISK/DISCOMFORTS:
There are no anticipated risks to the participants.

BENEFITS:
Your participation will contribute to the understanding of the roles and responsibilities of
individuals involved in the online course design process and their impact on the timely and
satisfactory completion of online course design and development tasks.

VOLUNTARY PARTICIPATION AND RIGHT TO WITHDRAW:

Your participation in this research study is completely voluntary and will be greatly appreciated.
If you decide not to participate, there are no penalties, and you will not lose any benefits to
which you would otherwise be entitled. You can stop participation in the study at any time,
without any penalty or loss of benefits.

CONFIDENTIALITY:
The information you provide via this online questionnaire will remain confidential and
anonymous. By completing this questionnaire, you are providing your consent and
acknowledging that the data provided anonymously by you can be accessed by the researcher.

COMPENSATION:
You will not receive any payment or other compensation for participating in this study.

IF YOU HAVE QUESTIONS OR CONCERNS:

You can ask questions about this research study at any time during the study by contacting Veena
Radhakrishnan at (443) 514-6007 or vradhak1@jhu.edu. If you have questions about your rights
as a research participant or feel that you have not been treated fairly, please call the Homewood
Institutional Review Board at Johns Hopkins University at (410) 516-6580. Your identity,
questions, and concerns will be kept confidential.

164
IF YOU WOULD LIKE, YOU CAN PRINT THIS PAGE FOR YOUR RECORDS.
If you would still like to take part in this study, please click the Yes button below.

WHAT YOUR AGREEMENT MEANS:

By clicking "Yes" below, you agree that you understand the information in this consent form. By
agreeing to this consent form, you have not waived any legal rights you otherwise would have as
a participant in a research study.

o Yes, I would like to participate

o No, I would not like to participate

165
 Appendix C

Intervention Logic Model

166
 Appendix D

Intervention Data Collection Instrument

Demographic Questionnaire:
The following questions ask about your background and your demographic characteristics. The
purpose of this information is only to describe the group of all respondents for the purpose of
comparison with other research studies. Individual responses will not be disclosed or shared with
any person working in your institution. Your answers will be kept strictly confidential and will
not be used to identify you or any of your responses in this study. You have the right not to
answer any questions, should you feel uncomfortable.

1. Choose your role from the following (check all that apply):
o Faculty
o Instructional Designer
o Instructional Technologist
o Project Manager
o Executive Team Member (Director etc.)
o Other (please specify): ______________________________

2. At what type of institution do you work?

o Public, four-year institution
o Private, four-year institution
o Public, two-year college
o Private, two-year college
o For-profit institution

3. Please select your appropriate age range:

o 18-30
o 31-40
o 41-50
o 51 and above

4. What is your gender?

o Male
o Female

5. What is your ethnicity? (Choose all that apply)

o White (non-Hispanic)
o Hispanic/Latino
o African American
o Asian
o Native American
o Native Alaskan or Pacific Islander
o Other (please specify): ______________________________

167
 6. What is the highest level of education you have completed?
o High school/GED
o Some college
o 2-year college degree (Associate’s)
o 4-year college degree (Bachelor’s)
o Advanced degree (Master’s or other)
o Other (please specify): ______________________________

7. How many years of professional experience do you have in your field?

o 1-2 years of professional experience
o 2 to 4 years of professional experience
o 4 to 5 years of professional experience
o 5 to 6 years of professional experience
o 6 or more years of professional experience

Directions:
Please read the instructions carefully before responding. People have different feelings about
their roles. Statements describing some of them are given below. Use the answer sheet to write
your responses. Read each statement, and indicate how often you have the feeling expressed in
the statement in relation to your role in online course development. Use the numbers given below
to indicate your feelings.

If you find that the category to be used in answering does not adequately indicate your feelings,
use the one closest to the way you feel. Do not leave any item unanswered. Answer the items in
the order given below.

Write 0 if you never or rarely feel this way.

Write 1 if you occasionally (a few times) feel this way.
Write 2 if you sometimes feel this way.
Write 3 if you frequently feel this way.
Write 4 if you very frequently or always feel this way.

Role Ambiguity Questionnaire

_____ 1. I am not clear on the scope and responsibilities of my role (job).

_____ 2. I do not know what the people I work with expect of me.

_____ 3. Several aspects of my role are vague and unclear.

_____ 4. My role has not been defined clearly and in detail.

_____ 5. I am not clear as to what are priorities in my role.

168
 Appendix E

Email Sent to the Director of the Instructional Design Team Requesting Approval

Dear [Director’s Name],

I am a doctoral student at JHU School of Education. My research focuses on roles that
instructional design team members play in the design of online courses. Specifically, I am
interested in exploring the potential for role stressors, especially role ambiguity, to exist within
instructional design teams and how these stressors could affect the quality of the online course
design process, and consequently, the quality of online courses delivered by higher education
institutions.

I would like to invite you and your team to participate in a study I am conducting as part of my
doctoral dissertation. The study involves the implementation of a role analysis exercise in
instructional design teams. Engaging in this exercise along with your team members may give
you the opportunity to: 1) analyze key roles within the context of your existing instructional
design processes, 2) identify weaknesses in structures, policies, and processes that could affect
role effectiveness, and 3) develop clarity around the specific roles of instructional design team
members in contributing to instructional design processes and the overall quality of online
courses.

Participation in this study will require attending a face-to-face, half a day workshop and
completing a few activities that will allow me to collect data regarding participants’ experiences.

I greatly appreciate your consideration and hope that you and your team will choose to
participate in this study. I look forward to hearing from you.

Best regards,
Veena

169
 Appendix F

Sample Approval Letter from the Director of the Instructional Design Team

From
[Name]
[Title]
[Institution]
[Street Address]
[City, State, Zip]

To
Veena Radhakrishnan
Johns Hopkins School of Education

RE: Permission to Conduct Research at the [Institution Name]

Dear Veena,
I am very interested in your topic of dissertation, “A Role Analysis Exercise to Minimize Role
Ambiguity and Promote Role Clarity in Instructional Design Teams.” I believe the data collected
for this study will be beneficial to the improvement of the [Instructional Design Team Name].
Therefore, I am pleased to give my support for this valuable research and allow you to conduct
the role analysis exercise with online faculty and [Instructional Design Team Name] staff and
collect data for your research study. However, please note that it is the decision of the individual
faculty and [Instructional Design Team Name] staff to choose to participate in the study.

I hope you will share with me your experiences and the results of the study. I anticipate that the
results will enhance the roles and responsibilities of the [Instructional Design Team Name] and
the related instructional design processes. I would be grateful if a summary of your key research
findings can be submitted to the [Department Name] at the completion of your doctoral studies.

Thank you very much for choosing the [Instructional Design Team Name] at the [Institution
Name] to implement your research. I wish you all the best.

Sincerely,

[Name]

170
 Appendix G

Recruitment Email Script

Dear [Instructional Design Team Member],

My name is Veena Radhakrishnan and I am a doctoral student at the Johns Hopkins School of
Education. My research focuses on roles that instructional design team members play in the
design of online courses. Specifically, I am interested in exploring the potential for role stressors,
especially role ambiguity, to exist within instructional design teams and how these stressors
could affect the quality of the online course design process, and consequently, the quality of
online courses delivered by higher education institutions. Being part of a team involved in online
course development, I am sure you are aware of the key roles that members of instructional
design teams play in designing and developing online courses.

I would like to invite you to participate in a study I am conducting as part of my doctoral

dissertation. The study involves the implementation of a role analysis exercise in instructional
design teams. Engaging in this exercise along with your team members may give you the
opportunity to: 1) analyze key roles within the context of your existing instructional design
processes, 2) identify weaknesses in structures, policies, and processes that could affect role
effectiveness, and 3) develop clarity around the specific roles of instructional design team
members in contributing to instructional design processes and the overall quality of online
courses. The research findings will be beneficial to instructional designers, faculty, and other
staff involved in the online course development process.

Participation in this study will require approximately four hours of your time (one half-day,
face-to-face workshop), completion of a five-question pre- and post-survey to measure role
ambiguity (approximately 5-10 minutes to complete), participation in written and verbal
reflection exercise about your experience (approximately 45 minutes to complete), and a 30-
minute follow-up interview (approximately three weeks after the workshop).

You will be provided light snacks/refreshments during the study.

There are no anticipated risks to the participants. Your participation is completely voluntary and
any data collected during the study will be kept secure and confidential.

I am attaching the informed consent form for your review. If you are interested in participating in
this study, please sign the form and return it to me via email. If you have any questions about the
study, please do not hesitate to contact me at (443) 514-6007 or vradhak1@jhu.edu. You may
also contact the principal investigator, Wendy Drexler, at wdrexle1@jhu.edu or (813) 309-4090.

I greatly appreciate your consideration and hope that you will choose to participate in this study.
I look forward to talking with you in the future.

Sincerely,
Veena Radhakrishnan

171
 Appendix H

Informed Consent Form

Johns Hopkins University

Homewood Institutional Review Board (HIRB)
Informed Consent Form
Title: A Role Analysis Exercise to Minimize Role Ambiguity and Promote
Role Clarity in Instructional Design Teams

Principal Investigator: Dr. Wendy Drexler, Johns Hopkins School of Education

Date: January 11, 2018

PURPOSE OF RESEARCH STUDY:

The purpose of this study is to explore the potential of a role analysis exercise to minimize role
ambiguity and promote role clarity among members of instructional design teams. The findings
of the study may result in an understanding of how role stressors, especially role ambiguity, can
exist in instructional design teams (IDT) and may create opportunities for IDTs to consciously
recognize role stress as a significant issue that has the potential to impact the online course
design process negatively. The research findings will be beneficial to instructional designers,
faculty, and other staff involved in the online course development process. We anticipate that
approximately 60 people will participate in this study.

PROCEDURES:
The study consists of participation in a role analysis exercise and will require approximately
four hours of your time (one half-day, face-to-face workshop). Participation in this study also
requires the completion of a five-question pre- and post-survey to measure role ambiguity
(approximately 5-10 minutes to complete), participation in written and verbal reflection exercise
about your experience (approximately 45 minutes to complete), and a 30-minute follow-up
interview (approximately three weeks after the workshop). Participation in the interview will be
audio recorded.

RISKS/DISCOMFORTS:
The risks associated with participation in this study are no greater than those encountered in
daily life.

172
BENEFITS:
There are no direct benefits to you from participating in this study. Your participation may
contribute to the understanding of the roles and responsibilities of individuals involved in the
online course development process and their impact on the timely and satisfactory completion of
online course development tasks.

VOLUNTARY PARTICIPATION AND RIGHT TO WITHDRAW:

Your participation in this study is entirely voluntary: You choose whether to participate. If you
decide not to participate, there are no penalties, and you will not lose any benefits to which you
would otherwise be entitled. If you choose to participate in the study, you can stop your
participation at any time, without any penalty or loss of benefits. If you want to withdraw from
the study, please notify the student investigator or the principal investigator at any time at the
email address and phone number provided below. Any data collected will be discarded.

CIRCUMSTANCES THAT COULD LEAD US TO END YOUR PARTICIPATION:

Under certain circumstances, we may decide to end your participation before you have
completed the study. Specifically, we may stop your participation if we determine that you have
not been involved in the development of online courses.

CONFIDENTIALITY:
Any study records that identify you will be kept confidential to the extent possible by law. The
records from your participation may be reviewed by people responsible for making sure that
research is done properly, including members of the Johns Hopkins University Homewood
Institutional Review Board and officials from government agencies such as the National
Institutes of Health and the Office for Human Research Protections. (All of these people are
required to keep your identity confidential.) Otherwise, records that identify you will be
available only to people working on the study, unless you give permission for other people to see
the records. All records will be stored in a locked file cabinet in a locked room. Only the
investigator and members of the research team will have access to these records.

COSTS:
You are not responsible for any research-related costs.

COMPENSATION:
You will not receive any payment or other compensation for participating in this study.

173
IF YOU HAVE QUESTIONS OR CONCERNS:
You can ask questions about this research study now or at any time during the study by
contacting Veena Radhakrishnan, Student Investigator, at (443) 514-6007 or vradhak1@jhu.edu.
You may also contact Wendy Drexler, Principal Investigator, at (813) 309-4090 or
wdrexle1@jhu.edu. If you have questions about your rights as a research participant or feel that
you have not been treated fairly, please call the Homewood Institutional Review Board at Johns
Hopkins University at (410) 516-6580.

SIGNATURES

WHAT YOUR SIGNATURE MEANS:

Your signature below means that you understand the information in this consent form.
Your signature also means that you agree to participate in the study.

By signing this consent form, you have not waived any legal rights you otherwise would
have as a participant in a research study.

Participant's Signature Date

Signature of Person Obtaining Consent Date

(Investigator or HIRB Approved Designee)

174
 Appendix I

Observational Protocol and Recording Sheet

(Adapted from Lodico, Spaulding, & Voegtle, 2010)

Date of observation:
Time of observation:
Setting:
Participants:
Observer:

RQ1: To what extent did the role analysis exercise result in a decrease in role ambiguity among
instructional design team (IDT) members, as measured by the Role Ambiguity (RA) subscale in
Pareek’s (1983) Organizational Role Stress (ORS) scale?

Sub-questions:
 What types of interactions do participants have within the team during the role analysis
exercise?
 What types of interactions show evidence of greater clarity around specific roles of IDT
members in contributing to online course development processes?
 What types of interactions show evidence of increased awareness of role effectiveness,
related processes, and alignment with the team and/or organizational components?

Participant Interactions:

Observer Reflections:

175
 Appendix J

Interview Protocol and Recording Sheet

(Adapted from Crewswell, 2017)

Date of interview:
Time of interview:
Place:
Interviewer:
Interviewee:

RQ2: What were instructional design team members’ perceptions of the value of a role analysis
exercise and its potential to decrease role ambiguity?

Questions:

1. What was your overall experience like taking part in the role analysis exercise?

2. To what extent did the role analysis exercise contribute to decreasing role ambiguity
around your role(s) and the roles of your team members?

3. Following the role analysis exercise, has your team utilized or plan to utilize the newly
established roles as part of the online course development processes?

4. What are some of the ongoing challenges that you predict might hinder you and your
team in establishing clarity around the roles and responsibilities of team members?

5. What support and/or resource would be needed to create further clarification in the roles
and responsibilities of individuals involved in the online course development process?

176
 Appendix K

Debrief Activity Questions

The debrief activity will follow the role analysis exercise and is an opportunity for participants to
share their experiences and provide feedback for the facilitator. The following questions were
devised to guide participants’ open-ended discussions.

RQ3: To what extent were all elements of the role analysis exercise based on Dayal and
Thomas’s (1968) role analysis technique (RAT) implemented as planned?

1. What was your experience like taking part in the role analysis exercise?

2. Were there any specific components of the role analysis exercise that were especially
beneficial?

3. To what extent do you have a clearer understanding of your own roles and your team
members’ roles?

4. What questions, confusions, or insights did the exercise bring up for you about your own
and your team members’ roles?

5. Were there any challenges or constraints you faced in completing the activities involved
in the role analysis exercise?

6. Do you have any recommendations for improving the exercise?

177
 Appendix L

Role Expectations Worksheet

(Adapted from Carter et al., 2005)

Below, write a brief description of your roles and responsibilities as individual members of the
team and how you perceive others’ expectations of you. Feel free to add additional circles to
represent more roles.

Name: Individual Roles and What Others Expect of Me

Responsibilities

ROLE ROLE
TITLE

ROLE ROLE

ROLE ROLE

Additional Comments:
______________________________________________________________________________
______________________________________________________________________________
______________________________________________________________________________

178
 Appendix M

Role Profile Worksheet

(Adapted from Srivastav 2012)

Main Purpose of Role:

______________________________________________________________________________

Role Expectations (Key Responsibilities and Accountabilities)

1. ____________________________________________________________________________

2. ____________________________________________________________________________

3. ____________________________________________________________________________

4. ____________________________________________________________________________

5. ____________________________________________________________________________

Critical Attributes

Educational Qualifications: _______________________________________________________

____________________________________________________________________________
Knowledge: ___________________________________________________________________
____________________________________________________________________________
Technical Skills: ________________________________________________________________
____________________________________________________________________________
Behavioral Skills: _______________________________________________________________
____________________________________________________________________________
Managerial Skills: ______________________________________________________________
____________________________________________________________________________
Experience: ___________________________________________________________________
____________________________________________________________________________

179
 Appendix N

Sample Workshop Agenda

Role Analysis Workshop

[Instructional Design Team Name] [School, University]
Date:
Time:
Location:

Meeting called by Veena Radhakrishnan

Workshop Facilitator [Facilitator Name]
[Title, Institutional Affiliation]
Attendees: [Participant Name]
[Participant Name]
[Participant Name]
[Participant Name]

9:30 a.m. – 9:45 a.m. Welcome

Participant Introduction
Purpose of the Study
Informed Consent
Pre-Workshop Survey
9:45 a.m. – 10:00 a.m. Introduction
Role versus Title: An Overview
10:00 a.m. – 10:15 a.m. Icebreaker Activity
Marshall Goldsmith’s Feedforward Exercise
10:15 a.m. – 10:30 a.m. Role Analysis Exercise: Activity 1
Individual Roles and Responsibilities
10:30 a.m. – 10:50 a.m. Role Analysis Exercise: Activity 2
Team Role Expectations
10:50 a.m. – 11:00 a.m. Break
11:00 a.m. – 11:30 a.m. Role Analysis Exercise: Activity 3
Role Clarification
11:30 a.m. – 12:00 p.m. Role Analysis Exercise: Activity 4
Revised Role Profiles
12:00 p.m. – 12:30 p.m. Debrief
Individual Reflections
Group Debrief
12:30 p.m. – 12:45 p.m. Closing
Follow-up Activity Reminders
12:45 p.m. – 1:30 p.m. Q&A, Open Discussion

180
 Curriculum Vitae

VEENA RADHAKRISHNAN
443-514-6007 | veena.radhakrishnan@gmail.com | linkedin.com/in/veenaradhakrishnan

ACADEMIC BACKGROUND
 Doctor of Education
Johns Hopkins University, School of Education
Specialization: Instructional Design for Online Teaching and Learning

PROFESSIONAL EXPERIENCE
 Director of Training Strategy & Design
International Society for Pharmaceutical Engineering
November 2018–Present
 Adjunct Professor, Digital Media & Web Technologies
University of Maryland University College
Mar 2013–Present
 Instructional Designer
Johns Hopkins Carey Business School
Jan 2016–October 2018
 Instructional Designer/Learning Technologist
Bowie State University
Jan 2015–Jan 2016

CERTIFICATIONS
 Online Teaching Certificate, Online Learning Consortium
Jun 2015–Present
 Certified Master Reviewer, Quality Matters (QM)
May 2015–Present
 Certified Peer Reviewer, Quality Matters (QM)
Jul 2013–Present

PRESENTATIONS
 Radhakrishnan, V., & Ariev, P. (2018). Moving beyond compliance and re-
conceptualizing instructional design: A context-specific framework for internal online
course reviews. American Educational Research Association 2018 Annual Meeting. New
York, NY.
 Radhakrishnan, V., & Dempsey, P. (2017). Implementation with intention: An integrative
framework for quality assurance in online higher education. Quality Matters Connect
Conference. Fort Worth, TX.
 Radhakrishnan, V., Dempsey, P., & Rennert-Ariev, P. (2017). A systemic approach to
quality assurance in online courses [Poster Presentation]. Online Learning Consortium
Conference. Orlando, FL.
 Radhakrishnan, V., Ariev, P., & Wachira, C. (2017). Moving beyond compliance: A
collaborative and context-specific framework for internal Quality Matters (QM) reviews.
Quality Matters Regional Conference. New York, NY.

181