Educational design research – Part B: Illustrative cases Chapter 39

Educational Design Research for

Collaborative Learning: Challenges and
Opportunities in Oman

David Porcaro & Thomas Reeves

SLO • Netherlands institute for curriculum development

Porcaro, D., & Reeves, T. (2013). Educational design research for collaborative learning: Challenges and opportunities in
Oman. In T. Plomp, & N. Nieveen (Eds.), Educational design research – Part B: Illustrative cases (pp. 827-850). Enschede,
the Netherlands: SLO.
Contents

39. Educational design research for collaborative learning:

Challenges and opportunities in Oman

Abstract 829

1. Introduction and context 829

2. Preliminary investigation 831

3. Design, development, and iterations 835

4. Project results 842

5. Reflections 844

Key sources 846

References 846

Credits

2013 SLO (Netherlands institute for curriculum development), Enschede

All rights reserved. This publication may be reproduced in whole or in part by photocopying or
any other means, provided the source is mentioned.
39. Educational design research for
collaborative learning: Challenges and
opportunities in Oman
David Porcaro & Thomas Reeves

Abstract
Constructivist teaching methods, such as Computer Supported Collaborative Learning, have
been championed by many educators and international organizations as a way to improve
education in developing countries and to prepare students for working within knowledge
societies. However, few studies have examined how personal, cultural, institutional, and
national factors contribute to the acceptance or rejection of these innovative methodologies,
especially in instructivist learning cultures. This chapter outlines one education design research
study which was conducted as a doctoral dissertation project that examined how to apply
collaborative learning methods to an undergraduate course for pre-service teachers in Oman.
This study takes a linguistic cognitive anthropology design research approach, focusing on the
role of culture in adopting research methods designed in one context and adapted to another.
In addition to describing design principles for preparing collaborative learning environments in a
setting like Oman, it outlines the challenges and opportunities of conducting long-term
international education design research as a graduate student.

1. Introduction and context

Many international organizations and donors have suggested the best way to develop
“knowledge societies” and increase development in the “information age” is to move from simple
information sharing, which is most common in recent Information and Communications
Technology (ICT) innovations as well as education practices throughout the world, to knowledge
building (UNESCO, 2005). If social and economic development is to progress in this manner, it
must be built on new ways of knowing - ways that no longer stress passive reception of
sanctioned information through memorization and recall, but instead on strategies that foster
skills in information gathering, evaluating source quality, collaborating, problem-solving, and
ultimately knowledge creation (Bereiter, 2002). For this reason, many educators have turned to
Computer-Supported Collaborative Learning (CSCL) as a way to bridge the distances amongst
learners (physically, temporally and interpersonally) as well as between learners and relevant
performance (Collis, 2008) with the goal of fostering collaborative knowledge creation
(Scardamalia & Bereiter, 1993). While it is clear that ICT does not always provide a simple
solution to education development’s complex problems (Vrasidas, Zembylas, & Glass, 2009)
and may further support inherent power-relations, innovations like CSCL provide great
opportunities for students to express themselves and become more empowered in creating
knowledge.

However, many of the nations that could benefit most as knowledge societies continue to
promote education practices that are largely instructivist in nature; that is, they rely on verbal
transmission of information through lectures and textbook readings and a heavy emphasis on
memorization of discrete facts that can be assessed through high stakes exams. In the Middle
East, traditional instructivist methods may even contribute to the gulf between education and the

 829
critical characteristics needed in the labor force, creating challenges as graduates are unable to
find work due to lack of relevant and practical skills (World Bank, 2008).

While many policy makers advocate a shift to constructivist-based social learning methods to
aid in educational development (cf. UNESCO, 2005), the debate about the relative effectiveness
of instructivist versus constructivist learning is still ongoing (see Donn & Manthri, 2010; Tobias &
Duffy, 2009). Nonetheless, the actual process of change from instructivist to constructivist
methods has been poorly investigated (Catterick, 2007). Specifically, in a traditionally
instructivist learning culture, such as Oman, there is little research into how personal and
institutional factors affect the adoption of CSCL.

The purpose of this study, conducted by the first author as a doctoral student in the USA and
supervised by the second author, was to understand how students, teachers and institutions
adapt to collaborative knowledge-building technologies and practices, as well as to understand
what design principles could lead to a CSCL environment that is contextually appropriate for an
undergraduate course at Sultan Qaboos University (SQU) in Oman. For this multi-semester
international design research study, the following questions were addressed:
1. a. What changes (attitudinal, cognitive, epistemological, habitual, etc.) occur in teachers
and students when a knowledge-building CSCL environment is introduced in an Omani
undergraduate course? b. How do the students and teachers manage those changes?
2. What are the characteristics of a sustainable knowledge-building CSCL environment in an
Omani undergraduate instructional design course, which would take into consideration
those changes?
These research questions defined the research project as a whole and research sub-questions
were selected to guide the process (see Table 1 and 2 below for a listing of those sub-
questions). However, the sheer amount of data collected was enormous, as often happens in
design research. This chapter addresses one dimension of those research questions, namely
the role culture played in the application of collaborative learning techniques to Oman.

Why educational design research?

Educational technology has been noted as “first and foremost a design field”, and as such, “the
paramount research goal of education technology should be solving teaching, learning, and
performance problems, and deriving design principles that can inform future development and
implementation decision” (Reeves, 2006, p. 61). It was this search for design principles based
on a deeper understanding of teaching and learning to solve existing problems that guided this
study of the introduction of a knowledge-building CSCL environment in Oman.

Thus, selecting an appropriate research method for this goal was important. According to
Reeves (2000), there are at least six major types of research goals: theoretical, predictive,
interpretivist, postmodern, design/development, and action/evaluation. The developmental goals
of design research are especially useful when “existing knowledge falls short, as is often the
case with highly innovative curriculum improvement initiatives” (McKenney, Nieveen, & Van den
Akker, 2006, p. 74), or when researchers want to create new learning environments, develop
contextually-based theories of learning, understand design processes or increase education
innovation (DBRC, 2003). While CSCL studies have historically included laboratory,
ethnographic or design research methods to reach their goals (Stahl, Koschmann, & Suthers,
2006), the predictive (e.g., laboratory) and descriptive (e.g., ethnographic) methods of research
are limited in several important ways.

 830
For example, while experimental (or quasi-experimental) laboratory studies or randomized
clinical trials may be considered the “gold standard” of research because of their ability to
isolate variables and determine causal relationships (National Research Council, 2002), this
focus on only hypothesis testing limits the scientific process to merely summative evaluation
(Phillips, 2006) and may hinder innovation by prematurely proclaiming evaluative judgments
regarding the efficacy of educational interventions (DBRC, 2003). In addition, these methods
often de-contextualize innovations to the point of insufficiency.

Likewise descriptive methods also fail too often in producing positive educational change.
Surveys, along with other descriptive research (such as ethnomethodology, phenomenology,
ethnography and case studies) can constrain educational change within the observation stage,
and may limit the transformative agendas of educational researchers due to fears of the
Hawthorne effect on the observed phenomena (Barab & Squire, 2004).

Design research, however, has as a goal the application of both prediction and description in an
attempt to create sustainable educational change while seeking to understand learning in situ.
Educational design research describes learning situations using a variety of methods, fosters
the creation of innovative learning environments, and evaluates them through predictive testing
methods over several iterations (Stahl et al., 2006). Thus it is best situated to produce practical
and socially-responsible research (Reeves, Herrington, & Oliver, 2005).

Furthermore, design research addresses many of the shortcomings of theoretical and

postmodern research in that it creates solutions as a goal, rather than assuming that others will
convert theory and critique into change. Design research also transcends the goals of formative
(or action) research in that it constantly connects design with existing theory and generates new
contextually-grounded theoretical knowledge (Barab & Squire, 2004), rather than merely
evaluating the effectiveness of a learning practice or design. In this way, design research
occupies Pasteur’s quadrant in Stokes' (1997) matrix, assuming the dual goals of consideration
of use and fundamental understanding (Reeves, 2000). The interwoven goals of design
research best allowed this research project to address understanding the nature of collaborative
knowledge building as well as working with local Omani educators to create solutions that will
improve education in their country and lead to greater social and economic development. In the
case of this study, we sought greater understanding of the theoretical nature of learning, culture,
and change (research question 1a and b), while simultaneously pursuing a design goal of
building a learning environment (research question 2).

This study was guided by the beliefs and methods of what Bell (2004) calls “linguistic or
cognitive anthropology design-based research,” which places significant emphasis on the
localized nature of the practices and norms of the social groups investigated (Omani students)
as they actually occur in their specific settings. Bell claims that such a focus allows for detailed
study of how new designs are appropriated, resisted, or even repurposed by groups that are
assumed to already have significant cultural momentum before any intervention begins.
Analysis often documents unanticipated consequences or emergent practices that derive from
the cultural worlds of participants never anticipated by the educational designers.

2. Preliminary investigation
This research design began with an informed exploration, or what van den Akker (1999) calls
“preliminary investigating,” which involves reviewing the literature, developing theory, and
understanding the users’ needs (see Figure 1). This stage included extensively reviewing the

 831
literature of CSCL theory and practice as well as the use of collaborative learning in Arab culture
in general and Omani culture specifically. Trends in the literature were analyzed to create a
framework for describing the varying elements that shape the interaction between constructivist
and instructivist learning cultures.

 832
Figure 1: Research design, based on Bannan-Ritland's ILD (2003)

 833
This preliminary investigation led to the creation of a conceptual framework (see Figure 2), that
incorporated theories of knowledge-building communities, understanding of the Omani context,
and “thought experiments” (Gravemeijer & Cobb, 2006). This conceptual framework was
documented as a publishable manuscript (Porcaro, 2011), which more fully describes and
analyzes the interacting pieces. However, it is worth summarizing here the key features.

An educational innovation sits at the nexus of teacher, student, and institutional system, all
housed within a national context. The national context includes variables that affect how that
innovation might be assimilated within the context. For instance, economic conditions,
educational tradition, sociopolitical conditions, work force requirements, and location of schools
will largely shape how well an innovation is supported or received. The institution also brings to
bear resources, policy, and institutional culture that shape the success of the innovation. The
teachers and students bring to the interaction their own educational philosophies, which include
their epistemology, culture, individual difference, knowledge and skills, and academic habits. All
of these elements collide at the point of innovation and result in either congruence between the
elements (e.g., a teacher’s constructivist epistemology aligns with the institutional culture of
progress and innovation at the school) or conflict (e.g., a student’s individual differences do not
align with the academic habits of the teacher). Students, teachers, and institutions may respond
to this conflict by dismissing it or adapting to it, while congruence may be ignored or celebrated.
This reaction then feeds back to the educational philosophy of the students and teachers, either
strengthening them or altering them.

(e.g. economic conditions, educational

tradition, sociopolitical conditions, work
National Context force requirements, location of schools)

Institutional System
resources
policy
institutional culture

Teacher Innovation Student

epistemology epistemology
Educational Philosophy
Educational Philosophy

culture conflict congruence culture

individual difference individual difference

knowledge and skills ignore knowledge and skills

dismiss
academic habits adapt celebrate academic habits

Figure 2: Framework for introducing innovative pedagogies

For this study, the most difficult aspect of undertaking international design-based research as a
graduate student was the limited ability to travel to the country during the preliminary phase to
undertake needs assessments and collaboratively design the intervention with local
practitioners. However, to overcome that distance during this phase, it was necessary to engage
a key informant, Mohamed Eltahir Osman, with whom we could communicate, discuss practical

 834
details, and co-design the study. This key informant was the essential local practitioner who
helped ensure the project was designed collaboratively. Many phone calls and emails were
exchanged to answer questions about the learning context, align our joint expectations, and
clarify the needs of the project. That being said, as noted above the bulk of the preliminary
analysis of the project was done via literature review from the United States, as well as building
from the researchers’ personal experiences from having lived or traveled in other Middle
Eastern contexts.

From here, we could use the designed conceptual framework to determine which points of my
design were flexible and could be altered, and which characteristics the learners should be
expected to change. For example, from the literature review, it was discovered that when
applying CSCL projects to non-Western settings, a variety of institutional and individual
reactions must be considered (Arnseth & Ludvigsen, 2006). Other researchers documented
teachers struggling to cede control of their classrooms to their students, as well as students
lacking self-directedness and appearing to be motivated only to “make the grade” (Hung, Bailey,
& Johnson, 2003). We grouped this factor with others related to epistemology of teachers, which
may stand in conflict or congruence to the epistemology of students.

As we reviewed the literature, we noted the possible reactions to differences in students’ and
teachers’ educational philosophies and how they fit within an institutional structure and national
context. From there, we could determine what factors we should attempt to control within the
intervention design (for instance accounting for individual differences in the way we designed
collaborative assignments).

As noted above, we continued reviewing how CSCL projects had been initiated in Oman and
the Middle East, and documented this literature review as a publishable manuscript (Porcaro,
2011). As noted above, being distanced from the setting we were reliant on the published
literature, as well as informal communications with the key informant in Oman to understand
what the state-of-the-art for collaborative learning in Oman was. While studies of CSCL
applications are common in Western countries, documented cases are rarer in developing
countries, especially the Arab Middle East. We did find, however, several reports focused
mostly on cooperative work or computer-mediated communication, leading us to see the
necessity of a study of this type considering the dearth of research on the subject.

After reviewing the current cultural and educational setting of Oman, we then verified our
understanding of the current context and needs with our key informant via telephone. We sent
copies of the conceptual framework for him to review to ensure they were culturally sensitive
and reflected reality.

3. Design, development, and iterations

Based on the conceptual framework (see Figure 2), we created an initial design for a
collaborative knowledge-building course (prototype 1) for undergraduates at Sultan Qaboos
University in Oman (with different students each semester). Through collaboration with Omani
educators, we felt that introducing such a CSCL environment would help to move the students
to gaining collaborative problem-solving skills that they would need after graduation and allow
them to move their learning from passive knowledge acquisition to knowledge building. The first
author was fortunate enough to receive funding from a Fulbright Fellowship that allowed him to
spend 10 months in Oman to implement and gather data.

 835
Once the work began in Oman, however, some of the key factors of our preliminary design had
to be changed. The course around which we had designed the intervention proved to be less
appropriate for practical considerations, and another course was chosen - a technology
integration course for pre-service teachers. This course not only allowed for greater
project-based learning, but was dedicated to pre-service English teachers, whose competence
in English (the medium of instruction for courses at SQU) allowed them to engage in
computer-supported collaborative learning with greater fluency. Further assumptions about
learning in Oman were challenged once we realized that expectations were far from reality (e.g.,
the amount of work students were expected to do outside of class, the nature of in-class
collaboration, and students’ technology proficiency). Thus once in country, a month was needed
to make design alterations before the course began was essential to successful implementation.
These included redesigning the course structure to meet the new subject, reworking how
students would be expected to collaborate, and modifying expectations for in-class and
out-of-class work. These changes did not, however, erode the initial conceptual framework, but
rather supported our initial impression that for successful implementation of innovation, some
changes were needed by the teacher as well as the innovation.

During this time the first author worked more closely with a second local key informant, Ali
Al-Musawi, who became a cultural translator as well as the lens through which we viewed the
design. As a Western-educated Omani, he was able to collaborate closely with the first author to
ensure the design was sustainable and more appropriate for Omani students. Many hours were
spent discussing things like the oral nature of Oman’s culture and how that affected course
reading assignments and writing tasks, or what sorts of collaborative interactions between male
and female students were appropriate. We were able to plan for to the first author to teach
during the Fall 2009 and Spring 2010 semesters, and build for sustainability by handing the
course over to the Ali in Fall 2010.

Course structure
In this course we introduced exercises based on Brown and Campione's (1996) Fostering a
Community of Learners (FCL) model, which we felt would best introduce Omani students to
opportunities for collaborative exchange and connections to real-world learning. During these
exercises students designed posters, videos, websites, presentations, and other instructional
media. They selected their own groups for each assignment and worked individually or in small
single-gender groups (3-5). The media they constructed were to supplement the national
curriculum for English language in Omani public schools, and had to reflect the realities of
technology in Omani classrooms.

The students chose roles and were divided into “jigsaw groups” - an essential element of the
FCL model - to learn about various aspects of educational technology. The roles were based on
different elements of a multimedia team, namely project manager, web specialist, graphic
designer, instructional designer, and audio/video specialist. Each student was responsible to
independently learn more about some aspect of their role and collaborate online with others
who shared their roles. They then rejoined their project groups and provided some level of
expertise in their role to the project. During one class session, the students rotated around the
room in either all-male or all-female “cross-talk” groups to allow them to share their knowledge
and experiences with students from other groups. Taking on specific roles and creating
authentic artifacts not only allowed the students to learn the skills of the English as a Foreign
Language community of practice (Lave & Wenger, 1991), but immersed them in a form of
cognitive apprenticeship (Collins, Brown, & Newman, 1987).

 836
During these tasks, students formed knowledge-building communities of their own using Moodle
(including discussion forums) and Wikis, as well as Future Learning Environments 4 (FLE4), an
open source plugin for WordPress blogs (see Muukkonen, Hakkarainen, & Lakkala, 1999).
FLE4 provided scripts for online collaboration in an attempt to guide their discussions and
artifact creation toward knowledge building, helping the students to “rise-above” other students’
comments as they built theory, questioned other students’ assumptions, and linked ideas to
relevant literature. Students’ collaborative work was conducted online, so that mixed-gender
groups could share ideas. Collaboration was conducted both synchronously (during class time)
and asynchronously, face-to-face (in single gender groups) and online.

Research methods
Over the course of these three interventions, and concurrent with implementation, a variety of
mixed methods (DBRC, 2003) were used to collect formative data about the course (van den
Akker, 1999). This allowed us to answer practical as well as theoretical questions on the
intervention. In order to control the extensive and sometimes overwhelming (Brown, 1992;
Dede, 2004) data collection, we tried to focus collection efforts on information richness and
process efficiency (van den Akker, 1999). The theoretical framework developed during
preliminary investigation became a useful tool for selecting appropriate methods, as well as a
filter through which to analyze collected data (see Table 1).

For the first research question, we sought to understand the ways institutional structures and
students’ educational philosophies affected the introduction of innovative curricula, such as
CSCL. To do this, we asked students to complete pre-, mid- and post-term questionnaires to
assess their educational philosophy and attitudes toward collaborative knowledge building. The
questions concerning epistemology were based on an English and Arab language questionnaire
developed by Karabenick and Moosa (2005) for use by Omani university students.

In order to understand more deeply the reaction of students to the innovative pedagogy (Brown,
1992), the first author selected, based on their initial reactions to the course and variety in
beliefs, three students during the first iteration as cases for further study. The three students
were purposefully selected to provide maximum variation in their reaction to constructivist
th th
education, ranging from congruence to conflict. Each of the students was in their 4 or 5 year
of the English language teaching program at SQU and came from different regions of Oman.

 837
Table 1: Methods matrix for research question 1
Research Methods Matrix
What changes (attitudinal, cognitive, epistemological, habitual, etc.) occur in teachers and
students when a knowledge building CSCL environment is introduced in an Omani
undergraduate course? How do the students and teachers manage those changes?

Case student interviews (3)

Teacher/staff interviews
Student questionnaire
Observations/Course

Available documents
Student focus group
Research Methods

(pre, mid, post)

Sub-questions

On-Line Data

Assignments
debriefs
a. How do available
resources affect   
adoption?
Institutional System

b. How does current

policy affect adoption?
  

c. How does institutional

culture affect adoption?
 

a. How does student

epistemology affect   
adoption?

b. How does a student’s

Student Educational Philosophy

culture affect adoption?    

c. How do students’
individual differences
    
affect adoption?

d. How do students’
knowledge and skills
   
affect adoption?

e. How do students’
academic habits and
     
expectations affect
adoption?

These case students each participated in three one-hour semi-structured interviews - at the
beginning of the course, mid-term and at the end of the course. Following Seidman's (2006)
recommendations for conducting three-session phenomenological interviews, the first interview
focused on the students’ history and context, the second focused on their experiences in

 838
learning an innovative curriculum, and the third focused on the meaning the student made of the
experience.

During the second iteration, we sought greater breadth in understanding the student
experience. Therefore, instead of selecting case students, we chose to employ student focus
groups at the end of the semester. The first author conducted one male focus group (n=3
students) and one female focus group (n=4 students). The questions for this interview focused
on the students’ reactions to the course and their experiences with collaboration.
Since during the first two iterations the first author was both the teacher and researcher, he also
played the role of participant observer. His observations of the collaborative sessions were
discussed with other educators at SQU and he kept a regular blog of observations.
Furthermore, since many of the students’ collaborations and discussions were online, the
discussions on Moodle, course Wikis and FLE4 were recorded online, as well as copies of
artifacts from the courses were recorded.

In order to answer the second research question, we sought to assess the legitimacy, efficacy,
and viability (McKenney et al., 2006) of the learning environment through formative evaluation
methods (see Table 2). As part of classroom observations, the first author monitored the
students’ use of the learning environment. The post-term survey also included questions about
students’ experiences with the environment and any refinements they might suggest.

 839
Table 2: Methods matrix for research question 2
Evaluation matrix
Research Question Two: What are the characteristics of a sustainable knowledge-building
CSCL environment in an Omani undergraduate instructional design course?

Observations/course debriefs
Evaluation sub-

Case student interviews (3)

Student questionnaire (pre,

questions

Teacher/staff interviews
Student focus groups
Expert review

Assignments
On-line data
mid, post)
a. Is the course design
harmonious, consistent    
Legitimacy

and coherent?

b. Does the course reflect

current scientific insights  
or best practice?

a. Did the course help

students build knowledge      
collaboratively?

b. What knowledge was

learned by students?     
Efficacy

c. What skills were

developed by students?     

d. How did the course

appeal to students?    

a. Is the course practical

for SQU?   

b. Is the course relevant

Viability

to the needs of SQU     

students?

c. Is the course
sustainable and/ or
  
scalable within Omani
higher education?

Case student and focus group interviews also included questions evaluating the students’
experience with the course and course tools. In addition, at our request, seven external

 840
reviewers from various countries, chosen for their understanding of Omani higher education,
collaborative knowledge building, or teacher training, reviewed the course.

Through micro-cycles of “thought experiments” and “instruction experiments,” the learning

environment was developed and a local instructional theory emerged (Gravemeijer & Cobb,
2006). At the end of each prototype, we reviewed the results of the implementation through
what Edelson (2002) calls “retrospective analysis,” and refined the design in preparation for a
second, and a third implementation (see Figure 3). We compared issues of feasibility and
project implementation to examine what areas of conflict or congruence existed, based on the
conceptual framework. From there, we noted areas that we felt needed to be changed in the
course design (Innovation), the teacher, or the student.

For example, from interviews and questionnaires it was discovered that students had a limited
capacity to write individual reports on their assigned roles, due to the oral nature of Omani
culture. Here the students’ culture did not align with the teacher’s, so changes were needed in
the course innovation. Thus we decided that a group wiki would allow students to share the
burden of writing while still producing a substantial course artifact. However, the results of the
second semester observations, artifact review, and focus groups with students demonstrated
that this still exceeded the academic habits and expectations of students, who felt the workload
was excessive. Thus there was conflict with the teacher’s academic expectations, so changes
were needed in both the innovation and the teacher’s educational philosophy. At the end of
each cycle, the preliminary findings were presented at regional education conferences to further
discuss the results with peers in the Middle East to gain additional insight and feedback.

Semester 1 (n=27) Semester 2 (n=21) Semester 3 (n=33)

Design Refine Transfer
•Jigsaw groups for media roles •Students expressed technical •Limited teacher preparation
•Students prepare report on and user difficulty with wiki: and tool maintenance time:
new knowledge gained about moved to Moodle platform removal of wiki and FLE4-
role •Limited capacity in individual enabled blogs, Moodle
•Consequential task: build report writing and information discussion only.
multimedia kit in groups of 4-5, literacy: moved to group Wiki •Excessive workload for
using new knowledge from role writing. students: course requirements
•Content: Unit of instruction •Limited use of online tools for for consequential task
from Omani national group discussion: tools were simplified.
curriculum for English language available but optional for group •Change in teaching style:
•Used wiki and FLE4-enabled discussions, face-to-face and flexibility in implementation of
blogs for jigsaw group and other methods (e.g. SMS) were FCL, student roles and
multimedia design group encouraged. consequential task.
discussions •Limited understanding of •Limited familiarity with
•One day in the semester knowledge building, and use of knowledge-building concepts:
devoted to cross-talk, between tools: greater teacher support, More instruction on
group discussion (all male, and one-on-one consultation with collaborative and student-
all female) students. centered methods at beginning
of course, and weaved
throughout the course.

Figure 3: Major changes made between iterations

In order to improve validity and reliability, or what Lincoln and Guba (1985) call trustworthiness
and credibility, we attempted to triangulate the most theoretically important data (Stake, 2000)
through multiple data types and sources (DBRC, 2003), as well as by working collaboratively
with Omanis and other experts in joint evaluation (Van den Akker, 1999) and member checking
of findings (McKenney et al., 2006). We also attempted in the description of the case to be

 841
“thick” enough to allow the reader to determine the generalizability of the study and usefulness
of the findings (Stake, 2000).

The evaluation criteria were analyzed after the final (third) implementation, and a research
report was generated focusing on what Bannan-Ritland (2003) calls broader impact, in an
attempt to generalize the results to other contexts. This is available in two versions, a technical
report (Porcaro, 2012), and a practitioner’s guide (Porcaro & Al-Musawi, 2011) which broadly
details the lessons learned from the study. The second of these was co-authored by Ali
Al-Musawi, a key Omani collaborator. We felt this co-authorship was essential to the long-term
sustainability of the ideas and the validity of the findings.

4. Project results
In undertaking this research study, we hoped to discover design principles for both process and
product. The former were focused on the theoretical implications of the intervention, such as the
role of culture and educational philosophy in educational change, and what elements of CSCL
and knowledge building were well aligned to this setting. The results of these efforts are outlined
in more detail elsewhere (Porcaro, 2013; Porcaro & Al Musawi, 2012).

In terms of process, this study developed a greater theoretical understanding of the role of
culture in collaborative learning. Because of the extensive time needed to undertake this
inductive analysis, and due to the limited time during project implementation (since the first
author was also designing and implementing the course at the time), analysis between iterations
focused solely on formative design modifications, and less on deepening theory. Understanding
of the role different conceptual framework concepts, such as culture, played in the design
application were examined ex post facto, and thus design modifications based on theme were
limited. This points to the need for further modifications and testing of the design in the future.

While there was not enough time in the dissertation study to analyze deeply all of the
framework’s variables, one topic (culture) was selected for focused attention. Other variables
were left open for potential future analysis and reporting. We began the case-study analysis by
coding the interviews and artifacts according to the categories of the conceptual framework,
then analyzed the data inductively across the case to further define and describe the key
themes. This lead to the creation of subthemes, and all project data was coded by those sub
themes. Data from the case, such as artifacts and interviews, were used to bring meaning to the
sub-themes.

While modifications occurred between iterations, it was not until the project’s completion that we
began to see more clearly the role of culture in these changes. Culture affected the students,
the teacher, and the design in several ways. For instance, we noticed during the first iteration
that students were unwilling to engage in text-based assignments, or spend much time reading.
We began to shape a theory around the idea of oral learning, but it wasn’t until the deeper
analysis of the whole case after implementation that our understanding began to take shape,
and we could see that the reliance of students on oral over textual learning meant that students
were less likely to adapt to written assignments and text-based collaboration. (A detailed
description of these findings can be found in Porcaro, 2013). We realized that this was
something the teacher needed to recognize and adjust expectations accordingly. This idea,
along with other examples of culture’s role in the design, were organized into design principles
for each sub-theme (see Figure 4).

 842
To further outline the product-related design principles, we focused on the practical
considerations of the intervention and sought to explain to practitioners the implications of
designing and implementing a similar course in the Middle East or similar learning context. The
second set of design principles developed focused on the study as a whole and described
considerations practitioners should keep in mind when designing, implementing, and providing
technical support to a CSCL project in a similar setting (see Figure 5).

Further testing and refining of the course and design themes are necessary, but can be used to
enhance understanding of the changes that occur in teachers and students when introducing
knowledge-building CSCL environments in Oman and the way students and teachers manage
those changes, as well as some of the key design principles necessary for a sustainable CSCL
course in Oman.

Oral Learning
•Online discussion may not be preferred due to large amounts of writing and reading. Where possible, allow
face-to-face collaboration options.
•Students are often not skilled in writing traditional research projects. Use alternative ways for students to
report, especially orally, such as presentations, etc., or provide strong scaffolding.
Physical Setting
•While male and female entrances may determine where students are comfortable sitting, where possible
arrange seating in small groups, rather than in rows. In order to make the students feel comfortable, this may
mean dividing the class into two halves.
Teacher as Authority
•Provide opportunities for the students to take initiative and control their learning. This may include the
teacher helping the students choose topics for jigsaw specialization, or allowing students to choose between a
selection of project options. Students may not be used to this, so the teacher must provide extra support and
scaffolding.
•Where possible, let students teach each other, and the class, formally or informally. Teachers may feel it
difficult to learn from their students, so moving from a teacher-centered classroom to a student-centered
classroom may take time.

Gender
•Male and female face-to-face interactions will make many students uncomfortable. Where possible, provide
opportunities for students to discuss online, or share their ideas with the whole class.
•Realize that many females may feel uncomfortable presenting in front of male students. Build their confidence
by allowing them to pesent either in small groups with other female students first, or online.
•Since many students feel comfortable with the anonymity of online discussion, allow students to post things
anonymously or using pseudonyms of their choosing.
Acceptance of Technology
•While many students may welcome technology, others may need the teacher to demonstrate how technology
supports local values
•Be aware that many students come with very limited technological exposure. So allow alternative assignments
based on different levels of technology, or allow them to work with other students.
Religion
•When designing the course, be sensitive to the values of Islam. While varying interpretations of Islam exist, if
the teacher shows respect for religion, the students will be comfortable with any differences between them
and the teacher.
Collaboration in Society
•Arab society is built on collaboration and helping others. Where possible build upon students' experiences in
collaboration within the course activities.
•While students may not be comfortable disagreeing with teachers or other students, they often have real-
world experience arguing points and negotiating solutions. Build students' collaboration skills from this base.
•Some students may work only with students from similar tribes or regions. Allow students to form their own
groups, but provide opportunities for students to work with others in the class.

Figure 4: Design principles around theory for the concept culture

 843
 Design Considerations
•Build on the theoretical foundation of FCL: activity, reflection, collaboration, and community.
•Apply to practical courses or subjects; coordinate with practicum or internship courses as much as
possible
•Utilize with senior students, or students proficient enough in the language of instruction to argue
their position through online discussion
•Explain purposes and processes of collaboration thoroughly before beginning projects
•Allow ample early practice with CSCL tools before undertaking consequential task
•Provide in-class time to collaborate and limit out-of-class expectations for meeting and
collaborating
•Place emphasis on performance or production assessment rather than traditional assessment
•Provide ample opportunities for students to share ideas in class as well as across genders online
(cross-talks, jigsaw groups)
•Form groups early and provide preliminary small-scale projects to facilitate relationship building

Teacher Considerations
•Be committed to collaborative knowledge-building principles
•Create peer-based relationships with students
•Motivate and provide enthusiasm for students
•Help students feel comfortable to express themselves freely
•Monitor group work in class; frequently cycle through groups and discuss group progress
•Provide students with individual advisement on their project roles
•Give feedback and support in online discussions

Technical Considerations
•Provide online space for mixed-gender idea sharing
• Allow students to maintain anonymity or choose a pseudonym
• Provide spaces for groups to discuss their work privately
•Use easy-to-use or widely-used CSCL tools where possible (e.g. LMS discussion boards, and to a
lesser extent blogs and wikis)
•Plan for low bandwidth or limited online access; be flexible in when or how students use tools
•Ensure computer labs are available after hours for out-of-class collaboration; ensure labs are
staffed during operating hours with knowledgeable support personnel
•Provide technical support for CSCL design and maintenance (e.g. knowledgeable staff that can
create and maintain a WordPress.org blog on the university’s server)
Figure 5: Design principles for practitioners

5. Reflections
As with any change project, this one did not proceed exactly as anticipated at the outset. As
soon as the first author arrived in Oman, we realized our understanding of the Middle East and
expectations of how to implement this research had to evolve. Furthermore, helping others to
buy in to the importance of what we were trying to do was also a major challenge throughout
this study. Doing educational design research as a graduate student is particularly difficult.
Often the goals of a dissertation do not always align well with the intense collaboration,
expertise, and large data collection efforts of successful educational design research projects.
This was intensified by the fact that we designed the course while still in the United States, and
then later travelled to Oman to implement it. This meant that the collaboration necessary from
the beginning - people sitting around a table and drawing up a vision for the project - was not
there. While everyone in Oman was extremely supportive, there was a sense throughout the
study that this was the authors’ study, and our Omani collaborators would accommodate

 844
anything we felt was necessary, rather than this being our study where we accommodated
whatever they thought was necessary.

This pointed to the fact that too often graduate dissertations are theory driven, and not needs
driven, as was the case with this dissertation. Thus the lack of a bottom-up needs assessment
to undergird the design can be seen as one of this study’s greatest limitations. While much was
done to work with local collaborators beforehand to identify appropriate interventions, from the
beginning the project suffered from too much reliance on applying an innovative theory to a
novel setting, without much regard of whether that intervention was truly necessary.

In the end, the constraints of working in Oman left us wondering if the students ever did build
knowledge collaboratively or whether we had indeed fostered a community of learners. Had this
study failed to meet its most fundamental goal? Additionally, did we make any lasting impact on
education in Oman? Would these ideas die as soon as we stopped fanning the flames?

For us, Schoenfeld's (2004) commentary on four FCL projects that likewise lacked confidence in
their results and which also faced major difficulties was instructive:
Put these four sets of difficulties - finding big ideas, finding a plausible decomposition for
purposes of jigsawing, designing non-standard instruction based on the jigsaw, and teaching in
ways that foster the development of intellectual communities - together and you have a
monumental challenge. This would be a major challenge for a seasoned design team and
teachers who are well accustomed to 'reform'. To ask teachers without much design experience
and relatively unseasoned facilitators to take on this challenge is to ask a great deal. The efforts
of the four teams discussed here were little short of heroic, and their limited accomplishments
should be judged accordingly. It should be understood that pioneering attempts demand heroic
efforts, and that prototypes (both of process and of product) necessarily have lots of rough
edges. What has been presented in this set of papers is a collective design experiment, where
the goal was to create something important in order to examine and improve it. As has been
seen, the outcomes fell short of expectations (for all of the reasons identified above). However,
they can be considered improvements in both process and product over much traditional
instruction, and catalysts for professional growth as well. On their own terms, those are
non-trivial achievements. (p. 247)

Reading this, we began to look at this project in a new light. Instead of seeing what went wrong
in this study, we began to see what went right. That was when we saw the heroics of this
project, and the value of our research.

What went right

While not all students collaborated in the way we had envisioned, some did. Some students
were prepared to take on this course and move forward their epistemology, knowledge, skills
and perspective. We saw students arguing with other students and defending ideas. We saw
students questioning the value of textbooks as the sole source of information. We saw students
realizing their teachers were more their peers than their judges. We saw students creating,
solving, producing, working together, and discovering new ways of teaching and learning. While
not all students reached this level, some did. And several of those students mentioned that they
wanted to apply these ideas in the future in their own teaching.

Additionally, students began to see a crumbling of the barriers that prevent information-sharing
across gender. Male and female students collaborated online and shared ideas. We found

 845
instances of students offering feedback, support, or inspiration to each other, from one gender
to the other. For many students this was new, and for many, this was exciting.

While the change we saw was incremental, we did see change. The first author spoke with
several faculty members about the ideas, and we saw many think about ways they could adopt
them in their own classes. The fact that one faculty member tried them out in his own course,
with a high degree of success, was enough to sense a change. In fact, we heard recently that
one female faculty member at SQU even introduced in-class, face-to-face discussions between
males and females, with some success. Change in any educational context is slow. But good
ideas spread eventually, and as students become teachers, their students will experience
change in their learning, and eventually Omani education will include more of the kinds of
learning experiences that will prepare Omani knowledge workers for the future.

Key sources
Porcaro, D. (2011). Applying constructivism in instructivist learning cultures. Multicultural
Education & Technology Journal, 5(1), 39-54.

Porcaro, D.S. (2011). Reviewing the literature of computer-supported collaborative learning

(CSCL) to determine its usefulness in Omani education development. International Journal of
Education and Development using Information and Communication Technology, 7(3), 102-120.

Porcaro, D.S. (2013). Educational change in Oman: A design research study of personal,
institutional, and societal reactions to collaborative knowledge building. Technology, Pedagogy,
and Education (manuscript submitted for publication).

Porcaro, D.S., & Al-Musawi, A.S. (2011). Lessons learned from adopting computer-supported
collaborative learning in Oman. EDUCAUSE Quarterly, 34(4).

References
Arnseth, H., & Ludvigsen, S. (2006). Approaching institutional contexts: Systemic versus
dialogic research in CSCL. International Journal of Computer-Supported Collaborative Learning,
1(2), 167-185.

Bannan-Ritland, B. (2003). The role of design in research: The Integrative Learning Design
framework. Educational Researcher, 32(1), 21-24.

Barab, S., & Squire, K. (2004). Design-based research: Putting a stake in the ground. The
Journal of the Learning Sciences, 13(1), 1-14.

Bell, P. (2004). On the theoretical breadth of design-based research in education. Educational

Psychologist, 39(4), 243-253.

Bereiter, C. (2002). Education and mind in the knowledge age. Mahwah, NJ: Lawrence Erlbaum
Associates.

Brown, A.L. (1992). Design experiments: Theoretical and methodological challenges in creating
complex interventions in classroom settings. The Journal of the Learning Sciences, 2(2), 141.

 846
Brown, A.L., & Campione, J.C. (1996). Psychological theory and the design of innovative
learning environments: On procedures, principles, and systems. In L. Schauble, & R. Glaser
(Eds.), Innovations in learning: New environments for education (pp. 289-325). Mahwah, NJ:
Lawrence Erlbaum Associates.

Catterick, D. (2007). Do the philosophical foundations of online learning disadvantage

non-Western students? In A. Edmundson (Ed.), Globalized e-learning cultural challenges (pp.
116-129). Hershey, PA: Information Science Publishing.

Collins, A., Brown, J.S., & Newman, S.E. (1987). Cognitive apprenticeship: Teaching the craft of
reading, writing, and mathematics. Technical Report No. 403. Retrieved from
http://www.eric.ed.gov/ERICWebPortal/contentdelivery/servlet/ERICServlet?accno=ED284181

Collis, B. (2008, March 19). Reinterpreting distance: What higher education and corporate
education can learn from each other. Presentation given at The Institute of Higher Education,
the University of Georgia, Athens, GA.

Dede, C.J. (2004). If design-based research is the answer, what is the question? A commentary
on Collins, Joseph, and Bielaczyc; DiSessa and Cobb; and Fishman, Marx, Blumenthal, Krajcik,
and Soloway in the JLS special issue on design-based research. The Journal of the Learning
Sciences, 13(1), 105-114.

Design-Based Research Collective. (2003). Design-based research: An emerging paradigm for

educational inquiry. Educational Researcher, 32(1), 5-8.

Donn, G., & Manthri, Y.A. (2010). Globalisation and higher education in the Arab Gulf States.
Symposium Books. Oxford: Didcot.

Edelson, D.C. (2002). Design research: What we learn when we engage in design. The Journal
of the Learning Sciences, 11(1), 105-121.

Gravemeijer, K., & Cobb, P. (2006). Design research from a learning design perspective. In J.
van den Akker, K. Gravemeijer, S. McKenney, & N. Nieveen (Eds.), Educational design
research (pp. 17-51). New York: Routledge.

Hung, W., Bailey, J.H., & Jonassen, D.H. (2003). Exploring the tensions of problem-based
learning: Insights from research. New Directions for Teaching and Learning, 2003(95), 13-23.

Karabenick, S.A., & Moosa, S. (2005). Culture and personal epistemology: US and Middle
Eastern students’ beliefs about scientific knowledge and knowing. Social Psychology of
Education, 8, 375-393.

Lave, J., & Wenger, E. (1991). Situated learning. Cambridge, UK: Cambridge University Press.

Lincoln, Y.S., & Guba, E.G. (1985). Naturalistic inquiry. Beverly Hills, CA: Sage.

McKenney, S., Nieveen, N., & Van den Akker, J. (2006). Design research from a curriculum
perspective. In J. van den Akker, K. Gravemeijer, S. McKenney, & N. Nieveen (Eds.),
Educational design research (pp. 67-90). New York and London: Routledge.

 847
Muukkonen, H., Hakkarainen, K., & Lakkala, M. (1999). Collaborative technology for facilitating
progressive inquiry: Future learning environment tools. In C. Hoadley & J. Roschelle (Eds.),
Designing new media for a new millennium: Collaborative technology for learning, education,
and training (pp. 406-415). Presented at the CSCL ’99: The Third International Conference on
Computer Support for Collaborative Learning, Mahwah, NJ: Lawrence Erlbaum Associates.
Retrieved from http://www.helsinki.fi/science/networkedlearning/texts/muukkonenetal1999.pdf

National Research Council. (2002). Scientific research in education. Washington, D.C.: National
Academy Press.

Phillips, D.C. (2006). Assessing the quality of design research proposals: some philosophical
perspectives. In J. van den Akker, K. Gravemeijer, S. McKenney, & N. Nieveen (Eds.),
Educational design research (pp. 93-99). New York and London: Routledge.

Porcaro, D. (2011). Applying constructivism in instructivist learning cultures. Multicultural

Education & Technology Journal, 5(1), 39-54. doi:10.1108/17504971111121919

Porcaro, D.S. (2011). Reviewing the literature of computer-supported collaborative learning

(CSCL) to determine its usefulness in Omani education development. International Journal of
Education and Development using Information and Communication Technology, 7(3), 102-120.

Porcaro, D.S. (2012). Educational change in Oman: A design research study of personal,
institutional, and societal reactions to collaborative knowledge building. Manuscript submitted for
publication.

Porcaro, D.S., & Al-Musawi, A.S. (2011). Lessons learned from adopting computer-supported
collaborative learning in Oman. EDUCAUSE Quarterly, 34(4).

Reeves, T.C. (2000). Socially responsible educational technology research. Educational

Technology, 40(6), 19-28.

Reeves, T.C. (2006). Design research from a technology perspective. In J. van den Akker,
K. Gravemeijer, S. McKenney, & N. Nieveen (Eds.), Educational design research (pp. 52-66).
New York: Routledge.

Reeves, T.C., Herrington, J., & Oliver, R. (2005). Design research: A socially responsible
approach to instructional technology research in higher education. Journal of Computing in
Higher Education, 16(2), 96-115.

Scardamalia, M., & Bereiter, C. (1993). Computer support for knowledge-building communities.
Journal of the Learning Sciences, 3(3), 265.

Schoenfeld, A.H. (2004). Multiple learning communities: Students, teachers, instructional

designers, and researchers. Journal of Curriculum Studies, 36(2), 237-255.

Seidman, I. (2006). Interviewing as qualitative research: A guide for researchers in education

and the social sciences (3rd ed.). New York: Teachers College Press.

 848
Stahl, G., Koschmann, T., & Suthers, D. (2006). Computer-supported collaborative learning: An
historical perspective. In R.K. Sawyer (Ed.), Cambridge handbook of the learning sciences (pp.
409-426). Cambridge, UK: Cambridge University Press. Retrieved from
http://www.cis.drexel.edu/faculty/gerry/index.html

Stake, R.E. (2000). The art of case study research. Thousand Oaks, CA: Sage Publications.

Stokes, D.E. (1997). Pasteur’s quadrant: Basic science and technological innovation.
Washington, D.C.: Brookings Institution Press.

Tobias, S., & Duffy, T.M. (Eds.) (2009). Constructivist instruction: Success or failure? New York:
Routledge.

UNESCO (2005). Towards knowledge societies. Paris: United Nations Educational Scientific
and Cultural Organisation.

Van den Akker, J. (1999). Principles and methods of development research. In J. van den
Akker, N. Nieveen, R.M. Branch, K.L. Gustafson, & T. Plomp (Eds.), Design methodology and
developmental research in education and training (pp. 1-14). Dordrecht, The Netherlands:
Kluwer Academic Publishers.

Vrasidas, C., Zembylas, M., & Glass, G.V. (2009). ICT for education, development, and social
justice. Charlotte, NC: Information Age Publishing.

World Bank (2008). The road not traveled (MENA Development Report). Washington, D.C.: The
International Bank for Reconstruction and Development/The World Bank. Retrieved from
http://siteresources.worldbank.org/INTMENA/Resources/EDU_Flagship_Full_ENG.pdf

 849
 David Porcaro is a senior instructional designer at Seward Incorporated in
Minneapolis, MN. His work has centered on using technology to improve
teaching and learning in a variety of settings, and the role of culture in adopting
innovative educational practices. He currently leads several teacher professional
development and education technology projects funded by USAID, NSF and
other organizations in Jordan, Malawi, and the US. His dissertation, which was
completed in 2011, was based on field work done in Oman as part of a Fulbright
Fellowship, and was supervised by Thomas C. Reeves and Janette R. Hill. David
received a PhD from the University of Georgia (2011) in Learning, Design, and
Technology, a MPhil from Cambridge University (2004) in Oriental Studies, and a
BA from Brigham Young University (2003) in Near Eastern Studies.

Email: [email protected]

Thomas C. Reeves, Professor Emeritus of Learning, Design, and Technology

at The University of Georgia, has designed and evaluated numerous interactive
learning programs for education and training. In 2003, he received the AACE
Fellowship Award and in 2010 he was made an ASCILITE Fellow. His books
include Interactive Learning Systems Evaluation (with John Hedberg), a Guide
to Authentic E-Learning (with Jan Herrington and Ron Oliver), and Conducting
Educational Design Research (with Susan McKenney). His research interests
include:
educational technology evaluation,
authentic tasks for online/blended learning,
educational design research, and
educational technology in developing countries.

Email: [email protected]