Themes in eLearning, 14, 13-30, 2021

Undergraduate students’ views of and difficulties in online

exams during the COVID-19 pandemic

Gürbüz Ocak1, Gülçin Karakuş2

[email protected], [email protected]

1
Afyon Kocatepe University, Faculty of Education, Turkey
2
Ministry of Education, Sivas, Turkey

Abstract
The coronavirus pandemic, which affected every aspect of life around the world, has led to radical
changes in teaching and learning methods. It is no longer healthy for students being together for a
long time in classroom. For this reason, online education applications have started to be implemented
rapidly around the world. Not only the education process but also the assessment processes have
been moved online. Online exams are in many ways differ from face to face exams. In this context,
the aim of this study was to examine undergraduate students’ views about online exams. A semi-
structured interview form was used as data collection tool. The study group of this research were 55
undergraduate students selected by convenience sampling method and data were analysed using
content analysis. Results of the study revealed that students mostly had technical problems such as
internet connection and sudden log out. Moreover, they also encountered different problems such as
difficulty in being motivated to online exams, lack of time or insufficient feedback.
Keywords: Online exams, undergraduate students’ views, problems

Introduction

Internet technology has been used to support educational settings for many years. For example, it
moved educational activities out of classroom. Students are provided with access to more materials
and gain ability to use internet technologies (White & Hammer, 2000). With transition of lessons from
traditional classroom to online, student and teacher interaction, communication, educational
paradigms, learning processes and assessment techniques are fundamentally changing (Robles &
Braathen, 2002). Educators and educational institutions must adapt to these rapid changes in
information technologies (Çelen, Çelik & Seferoglu, 2011).
Resources available in internet are quite rich, applications are unlimited compared to traditional
resources. This diversity offers learning opportunities in ways that would not be possible with
traditional classroom resources. However, educators aiming to take advantage of extensive resources
and usability of internet, face many problems during inclusion of these new resources into their
classes. One of these problems is assessment which reflects education, practices, how students are
taught and what students learn (Galante, 2002). Although online education technologies are mostly
adopted and used, online assessment processes haven’t been carried out in targeted way yet. For this
reason, educators should also pay attention to assessment of their education in online education
process (Robles & Braathen, 2002).
The COVID-19 pandemic has affected the world and caused education to be halted. As the world’s all
over, pandemic has affected education in Turkey and online education has started. However,
transition to online education suddenly and students’ not being ready for online assessment system
caused some problems. In order to solve problems related to this sudden change, these problems
should be determined. In this context, aim of this study is to determine problems experienced by
undergraduate students’ online exams.
14 G. Ocak, G. Karakuş

Online education

Keegan (1988) defines online education as a process in which teachers and students participate in
educational activities in different places, unlike face-to-face education, individual work is at the
forefront, content is followed in computer environment, and students communicate with their
teachers and each other through computer technologies. Aparicio, Bacao & Oliveira (2016) adds that
online education has three basic elements; people, technologies, and services. In addition to this
definition, Gunawardena, Wilson & Nolla (2003) added a cultural aspect of online education. In online
education, it is important that this communication and interaction, which is not face-to-face should
be appropriate for social culture. For example, as long as student participating in online courses and
answering a question is perceived as normal in that culture, there is no problem for adapting online
education. On the other hand, basic elements of online education cannot be provided as long as online
education is perceived as different, difficult and strange in the society. Moreover, Moreno-Ger,
Burgos, Martínez-Ortiz, Sierra & Fernández-Manjón (2008) focused on the pedagogical needs of online
education such as internal motivation. Keegan’s (1988) definition is the most relevant to the aims of
this study. Although other definitions are contributing to clear understanding and deeper
interpretation, this study generally focuses on online education and online assessment specifically
Keegan’s (1988) definition is therefore more relevant to current study.
Online education activities are increasingly used in educational setting, and it is aimed to develop
traditional educational methods (White & Hammer, 2000) and to contribute to solutions of some
problems encountered in traditional education such as difficulties experienced due to education in
very crowded classrooms or inability to listen to lesson again (Karaağaçlı, 2002). Moreover, online
education both changed basic structure of information exchange and enabled teachers to make
changes in teaching techniques (White & Hammer, 2011). Today qualified online education is much
more important that it should be at same level as face-to-face education (Dirks, 1998). Online
education has benefits such as flexibility, ease of access to education at appropriate time, and learning
anytime, anywhere (Robles & Braathen, 2002; Keskin & Guneş, 2015). However, in addition to these
facilities, it has some shortcomings also. In online education, since instructor and students are not
physically in same environment and face-to-face communication cannot be achieved, assessment is
done differently. Unless student is actively participating in something in virtual classroom, teaching
becomes difficult. It is important for student to participate in activities such as doing homework,
performance task, quizzes or group work. This situation requires use of alternative assessment
methods and assessment process needs to be more transparent and support learning since
assessment helps determining quality of learning (Arend, 2007). Literature review indicated that
researchers focused on different aspects of online education such as instructor and student views on
online education applications (Tanyıldızı & Semerci, 2005; Torkul, Kibar & Tasci, 2004; Tufekci, Ekinci
& Kose, 2013), factors affecting online education (Aragon & Johnson, 2008), student learning
strategies in online courses (Arend, 2007), problems in online education (Çelen, Çelik & Seferoglu,
2011), technology in online education (Hillier & Fluck, 2017) and motivation in online education
(Kawachi, 2003). In these studies, the time shortage of instructors, unethical behaviours in online
education, lack of information about online education, and deficiencies in technology literacy are
generally emphasized. Some other studies focus on online assessment which are presented below.

Online assessment

Online assessment is defined as web based or intranet assessment (Ayo, Akinyemi, Adebiyi & Ekong,
2007) and is more difficult than traditional assessment. Students’ attendance, doing assignments,
participating in courses actively are recorded in technological environment. However, the important
thing is to be able to determine what and how much student learned with online education. In online
assessment before developing an effective assessment tool, instructor must have a necessary
Undergraduate students’ views of and difficulties in online exams during the COVID-19 pandemic 15

knowledge of course content and academic competence. Since teaching and presentation method
changes, instructors also need to change their methods of determining teaching and learning
effectiveness, and assessment activities (Robles & Braathen, 2002).
Web-supported applications in online assessment process offer many conveniences for instructors.
For example, student responses can be recorded in a digital file and feedback can be given easily.
Student answers can be compared. Statistical calculations of correct and incorrect answers can be
figured faster (Dickinson, 1997). Some other advantages of online assessments are; using more visuals,
not using too much paper, easy to analyse results, and conducting exams from one centre (Hols-Elders
et al., 2008). In online assessments, multiple choice questions, fill-in-the-blanks questions, matching
questions, short answer questions, numbering questions, right-wrong questions, etc. can be used.
Other assessment methods can be in the form of forum and homework modules, in-class activities,
end-of-term assessment questions or final written exams (Kınalıoğlu & Güven, 2011). Literary review
of online assessment revealed that while some studies focus on online assessment (Dirks, 1998;
Galante, 2002; Shraim, 2019), some others focus on comparison of online assessment and traditional
assessment (Luecht, Hadadi, Swanson & Case, 1998; Robles & Braathen, 2002; Ryznar, 2020; Sarrayrih
& Ilyas, 2013; Shraim, 2019; Stowell & Bennett, 2010) and security in online exams (Cluskey, Ehlen &
Raiborn, 2011; Alessio, Malay, Maurer, Bailer & Rubin, 2017; Colby, 2018).
Online education and online assessment were implemented before COVID-19 pandemic. However, it
was conducted in a limited number of lessons and limited hours. With Covid pandemic, online
education started to be implemented in all courses and at all grade levels. Depending on quarantine,
online education is now carried out everywhere. This situation obliged universities to review their
previous infrastructure and facilities and to make necessary changes. Therefore, online education
after the COVID-19 outbreak is different.

Online assessment in Turkey

In Turkey, online assessment applications have started to gain momentum in recent years. In
particular, the COVID-19 pandemic has made it necessary to accelerate this process and identify
problems encountered.
Depending on decision of government, in all levels, face to face education in Turkey was suspended in
March 2019. After then online education started. For about 10 months, online education was
conducted in universities. During period of March 2020 and June 2020, since universities were not
prepared for online education, online courses were asynchronous. Course documents were uploaded
to the system. Lessons were supported with homework and problems arose from time to time. In
September 2020, many universities switched to synchronous education in order to be more
professional and to further reduce technical problems. In Turkey, online assessments are mostly
conducted in the form of multiple choice questions. And they were carried out using Adobe Connect
and Microsoft Teams systems. However, there were also assessment methods made in the form of
homework. While one midterm exam and one final exam are held in traditional exams, online
assessment is made as two midterm exams and one final with the decision of universities in online
assessment. However, instructor of that course decided the type of exam in general. The rapid
transition to online education due to the pandemic caused many problems in online assessment
process. In online assessment, similar to face to face exams, some problems can be seen such as
insufficient knowledge about assessment processes, instructors’ lack of time, insufficient assessment
studies in institutions (Çallı, Torkul & Taşbaşı, 2003). In their study Keskin and Güneş (2015) suggested
a security system for online assessment. Similarly, Çallı, Torkul & Taşbaşı (2003) suggested online
assessment system designs and Bozkurt and Uçar (2018) suggested biometrics for identity verification.
In these studies, assessment and security in online exams are emphasized. Literary review also
indicated that there are some studies about students’ views on online education. Kuo, Walker, Belland
16 G. Ocak, G. Karakuş

& Schroder (2013) found that learner-instructor interaction, learner-content interaction, and internet
self-efficacy were good predictors of student satisfaction while interactions among students and self-
regulated learning did not contribute to student satisfaction in online education. In their study Yang
& Cornelius (2004) state that delayed feedback from instructors, unavailable technical support from
instructors, lack of self-regulation and self-motivation, the sense of isolation, monotonous
instructional methods, and poorly-designed course content were negative results of online education.
Tello (2007) analysed student persistence in online education and addressed online instructional
techniques, faculty development, technology development, and program development. Most
research has focused on students’ views on online education, but there is still a lack of robust research
on undergraduate students’ views on online assessment.
With this study, problems experienced in online assessment process can be determined by obtaining
student views on online assessment process and measures can be taken to eliminate these problems,
so that quality and effect of online education can be increased, universities can develop their policies
for online education and create necessary regulations. In addition, reasons for students’ behaviour,
attitudes, and perspectives towards online course can be better understood in online education.
Determining difficulties undergraduate students experience in assessment will also shed light on
difficulties experienced in online education during pandemic periods. For this reason, this study aims
to examine undergraduate students’ views about online exams and expected to contribute to
literature.
The research problems of this study are:
 What are the difficulties experienced by undergraduate students in the preparation process for
online exams?
 What are the difficulties that undergraduate students encounter during online exams?
 What are the measures that can be taken to solve the problems encountered in online exams?

Method

Because the aim was to examine undergraduate students’ views for online exams more deeply,
research methodology of this study was a case study. In a case study, researcher is primarily focused
on understanding a specific individual or situation (Fraenkel, Wallen & Hyun, 2012). In this study, aim
is to understand students’ views about problems they encountered in online exams, to examine this
specific case more deeply with a specific group of participants (in this study undergraduate students),
to obtain in-depth data about problems in online education in a real life situation. For this reason, this
study is a case study.

Study group

The study group consists of 55 undergraduate students studying at two different universities (one of
them is from the east of Turkey, Sivas Cumhuriyet University and the other one is from the west part,
Afyon Kocatepe University). For sampling, convenience sampling method was used. In convenience
sampling method, researcher chooses a situation that is close and easy to access. The probability of
participants being selected as a study group is unknown (Dawson & Trapp, 2001).
Before study was conducted, participants were given detailed information about process by pre-
interviewing. Ethics committee approval and participant approval were obtained for study. In this
study, undergraduate students were chosen as a study group. The reason is that in Turkey online
exams were held only at universities, not at other grade levels (pre-K12). Since problems experienced
in online exams are the focus of study, the fact that participants have taken online exams is main
Undergraduate students’ views of and difficulties in online exams during the COVID-19 pandemic 17

element of sample selection. Moreover, this study included education faculty undergraduate
students. Undergraduate students were selected from different departments, aiming to provide as
much diversity as possible in study group. Undergraduate students were from Science (15),
Mathematics (13), Social Studies (14) and Primary School (13) preservice teachers. For grade levels,
seniors, sophomores and juniors were included, freshmen were not included. Since they had not yet
taken face-to-face exam at university and therefore they could not make comparison between face to
face exam and online exams they were not included. Students in study group have attended online
exams using online exam systems at least twice. This study is limited to opinions of participants.

Data collection tool and process

Data of this study was collected via semi structured interview form developed by researchers. A semi
structured interview was conducted during research process. With interviews, the aim was to enable
participants to detail their views and to reach more elaborated data. Interview helps researcher
achieve more in-depth, complex and difficult information about participants’ views (Kumar, 2005).
With eight open ended questions included in semi structured interview forms, students were given
freedom of expression and opportunity to make comments. Questions in semi-structured interview
form were prepared by researchers on the basis of literary review of studies and research problems
of this study. Two experts in the field of qualitative research were asked to examine questions and
expert views were received. Based on expert opinion, one question depending on the view that it does
not fully reflect purpose of research was removed and one question was shortened. Forms and
expressions in two questions were changed to be more understandable (Walton, 2010). Pilot study
was carried out with two students. Interview form has been finalized according to expert opinions and
results of pre-application. The first three questions in semi-structured interview form were prepared
in order to determine difficulties students encounter during exam. The fourth, fifth and sixth questions
were aimed at determining difficulties they encountered during exam. The last two questions were
prepared in order to determine thoughts of undergraduate students on the solution of these
problems. The study was conducted in fall semester of 2020-2021 academic year for two weeks
according to pre-prepared interview schedule. Interviews were conducted online in order to prevent
risks that may occur due to the pandemic. Each interview with undergraduate students lasted an
average15-20 minutes. One of researchers was also an interviewer. Participants were also informed
about this. Audio recordings were recorded after getting participant permissions. At the end of each
interview, audio recordings were transcribed and made ready for analysis.
Sample questions from semi-structured interview form:
 “What kind of problems do you encounter before online exams?”
 “How do you prepare for online exams?”
 “What are the most common difficulties you encounter during online exams?”
 “What kinds of problems prevent successful application of exam?”
 “What can be done before online exam so that online exams run smoothly?”
 “What precautions should instructors or institutions take before online exams to make exam
process more comfortable for students?”

Data analysis

Data collection and data analysis processes were carried out together. The three-stage qualitative
analysis method of Miles and Huberman's (1994) was used in data analysis. These stages are: "Data
condensation", "Data display" and "Conclusion drawing/verification". Data condensation was
collecting data. Data display stage included determining similar sentences by examining raw data.
18 G. Ocak, G. Karakuş

Then codes themes and categories were formed. In verification stage deductions were structured. In
conclusion results were compared to studies in literature. Data were analysed after each interview.
The study was carried out by taking codes from textual expressions as a conventional qualitative
analysis. Without making assumptions about codes and categories, researchers analysed content of
data (Merriam, 2001). Data analysis was mainly carried out to interpret data.
Before data analysis participants were given codes as P1, P2, P3. Researchers first read data many
times and formed a general perspective. Then, data was analysed deeply. The focus was on explicit
and implicit expressions of participants. Key concepts stated in participants’ expressions have been
identified. Codes suitable for each key concept were determined and expressions were classified
according to these codes. Categories were determined from codes that have common features.
Categories were compared, and themes were created considering their similar and different features
(Pazargadi, Khatiban & Asktorab, 2009; Schwandt, 1997). Codes, categories and themes were
discussed continuously. Researchers peer checked results. To illustrate important parts of interviews
quotations were given. The aim in content analysis is to reach connections among expressions and
basic concepts (Patton, 2014). Themes and categories obtained from data through research questions
of study were presented in findings section according to each research question. In coding for
instance, it was concluded that the most repeated expressions of the first research problem were
“checking”, “lack of”, “not reaching” and “not knowing” expressions. After these statements were
determined as codes, it was determined that all these codes constitute “precaution” theme before
online exam. Considering problems experienced by undergraduate students before online exam with
codes and categories, it was decided to present these findings in “pre-exam stress” theme.
In this study main purpose of data analysis was to present findings in detail, clearly and succinctly.
Content analysis was made through themes and codes obtained. A general conclusion has been
reached from codes and themes. Care was taken not to present data directly in a descriptive manner,
and quotations were only included to increase credibility. For the reliability of study, views of
researchers were compared and similarities and differences were determined. The following formula,
proposed by Miles and Huberman (1994), was used to determine agreement between researchers:
“Percentage of compromise = [consensus / Na (consensus) + Nd (disagreement)] x 100”.
As a result, compliance percentage of study was calculated as 82%. According to Miles and Huberman
(1994) compliance percentage as 70% and above indicates that study is reliable.

Findings

Findings obtained indicate three main themes; pre-exam stress (three categories), application process
(three categories) and suggestions for solving online exam problems (five categories). According to
the first research problem of “What are the difficulties experienced by undergraduate students in
preparation process for online exams?” Pre-exam Stress theme extracted from interviews and
Informing Problems, Problems in Preparing for Exam and Precaution categories were specified.
Table 1 indicates that in Precaution category students control hardware, software and social
environment. In Problems in Preparing for Exam category, students have problems with Lack/
Insufficiency of Course Documents/Materials, Technological Tools, Affective Features and Time. In
Informing Problems Category, students have problems about exam details as duration, variability and
scope of exam.
A sample view from checking course document is given below:
P11: “Here I have advantage of re-watching online lessons and learning topics I don't understand. I
follow these and prepare all notes I have taken, if I have missing lecture documents, I complete them
and work before exam.”
Undergraduate students’ views of and difficulties in online exams during the COVID-19 pandemic 19

Table1. Basic themes, categories and codes regarding the preparation process for online exams
Theme Category Sub Category Code Participants f
Checking course P3,P6,P8,P11,P12,P14,P16,P17,
materials/documents (word/pdf) P20, P28, P35, P46,P47,P53 14

Checking whether technological P3,P7,P11,P12,P14,P16,P32,P47,

Hardware/ tool to be used is working or not P55 9
Software
Precaution

Checking internet connection P8,P10,P17,P27,P39 5

Providing technological tool in P7,P14,P22,P39
advance 4

Creating a silence environment P12,P26,P34 3

Social Control Warning family members about P5,P9
using Internet unnecessarily 2

Lack of course documentation P5,P6,P11,P23,P36,P50,P51

Lack/ 7
Insufficiency Not reaching course notes P5,P8,P18,P24,P33,P51 6
of Course
Documents/ Not reaching instructor for P6,P10,P14,P37,P48
Materials resources 5

Internet problem P1,P4,P5,P11,P16,P23,P24,P35,

14
P39,P41,P45,P50,P52,P53
Not knowing how to use P3,P16,P19, P25,P33,P47,P54
Problems 7
computer program for exam
Regarding
Problems in Preparing for Exam

Problems arising from home P1,P4,P10,P22,P35,P46

Technological
environment (Power cut-Sound 6
Tools
Pre-Exam Stress

etc.)
Insufficient number of computers P2,P11,P16,
3
Not focusing on online exams P5,P8,P11,P13,P14,P19,P20,P31,
P34,P41,P49 11

Affective Not benefitting from online P7,P12,P17,P18

Features exams 4

Inability to study for online exam P1,P9,P13

3
Late loading / not uploading of P9,P22,P31,P49,P54
course materials to system by 5
instructor

Time Downloading course materials P8,P17,P21,P55

over a long time 4

Access to course materials in a P10,P14

long time due to system 2

Insufficient information about P6,P11,P28,P21,P29,P30,P51,P27,

scope of exam P31,P36,P40,P45,P49,P52 13
Informing Problems

Variability in exam date P3,P17,P22, P32,P44,P45,P52,P55

8

Informing Lack of information about exam P9,P12,P15,P18,P20,P23

6
About Exam
Not providing information about P2,P3,P5,P9
the duration of exam 4

Providing information only about P8,P16,P20

the type of exam 3
20 G. Ocak, G. Karakuş

The following samples of student view concern the categories of Problems with the Technological
Tools, Lack of course material and Information about the exams:
P35: “I check internet many times. I take care that my phone works.”
P 5: “I was in trouble to find enough resources to study for exam. Questions in exams can be either very
difficult or very easy. Therefore, I had to work from many sources, but the resources I could reach are
not enough.”
P45: “Information was given about the date of exams, but when exam week approached, dates exams
were changed and moved earlier, and these changes were notified late.”
In terms of the second research question, regarding students’ difficulties during the online exams, the
Application Process theme and the categories of Technical Issues, Structure of Exam and Personal
Problems were specified (Table 2).

Table 2. Basic themes, categories and codes about the application process for online exams
Theme Category Sub Category Code Participants f

P2,P7,P11, P13,P15,P19, P21,

Inability to connect to internet 9
Internet P20,P42
Connection
Internet outage problem P5,P7,P12,P20,P29,P32,P35,P42 8
Problem
Insufficient internet quota P5, P7, P10, P17 4

Inability to return to previous P5, P7, P8,P11,P14,P15,P18,P22,

13
exam questions P23,P25,P30,P33,P34
Not being able to enter exam
P5,P6,P13,P19,P32,P50 6
because of system density
Technical Issues

Having difficulty to view

P6,P15,P28,P36,P49 5
System Related questions
Problems
Missing questions P4,P7,P19,P26 4

Expulsion from system P4, P11, P12 3

Incorrect coding of answers

P1, P9, P38 3
Application Process

(due to system)
Problem of recording answers P1, P15 2
Software deficiencies of exam
P22,P37 2
Problems With device
Device Hardware deficiencies of exam
P5 1
device
Often having multiple choice
P7,P28,P39,P44,P48 5
questions
Online exam questions being
Features of more difficult than classical P10,P31,P49,P52 4
exams
Structure of Exam

Question
Questions from same topics P3,P27,P55 3
Questions not compatible with
P6,P14 2
course content
Limited time to respond P1,P4,P5,P7,P8,P12,P13,P16,P23,
16
questions P24,P32,P33,P41,P47,P53,P55
Duration
Inability to view questions in
P7,P11,P12 3
sufficient time
Undergraduate students’ views of and difficulties in online exams during the COVID-19 pandemic 21

Table 2. (Continued)
Theme Category Sub Category Code Participants f

Inadequate time due to slow

Duration P11,P24,P39 3
operation of system
Inability to convey problem
Structure of Exam

experienced in exam to P6,P9,P16,P23,P24,P45,P54 7

Communication instructor

Lack of face-to-face interaction P10,P16,P19,P24,P38,P46 6

More likely to cheat in online

P4,P13,P37,P48 4
exam
Exam Security
Keeping notebooks and books
P7,P13,P17 3
open
Concern about whether
P3,P5,P8,P14,P15,P20,P28,P31,
answers were recorded 9
Application Process

P52
correctly
Pedagogical
Concentration problem in
Effectiveness P11,P14,P26,P29,P34,P49 6
online exams
Feeling of not fully learning the
P2,P8,P15 3
subject in online classes
Personal Problems

More excitement in online P6,P9,P10,P15,P22,P27,P34,P36,

11
exams P43,P44,P47

Feeling demoralized by
technical problems during P1,P5,P13,P23,P35,P48,P49,P54 8
exam
Motivation-
Anxiety Less attention to online exam P4,P10,P16,P23,P35,P40 6

Increasing online exams anxiety P8,P13,P14,P22 4

Stress of having an exam in

P6, P27 2
front of camera

Table 2 indicates that in Technical Issues category students have problems with Internet connection,
system and device. In structure of exam category students reported problems with the features of
questions, duration and communication and exam security. In personal problems category students
have problems as pedagogical effectiveness and motivation-anxiety.
Students’ views about the sub categories of Internet Connection Problem and System Related
Problems is the following:
P 7: “I take care to have an internet quota in order not to have any problems with online exam. I attend
exams on phone. I pray that not to have calls during exam, and I try to deal with systemic difficulties.”
The transcript below was classified to the Duration category:
P4: “The unfairness of exam is challenging me. Also, as a student of a mathematics department, it
takes a lot of time to solve and save question. Even system that sends questions mixed, sometimes only
sends difficult questions and one type of questions. The time given for us to answer questions is not
enough.”
Student view from Feeling demoralized by technical problems during exam code is given below:
P35: “When I have a problem with computer during exam, I get very excited and this situation is very
demoralizing for me.”
22 G. Ocak, G. Karakuş

The following example represents student concerns if their answers were recorded correctly:
P5: “During the exam, sometimes the system was very intense, so answers to questions may not be
recorded, internet may be cut off. So when I have this kind of problem I wonder whether my answers
were recorded correctly.”
View from Inability to convey problem experienced in exam to instructor code is given below;
P24: “I take a screenshot during exam and send an e-mail to instructor after exam. Unfortunately,
when there is a problem during exam, there is no solution at that time”.
According to the third research problem of “What are measures that can be taken to solve the
problems encountered in online exams?”, Suggestions for Solving Online Exam Problems theme
extracted from interviews and Assessment, Technical System, Content of Exam, Communication, Time
categories were specified (Table 3).
Table 3 indicates that in assessment category students suggest assigning homework, fair distribution
of questions, announcing exam earlier, asking short answer questions and make-up exams. In
Technical System category students suggest returning to previous questions in exam, taking measures
in advance for problems related to system, strengthening the substructure of system, controlling
technological device to be used in exam, checking internet connection, reducing number exams held
at the same time and doing a short mock test before exam to determine if system is working. In
Content of Exam category students suggest providing necessary information, ensuring compatibility
of exam questions with lecture taught in distance education, being informed about all the details
(content, type, duration of the exam, etc.), taking exam in a small number of groups, not including
unfinished subject to exam content, checking exam items in randomly assigned questions. In
communication category students suggest reaching correct answers, providing sufficient feedback,
having contact with lecturer during exam. In time category they suggest more time to complete exam.
The transcript below represents an example of students’ views about;
Assigning homework code:
P42: “I don't think online exams are fair. I think it is not suitable for equal opportunity in education. I
believe that having final exams with homework as last year will be better especially for students with
limited opportunities.”
Returning to previous questions in exam and Fair distribution of questions as easy and difficult:
P43: “Practices such as not being able to return to questions have really worn out many of my friends,
especially me. I always wonder if questions are too difficult, how long should I devote to this question,
I find myself in an unnecessary stress. In some exams, when my first question is a difficult question, I
lose all my enthusiasm and I have to worry about education. This process was really exhausting.”
The transcript below shows an example of students’ views related to the Contact lecturer code:
P15: “An arrangement should be made in order to contact to instructors faster during exam.”
The next transcript indicates students’ need for feedback to their responses and mistakes after the
exams:
P37: “There should be a return between questions and we should see our mistakes after exam. After
all, if we do not know what we are doing wrong, we will continue with mistakes we know are right.”
Following, an indicative example of students’ need for providing necessary information before the
exam is given:
P2: “Assessment should be made before exams and it should be ensured that time will be sufficient for
questions. Information should be given about the number of questions and their content.”
Undergraduate students’ views of and difficulties in online exams during the COVID-19 pandemic 23

Table 3. Basic themes, categories and codes regarding problems encountered in online exams
Theme Category Code Participants f

Assigning homework instead of multiple P4,P6,P8,P9,P10,P11,P13,P14,P17,P21,

18
choice exams P23,P25,P26,P30,P34,P38,P41,P42

Fair distribution of questions as easy P1,P3,P5,P6,P21,P29,P30,P32,P43,P49

10
and difficult
Assessment
Not announcing exam results too late P6,P9,P14,P16,P22,P38,P44 7

Asking short answer questions P4,P7,P13,P26,P34 5

Make-up exams should be done P5,P17,P28 3

Returning to previous questions in exam P1,P2,P4,P5,P11,P13,P16,P17,P22,P25,

17
P26,P37,P38,P39,P43,P53,P54,

Universities should take measures in P3,P4,P7,P11,P16,P17,P21,P24,P29,

14
advance for problems related to system P30,P32,P36,P39,P47

Strengthening substructure of system P2,P3,P7,P13,P17,P19,P22,P25,P36,

11
P42,P51
Technical Controlling technological device to be P3,P8,P9,P15,P28,P33,P49
System 7
used in exam
Suggestions for Solving Online Exam Problems

Checking internet connection. P7,P12,P28,P31,P45 5

To reduce the number of exams held at P9,P17,P23

3
the same time

Doing a short mock test before exam to P32,P55

determine if system is working 2

Providing necessary information before P2,P3,P7,P8,P11,P2P4,P16,P21,P23,P2,

18
exam P32,P33,P34,P42,P45,P50,P52
Compatibility of exam questions with P7,P18,P25,P29,P36,P44,P47,P52,P55
9
lecture taught in online education

Students should be informed about all P5,P8,P19,P21,P38

Content of the details (content, type, duration of 5
Exam the exam, etc.)

Takin exam in a small number of groups P11,P42 2

Not including unfinished subject to P4,P18,

2
exam content

Checking exam items in randomly P2,P36

2
assigned questions

The opportunity to reach correct P1,P4,P8,P16,P20,P22,P23,P27,P29,

16
answers should be offered after exam. P32,P33,P37,P45,P52,P54,P55

Sufficient feedback should be provided P4,P5,P7,P17,P25,P28,P29,P32,P33,

15
by instructor about exam P36,P47,P49,P50,P51,P54
Communication It should be possible to contact P9,P15
2
instructor during exam

If there is a problem during exam, P7,P17

feedback should be obtained for 2
solution

Time Longer time to complete exam P6,P14,P23,P40,P52 5

24 G. Ocak, G. Karakuş

Conclusions

During preparation process of online exams, the most frequently expressed problem by
undergraduate students were checking course materials/documents, internet problems, not focusing
online exams and insufficient information about exams. Less expressed problems were checking
whether technological tool to be used is working or not, lack of course document and variability in
exam date. It is important for students to control hardware, software and social environment before
exam. The reason for this may be that students want everything ready before exam in order not to
encounter any problems during exam, and these checks make them feel more comfortable. Certainly
course materials facilitate learning and ensure its permanence (Bozpolat & Arslan, 2018). The more
qualified and problem-free software and hardware in learning environment, helps effective education
(Hentea, Shea & Pennington, 2003).
Internet related problems can be considered as the worst problem since it’s out of control of students.
For example, loading of exam questions can be delayed, students may not find exam questions on
webpage and for students there is nothing to do at that moment. By all means, after studying long
hours, these kind of trouble is a problem. This situation may cause student not to attend online exam
or exam may be declared invalid (Çallı, Torkul & Taşbaşı, 2003). Internet connection and technical
infrastructure problems related to system should be resolved for effective teaching and assessment.
Similar to this finding, internet related problems and necessity of solving these problems are stated in
literature frequently (White & Hammer, 2000; Çelen, Çelik & Seferoglu, 2011; Kınalıoğlu & Güven,
2011).
Students have problems about focusing on online exams. The reason for this may be that students are
in a different place and communication is more difficult in online exams, unlike traditional assessment.
Due to difference in time, place and features, a new and different exam for student may not be
adopted very quickly. Being assessed with an exam different from exams students used to attend for
many years is a difficult process. Instructors need to know these pedagogical features of student and
prepare and apply online exams accordingly. Only in this way online assessment can be used
effectively both for learning objectives and for applying knowledge. The instructor should know
pedagogical reasons of not focusing on online exams and they should also use assessment techniques
accordingly or existing traditional assessment techniques can be modified to be suitable for online
assessment (Robles & Braathen, 2002). Kerka & Wonacott (2000) claim that online assessment is more
difficult than face-to-face assessment since student’s non-verbal cues cannot be seen online. In online
exams student often feels isolated, thinks that assessment process is not personalized and that
assessment is out of teacher’s control. Student also feels anxious about technical problems he may
experience during exam. Moreover, constantly looking at computer screen may cause problems for
student to concentrate after a while.
Another problem for students is insufficient information about exams. Getting information about
exam, knowing which content is included in exam helps draw the boundaries of exam and facilitates
exam preparation process. Mostly students want to know what subjects will be included in exam, how
many questions will be asked, and how much time will be given for answers. Otherwise, if exam
includes an uncertainty for student, both anxiety level increases and desire to work decreases. For this
reason, students should be informed beforehand about online exam system and its usage, so they
know what they can do at the time of exam (Torkul, Kibar & Tasci, 2004). To solve this problem,
moreover, students should have opportunities to contact with instructor. In fact, especially with
developing technology, students can communicate more easily with instructor. From time to time, it
will also be useful for students to meet with instructor (Hentea, Shea & Pennington, 2003) and have
information about exam details (Tüfekçi, Ekinci & Köse, 2013). To make online education to be
permanent, interaction between students and educators should be increased (Celen, Çelik &
Seferoğlu, 2011).
Undergraduate students’ views of and difficulties in online exams during the COVID-19 pandemic 25

Moreover, as another problem variability in exam date has also a negative effect for students.
Students are disturbed by sudden change in exam date, because this change may not be suitable for
students’ own study schedule. An early date for an exam to be held on a later date may result in
student’s inability to complete his studies or delay in exam date to may cause student’s effort to be
wasted. Student may think that they will forget what they know or work in vain too early.
Students also need to check whether technological tool to be used is working or not before exam. The
reason for this situation is a desire not to experience negative impact of any technological problem
during exam. It is obvious that online education requires much more time and technology than
traditional education (White & Hummer, 2000). So students should be provided with necessary
knowledge. Hillier & Fluck (2017) claim that technological systems used in online exams must be
flexible, common and compatible. Technological tools’ quality creates greater success of online exam
system.
In application process of online exams, problems the most frequently expressed by undergraduate
students were inability to return to previous question, limited time to respond questions and more
excitement. Less expressed problems were feeling demoralized by technical problems, concern
whether answers were recorded correctly, inability to convey problem experienced in exam to
instructor.
In application of online exam, students complain about inability to return to previous question and
they also suggest to return to previous question. They also have problems with exam duration. In
traditional exams, when a student cannot find the answer to a question at the time of exam, he passes
that question and returns to question at the end of exam. Exam duration is long enough for revision
of answers. However, it is an important disadvantage for a student to see question only once in online
exam and not to return to question he could not answer. The reason of not allowing to return to
previous questions and giving limited time for answers may be possibility of cheating. Students are
likely to cheat in online exams. For example, they can communicate with each other on social media
during exam or they can search for answers to questions asked in exam from another laptop (Ryznar,
2020). Today, technological developments cause students to cheat more easily in exams. In addition,
another reason is that student can search for answer to unanswered question by using another
browser or technological device. He can go back and answer that question before exam finishes. This
harms exam security considerably. Instructors and universities can control these unethical behaviours
for example, by using cameras during online exams or they can check whether student has taken exam
himself or not from their IP addresses (Cerimagic & Hasan, 2019). Although it is a problem for student
to not return to previous questions, ensuring exam security is the first priority for institutions, equality
and justice for all students should be given importance in online exams (Robinowitz, 1995; as cited in
Robles & Braathen, 2002; Shraim, 2019; Williamson, 2018).
Students stated that they feel more excited in online exams compared to traditional exams. In online
exam, student attend exam without face to face communication, in a separate environment from his
friends and instructor, and in a relatively shorter time so, if he encounters any problem, he may worry
that he will not be able to express himself and may feel uneasy. As students stated lack of
communication is an important deficiency of online exams. Hentea, Shea & Pennington (2003) and
Kınalıoğlu & Güven (2011) claim that lack of face to face communication affects both distance
education and assessment negatively. Similarly, Aragon & Johnson (2008) and Bunn (2004) stated that
poor communication prevents students attending online education programs. They view online
communication as incomplete or ineffective. They consider lack of face-to-face communication as a
barrier which creates anxiety. Moreover, different environment can be the most important reason for
online exam anxiety (Tüfekci, Ekinci & Köse, 2013). Stowell & Bennett (2010) stated that students who
have exam anxiety in face-to-face exams, increase their anxiety levels more in online exams. And they
also suggest doing online exams in classroom can be effective in face-to-face education process to
reduce online exam anxiety. Infect, it is possible to increase success of student in online exam because
26 G. Ocak, G. Karakuş

with online exam, student can perform better by taking exam at any time according to his / her sleep
routine and mood (Hartley & Nicholls, 2008).
In a study conducted by DeSouza & Fleming (2003), students who attended online exams were more
successful than those who attend exams in classroom. Another reason for excitement in online exams
can be related to online courses. In this study, some students stated that they felt as if they had not
fully learned subjects in online courses they think they are not ready for online exam. So, they feel
anxious. In a study conducted by Fakcharoenphol, Potter & Stelzer (2011) students with high academic
success were more eager to take online exams. So, a student with low academic success may not want
to take online exam, fearing that she may get low marks in online exam. A student who thinks she has
learned will have no problem taking online exam.
Students who are worried about online exam may also feel nervous due to problems such as technical
difficulties and connection issues. As stated above findings of this study may cause anxiety; that feeling
demoralized by technical problems, concern whether answers were recorded correctly and inability
to convey problem experienced in exam to instructor. Minimizing these problems can also reduce
anxiety in online exam. And midterm exams can be done online as formative assessment, they can
help students see their weaknesses and strengths and reduce their anxiety (Fakcharoenphol, Potter &
Stelzer, 2011).
Not having contact with instructor is another problem. Instructor is very effective as a connection
between student and course, can solve any problem related to exam, end student’s worries, and affect
student’s attitude towards course. Therefore, instructor is the most competent person regarding
exam. Students want to reach instructor about a problem related to exam at any time. Aragon and
Johnson (2008) claimed that not having contact with instructor can affect online education negatively
as withdrawing from online course. To solve this problem, Kashy, Thoennessen, Tsai, Davis & Wolfe
(1997) stated that technological opportunities can be used to have contact with students and
instructors. Certainly, it is not possible for all students to reach instructor, but by determining class
representatives, problems about online exam can be delivered to instructor with technological tools.
The effective use of technology in educational environment provides instructors with considerable
convenience in terms of time, grading and recording, while increasing success of student.
In suggestions to solve these problems, the most frequently expressed suggestions were, assigning
homework instead of exams, returning to previous questions, providing necessary information before
exams, sufficient feedback from instructor, strengthening substructure of system, fair distribution of
easy and difficult questions.
Students stated that they prefer homework instead of multiple choice questions. The reason for
instructors for using mostly multiple choice questions can be usefulness of multiple choice exams. In
online exams, questions and answer options can be designed to be different for each student, but this
is more time consuming in traditional exams. In addition, educators make exam analysis easier
(Dickinson, 1997). For example, after a multiple-choice online exam, results of exam can be projected
on a screen that students can see after a short time, and answers to questions can be examined.
Student can immediately get feedback about exam. At the same time, this situation provides
convenience for instructor in terms of time and effort (Wolfson, Veloski, Robeson & Maxwell, 2001;
Dirks, 1998).
Suggestions about returning to previous questions and providing necessary information before exams
were mentioned above. Another suggestion was having sufficient feedback from instructor. As an
important problem it is clear that students need to get feedback after online exams. The students’
making this suggestion shows that they could not get enough feedback. This may be due to high
workload of instructors after online exams, students not being able to go to campuses due to the
pandemic, and not being able to meet instructor. The educational setting can be face to face or online,
but assessment should provide immediate feedback to student. In this way, student can understand
that he has learned subject or not. In online exams, instructors need to make their assessment
Undergraduate students’ views of and difficulties in online exams during the COVID-19 pandemic 27

according to a specific strategy and make sure that feedback given to student is effective and
understandable by student (Arend, 2007). The basic criteria of online courses are to achieve goals of
course, and main purpose of online assessment should be to measure learning outcomes and provide
feedback to student (Robles & Braathen, 2002; Kerka & Wonacott, 2000; Kashy, Thoennessen, Tsai,
Davis & Wolfe, 1997).
Problems about online exams and suggestions of students also refer to exam security. Compared to
face-to-face exams, students think that security of online exams is low. Due to security problems that
may occur in online exams, many institutions make their exams in face-to-face environments.
However, these exams are in conflict with distance education’s understanding of education
independent of space and time (Bozkurt & Uçar, 2018). This problem can be solved with the latest
developments in technology with the help of exam security systems (Kınalıoğlu & Güven, 2011; Glater,
2006; Cerimagic & Hasan, 2019) or biometrics face recognition for security of online exams can be
used (Sarrayrih & Ilyas, 2013; Smith, Clarke, Carmona & Cerimagic, 2017). Using such a system will
both speed up identity control process, and exam security will be provided by fingerprint or face
recognition. Students also think that biometric and information-based identification methods are
more reliable in terms of exam security (Bozkurt & Uçar, 2018; Keskin & Güneş, 2015). To solve security
problems that may occur in online exams Williamson (2018) suggests some measures such as not using
exam questions available on internet, ensuring students answering only one question on each page in
exam, asking questions in form of short articles, giving students enough time to answer questions,
limiting students’ screen for not seeing questions as a whole, and constantly changing questions and
choices. However, Clyman & Orr (1990) and Luecht, Hadadi, Swanson & Case (1998) claim that online
exams are safer than traditional exams. The reason for this situation may be that these studies were
in the past years and they were assessed according to technological conditions of those periods.
Strengthening substructure of system is another suggestion of students. Substructure expression here
refers to technology that includes internet and online exam system. Technical problems can be highly
annoying and stressful for students and for this reason online exam process can be viewed as more
difficult for them. For this reason, in order to avoid problems in the online exam, which is directly
connected to technology, institutions should offer all their facilities without any problems and not
expose students to technical problems. Williamson (2018) states that potential possibilities of online
education cannot be reached with traditional teaching and assessment methods. Although
universities that have shifted from traditional to online education have started to make some changes
in their teaching methods, they are still not aware of what kind of problems they may encounter with
online assessment. For example, one of these problems is possibility of resorting to unethical
behaviour by students who take exams on their own computers at home. Therefore, universities and
institutions should take necessary precautions for problems in online exams.
An important suggestion of students was fair distribution of easy and difficult questions. It is clear that
students have problems with features of question. In order to solve problems about features of
questions instructors may check their colleagues’ questions before exam. In online exam system
developed by Tüfekçi, Ekinci Köse (2013) teachers were enabled to comment under questions in
question bank consisting of questions prepared by instructors. The implementation of a system in this
way can enable questions prepared in online exams to be examined by different instructors.
Students also stated that in online exams sometimes they encounter questions about topics not
covered in online course. There seems incompatibility of questions and course content. Similarly,
Robles & Braathen (2002) stated that, students are asked questions about topics that are not among
objectives of course in online exams, and this is an important problem. This situation is both very
disturbing for student and causes instructors to go beyond the goal of assessment. To solve this,
student should be informed about goals set at the beginning of lesson and should be assessed only in
accordance with these goals. The compatibility of course and assessment process with each other will
increase success of online education and online exams.
28 G. Ocak, G. Karakuş

As students stated questions in online exams mostly in form of multiple choice. However, using
different question types will be more effective for assessment since it has a multidimensional purpose,
multiple methods and different question types should be used in online exams (Galante, 2002; Kerka
& Wonacott, 2000; Tüfekçi, Ekinci & Köse, 2013).
Although online education was used before the COVID-19 pandemic, it was used compulsorily in the
pandemic period all over the world and in every different discipline. Although users of online
education before the pandemic had basic information (Keegan, 1988), what kind of knowledge both
students and teachers had on this subject during pandemic period and how they tried to adapt to this
difficult process were obtained from results of this study.

Recommendations

Online education environment offers students much more than textbooks and classroom education.
Although many educators use online education and internet technologies effectively, they prefer
traditional test method in assessment phase. Results of this study indicated that students have some
problems in online exams. Further research can be done about online exams. For instance, alternative
assessment processes that can be done in online education should not be approached with prejudice.
Online assessment can be done with different assessment tools such as discussion groups, electronic
portfolios, self-assessment tests, and writing assignments. Additional research can be conducted. Self-
assessment is also an important element that positively affects assessment process in online courses.
While instructor’s assessment of students is basis for assessing students, it is also necessary for
students to assess their own learning (Robles & Braathen, 2002). For further research self-assessment
can also be conducted. In this way, student will realize whether he will be able to move on to next
stage and will be able to repeat lesson if necessary, and will be able to measure his own learning and
success. For this reason, it would be useful for teachers to include self-assessment tests that student
can assess himself. Moreover, studies and results obtained for online education during pandemic
period on a country basis can be compared in future studies.

Limitations

This study is limited to the views of undergraduate students studying at two different universities in
2020-2021 academic year in Turkey. Since this study was conducted based on views of 55
undergraduate students, obtained results are limited only to views of these participants. The study
has no aim to reach generalization.

References
Alessio, H.M., Malay, N., Maurer, K. Bailer, A.J., & Rubin, B. (2017). Examining the effect of proctoring on online test scores.
Online Learning, 21(1), 146-161.
Aparicio, M., Bacao, F., & Oliveira, T. (2016). An e-learning theoretical framework. Educational Technology & Society, 19(1),
292-307.
Aragon, S. R., & Johnson, E. S. (2008). Factors influencing completion and non-completion of community college online
courses. The American Journal of Distance Education, 22(3), 146-158.
Arend, B. D. (2007). Course assessment practices and student learning strategies in online courses. Journal of Asynchronous
Learning Networks, 11(4), 3-17.
Ayo, C. K., Akinyemi, I. O., Adebiyi, A. A., & Ekong, U. O. (2007). The prospects of e-examination implementation in Nigeria.
Turkish Online Journal of Distance Education, 8(4), 125-134.
Bozkurt, A., & Uçar, H. (2018). E-Öğrenme ve e-sınavlar: Çevrimiçi ölçme değerlendirme süreçlerinde kimlik doğrulama
yöntemlerine ilişkin öğrenen görüşlerinin incelenmesi. Mersin Üniversitesi Eğitim Fakültesi Dergisi, 14(2), 745-755.
Bozpolat, E., &. Arslan, A. (2018). Öğretmen adaylarının öğretim teknolojileri ve materyal tasarımı dersine ilişkin görüşleri. E-
Uluslararası Eğitim Araştırmaları Dergisi, 9(3), 2018, 60-84. DOI: 10.19160/ijer.463977.
Undergraduate students’ views of and difficulties in online exams during the COVID-19 pandemic 29

Bunn, J. (2004). Student persistence in a LIS distance education program. Australian Academic Research Libraries, 35(3), 253-
270.
Çelen, F. K., Çelik, A., & Seferoglu, S. S. (2011). Yükseköğretimde çevrim-içi öğrenme: Sistemde yaşanan sorunlar ve çözüm
önerileri. Journal of European Education, 1(1), 25-34.
Cerimagic, S., & Hasan, M. R. (2019). Online exam vigilantes at Australian universities: Student academic fraudulence and the
role of universities to counteract. Universal Journal of Educational Research, 7(4), 929-936. DOI:
10.13189/ujer.2019.070403.
Clyman, S. G., & Orr, N. A. (1990). Status report on the NBME's computer-based testing. Academic Medicine, 65(4), 235-241.
DOI:10.1097/00001888-199004000-00005.
Cluskey Jr, G. R., Ehlen, C. R., & Raiborn, M. H. (2011). Thwarting online exam cheating without proctor supervision. Journal
of Academic and Business Ethics, 4(1). Retrieved 3 January 2021, from https://www.aabri.com/manuscripts/11775.pdf.
Colby, M. (2018). ProctorU Auto – Canvas Demo Days [Webinar]. Retrieved 20 March 2021, from
https://www.youtube.com/watch?v=mqfVrFsGEGk&feature=youtu.be.
Çallı, İ., Torkul, O., & Taşbaşı, N. (2003). İnternet destekli öğretimde kullanılmak üzere web erişimli veri tabanı yönetim
sistemiyle ölçme ve değerlendirme sistemi tasarımı. The Turkish Online Journal of Educational Technology, 2(3), 108-115.
Dawson B., & Trapp R. G. (2001). Basic & Clinical Biostatistics (3rd edition). New York: Lange Medical Books/McGraw-Hill.
DeSouza, E., & Fleming, M. (2003). A comparison of in-class and online quizzes on student exam performance. Journal of
Computing in Higher Education, 14(2), 121-134.
Dickinson, K. (1997). Distance learning on the internet: Testing students using web forms and the computer gateway
interface. TechTrends, 42(2), 43-46.
Dirks, M. (1998). How is assessment being done in distance learning? Paper presented at the NAU/Web.98 Conference.
Retrieved 3 January 2021, from https://eric.ed.gov/?id=ED423273.
Fakcharoenphol, W., Potter, E., & Stelzer, T. (2011). What students learn when studying physics practice exam problems.
Physical Review Special Topics - Physics Education Research 7, 010107. DOI: 10.1103/PhysRevSTPER.7.010107.
Fraenkel, J. R., Wallen, N. E., & Hyun, H. H. (2012). How to design and evaluate research in education (7th edition). New York:
McGraw-Hill.
Galante, D. J. (2002). Web-based mathematics: An examination of assessment strategies implemented in the online
mathematics classroom (pp. 1-189). Illinois State University. Retrieved 3 January 2021, from
https://www.proquest.com/docview/305573922.
Glater, J. (2006). Colleges chase as cheats shift to higher tech. The New York Times (May 18), A1-A24.
Gunawardena, C. N., Wilson, P. L., & Nolla, A. C. (2003). Culture and online education. In G. M. Moore & W. G. Anderson
(eds.), Handbook of Distance Education (pp. 753-775). New Jersey: Lawrence Erlbaum Associates, Inc.
Hartley, J., & Nicholls, L. (2008). Time of day, exam performance and new technology. British Journal of Educational
Technology, 39, 555-558.
Hentea, M., Shea, M. J., & Pennington, L. (2003). A perspective on fulfilling the expectations of distance education.
Proceedings of the 4th Conference on Information Technology Curriculum (pp. 160-167). Retrieved 3 January 2021, from
https://dl.acm.org/doi/abs/10.1145/947121.947158.
Hillier, M., & Fluck, A. (2017). Transforming exams-how IT works for BYOD e-Exams. In H. Partridge, K. Davis, & J. Thomas
(eds.), Proceedings of the 34th International Conference on Innovation, Practice and Research in the Use of Educational
Technologies in Tertiary Education – ASCILITE 2007 (pp. 100-105). University of Southern Queensland. Retrieved 3
January 2021, from https://2017conference.ascilite.org/program/conference-proceedings.
Hols-Elders, W., Bloemendaal, P., Bos, N., Quaak, M., Sijstermans, R., & Jong, P. D. (2008). Twelve tips for computer-based
assessment in medical education. Medical Teacher, 30(7), 673-678.
Kashy, E., Thoennessen, M., Tsai, Y., Davis, N. E., & Wolfe, S. L. (1997). Using networked tools to enhance student success
rates in large classes. In Proceedings of the 27th Annual Conference Frontiers in Education. Teaching and Learning in an
Era of Change (Vol. 1, pp. 233-237). IEEE.
Karaağaçlı, M. (2002). Mesleki eğitim ve teknoloji eğitiminde özel öğretim yöntemleri. Ankara: Nobel Yayın. Dağıtım.
Kawachi, P. (2003). Initiating intrinsic motivation in online education: Review of the current state of the art. Interactive
Learning Environments, 11(1), 59-81.
Keegan, D. (1988). On defining distance education. In D. Sewart, D. Keegan & B. Holmberg (eds.), Distance Education:
International Perspectives (pp. 6-33). London/New York: Routledge.
Kerka, S., & Wonacott, M. E. (2000). Assessing learners online. Practitioner File. Retrieved 3 January 2021, from
https://eric.ed.gov/?id=ED448285.
Keskin, Ö. G. K., & Güneş, A. (2015). Online sınav sistemlerinde güvenlik sorunları ve güvenli online sınav giriş uygulaması.
Eğitim ve Öğretim Araştırmaları Dergisi, 4(4), 48-54.
Kınalıoğlu, İ. H., & Güven, Ş. (2011). Uzaktan eğitim sisteminde öğrenci başarısını ölçülmesinde karşılaşılan güçlükler ve çözüm
önerileri. XIII. Akademik Bilişim Konferansı, 2 - 4 Şubat 2011 İnönü Üniversitesi, Malatya.
Kumar, R. (2005). Research Methodology – A Step by Step Guide for Beginners (2nd edition). London: SAGE Publications Ltd.
Kuo, Y. C., Walker, A. E., Belland, B. R., & Schroder, K. E. (2013). A predictive study of student satisfaction in online education
programs. International Review of Research in Open and Distributed Learning, 14(1), 16-39.
Luecht, R. M., Hadadi, A., Swanson, D. B., & Case, S. M. (1998). Testing the test: A comparative study of a comprehensive
basic sciences test using paper-and-pencil and computerized formats. Academic Medicine, 73(10), S51-53.
Miles, M. B., & Huberman, A. M. (1994). Qualitative data analysis: An expanded sourcebook. Thousand Oaks, CA: Sage.
30 G. Ocak, G. Karakuş

Merriam, S. B. (2001). Qualitative research and case study applications in education. San Francisco: Jossey-Bass.
Moreno-Ger, P., Burgos, D., Martínez-Ortiz, I., Sierra, J. L., & Fernández-Manjón, B. (2008). Educational game design for online
education. Computers in Human Behavior, 24(6), 2530-2540.
Patton, M. Q. (2014). Qualitative research and evaluation methods: Integrating theory and practice. CA: Sage publications.
Pazargadi, M., Khatiban, M., & Asktorab, T. (2009). Performance evaluation of nursing faculty members: A qualitative study.
Iranian Journal of Medical Education, 8(2), 213-227.
Robles, M., & Braathen, S. (2002). Online assessment techniques. Delta Pi Epsilon Journal, 44(1), 39-49.
Ryznar, M. (2020). Giving an online exam. Indiana University Robert H. McKinney School of Law Research Paper No. 2020-
16, Retrieved 3 January, from https://ssrn.com/abstract=3684958.
Sarrayrih, M. A., & Ilyas, M. (2013). Challenges of online exam, performances and problems for online university
exam. International Journal of Computer Science Issues, 10(1), 439-443.
Schwandt, T. A. (1997). Qualitative inquiry: A dictionary of terms. Thousand Oaks, CA: Sage.
Shraim, K. (2019). Online examination practices in higher education ınstitutions: learners’ perspectives. Turkish Online
Journal of Distance Education, 20(4), 185-196.
Smith, E., Clarke, L., Carmona, P., & Cerimagic, S. (2017). Academic fraudulence in online degrees and exams at Australian
universities. International Journal of Research and Development Organization, 3(2), February 2017.
Stowell, J. R., & Bennett, D. (2010). Effects of online testing on student exam performance and test anxiety. Journal of
Educational Computing Research, 42(2), 161-171.
Tanyıldızı, M., & Semerci, Ç. (2005). Çevrimiçi eğitim uygulamalarına ilişkin öğretim elemanı ve öğrenci görüşlerinin
belirlenmesi. Türk Eğitim Bilimleri Dergisi, 3(2), 192-216.
Tello, S. F. (2007). An analysis of student persistence in online education. International Journal of Information and
Communication Technology Education, 3(3), 47–62.
Torkul, O., Kibar, A., & Tasci, T. (2004). Web tabanlı sınav sistemleri [Web Based Examination Systems], Paper presented at
the 1st International Conference on Informatics, Çeşme, Turkey.
Tufekci, A., Ekinci, H., & Kose, U. (2013). Development of an internet-based exam system for mobile environments and
evaluation of its usability. Mevlana International Journal of Education, 3(4), 57-74.
Walton, D. (2010). Appeal to expert opinion: Arguments from authority. Pennsylvania: Penn State Press.
White, R. J., & Hammer, C. A. (2000). Quiz-o-Matic: A free Web-based tool for construction of self-scoring on-line
quizzes. Behavior Research Methods, Instruments & Computers, 32(2), 250-253.
Williamson, M. H. (2018). Online exams: The need for best practices and overcoming challenges. The Journal of Public and
Professional Sociology, 10(1), Art. 2.
Wolfson, P. J., Veloski, J. J., Robeson, M. R., & Maxwell, K. S. (2001). Administration of open-ended test questions by
computer in a clerkship final examination. Academic Medicine, 76(8), 835-839.
Yang, Y., & Cornelius, L. F. (2004). Students’ perceptions towards the quality of online education: A qualitative approach.
Proceedings of the 27th Annual Conference of the Association for Educational Communications and Technology (pp. 19-
23). Chicago, IL.

To cite this article: Ocak, G., & Karakuş, G. (2021). Undergraduate students’ views of and difficulties in online exams during
the COVID-19 pandemic. Themes in eLearning, 14, 13-30.

URL: http://earthlab.uoi.gr/tel