Master’s thesis

Possibilities of Artificial Intelligence

in Education
An Assessment of the role of AI chatbots as a communication
medium in higher education

Author: Marta Slepankova

Supervisor: Nuci Krenare
Examiner: Anita Mirijamdotter
Course Code:5IK50E, 30 credits
Subject: Information Systems
Abstract Artificial intelligence has grown in importance in many application areas. However,
the application in the education sector is in an embryonic state, where a variety of trials has
been conducted. The purpose of this master’s thesis is to investigate the factors that influence
the acceptability of AI chatbots by university students in higher education which might point
subsequently to the lack of usage. The study also suggests the most appropriate communication
areas of AI chatbot application in higher education suggested by students. For this study, the
unified theory of Acceptance and Use of technology 2 (UTAUT2) has been compiled with the
qualitative data gathered from semi-structured interviews and questionnaire surveys. The study
participants are university students from various countries (Sweden, Norway, Finland, Czech
Republic). The findings showed three primary constructs: Performance expectancy (PE), Effort
expectancy (EE), and a newly proposed construct, Nonjudgmental expectancy (NE), to
significantly predict intention to use AI chatbot technology without education intentionality.
Students suggested using AI chatbots for recap of course material, study material
recommendation, and exam and requirements information. Furthermore, this study provides a
rationale behind AI chatbot acceptability based on students' generation characteristics. The
results can guide universities to incorporate innovative solutions into their organization.

Keywords AI chatbot; conversational agent; Technology acceptability factors; Technology

Acceptance; UTAUT2, behavioral intention

2
Contents
1. Introduction _______________________________________________ 1
1.1 Research Setting _______________________________________________________ 1
1.1.2 Related work ______________________________________________________ 2
1.2 Purpose Statement and Research Questions __________________________________ 3
1.3 Topic Justification ______________________________________________________ 3
1.4 Scope and Limitations ___________________________________________________ 4
1.5 Thesis Organization_____________________________________________________ 4

2. Review of the Literature _____________________________________ 5

2.1 Literature review search strategy __________________________________________ 5
2.2 Brief history of chatbots _________________________________________________ 7
2.3 AI Chatbots in the education sector ________________________________________ 8
2.3.1 AI chatbot in education sector used as a communication medium _____________ 9
2.3.2 Challenges in AI chatbot adoption to the education sector __________________ 11
2.3.3 Other AI solutions in the education sector _______________________________ 11
2.4 Essential chatbot design concepts _________________________________________ 12
2.4.1 Conversational Skill ________________________________________________ 14
2.5 Characterizations of the university students' generations _______________________ 15
2.6 Gaps in knowledge found in Literature review articles ________________________ 15
2.7 Theoretical Framework _________________________________________________ 16

3. Methodology _____________________________________________ 19
3.1 Methodological Tradition _______________________________________________ 19
3.2 Methodological Approach _______________________________________________ 20
3.2.1 Research Design ___________________________________________________ 21
3.2.2 Case study _______________________________________________________ 21
3.3.3 Types of case studies _______________________________________________ 22
3.2.4 Participants’ Selection Criteria________________________________________ 22
3.3 Methods/Techniques for Data Collection ___________________________________ 24
3.3.1 Interview process __________________________________________________ 25
3.3.2 Informed consent __________________________________________________ 26
3.4 Methods/Techniques for Data Analysis ____________________________________ 26
3.5 Reliability, Validity or similar ___________________________________________ 28
3.5.1 Saturation ________________________________________________________ 29
3.6 Ethical Considerations__________________________________________________ 29

4. Empirical Findings ________________________________________ 30

4.1 General findings ______________________________________________________ 30
4.2 Findings from the themes _______________________________________________ 31

5. Discussion _______________________________________________ 35

6. Conclusion _______________________________________________ 41
6.1 Conclusions __________________________________________________________ 41

3
 6.2 Contribution _________________________________________________________ 42
6.3 Future Research _______________________________________________________ 42

References ___________________________________________________ 43

Acknowledgment _____________________________________________ 50

Appendix A.: Literature map _____________________________________ 1

Appendix B.: PRISMA flow ______________________________________ 1

Appendix C.: Survey questions ____________________________________ 2

Appendix D.: Interview questions __________________________________ 3

Appendix E.: Consent form template _______________________________ 5

Appendix F.: Themes, subthemes, codes ____________________________ 6

4
 1. Introduction
Education and educational institutions have undergone many changes in the last years, mainly
due to new technologies. The rapid evolution of internet technology has changed and is
constantly changing the ways of teaching and learning. Thus, new technologies transform how
teachers and educators interact with students, setting new teaching methods and altering the
traditional classrooms' look. Organizations depend heavily on Information Communication
Technologies (ICT) for the efficient completion of everyday tasks. Among these new
technologies, Artificial Intelligence (AI) is gaining popularity. It is becoming prevalent in our
lives with solutions such as AI chatbots.

Although the use of Artificial Intelligence is constantly expanding and enabling many
processes, AI chatbots, also known as conversational agents, are still regularly underused,
existing in the shadow of more frequently used rule-based chatbots. AI chatbots use Machine
learning (ML) and Natural language processing (NLP) techniques that enable them to mimic
human conversation through messaging platforms (Schuetzler, Grimes and Giboney, 2020).

Most typically, AI chatbots have been implemented into organizations to provide efficient user
inquiries, like responding to users' questions on 24/7 availability. It has been confirmed that
this solution can increase productivity, automate repetitive tasks, and lower the cost of customer
service support by handling multiple users in multiple languages simultaneously
(Adamopoulou and Moussiades, 2020). Artificial intelligence has grown in importance in many
application areas; however, the application in the education sector is in an embryonic state. A
variety of trials has been conducted. Some applications are available for commercial use, but
the technology is not yet widely used (Jaakkola, et al., 2020). Emerging artificial intelligence
technology is driving human education to a transition. These changes mean an alteration from
a traditional tutor-centered education system to a learner-centered, characterized by
personalized learning (Sandu and Gide, 2019). Therefore, the application of chatbots in
education is regarded as a possible essential assistant since they can provide considerable help
in learning processes (Molnár and Szüts, 2018). From this perspective, chatbots can initiate
communication based on the user's environment, the user's location, or clickstreams and provide
more personalized learning (Hien, et al., 2018). The pressure on higher education institutions
increases, while teacher's support for each student is reduced considerably, resulting in
ineffective learning and a high dropout rate (Hien, et al., 2018). At the same time, the high
dropout rates and delayed completion of studies are significant issues in higher education
worldwide (Baranyi, Nagy and Molontay, 2020).

The recent development of communication technology has accelerated the communication

process and enabled instant communication between people. Increased number of students
reflects to increased student-university communication (communication initiated by student
towards university staff). This research will first explore the studies addressing AI chatbots and
investigate how university students perceive them by pointing to factors of the acceptability of
AI chatbots in higher education and elaborate on what students-university communication topic
could be AI chatbots applied from students point of view.

1.1 Research Setting

Communication between students and the university is an important educational activity,

especially in online education. The term communication can vary in some cases; however,
researchers see the importance of communication between students and teachers. It might
highly influence the students' success (Bajracharya, 2015; Krahe, Escamilla-Fajardo and
López-Carril, 2021). However, the increased number of students makes adequate guidance
difficult for teachers; thus, the positive interaction between teacher and student is lowered
(Dimitriadis, 2020). It has been proven that chatbots can: a) repeat the course lessons, b)
gather feedback for a course to improve learning and teaching, c) answer frequently asked
questions about a specific subject, or d) support administrative tasks (Hien, et al., 2018).

Even though there is a possible solution to support the communication, the universities still lack
AI chatbots to be incorporated. Moreover, even though academics have researched the
application of artificial intelligence to education (AIEd) for more than 30 years, its adoption in
education lags behind other fields such as the applied sciences or finance. In this matter, more
than a decade ago, the researchers Rubin, Chen and Thorimbert (2010) suggested that it is both
timely and conceivable to consider adopting AI chatbots for informational, educational, and
assistive tasks which were considered trivial and repetitive. The article has been purposely
chosen to benchmark people’s perception of AI chatbots in research 10 years apart. Same as
master’s thesis, also not reflects the age nor gender, educational attainment, or socioeconomic
background. In the paper, researchers pointed to the importance of user preparedness and
acceptance. In addition, researchers advised that many information professionals believed that
AI chatbots might threaten the future of face-to-face service or considered AI chatbots as a
marketing gimmick. Most importantly, the study suggested an explicit requirement in user
awareness and familiarity with the systems for AI chatbots to overcome the mentioned negative
categorization.

Could it be that consumers still cannot imagine accepting AI chatbots into their day-to-day
lives? Could this be the reason for the lack of AI chatbots in the education field? Or the current
opinion differs from research done by Rubin, Chen and Thorimbert (2010) and AI chatbots are
now an acceptable awaited solution to provide such a service? The thesis utilizes the Unified
Theory of Acceptance and Use of Technology 2 (UTAUT2) model to find the predictors of AI
chatbots acceptability. However, there is a need to differentiate two terms: acceptability and
acceptance. Acceptability refers to perception technologies before use, whereas acceptance
refers to the perception of products after use. In other words, acceptability relies on personal
expectations that the product can provide one day, and the opinion is formed before its actual
use (Martin, Cojean and Ragot, 2020).

1.1.2 Related work

Regarding the acceptance, research of AI acceptance of AI chatbots in the United Kingdom by

Almahri, Bell and Merhi (2020) has been identified as the first study to investigate university
students' motivation for AI chatbots using UTAUT2. The results indicated that Performance
Expectancy, Effort Expectancy, and Habit are the main predictors of student intention to use
AI chatbots. The researchers pointed to factors that block and facilitate the acceptance of this
technology. Researchers conducted a quantitative survey-based approach to collect data from
431 students at Brunel University London. However, the study's outcome could not be
generalized, as the data came only from one particular university department. As future work,
the researchers proposed applying the UTAUT2 model across different countries and possibly
improving the prediction model by expanding it with other constructs. These particular future
research suggestions by Almahri, Bell and Merhi (2020) have led the author of the thesis to
focus on data collection from university students from different countries and search for
additional UTAUT2 constructs. Furthermore, to make the research more unique, the focus has
been placed on acceptability instead of acceptance.

2
Regarding other AI chatbot experiments in education, Rossi and Carletti (2011) explored the
possibilities to make online tuition sustainable and effective by evaluating an open-source
system containing a chatbot. This system supported university teachers' work by implementing
a knowledge base rather than pedagogical aspects specific to subject-specific. The AI chatbot
retrieved information already coded in the courses or originated in student activity logs, thus
answering the student's most common questions. Furthermore, the study showed that 42% of
all questions posted on university forums could be delegated to a software agent having access
to the course and students tracking data. The authors further suggested exploring the chatbot to
all university users through an internal messaging system enabling synchronous, on-demand,
first-level help desk service.

Also, Fonte, et al., (2016) presented an AI chatbot able to address students' questions regarding
their exams, questions about a course, and recommend study materials. The purpose of the
chatbot was to provide a friendly interface and support student's needs. The proposed solution
enabled students to ask questions (text or speech) about their study content. The answer would
be provided from the knowledge base stored in the repository. The application was based on AI
Markup language (AIML) and acted as intermediation between the students and server
platform, where the repository was located.

Research by Rieke (2018) focused on verifying and analyzing the relationship between AI
chatbot characteristics and the motivation of Portuguese millennials to use them. The results
showed that millennials seem to regard almost all of the indicated characteristics as indifferent.
One reason for this could be that the target group had limited knowledge of AI chatbots and
their preparation. They did not seem to value personality, emotions, or avatar characteristics in
this technology. Still, they valued features such as speed and accessibility.

1.2 Purpose Statement and Research Questions

This research seeks to understand the factors of acceptability of AI chatbots, which might
subsequently explain to lack of AI chatbots (without education intentionality) implementation
in higher education in contrast to student-university communication increase. The research is
focused on seeking the factors influencing the consumer's acceptability and pointing to
communication topics suitable for AI chatbots to handle from students’ point of view.

The research questions comprise three elements: a conceptual element, an IT element, and an
empirical element. Accordingly, this research aims to explore the following questions:

RQ1: What are the factors of the acceptability of AI chatbots by students in higher education?

RQ2: What types of communication areas can be assisted by AI chatbots to support student-
university communication from student perspective?

1.3 Topic Justification

Artificial intelligence is in the epicenter of the Information systems (IS) discipline as it stands
today as one of the most promising emerging technologies (Ågerfalk, 2020). However, there is
little research within the IS field, particularly in senior scholar's basket of journals (basket of
8). Based on those mentioned above, this study focuses on the IS theory of Unified theory of
acceptance and use of technology information (UTAUT2) and elaborates on AI chatbots
application in the educational setting as a medium in student-university communication. The
research is based on future work suggestions by Almahri, Bell and Merhi (2020). The

3
researchers proposed applying the UTAUT2 model across different countries and possibly
improving the prediction model by expanding it with other constructs. Furthermore, to make
the research more unique, the focus has been placed on acceptability instead of acceptance.

As the research focuses on acceptability factors (regardless of participant's gender or study

background). It suggests what student-university communication topics AI chatbots can assist
with. It is the primary social concern of this thesis. However, the characteristics of the student's
generations characteristics will be taken into consideration.

This thesis positions the research within Informatics research field with body of research within
human-computer interaction (HCI).

1.4 Scope and Limitations

AI chatbot application research is becoming popular also in the educational field. Numerous
papers have been written, however not many within senior scholars' basket of journals. A single
paper has not been found concerning the acceptability of AI chatbots in higher education using
UTATU2.

To better illustrate the research to the reader Creswell (2014) suggests designing a literature
map where links are made between concepts by drawing arrows. It is a visual picture of chosen
publications that illustrates how the thesis will extend or replicates the literature; thus, the map
visualizes the thesis within the larger body of research. The thesis literature map visualizes the
thesis in contrast to related work research (see Appendix A.: Literature map).

This master thesis study collects data through interviews and questionnaires surveys from
university students to understand and gain valuable insight into how students in higher
education perceive AI chatbots. Moreover, to the best knowledge, there is no research
identified that has focused on the acceptability of AI chatbots relying on UTAUT2, thus
predicting the intention of university students to use AI chatbots. The data will be compiled
with the UTAUT2 in a context (AI chatbots), consumers (university students), in a new cultural
setting (Sweden, Norway, Finland, Czech Republic). On top of that, the research focus on
discovering a new UTAUT2 construct of the predictive model while explaining the student's
acceptability intentions based on the age moderator. The mentioned above makes the research
unique.

1.5 Thesis Organization

Section 1: As shown in Figure 1, the introduction part introduces the reader to the topic,
background information, followed by the purpose statement and definition of research
questions, related work, topic justification, and study limitations.

Section 2: This part is the literature review section covering the search, brief history of chatbots,
current use of AI technology in the educational sector, and technical description of basic chatbot
design. It describes AI chatbots application in education, focusing on AI chatbots as a
communication medium. Further, it touches on the known challenges in AI chatbot
(conversational agents) adoption and characterizations of the university students' generations.
The gaps in the reviewed literature and theoretical framework is mentioned as well.

4
Section 3: After the literature review, the thesis moves into a research design section, describing
the philosophical tradition and methodological approach. Further explains how data will be
gathered and analyzed. The reliability and validity part specifies how the data could prove to
be valid and trustworthy. This part ends with ethical considerations describing how this research
addressed ethics and secured ethical considerations.

Section 4: The empirical findings part covers the interview and questionnaire survey analysis
outcome.

Section 5: In the discussion part, the research questions (described in the introduction part) are
discussed based on collected data and findings from the literature review. These parts lead up
to the conclusion.

Section 6: The conclusion part points out the study's conclusion, contribution, and future
research in this area.

Figure 1. Thesis structure

2. Review of the Literature

This part of the thesis includes the literature review, the gaps identified from selected literature,
and the theoretical framework adopted to conduct the research. According to Creswell (2014),
there are several goals of the literature review; for instance, it points to other studies closely
related to this thesis. It also provides a framework for the thesis. Literature reviews focus on
integrating or criticizing previous scholarly work or building bridges between related topics.
However, most literature review types aim to integrate the literature, organize it into a series of
related topics and summarize it. The literature review presents Brief history of chatbots, AI
Chatbots in education, the Application of other AI solutions in the education sector, Essential
chatbot design concepts, and the Characterizations of university student generations.

2.1 Literature review search strategy

There have been various papers regarding AI technology and AI-based solutions over the last
years concerning various fields such as Big data, finance, healthcare, to name a few. Therefore,
it should be mentioned that there is no lack of publications in this domain; however, this
literature review focuses mainly on the application of AI-chatbots in the education sector to
support communication between university and students and human-computer interaction.
Therefore, the thesis author followed the review process suggested by Creswell (2017) which
contains four steps: identification, screening, eligibility, and inclusion.

To ensure a transparent and complete reporting, the thesis author followed the PRISMA method
(Preferred Reporting Items for Systematic Reviews and Meta-Analyses) and visualized the
search process by the flow diagram (see Appendix B.: PRISMA flow). The search has been

5
conducted throughout January and February 2021 in five databases – Scopus, EBSCOhost,
IEEExplore, One Search, and Google Scholar. In each of the databases, the key search term has
been conducted individually. However, the main focus has been on papers published in senior
scholars' baskets of journals.

The thesis researcher followed the recommendation by Creswell (2014) and started by
identifying keywords. Final keywords were decided based on identifying a topic, often resulted
from preliminary readings of AI-related articles. The initial search has been done by the term
"Artificial intelligence", later by a combination of terms "AI" AND "education", additionally
by combination "AI" AND "education" AND "chatbot". Further has been the search extended
to terms as "bot" OR "tutor" "agent" OR "machine learning" OR "NLP". Thus, the search has
been done by combining the search terms with the Boolean operator "AND" and "OR". The
literature has also been searched by the source title "information systems". The initial searches
have focused purely on predefined journals. However, due to a lack of relevant literature, the
research also expanded to other journals. During this phase, short papers, non-computer
science-related papers, and duplicates were excluded. Publication whose title or abstract
consisted of these terms (Artificial intelligence, education, and chatbot) were preferred,
selected, and taken as potential papers for review.

For the comprehensive literature review, the author has chosen only the publications published
after 2015. Only newer papers were chosen. By this decision, some publications might have
been missed. Also, it is essential to note that the searches for this study were undertaken with
criteria specified by mentioned keywords and a general review process of the abstract and the
content of the selected articles; however, other keywords could be used. Furthermore, findings
were collected only from academic journals and conference papers in scholarly databases; thus,
other materials containing more information on this topic might have been excluded. For
instance, business promotional materials of existing devices taken from companies' web pages
could bring another level of information. However, the belief is that the author provided a
comprehensive review by pointing to general information on the work being accomplished in
the education sector. Some relevant articles from other databases such as Web of Science may
have been missed regarding the database search.

Starting the search with the keywords mentioned above, Scopus showed the most significant
number of results with 2,765, followed by EBSCOhost 1,563, IEEExplore 502, Google Scholar
413. However, each of the search combinations of words brought a percentage of the same
articles. At this point, it was crucial to eliminate and remove the duplicates. To complement the
chosen systematic review method, the author used both the forward and backward search
approaches. It is especially beneficial to identify other references by reviewing the citations and
references (Wohlin, 2014). Once this step was done, the remaining articles had to be screened
by their titles and abstracts. Only relevant articles remained (Liberati, et al., 2009). To ensure
that the papers selected for this study were related to the topic chosen, the inclusion and
exclusion criteria were defined as eligibility criteria set of predefined characteristics that help
decide which articles should be or not be included in the study. It was essential to define the
criteria wisely as they could impact the validity of the results. Inclusion criteria are used for
further study. The exclusion criteria help withdraw the articles from the study (Salkind, 2010).

Due to the relevance, only publications after the year 2015 were included as the technology is
developing fast, and older described solutions might have become obsolete. Hence, including
only the resources from recent years contributes to the internal validity of thesis research.

6
Inclusion criteria were (1) studies focusing on Artificial intelligence chatbots suitable for
student-university communication; (2) studies addressing services for university-related
communication purposes; (3) studies targeting chatbots/tutoring systems; (4) studies with the
full text is written in English and (5) studies published between 2015 and 2021.

The exclusion criteria were 1) the paper describes an Artificial intelligence-based solution not
applicable for university-student communication 2) papers as book chapters, a magazine or
letters, short papers 3) source of the paper focused outside of the information systems field or
pedagogy field.

As a next step, the texts of eligible publications were read to ascertain and confirm the relevance
of the articles to the subject focus. Then, the author made notes from each publication into the
shared document. As a result, each paper has been synthesized based on the following: the title
of the paper, author name, source, research purpose, research objective, concept/theory used,
data collection method, type, key findings, contribution to knowledge, and limitations. Finally,
the selected articles were analyzed to identify themes and gaps.

2.2 Brief history of chatbots

Figure 2. The evolution of chatbots

As shown in Figure 2, everything started with the first publicly known chatbot called ELIZA,
introduced by the Massachusetts Institute of Technology (MIT) in 1966. In contrast to modern
AI chatbots, ELIZA was based on simple pattern matching rules. These types of chatbots are
referred to as rule-based chatbots. Once a particular keyword is identified in the text, the chatbot
provides a predefined response. The primary purpose of the ELIZA chatbot was to act as a
psychotherapist (Mekni, Baani, and Sulieman, 2020; Adamopoulou and Moussiades, 2020).
Based on template-based responses, the chatbot was able to imitate the conversational style of
a non-directional psychotherapist. As a result, users thought that ELIZA was an actual person
and not a machine. Another example of a well-known chatbot is ALICE (Artificial Linguistic
Internet Computer Entity), released in 2001, as a natural language processing chatbot. This
chatbot was also built on a rule-based approach, which aims to discover significant parts of the
text, code them, and script language to create conversation scenarios.

Over the following years, the efforts in chatbot design aim to re-engineer or improve this
methodology. For example, in 2006, IBM released Watson, AI chatbot based on natural
language processing and machine learning. In the last decade, developers have deepened
computer interaction based on Artificial intelligence technologies and Natural language
processing (NLP). However, both types of chatbots are still frequently used, and therefore
distinction needs to be made between the AI chatbots (conversational agents) and the rule-based
chatbots. AI chatbots use algorithms to learn and understand unstructured information and learn
from the interaction with the users and the outcomes. Thus, the users co-produce and influence
7
AI chatbots while communicating with them (Shumanov and Lester, 2021). Recent
developments include virtual personal assistants like Apple Siri in 2010, Microsoft Cortana, or
Amazon Alexa in 2015 (Mekni, Baani, and Sulieman, 2020; Adamopoulou and Moussiades,
2020).

Further, Microsoft's XiaoIce was designed to answer questions and be a virtual companion to
users (Shum, He and Li, 2018). The improvement of text-to-speech and speech-to-text
communication has made AI chatbots more convenient, resulting in the popularity of chatbots
as assistants worldwide. AI chatbots allow users to access data and services and exchange
information by imitating a human conversation (Jansen et al., 2020). Nowadays, AI chatbots
are used in various public and private institutions for services and tasks such as checking the
user's calendar, making appointments, reading, writing, and sending emails (Dale, 2016).
Different parameters can classify today's chatbots (see Figure 3).

Figure 3. Categories of chatbots by Sandu and Gide (2019)

Following Figure 3, AI chatbots based on the assessed knowledge can be divided into chatbots
with open domains to address general topics and respond appropriately to general questions and
chatbots with closed domains to address specific knowledge. Their service can further classify
chatbots into those that offer interpersonal, intrapersonal, and inter-agent services. Another
classification is the chatbot's goal: informative, conversational, and task-based (Sandu and
Gide, 2019). Finally, another classification divides AI chatbots into a) with educational
intentionality and b) without educational intentionality (outside of teaching). Chatbots without
educational intentionality support students by performing educational and informational tasks
but do not teach students courses. The latter once is under the focus of this thesis.

2.3 AI Chatbots in the education sector

Although the term AI chatbot or conversational agent represents a wide variety of systems with
various capabilities and purposes, most researchers refer to a chat application supported by
artificial intelligence in a human-machine dialog (Mekni, Baani and Sulieman, 2020;
Dimitriadis, 2020). They enable users to access data and services and exchange information by
8
simulating human conversation (Janssen, et al., 2020). Their functional ability can vary from
simple questions to complex conversations (Molnár and Szüts, 2018).

Researchers addressed the AI chatbots as they believe that this technology can increase and
enhance teachers' capabilities and students' abilities (Ashfaque, et al., 2020; Lee, et al., 2020).
Some also noted that the AI chatbots have a general potential to help students, teachers, and
education staff by providing helpful information (Khin and Soe, 2020; Mekni, Baani, and
Sulieman, 2020; Molnár and Szüts, 2018). Furthermore, the researchers were trying to solve
the increased workload of the teaching staff (Lee, et al., 2020) and support college students who
recurrently navigate challenging tasks (Mekni, Baani, and Sulieman, 2020).

The use of AI chatbots has been recently implemented in universities to improve existing
services or introduce new ones (Quiroga, Daradoumis and Puig, 2020). AI chatbots are applied
in the educational sector with or without education intentionality. Chatbots with education
intentionality are examples of AI-led solutions and supplement the teacher in fostering teaching.
Chatbots without education intentionality are AI-supported solutions and are used in
administrative tasks such as student guidance and assistance and handling FAQs (Frequently
Asked Questions) (Sandu and Gide, 2019; Ranoliya, et al., 2017). Further, the researchers point
to personalization as an outcome of the AI chatbot adaptation. For example, they can assist
students separately by providing answers according to different student profiles and adjusting
to their needs (Lee, et al., 2020). However, some researchers advise further research as the
addition of ontology technology can help to improve a chatbot's performance and architecture
by giving basic concepts and relationships to capture and train relevant domain knowledge
(Hien, et al., 2018).

Further, AI chatbots can be equipped with voice recognition and text-to-speech capability so
that individuals will interact more successfully with them (Santoso et al., 2018). Furthermore,
AI chatbots can sense a users' feelings during interactions and target the response based on it.
In consequence, it provides a more pleasant service experience (Yang and Evans, 2019).

Although the suggested outcome of AI chatbot adaptation varies, the researchers often mention
the cost-saving aspect as the individual support to students is costly. In consequence, the use of
chatbots can not only improve an institution's services but also reduce labor costs (Hien, et al.,
2018; Choque-Díaz, Armas-Aguirre and Shiguihara-Juárez, 2018; Yang and Evans, 2019).
According to Yang and Evans (2019), the university bureaucracy workload could be reduced if
AI chatbots could automatically fill in users' request forms.

2.3.1 AI chatbot in education sector used as a communication medium

Many studies have provided a wide range of practical implications to support student-university
communication. Researchers Ranoliya, et al., (2017) addressed the time consumption
perspective in services provided by a human to a human. The growing number of student
questions increases the waiting time for the response and, consequently, results in poor client
satisfaction. As a solution, the researchers propose AI chatbots as a communication medium
due to their time effectiveness. Such an interaction between the students and the chatbot consists
of the following elements: initiation, response, and feedback (Fleming, et al., 2018).

The selected papers provide helpful insight into what is currently possible to delegate to
software. The analysis tried to capture the simultaneous use as a communication medium
mentioned in selected literature (see Figure 4). In other words, Figure 4 depicts communication
areas mentioned in research papers where AI chatbots are used as a communication medium.
9
 Figure 4. Area of application of AI chatbots as a communication medium mentioned in
research papers

Most of the AI chatbot prototypes were concerning course-related information support.

Solutions could assist students and provide a quick recap on course materials on multiple
common social platforms like Telegram or Facebook messenger. At the same time, it could
function as an online tutor reducing teachers' workload effectively (Lee, et al., 2020). In
addition, it can assist and help teachers identify a set of at-risk indicators among students, such
as consistently late assignments or lack of login activity (Mekni, Baani and Sulieman, 2020).
Another application was university-related information support, where AI chatbots represent
an alternative to an information desk (Stachowicz-Stanusch and Amann, 2018). For example,
experiments pointed to the possibility of students approaching the chatbot with inquiries
regarding building their graduation plans, learning about majors, and retrieving information
about scheduling (Mekni, Baani and Sulieman, 2020), as well as information about the
university location (Khin and Soe, 2020). In addition, the system can act as an intelligent
assistant and improve university services (Touimi, et al., 2020; Choque-Díaz, Armas-Aguirre
and Shiguihara-Juárez, 2018) as well as reduce labor costs and create new innovative services
(Hien, et al., 2018; Mekni, Baani and Sulieman, 2020). Furthermore, the AI chatbot can extract
the information from the university knowledge base and respond to the students and applicants
about admission services (Santoso, et al., 2018). Similarly, a question answering chatbot can
also support academic admissions by using a data set of conversations and help with admission
services (Chandra and Suyanto, 2019).

10
2.3.2 Challenges in AI chatbot adoption to the education sector

Despite the many opportunities that the application of chatbots can bring in education, some
challenges need to be addressed as they can have a negative impact on individuals and society
as a whole. This technological revolution will dramatically change higher education worldwide.
Therefore, a more humanistic perspective for this kind of education needs to be considered
before any actual changes (Popenici and Kerr, 2017).

Privacy and ethical issues are major topics that require more attention and to be discussed
further. Regarding that, AI systems demand vast amounts of data and information derived from
students and university staff that is confidential; this can cause serious privacy issues (Kengam,
2020; Benbya, Davenport and Pachidi, 2020). Further, another challenge concerns the
development of chatbots that depend on their users to enhance the communication process.
Insults, intimate questions, vulgar words, and other inappropriate topics can have a negative
impact on the chatbot's communication development (Stachowicz-Stanusch and Amann, 2018).
For this reason, chatbots should be aligned with an institution's policies and regulations
regarding data privacy and security (Yang and Evans, 2019). The results show that there are
always questions that AI chatbots cannot answer as it is impossible to predicate all potential
scenarios for interaction.

Nevertheless, to change and update chatbots' responses, it is necessary to have AI knowledge

and technical skills that the teaching and support staff often do not have (Yang and Evans,
2019). In addition to that, education institutions will have to equip with appropriate
infrastructure while ensuring the safety and credibility of AI-based systems. Additionally, the
cost of installing, maintaining, and repairing AI is considerably expensive, and only well-
funded educational organizations might be able to equip such high technology (Kengam, 2020).
Another factor is a fear of job loss that faculty members, librarians, or administrative staff might
experience. Moreover, in the future, AI applications in education might increase due to
addiction to non-human technology, limiting all personal interactions and personal connections.
Thus, AI addiction can hurt the learners instead of helping them (Kengam, 2020).

2.3.3 Other AI solutions in the education sector

Numerous researchers that focus on applying AI into the education sector belong to the
interdisciplinary community called The International Artificial Intelligence in Education
society (AIED), which combines computer science, education, and psychology. AIED
researchers point to several current trends, including intelligent tutor systems, adaptive
learning, and smart classrooms (Joshi, et al., 2021; Kengam, 2020). As for the thesis, the
research is positions within computer science research field with body of research within
Human-computer interaction (HCI).

Further, Zawacki-Richter, et al., (2019) notes the following application areas of AI in higher
education:

Profiling and Prediction – At the administration level, many AI applications can function as
learner models or profiles in a way that allows prediction concerned students in a) admission
decisions and course scheduling by evaluating students' academic performance, b) dropout and
retention rate by developing early warning systems to detect the performance of students while
still in their first year and c) future academic achievement by modeling academic behavior.

11
Intelligent Tutoring Systems – (ITS) is concerned with the teaching and learning of the student
and include functions as a) teaching course content as mathematics, business statistics,
accounting, b) diagnose strengths or gaps in students' knowledge c) counseling on student
materials by observing the student's behavior in the course and generating a student's profile,
d) facilitate collaboration between learners by supporting online collaborative learning
discussions and writing, e) assisting teachers for example with supervision and detection of
conflictive cases in collaborative learning.

Assessment and Evaluation – Address the teaching includes a) automated grading a wide-
ranging of courses b) evaluation of student understanding, engagement, and academic integrity,
as well as c) evaluation of teaching.

Additionally, other researchers propose Pedagogical Agents (Joshi, et al., 2021), which are
digital or virtual characters aiming at facilitating learning incorporating social, emotional, and
motivational aspects of technology while interacting with the students. The agents can have a
three-dimensional or two-dimensional shape.

2.4 Essential chatbot design concepts

There are various designing techniques to develop the source of communication between
computers and humans. The developers can choose among methods like pattern matching for
rule-based chatbots or AI Markup language (AIML) for AI chatbots (Benbya, Davenport and
Pachidi, 2020). Human-computer interaction (HCI) is a multidisciplinary field of study. The
main focus is on users and their interaction with computers. The interaction can be done on
messaging platforms by typing the text or speech dialogue using the voice (Khin and Soe, 2020;
Mekni, Baani and Sulieman, 2020; Følstad and Brandtzaeg, 2017; Benbya, Davenport and
Pachidi, 2020). According to Adamopoulou and Moussiades (2020), AI chatbot can be
considered one of the most elementary examples of intelligent Human-Computer Interaction.
Further, HCI focuses on the design and use of information with an explicit goal to improve user
experiences, task performance, and the connection of modalities (Khin and Soe, 2020;
Ranoliya, et al., 2017). Although the designers have been for years focused on rule-based
chatbots and designing graphical user interfaces, recently conversational agents designers
focused on natural language user interface and utility to enable chatbots to perform specific
tasks (Mekni, Baani and Sulieman, 2020; Følstad and Brandtzaeg, 2017). Based on chatbots'
leading functional focus, developers decide on algorithms or platforms to use (Adamopoulou
and Moussiades, 2020; Khin and Soe, 2020. According to Adamopoulou and Moussiades
(2020), some fundamental concepts are mentioned to better understand AI chatbot technology.

Pattern-matching (also called rule-based approach). This means that the system matches the
words from a predefined dictionary. This method has been used in Eliza and ALICE chatbots;
however, this approach has many disadvantages. For instance, the responses are relatively
predictable and repetitive and have a conversational looping effect due to missing the storage
(to store previous responses). Figure 5 depicts the example of the conversation between a rule-
based chatbot and an AI chatbot.

12
Figure 5. Example of conversation between basic rule-based chatbot and AI chatbot. Inspired
by Agrawal (2020)

The main difference visible for the consumer is the understanding of the chatbot. For example,
suppose a consumer asks a question that a rule-based chatbot has not been trained to answer. In
that case, it is confusing and unable to provide the information. On the contrary, AI chatbots
enable users to engage in natural, human-like conversation.

The Artificial Intelligence Markup Language (AIML) tag-based technique is applied to natural
language modeling. It is based on basic units of dialogue called "categories". General questions
or template-based questions such as greetings can be answered by using AIML. In contrast, AI
chatbots use Latent Semantic Analysis (LSA) to discover likenesses between words for more
advanced questions. However, every system contains a set of predefined tailored neutral
answers when the input is not understood.

Figure 6. NLP processing of user's input. Inspired by Mallya (2021)

13
Figure 6 depicts the Natural Language Processing (NLP) technology used for AI chatbots. NLP
is a field of artificial intelligence that focuses on how computers understand and translate human
language. Thus, NLP aims to ensure that complex and highly diverse human language is easy
to understand for machines. NLP converts the user's speech or text into structured data; the data
are then utilized to give a relevant answer. Many NLP techniques are based on Machine
learning (ML), which is the computer's ability to learn by training enormous data and improve
over time without being explicitly programmed. However, the core technique for the NLP task
is Natural Language Understanding (NLU). When a user provides input to the AI chatbot, the
NLU extracts the context and meanings from natural language. Once specific intent is triggered,
the AI chatbot responds by corresponding action.

Another critical component is an entity. The entity is a tool for extracting values from the natural
language. Once the user requests the chatbot, the system aims to infer the user's intention and
the associated information. For instance, a user might type a request "Translate orange," where
translate is the intended value and orange are the entity (associated information). Alternatively,
as another example, a user might ask, "What is the weather in Sweden?". The user's intent is to
obtain information about the current weather forecast; the entity value is the word Sweden.
Then, the chatbot proceeds with the execution of the action or asks for additional information
or clarification. Machine learning gives AI chatbots the ability to learn from the experience so
that AI chatbots can execute the task based on the previous conversation (Adamopoulou and
Moussiades, 2020). Additionally, chatbots could be developed by programming languages like
Java or Python or be built on cloud-based state-of-the-art platforms such as Google's
DialogFlow, IBM Watson, or SAP Conversation, all empowered by machine learning
(Adamopoulou and Moussiades, 2020).

2.4.1 Conversational Skill

The rapid growth in the last decade in the development of natural language processing (NLP) s
overcome many NLP technical challenges; however, the research in behavioral aspects of
communication skills of AI chatbots is still ongoing. Conversational skill is an essential
capability for chatbots, to the extent that it might impact consumer perceptions, especially when
organizations pass on more significant roles and tasks on chatbots, which subsequently start to
act like employees (Schuetzler, Grimes and Giboney, 2020). As developers aim to develop
chatbots that are indistinguishable from humans, the last decade has been characterized by
significant advancement in the technical capabilities of conversational agents. Besides, the
research focused on technical capabilities rather than understanding how the different designs
impact the perception. It's crucial to understand when it is more suitable to apply sophisticated
AI chatbots and when the system's sophistication may lead to adverse effects, and a relatively
simpler rule-based chatbot would be more appropriate. Further, Schuetzler, Grimes and
Giboney (2020) also determined that once a chatbot can act more human-like, consumers
increase their expectation of the device capabilities and see the chatbot as more engaging than
a less skilled one. This led to blurred differentiation between behaviors in human-human and
human-computer conversations.

The level of conversational skill consists of many different behaviors. Consumers with high
conversational skills often give relevant responses, do not speak too slow or too fast, and exhibit
variety in their speech. These skills influence how others see the speaker during an interaction.
The same principle is applied to chatbots. The researchers distinguish between style and content
of chatbot conversation skills. Style is how the intent is expressed, while content is seen as the
main intent expressed in a message. The chatbot's predefined set of tailored responses represents
content, and response variety represents a style (Schuetzler, Grimes and Giboney, 2020).
14
According to Schuetzler, Grimes and Giboney (2020), the improvements in conversational
skills can positively influence perceptions of AI chatbots, as tailored responses and variety in
responses improve engagement and increase the sense of connection between consumers and
conversational agents. The authors suggested that organizations aiming to engage with their
consumers should incorporate tailored responses for their chatbots.

2.5 Characterizations of the university students' generations

In addition to the AI and chatbot-related information, there may well be critical generational
characteristics information. For example, most current university students belong to Generation
Y (also referred to as the millennials) and Generation Z (also referred to as the iGeneration).

According to Linnes and Metcalf (2017) generation Y (born between 1981-1995) is defined as
digital natives, and fast communication is an essential aspect of their school and daily lives.
When decisions need to be made, this generation heavily relies on peer advice by asking for a
recommendation in their network of "friends" and "followers" rather than researching other
sources. The millennials generation was the first to experience the Internet's possibilities and
witness the development of many types of technology such as mobile phones. Further, this
generation prefers online or mobile communication and social media, and their signature
product can be described as a tablet and smartphone.

Further, Linnes and Metcalf (2017) advise that the communication process accelerated with the
technological developments such as reading the emails moved from computer desktops to
smartphones. In practice, students can approach their peers or university staff outside of their
consultation hours and dispel their doubts any time over the Internet. However, this led to a
situation where students representing the millennial generation often expect an immediate
response. Students of this generation approach the teachers with questions about the practical
application of the course content. They seek to have a closer relationship and extra personal
attention from the teacher than previous generations. According to Schrodt, Whitt and Truman
(2007) the student-teacher relationship involves a degree of social influence. Teachers have the
power to influence students in many ways. Teachers reward students with extra course credits
or other forms of positive feedback. Students are motivated to get a reward, so they aim to meet
the teacher's requirements. Students are also aware that teachers have the power to punish in
the form of penalties for late papers or unexcused absences.

Generation Z is the generation born after the millennials (born from 1995 to the present). It is
the first generation accompanied by technology since birth. This generation is very dependent
on IT, fond of innovations with signature products as nano computing and 3D printed products.
They demand customer orientation, high priority to customer satisfaction, and a "customer is
always right" attitude. Their communication mainly takes place on hand-held devices, and the
generation moves the favor from "older "communication platforms such as Facebook and
"ancient" forms of communication such as email on immediate social platforms such as
Snapchat (Linnes and Metcalf, 2017).

2.6 Gaps in knowledge found in Literature review articles

After summarizing the review assemblies, the thesis researcher, was able to distinguish some
gaps in the themes and point out some of the deficiencies. For example, although Artificial

15
Intelligence is an important subject within the IS field, the researcher believes that research
concerning AI chatbots application in higher education communication has not been addressed
fully in the senior scholars' basket of journals (basket of 8).

Another deficiency in the reviewed literature is the perceived lack of acceptance metrics. None
of the reviewed papers referred to IS research theory, such as the Unified theory of acceptance
and use of technology information (UTAUT). Besides, the experiments have been only
conducted in one particular country and not in different countries. The studies also do not
address the consumer behavioral characteristics based on their generation characteristics.
Generally, the papers lack extensive evaluation to support the proposed solutions or justify the
proof of concepts.

Furthermore, as the implementation of AI chatbots in the education sector has not been fully
realized, there has not been found studies with phenomenological research design used by
researchers. There could be a reason, as phenomenology attempts to build the essence from
participant's experience (Creswell, 2017) and currently mentioned technology is not being
widely accepted and incorporated particularly into the education sphere.

The last gap has been perceived in the lack to mention of ethics or security-related aspects of
AI chatbots application in higher education experiments. The researchers often did not address
or take into consideration these issues during their experiments. AI chatbot might be an
effective communication enabler; however, the innovation brings challenges. This creates a
paradoxical situation when benefits are accompanied by unintended consequences such as
security risk. However, the authors of studies did not address such issues.

2.7 Theoretical Framework

Technology acceptance is vital for designers; consequently, researchers also address acceptance
and acceptability in their research. Both are crucial for the further development of any new
technology. Furthermore, when an organization is planning to facilitate their platform to
increase, for instance, personnel efficiency or profitability, or cost reduction or to improve
marketing, it is important to recognize the consumers' needs and attitude towards the solution
in the beginning stage. Therefore, understanding the issues that influence consumers' decision
to use technology is helpful to consider already during the development phase. Consequently,
it is also important for the successful implementation of such technology. Furthermore, the
impact on individual willingness to use the technology plays the technology's features.
Therefore, understanding the consumers' perception towards adoption helps facilitate the rise
of implementing a new solution (Taherdoost, 2019).

Various studies have been conducted to predict the adaptation and acceptance of technological
solutions. Many recent studies have relied mainly on models such as the technology acceptance
model (TAM) (Davis, 1989) and the unified theory of acceptance and use of technology
(UTAUT). UTAUT has been proposed by Venkatesh, et al., (2003) and consists of eight models
(TAM is one of the models included). This theory helps researchers evaluate the probability of
success of new technology and understand the factors for its acceptance. According to
Venkatesh, et al., (2003) it also points to the mitigation of the impact of skeptics and less prone
users to adopt the new technology, for instance, by marketing or training interventions. The
model consists of four variables: performance expectancy (PE), effort expectancy (EE),
facilitating conditions (FC), and social influence (SI). There is a difference between UTAUT
and UTAUT2 introduced by Venkatesh, Thong and Xin (2012). The main focus of UTAUT is
organizational context, aiming to predict if the employees will accept and use the technology.
16
On the other hand, the UTAUT2 has been proposed to understand consumer behavior. Both
include the moderators of age, gender, and experience. Also, both predicting models consist of
performance and effort expectancy, social influence, and facilitating conditions. On top of that,
UTAUT2 contains three additional factors: hedonic motivation (HM), price value (PV), and
habit (HA) to explain consumer behavior.

According to researchers Gharrah and Aljaafreh (2021) and Huang and Kao (2015), the seven
factors are further described:

Performance expectancy (PE) can be explained as a personal belief on how the technology will
support a specific activity or attain a gain in job performance. In other words, the extent to
which new technology usage can bring benefits to consumers while performing activities. This
construct further consists of four criteria: the perceived usefulness, the job fit, the extrinsic
motivation, and the relative advantage. Perceived usefulness means a degree to which people
believe that using new technology can improve their job performance. The job fit criterium
means the potential of new technology to increase job performance. Extrinsic motivation means
realizing if consumers would like to perform an activity linked to a reward or linked to a valued
outcome. Finally, relative advantage criterium refers to benefits coming from the new
technology adoption compared to the costs. This factor points to advantages that university
students would gain by using AI chatbots in the thesis context. Further, the students might
expect the AI chatbot's performance to enhance their study performance. Based on this reason,
the performance expectancy factor impacts the behavioral intention of university students to
use AI chatbots.

Effort Expectancy (EE) can be described as the extent of the effort of a person who used the
system. This construct consists of three criteria: complexity, the perceived ease of use, and ease
of use. The complexity criterium means if the technology is easy or difficult to use and
understand. The perceived ease of use refers to consumers believing that the technology will be
free of effort. Finally, the ease of use criterium means the likeness if the consumers see the
technology as easy or difficult to use. In the thesis context, these factors point to, for instance,
the degree of ease exerted by university students using AI chatbots.

Social influence (SI) can be explained as the importance of others' opinions to use the
technology. Further, this construct consists of the image, the subjective norm, and the social
factor. The image shows what consumers think the technology will enhance an individual's
status in a social organization. A subjective norm is a social pressure to behave in a certain way.
It is the consumer's internalization from the social system's subjective culture. For instance, the
thesis context would be the importance of what friends of university students think about A
chatbots and if they accept and support using AI chatbots.

Facilitating conditions (FC) can be explained as a user's confidence in the organization and
technical infrastructure to be able to support the technology. In the thesis context, it would be
the ability of the universities to provide and facilitate sufficient infrastructure and resources
such as internet connection.

Hedonic Motivation (HM) can be explained as the comfort, stimulation, or pleasure a person
feels when using the technology. In other words, it is a motivation based on consumer
satisfaction. In the thesis context, this would, for instance, mean that university students are
having fun using AI chatbots.

17
The habit (HA) factor can be explained as routine when a person performs the task
automatically as a custom. This construct consists of three criteria: reflex behavior, past
behavior, and individual experience. The reflex behavior means consumers' daily life customs.
The past behavior refers to consumer behavior in the past. Finally, the individual experience
refers to consumers' routines linked to using technology. For instance, in the thesis context,
university students' habits affect their intentions to use AI chatbots.

The price value (PV) factor indicates the tradeoffs between the benefits and monetary costs of
using the technology. However, this factor has been removed from the model. It is not relevant
as it is assumed that any university would provide such service for students free of charge.
Furthermore, the factors lead to a state when the consumer has made his mind and has
formulated a conscious decision to use or not use the technology in the future. This stage is
called behavioral intention. The six mentioned constructs are the predictors of behavioral
intention in this study. The thesis author decided to apply this IS theory. It has been recently
used in the research of AI Tools adaptation by Venkatesh (2021) and in pedagogical research
(Almaiah, Alamri and Al-Rahmi, 2019) addressing new technology acceptance in higher
education. Further, the thesis author decided to rely on the UTAUT2 version used in
quantitative research by Almahri, Bell and Merhi (2020) where researchers excluded the price
value factor. The authors also did not address any moderators in their study (age, gender,
experience); however, they suggested that the future work should focus on modifications of the
prediction model by including other

Figure 7. UTAUT2 model by Almahri, Bell and Merhi (2020) enriched by age moderator

constructors such as security, trust, and system quality. Consequently, this study would use the
same prediction model but consider one of the moderators (age) and look for the other
constructs that might improve the current model (see Figure 7).

18
 3. Methodology

This chapter focuses on the research methodology adopted to answer the research questions.
The chosen methodological tradition is based on the recommendations mainly by Creswell
(2018) and elaborates on the research methodologies. This part is divided into the following
subsections: firstly, there are presented the methodological traditions and explained the
philosophical idea (or else the worldview) that this study would adopt; secondly, there is the
methodological approach where the research approach will be discussed, and last, the research
design, data collection, and analysis methods that will be used to conduct this research.

3.1 Methodological Tradition

The meaning of philosophical ideas, or else what is often called worldviews (or ontologies and
epistemologies), has to do with the general philosophical orientation that the researcher brings
to the research study about the world and the nature of the research. Respective paradigms
supplement worldviews can be described as an intellectual framework embodying a tradition
of scientific research. It helps to explain why the researcher chose a particular research approach
in the study. Worldviews are developed in individuals according to their discipline orientation
or another subject area, past research experiences, research communities, etc. (Creswell and
Creswell, 2018). Ontology is a concept that aims to answer the questions "what exists in
reality?". It is built on a belief that there is only one single reality, this reality is constantly
changing and that multiple realities exist. It is seen as the reality perceived by the researcher.

On the contrary, epistemology aims to answer questions "how do we know?" and "how can we
acquire knowledge?". It builds on assumptions that knowledge is measurable and that reality
needs to be interpreted and examined in the best way possible with the right tools. Researchers
following epistemology are trying to find the knowledge from reality (Creswell, 2014).

Four worldviews are broadly discussed in the literature: postpositivism, constructivism,

transformative, and pragmatism. Postpositivism holds a deterministic philosophy and regards
that the reality is objective and develops numeric observation measures, setting it as a more
suitable for a quantitative approach. Constructivism or else social constructivism considers the
reality of being subjective. Individuals develop meanings of their experiences, and it is more
appropriate for a qualitative approach. Transformative regards that the research has to be closely
tied with politics, and the research contains a plan of action to change the lives of the
participants that the researcher studies. Pragmatism holds the belief that truth is what works at
the time and reality is in constant change (Creswell, 2014).

This study adopts the constructivist worldview as the aim is to explore and capture the various
meanings of students in higher education institutions. The goal is to capture the individuals'
views and meanings of people in a setting. Hence, the researcher collects a diversity of views
from as many participants as possible (as the meanings of people can vary). In the constructivist
worldview, the researcher intends to interpret and make sense of the meanings of others about
the world. Further, according to Creswell (2014), constructivism or social constructivism is an
approach to qualitative research. Researchers applying this perspective believe that individuals
want to understand the world where they live and work. These individuals develop subjective
views - meaning directed toward particular objects or things. Accordingly, researchers rely
heavily on the participant's views by asking them open-ended questions. This approach can be
justified as the most suitable for this study because it is based on literature review and
qualitative interviews and surveys. As literature review analysis points to various findings of
19
other researchers and interviews and questionnaire survey seeks to understand the topic from
the viewpoint of consumers.

3.2 Methodological Approach

There are three research approaches: quantitative, qualitative, and mixed methods (Creswell
and Creswell, 2018). The quantitative approach tests objective theories by studying the
relationship among measurable variables where the collected data can be analyzed using
statistical procedures. Researchers who conduct quantitative approach test theories deductively,
looking for alternative explanations, limiting bias, and generalizing the findings. The literature
in such studies justifies the research problem and suggests possible questions or hypotheses.
Quantitative studies include variables and hypotheses. A variable can be described as a
measurable characteristic or attribute of an individual that varies among the people under study.
Hypotheses are predictions about what the results will show. A theory is placed into the study
in the form of an argument, a discussion, a figure, or a rationale. The theory then tries to explain
the phenomena that occur in the world; the researchers often test theories as an explanation to
answer research questions. The final report has a rigid structure (Creswell and Creswell, 2018).

The qualitative approach aims to understand the meaning of individuals or groups of people of
a problem while the research process includes questions and procedures. Thus, non-numerical
data are collected in the participants' environment, and their analysis is done inductively,
building from particular to general themes. One of the suggested methods for data collection in
this approach is qualitative unstructured interviews. Creswell and Creswell (2018) suggest that
the questions are generally open-ended, few in number, and intended to elicit opinions from the
participants. Researchers conducting qualitative research are looking at research that follows
an inductive style. Their main focus is on the individual meaning and rendering the complexity
of a situation. However, the literature review helps to justify the research problem does not
constrain the views of participants. The Review is usually in a separate section of the thesis.
The use of theories in qualitative research varies. For instance, some researchers place the
theory as a lens at the beginning of the study, and some generate the theory as a final outcome
of the study. The theory is in the qualitative study generally referred to as patterns, naturalistic
generalizations, or theoretical lens. The final report has a flexible structure (Creswell and
Creswell, 2018).

Last, the mixed methods approach involves collecting quantitative and qualitative data; later,
these forms of data can be integrated into a study. A qualitative approach is appropriate when
a concept or a phenomenon needs to be explored and understood because little research has
been done. There is a belief that a combination of both approaches provides a more
comprehensive understanding than either approach alone (Creswell, 2014). This approach can
include deductive theory testing quantitatively or inductive qualitative theory, or pattern
emerge. Researchers can, in this approach, decide to use either a qualitative or a quantitative
approach to the literature review.

This study follows a qualitative research approach as the purpose is to record, understand and
interpret the meanings, experiences, and opinions of the individuals involving the collection of
non-numerical data. Additionally, Myers and Avison (2002) note that it is designed to help
researchers understand the people and the social and cultural context. The approach has been
chosen based on Creswell (2014) suggestion to choose this approach if the topic is new and the
subject has never been addressed before with a particular sample or group of people. When
existing theories do not apply to the particular sample or group under study. This

20
recommendation suits this study's characteristics. The qualitative approach is also particularly
useful when the researcher does not know the necessary variables to examine.

3.2.1 Research Design

Researchers not only select research methods to conduct the study (qualitative, quantitative, or
mixed methods), they also must decide on a type of study within these three approaches.
Following the section of the methodological approach is the section of the research design (or
else strategy of inquiry). The research design has to do with the structure of the research project,
the plan, or the framework for carrying out an investigation focused on a research question
(Schwartz-Shea and Yanow, 2013). For example, the design associated with quantitative
research includes experimental designs such as true experimental or nonexperimental designs,
such as surveys. If researchers adopt a qualitative approach Creswell (2017) further
recommends among the possibilities of design such as narrative, phenomenology, ethnography,
case study, and grounded theory.

To present briefly, in narrative research, the researcher tries to analyze and understand stories
provided by individuals. Here, the researcher focuses on one or more individuals and gathers
data by collecting their stories. Then, the researcher reports the experiences of these individuals
that are ordered accordingly chronologically. Often, in the end, a narrative is presented with are
combined views from the participants and researcher. Phenomenological research involves
describing the meaning of several individuals of their lived experiences about a concept or a
phenomenon. The researchers in a phenomenological study focus on describing what all
participants have in common as they experience a phenomenon. This type of research aims to
reduce the meaning of the individual experiences about a phenomenon to a description that
provides an understanding of the actual phenomenon like, for example, insomnia, something
that Creswell (2007) calls the description of the universal essence. In the grounded theory, the
researcher moves beyond description and derives a general, abstract theory of a process, action,
or interaction based on the participants' views. All the participants in this study should have
experienced this process, and the development of a theory can help provide a framework for
further research. Additionally, ethnography is a design where the researcher studies shared
patterns of behaviors, language, and actions of a cultural group in its natural setting for an
extended time period. Last, a case study analyzes in-depth a case, an activity, or a process of
one or more individuals. The researchers, therefore, collect detailed information by using a
variety of data collection procedures for a sustained period of time (Creswell, 2007; Creswell
and Creswell, 2018).

The thesis researcher decided to collect data from various sources to explore activities with a
detailed description of settings. The most suitable is to follow the case study design.

3.2.2 Case study

Creswell (2007) notes that researchers in case studies explore a case through detailed data
collection, including multiple sources of information such as observations, interviews, audio-
visual materials, documents, reports, and questionnaires. The case is seen as a real-life,
contemporary bounded system, and the researchers explore the case through multiple detailed
sources of data collection. One of the characteristics of a case study is the unit of analysis and
not the topic of investigation. For example, they are analyzing the experiences of a specific
group (Merriam, 2002). Further, Merriam (2002) provides an example to understand how the
unit of analysis characterizes a case study. The example involves a study of women’s
experiences in welfare-to-work training programs. Although it can be a qualitative study, this
21
study does not necessarily mean that it should be a case study. To explain, the unit of analysis
is the women’s experiences, and there could be an indefinite number of women selected for this
study. However, for that to be a case study, one particular program (bounded system) has to be
selected. A bounded system is typically unique, experimental, and it is the unit of analysis. For
instance, a case study could also be conducted by recording the experiences of a single woman.

In this thesis, the case study records the experiences of university students (who might already
have used chatbots in different circumstances); now focused on their thoughts on the potential
use of AI chatbots in particular bounded systems of higher education institutions. This study,
therefore, examine the application of AI chatbots in general but in a specific setting.

3.3.3 Types of case studies

There are different types of case studies mentioned in the literature. For example, according to
Yin (2009) a case study aims to answer “how,” “why,” and “what” questions to explore a
phenomenon. The author further distinguishes between three types:

a) Explanatory - is used when the researcher seeks to answer a question that aims to
explain the presumed causal links in real-life interventions that are too complex for the
survey or experimental strategies. This type of case study seeks to answer “How” and
“Why” questions
b) Exploratory - is used to explore situations in which there is no single set of outcomes.
This type aims to answer “How” and “What” questions.
c) Descriptive - is used to describe a phenomenon and the real-life context that it occurred.

Further, Stake (2006) makes the following distinction among case studies:

a) Intrinsic - a case study is characterized as intestacy when the researcher has an intrinsic
or else enduring interest for the case itself that is not necessarily a means of learning
about general problems.
b) Instrumental - is when the purpose of the case study is to go beyond the case, meaning
that there is a need to understand the case to understand a bigger research question at
hand. The goal of this case study is to understand something more by understanding it.
c) Collective - or else called multiple cases. Each of them is useful for the learning
experience. The researcher selects multiple cases to illustrate an issue

This study follows an exploratory case study (students' thoughts on the potential use of AI
chatbots in particular bounded systems of higher education institutions). It explores the
acceptability of AI chatbots in higher education as a communication medium to inquiries
addressed to the university by students. Additionally, the thesis researcher regarded that there
is not a single set of outcomes. To further justify the chosen type, a detailed description of the
situation is established by recording the meanings and opinions of the relevant stakeholders.
The study seeks to answer the questions: "How" can AI chatbot technology be used in the higher
educational setting (usage area) and “What” are the factor behind the acceptability. Therefore,
the thesis researcher considers the study exploratory.

3.2.4 Participants’ Selection Criteria

Bounded systems in this thesis refer to cases being bound by time and place. The phenomenon
of the study is the acceptability of AI chatbots by university students (consumers). Real-life
contemporary bounded system (university student acceptability AI chatbots). The researcher
purposely decided to explore university students' perceptions from different countries to obtain
22
in-depth data, not limited to a particular country. Before selecting the population, the thesis
researcher considered several aspects. For instance, the chosen population must be aligned with
a chosen social problem. The researched chosen social problem is the lack of incorporated AI
chatbots into educational settings. The purpose is to understand how university students view
AI chatbots (if their beliefs vary from the previous research). Therefore, the thesis population
includes the students interviewed or approached to answer the open-ended self-administered
questions in the online survey tool MonkeySurvey. Thus, the selected sample of the population
has knowledge regarding the phenomenon which is being researched.

A population can be described as a set of cases from which a sample is created. Due to time and
resource limitations, the researcher of this thesis cannot analyze the entire population. The
number of cases has been reduced and narrowed down to a specific group - university students
(Taherdoost, 2016). The thesis researcher considered the main aspects to justify the choice of
university students as this study’s population. Firstly, university students' inclusion is relevant
as they are seen as consumers (target users). Thus, their view on this technology is essential for
the success of the implementation. Secondly, one of the research questions elaborates on
student-university communication.

The population location (Sweden, Norway, Finland, Czech Republic) has been predetermined
based on the researcher’s personal network pool. All participants come from predefined
countries, are enrolled in a university, and are between 20-32 years old (Generation Y or
Generation Z). The age of participants is one of the moderators of the acceptability study. In
addition, the researcher delineated that the selected participants must be enrolled in a bachelor
or master's study program (not in doctoral or a single standing course).

Their gender, university program, and socioeconomic status vary; however, this supports the
choice of the sample as students in real life are not heterogeneous. In other words, this sample
is representative of the population.

To meet the participant’s criteria (university students), the qualitative sampling could not be
done randomly. However, one of the sampling techniques called snowball recommended by
Creswell (2013) has been used to identify the participants. Besides the social media groups, the
researcher asked people in her network to help them to identify others who fit these criteria and
decided to conduct interviews with 3 participants and approach the rest of the 17 participants
with the questionnaire survey until data saturation occurs. However, the researcher has not been
engaged prior with the interview participants and, as the questionnaire survey is done
anonymously, is not aware of any relationship to the population.

The researcher does not purposely distinguish between gender (meaning that this study does
not seek to understand if particular gender has higher acceptability of AI chatbots than others).
Instead, the researcher seeks to reveal the readiness to accept this technology to reveal the mood
among university students to use AI chatbots in general, not to point who should be in the pilot
group and act as an early bird to implement the solution.

Generally, with more diversity between the participants, more cases are needed for data
saturation than in a more homogeneous sample (Taherdoost, 2016). There could also be other
factors that might distinguish participants into a more heterogeneous sample. For example, the
differences between private or public university students, digital readiness or strategy of each
university or country, different funding possibilities of each university to maintain digital
technology. Besides, some countries might be more digital-oriented than others. Therefore, the
thesis researcher decided to consider the chosen participants as a homogenous sample, even

23
though more diversity could be found if looked for. As the As the thesis follows qualitative
research, which is not generalized to the wider population (does not imply a large number of
participants) the researcher believes that such collected information (gender, demographics, and
background) might lead to wrongly understanding assumptions only because of limited
examples. Hence, due to the limited number of participants, this study has not covered the
UTAUT2 moderators of gender and experience. The researcher did not want to influence the
research based on personal bias and assumptions from information based only on a few
examples. Roberts (2020) points out that a qualitative researcher might disseminate findings
based on personal bias and prejudice if proper preparation is skipped. The researchers that
approach the research as if they know the answer to the research question might be tempted to
guide the participants to provide only responses that support their predetermined ideas and
beliefs.

3.3 Methods/Techniques for Data Collection

This section highlights the methods used to collect the data for this thesis research. As the thesis
researcher explores a phenomenon, the primary type of data collected is the participants'
responses. However, as the case study has been chosen, the data should be triangulated,
meaning to be collected from other sources to completely understand the phenomenon. The
thesis researcher also incorporated field notes from the interviews into data collection, as some
of the notes provided additional information that later helped put the interview responses into
context. A case study employs multiple methods for data collection of one or more people
(Myers and Avison, 2002). In this study, two methods are used for data collection: a primary
suggested by Merriam (2002) semi-structured interviews with open-ended questions and
second surveys with open-ended questions (Saris and Gallhofer, 2014).

According to Alshenqeeti (2014) qualitative research provides detailed descriptions of

individuals, events, and natural settings. The interview is generally thought of as a key factor
in research design and communication with people as one of the most effective methods for
exploring constructs. Data gathered during an interview can extend the scope of understanding
investigated phenomena. It is a rather naturalistic than structured data collection tool. Further
describes three types of interviews: structured, unstructured, and semi-structured. The critical
feature of structured interviewing is a set of predetermined direct close-ended questions that
require an immediate response. For instance, the ‘yes’ or ‘no’ type of response. Unstructured
interviewing is characterized by flexibility and freedom in terms of organization and
implementation and is offered to both interviewers as well as to interviewees.

This thesis uses semi-structured interviews as this type is considered the most appropriate in a
case study, according to Hancock and Algozzine (2017). Researchers can ask predetermined
but flexible questions followed by further unstructured questions, allowing them to express their
perspectives openly. Data collection through interviews is a standard method in case study
research as interviewing individuals or groups allows the researcher to obtain rich and
personalized information. Therefore, the researchers tend to use open-ended questions so that
interwebs can share their views (Creswell and Creswell, 2018). Also, researchers follow the
guidelines by Hancock and Algozzine (2017) to perform a successful interview. These authors
suggest that a researcher firstly identifies key participants whose knowledge and opinion
provide helpful insight. Secondly, the researcher should formulate an interview guide/protocol
with open-ended questions to ask the interviewees. Thirdly, the researcher needs to consider
the setting where the interviews is conducted. Fourthly, a researcher must record the session to

24
prevent the loss of valuable information. Fifthly, the researcher must consider the legal and
ethical requirements that concern the involved people.

The thesis participants are university students from multiple locations (Sweden, Norway,
Finland, Czech Republic). These countries have been purposely selected, as the researcher has
her personal network pool in these countries. Regarding the settings, due to the ongoing
COVID-19 pandemic, the interviews have been, as suggested by Merriam (2002) held online.
Additionally, the participants have been asked if they consent to record the interview. The
recording allowed the researcher to analyze the interviews and ensure that no data was lost.
Further, the thesis researcher considered ethical requirements.

Another qualitative research method is a survey with open-ended questions (Saris and
Gallhofer, 2014). Surveys are a usual method in quantitative studies that aim primarily to
describe numerical distributions of variables. However, a qualitative survey aims to determine
the diversity of some topics of interest within a given population (Jansen, 2010). That kind of
survey can be performed in the context of critical theory or constructivist projects. When it
comes to case studies, a qualitative survey can be addressed in the analysis as a multiple case
study (Jansen, 2010).

The participants have been engaged on social media (Facebook) university student groups or
engaged via messaging platform (Messenger) from the researcher's personal network pool.
Further, other participants were addressed based on the participants' recommendation based on
the snowball technique recommended by Creswell (2013). The primary data collection was
completed through a survey and interviews held in March 2021, and participation was
voluntary. A reward of any type has not been provided. However, participants were informed
about the aim of the thesis before the data collection. In addition, the information about the
possibility of withdrawing (right to step out of the interview/survey and not participate further)
has been provided. In general, questionnaire survey participation required complete 10 open-
ended questions that inquired about the participants’ perception of AI chatbot's potential usage
in higher education (see Appendix C.: Survey questions). The participants have been later
referred to as respondents (R1-R17). As per the MonkeySurvey site, the average response time
of each participant was 10 minutes. On the contrary, interview participation has been held over
online conference calls (Zoom), recorded, and the average time spent answering the open-ended
interview questions was circa 45 minutes.

3.3.1 Interview process

Roberts (2020) advises that the most important aspect of the interview is to keep in mind that
the purpose is not to get the informant to answer the questions but to listen to participants to
acquire the meanings associated with these experiences. Researchers should have an attitude to
see the participant as an expert on their experiences, and only a participant possesses the needed
knowledge to answer the research questions.

The three interviews took place on 26th March 2021 over an online conference call (Zoom).
Each took approximately 45 minutes to conduct. As the researcher proceeded with semi-
structured interviews, a set of 20 questions was prepared before the meeting (see Appendix D.:
Interview question). However, some questions deviated from the list and have been asked
spontaneously to clarify the answers. During the call, the list of the questions has been used to
reference what needs to be addressed and to keep the interview structured but not framed to
limit the focus. Roberts (2020) suggests developing a list of main questions to break the topic
into components. The thesis researcher divided the interview list into 6 parts, 5 consisting of
25
main questions and 1 part with screening questions. Also, probs have been incorporated into
the interview, however not written down, but kept in mind to use during the interview meeting.
For example, the thesis researcher used probes like “Go on,” and “That is interesting, could you
tell me more,” or “Could you go back and tell me about... “ Probs, as described by Roberts
(2020) support the researcher in keeping the interviewee engaged, summarizing the topic,
managing the flow efficiently, and checking for understanding.

The initial "meeting greetings" conversation has not been recorded. Instead, the recording
started once the initial conversation ended and moved to the interview questions. The initial
phase made participants feel more comfortable and more open to answering the questions. The
recording ended with the last answer; however, the interview continued for a while longer as
the thesis researcher thanked each participant for their contribution. In notes, the participants
have not been referred by their name but by a personal identifier assigned to each person (P1,
P2, P3); their field of study and location have been written down. Next, the researcher used the
otter.ai solution to transcribe the voice interviews into the text automatically. Following
transcription, the researcher engaged the interview participants with a transcript as an email
attachment. This process step is called member checking and aims to review the transcribed
interview to double-check and support the accuracy of captured information. The feedback for
the member checking did not point to any changes in the text.

3.3.2 Informed consent

Informed consent can be described as a process of providing clear and truthful information
about the research to participants. According to Creswell (2014) informed consent form
acknowledges human rights and protects participant’s data. It contains a standard set of
elements such as a guarantee of confidentiality, identification of the researcher, assurance that
the participant can withdraw at any time, and provisioning of contact details if questions arise.
Furthermore, the participants can decide not to participate if they disagree with the instructions
for consent provided by the researcher. Also, no participant can be forced into signing the
consent (Halse and Honey, 2005).

Regarding questionnaire survey participants: informed consent was sent within the messaging
tool (Messenger) requesting participation. Once the potential candidate expressed interest in
being part of the study, the text was sent to participants (see Appendix E.: Consent form
template). Three of the participants have volunteered for an interview, and the consent has been
again mentioned at the beginning of the call. In addition, all of the interview participants
expressed their permissions to be contacted for additional information and for member
checking. All participants have also been advised that the data are stored in an encrypted share
drive used only by the thesis researcher for 5 years based on Peng (2017) recommendation for
computer science studies.

3.4 Methods/Techniques for Data Analysis

This section deals with the chosen method for data analysis and the justification of this
particular model. According to Creswell (2014) data analysis is an ongoing process, and it
consists of participant information analysis. Creswell (2014) further recommends that
qualitative research adopting a constructivist worldview proceed with inductively data analysis,
building from particular to general themes. This study uses two types of analysis – thematic and
content analysis. The general thematic analysis includes the organization of data, initial reading
through the transcript text, coding the data, and formed description. It is a systematic technique
recommended by Braun and Clarke (2006) for identifying, analyzing, organizing qualitative

26
data sets and further describing them. The method is relatively flexible. It is not tied to a
particular epistemological or theoretical perspective. Furthermore, it is agnostic, as it can be
used in inductive (data-driven) and deductive (theory-driven) analysis as well as to capture
semantic(explicit) or latent (underlying) meaning. In semantic analysis, the researchers focus
purely on what participants said or what information was provided. On the contrary, latent
analysis seeks to go beyond what has been said to examine underlying ideas and assumptions.

The analysis is conducted within a constructionist framework focusing on latent themes rather
than semantic, as the study aims to seek the motivation of acceptability of the AI chatbots by
university students (consumers). On the other hand, an analysis focusing on semantic themes
does not seek to focus on motivation but seeks to theorize the socio-cultural contexts. Further,
latent theme analysis tends to be more constructionist. In more detail, in the inductive content
analysis, the researcher is looking for patterns, and therefore it is also called data-driven analysis
(Graneheim, Lindgren and Lundman, 2017). Data are strongly linked to the data themselves.
Researchers are coding the data without trying to fit it into a pre-existing frame; thus, the
process is not driven by the researcher’s analytic preconceptions (Braun and Clarke, 2006).

In contrast, in the deductive analysis known as concept-driven, the researchers test existing
theories about a phenomenon under study against the collected data (Graneheim, Lindgren and
Lundman, 2017). Thus, an analysis is driven by the researchers’ theoretical or analytic interest;
however, the outcome has a less detailed description of the overall data (Braun and Clarke,
2006). Clarke and Braun (2013) further advised not to use the main interview questions as the
themes, hence only summarizing and organizing the dataset instead of truly analyzing it.

Once the member checking was completed, the thesis researcher decided to use the qualitative
data analysis software (NVivo) to facilitate coding and sorting data sets from questionnaire
surveys and interview transcripts into the initial results themes. The software is based on
Natural language processing (NLP), combining the researcher’s several rounds of coding to
determine categories of themes (Bazeley and Jackson, 2013). In this study, a qualitative
approach was employed using online interviews and questionnaire surveys. The thematic
analysis has been performed in NVivo by attaching codes to data units while analyzing the files.
The content analysis has been performed manually by the thesis researcher. The collected data
has been analyzed, focusing on latent themes.

As the NVivo tool was not used before, the thesis researcher decided to perform an additional
manual content analysis to analyze the text. Even though the procedure has been more time-
consuming, the researcher aimed to cross-check the outcome of the manual analysis with the
software support analysis to avoid any missing themes. The content analysis helped to analyze
large amounts of data collected from interviews and survey answers quantitatively. The aim
was to attain a summarized and broad description of a phenomenon. First, the researcher
extracted words into content-related categories to build up a conceptual system, as suggested
by Elo and Kyngäs (2008). Then, the data has been organized into specific units of words.

Further, the transcripts have been read over to narrow down the major themes that emerge. As
a next step, the text has been coded into categories concerning the research questions. The
researcher of this thesis was looking for similarities or differences in the collected data, which
were then described in categories to point to theoretical understanding. Once the final categories
of themes had been determined, the researcher conducted the relational application to the
UTAUT2 model.

27
3.5 Reliability, Validity or similar

Qualitative research validity refers to determining the accuracy of the findings from the
researcher's viewpoint, the participants, and the readers of the research. Various terms can
define validity as trustworthiness, authenticity, and credibility (Creswell, 2014). To ensure the
validity of the study, the researcher applied different strategies and procedures. Creswell (2014)
recommends the use of multiple strategies to increase the accuracy of the results. The frequently
used strategies are eight. The first strategy is the triangulation of different data sources. The
second strategy adds to the study's validity to establish themes based on several data sources or
perspectives from participants. For instance, the researcher can use member checking by taking
the final reports or the themes back to the participants and ascertain if the participants feel that
the themes are accurate. The third strategy describes the findings abundantly and in detail to be
realistic, and the readers are transferred to the setting. The fourth strategy explains the bias
brought in the study and reflects on how their interpretation was influenced by various factors
like the researcher’s background, gender, and culture. The fifth strategy refers to differences in
real-life opinions of people. Opinions vary, for instance, due to different perspectives.
Therefore, it is advantageous to present information that contradicts the perspective of the
themes, making the report more valid. The sixth strategy advises spending a long time in the
field so that the researcher is able to develop a deep understanding of the studied phenomenon
to be able to communicate in detail the existing situation and the people. The seventh strategy
aims to find/use/locate a person who to review and ask questions about the study to include
another person’s interpretation. Last, and the eighth strategy advices to use an external editor
to review the entire project meaning to use/utilize a person who is not familiar with the study
and thus provide a more objective assessment of the project (Creswell, 2014). However the
thesis researcher did not followed all the suggested strategies, for instance triangulation has not
been adopted for this thesis nor spending a long time in the field.

Reliability in qualitative research shows that the researcher’s approach is consistent among
other researchers and projects. For example, Creswell (2014) mentions two known researchers
on how they established the reliability of their studies. The first known researcher who
suggested that researchers in a qualitative study should document the procedures of their case
study was Yin (2009). He suggested that researchers should describe as many steps of the
process as possible and establish a detailed case study protocol so that others can follow the
procedures. The other known researcher is Gibbs (2007). He proposed to eliminate transcript
mistakes by constantly comparing the data with the codes. Also, the researcher can find another
person who can cross-check their codes.

The background of the researcher and position might affect what they chose to investigate. It
could influence the angle of investigation, the methods selected for this purpose, the findings
considered most appropriate, and the conclusions (Malterud, 2001). In addition, a researcher
often has certain opinions about what it is all about. Therefore, different researchers can have
different interpretations and representations depending on each one’s perspective and position
of a studied situation (Malterud, 2001).

Furthermore, the validity and reliability of this study can be supported by correctly using
scientific methods. For instance, the data collection, analysis, and interpretation are made in
line with scientific methodologies. Moreover, the guidance and feedback from the course
examiners, the supervisor, and peer validation, help the thesis researcher contribute to valid and
reliable characteristics. For example, the literature review could be validated through cross-
checking the papers listed in the reference list. Further, the thematic analysis has been done
manually as well as by software support to obtain accurate and dependable results. Regarding
28
the validity of the interview, the researcher has followed interview recommendations for the
novice by Roberts (2020). For instance, the researcher crafted the interview questions with
support from the supervisor and prepared a protocol as a guide. Later on, the analysis reflected
on the effectiveness of the interview questions. Further, some steps of the Interview Protocol
Refinement Framework (IPR) by Castillo-Montoya (2016) have been followed to ensure that
the open-ended questions align with the research question and that the questions can simulate
the real conversation, however, focused on the study objective.

As a limitation, only qualitative methods were followed due to the chosen methodology
approach. The qualitative interview also has some limitations, as it provides indirect
information, filtered by the participants' view. Further, researchers' presence might bias
responses. Also, not every participant has articulation and perceptive skills (Creswell, 2014).
On the other hand, expanding the methods to qualitative observation might bring firsthand
experience with participants. One of the critical limitations was time, resource, and sample size,
and it might influence the results of this thesis. An extended time frame, additional researchers,
and more participants in the study might add additional perspectives and allow the researcher
to cover additional UTAUT2 moderators.

3.5.1 Saturation

According to Fusch and Ness (2015), saturation can enhance the validity of the qualitative
study. The study population consisted of 20 participants; 3 of them participated in the interview,
and the rest in the questionnaire survey. Creswell (2014) explains saturation as a state when the
researcher stops the data collection as the themes emerge and further data gathering no longer
brings new insights. Furthermore, saturation is reached when there is enough data to replicate
the study (Fusch and Ness, 2015). Data saturation occurred during content analysis at
participant n.16, with the emergence of common themes concluding. The thesis researcher
could not determine any new information, coding or a new theme from the rest of the
participants. However, the additional 4 participants from the questionnaire survey enhanced the
data-rich responses obtained, especially to confirm the confirmation of subthemes rather than
single standing primary themes.
3.6 Ethical Considerations

Research ethics concerns the value, standards, and institutional arrangements that regulate a
scientific study. During the conduction of research, certain moral principles need to be followed
while the research is conducted. These include the relations between the researcher and other
people, as well as the environment and society in a broader sense. An essential part of research
ethics is how the people who participate as subjects in research should be treated (The
Norwegian National Committee for Research Ethics in Science and Technology, 2016).

Furthermore, the author of this thesis anticipated the ethical issues which arise during studies.
Creswell further (2014) points out that ethical questions are evident in areas such as personal
privacy, authenticity, and credibility of the thesis. The researcher also worked in cross-cultural
contexts. Regarding the literature review, the Harvard referencing system has been followed.
The author's information captured in the literature was not manipulated and was expressed as
accurately as possible.

Additionally, the researcher needs to protect participants of the study by not causing any harm
to research subjects, promote integrity, avoid any misconduct, and remain responsible towards

29
society. The author of this thesis also followed Creswell (2014) suggestions of ethical issues
and how to address them. For instance, at the beginning of the study, the author identified a
research problem that benefits participants. Creswell (2014) further advises treating participants
equally, reporting multiple perspectives, communicating clearly, and reporting honestly. In
more detail, the researcher followed all the legal and ethical requirements in this research that
involved people (Hancock and Algozzine, 2017). The participants have not been deceived and
have been protected from any form of mental, physical, or emotional injury. Their data and
their privacy have been protected so that no names will be exposed in the future. Therefore, the
information obtained from interviews and surveys is anonymous and confidential. Additionally,
the participants were informed about the purpose of the study, and they provided their consent
in the participation of this research as voluntarily. Last, the interwebs have the right to withdraw
and end the interview at any time (Hancock and Algozzine, 2017).

4. Empirical Findings
This chapter presents the findings coming from the analyzed qualitative data. Through this
process, certain themes were identified by interpretation, as the researcher drew meaning from
the findings. Creswell (2014) suggests that such findings can be in the form of a lesson learned,
personal experience, or information in contrast to information obtained in the reviewed
literature.

4.1 General findings

To ensure that the participants have the necessary experience in student-university

communication, two screening questions were asked at the beginning of the interview and
questionnaire survey. The general findings from the interview and questionnaire survey data
showed that the university students are in frequent contact with the university staff – teachers
and admission office employees. The students are mainly approaching their teachers regarding
two areas. The first topic type involves questions about assignment submission dates, deadlines
of projects, and dates of exams. The second type consists of course-related questions about
requirements. These two areas can be explained as “when” it needs to be delivered and “what”
needs to be delivered to pass the course. These two areas are the most crucial for the student to
go through the university education, but they can be read in many ways. Firstly, it indicates that
the students are serious about their studies, and that is why they are approaching the university
staff to make sure they do not fall behind in their studies. Nevertheless, it can be a consequence
of insufficient information provided by the university. On the other hand, it can be caused by
the student's fruitless search for information; however, none of the participants suggested it.
The students also frequently communicate with the Admission office. The questions mainly
address the credit policy (needed amount of credits to continue), and the course retakes policy.

The presented data is coming from three interviews and the replies of 17 questionnaire surveys.
Respondents of the interviews are shown in Table 1. Because the survey tool does not
automatically gather the data about the participants, information such as gender or field of study
has been noted only from the interview participants.

30
 Table 1. Respondents of the interviews

Nr. Participants of interview Field of study (Major) Gender Country

1 P1 Nursing F Czech
2 P2 Marketing and Finance F Finland
3 P3 Computer science M Norway

The first screening question was: “How often are you engaging with teachers/university
staff/university library per month (email, forum, chat, face-to-face, etc.)?”. The results from
the screening question suggest that every participant is engaged in some kind of student-
university communication at least two to ten times per month. The second screening question
was: “With what questions/topics are you mostly addressing teachers/ the university office/
university library?”. As shown in Table 2, the results from the screening question suggest that
students approach their university with the seven most common types of topics, mainly with
questions about the dates of exams, followed by course and assignment requirements types of
questions and questions concerning the Admission office. As shown in Table 2, students do not
engage a university with one type of topic only; they engage university employees with several
topics based on a situation. For instance, a student can contact a university with recommended
literature questions and consultation questions within a month.

Table 2. Most common topic engagement

Number of participants
Engagement topic (out of 20) engaging a
university with this topic
Deadlines and dates of exams/retakes 16
Requirements (course, assignment) 14
Recommended literature, availability of books 6
Thesis supervision, consultation 7
Student funding, interlibrary loans 4
Admission office (credits, course retakes) 10
Other 3

4.2 Findings from the themes

Firstly the researcher has analyzed the data with software NVivo support. This pointed to three
primary themes: Fast support, Limited capabilities, and Lack of concerns. Then, the researcher
did the second analysis without software support and provided additional two primary themes
(No judgment and Simplicity kills). It might be caused by personal preferences or a lack of
familiarity with the software; however, the researcher believes that the second analysis
indicated the additional themes. Finally, coding resulted in five primary themes, three
subthemes, and 13 codes (see Table 3 and Appendix F.: Themes, subthemes, and codes). The
selected responses have been incorporated from different questions; however, they address the
same theme. Meaning, the responses are not coming from one particular question, but the codes
have been mentioned multiple times throughout the interview and questionnaire survey.

31
 Table 3. Identified themes, subthemes and codes

Theme n.1: No judgment. The researcher specifically asked about the negative aspects of
communication with humans, what feelings are associated with the communication, and the
reply. In total, 20 participants provided 39 related responses which could be linked to this
theme. Some of the participants of the interview (P1-P3) and respondents of the questionnaire
survey (R1-R17) advised that they have negative feelings associated with the communication
to the university employees and the awaiting the reply. Some participants mentioned feelings
such as shyness regarding approaching the teacher in front of other students and would prefer
to ask their questions anonymously. In contrast, others mentioned that they are worried about
how the teacher will perceive their questions.

These findings regarding behavioral and psychological aspects of students’ communication,

especially the negative ones, might be mitigated by the adoption of AI chatbots. By providing
a stress-free solution for particular student-university communication topics.

In more detail, this theme is divided into 3 subthemes. The first subtheme (Ideal state) is based
on responses advising that students prefer to keep identities secret for various reasons. The
second subtheme (Feelings) is based on what students feel while engaging the university staff.
The subtheme links to different feelings during the communication, not on the reason for the
communication. There are two very different codes under this subtheme. For example, some
students are experiencing a negative feeling of guilt of allocation of other time. While for some,
it is a relief as they see the engagement as a workaround to their shyness to ask out loud. It can
be argued that one of the feelings does not exclude the other; however, one prevails, as none of
the participants mentioned them together.

P3: “Sometimes, when I do not follow the course closely, I feel other students will think of me
poorly, so I rather ask the teacher separately as you cannot ask in the forum anonymously.”
R10: “I got the feeling that I am delaying someone from their work by my question.”

The third subtheme (Action reduction) is based on students’ responses suggesting that some
parts of the communication could be removed and eliminated. For example, students feel
obligated to be more polite while communicating with a university staff than others. The

32
behavior could be described as walking on eggshells, in contrast with the communication to the
AI chatbot, allowing students to go to the point without any decorum or exaggerated courtesy.

P1: “I try to be as nice as possible in emails, very formal, and very polite. I am trying to title
them properly, using please and thank you. Maybe it’s something in our post-communist culture
that teachers are above students, and they have the power to punish you. I don’t know; I just
feel this way. Anyway, in the past, teachers had the power to expel you and make sure you were
not able to get to university, not for your grades but political attitude, the cultural feeling might
still be there.”
R1:” With technology, there is no need to be super polite or shy.”

Some of the participants mentioned that they felt nervous and nauseous waiting for a reply. As
the latent analysis seeks beyond the responses to examine assumptions, the researcher got the
impression that university students fear being judged while seeking attention in parallel.
Figuratively speaking, the researcher got the impression that the students wanted to be served
with the information. Besides, the respondent should not have any right to opinionate or show
an attitude towards the asker.

P1: “Sometimes teachers' answers are quite short, and you can say “cold”. Stating a fact. The
worst is when they don’t understand what I’m trying to say, as the original info is in 2 emails
prior, or when they do not read the email thoroughly and answer me to something different.
Sometimes I feel like a child, wanting to reply “thanks for this advice, but I’m not 15 anymore,
I could have figured this by myself. I’m asking for more management decisions, not advice to a
kid”. But I have also come across a great response, so it's more about the personality of a
teacher.”
P3: “Yes, you might say sometimes I could feel from the answer that I have disturbed them in
the middle of the joke or cup of coffee.”
P3: “Sometimes, when I do not follow the course closely, I feel other students will think of me
poorly, so I rather ask the teacher separately as you cannot ask in the forum anonymously.”
P1: “Stressed and nervous. Sometimes I even think – was I too pushy? What if the teacher will
remember my name and get me harder exam questions. Maybe it's silly. Also, I’m thinking –
maybe it went to spam and they will not answer at all.”

Theme n.2: Fast support. The responses indicate the perceived benefits of using AI chatbots
(without the educational intention) in communication and information provisioning. The
researcher specifically asked about the opportunities that technology might bring to students
and important AI chatbot features. In total, 33 related responses have been linked to this theme.
Most of the participants in the interview (P1-P3) and respondents of the questionnaire survey
(R1-R17) highlighted the speed of the response and the 24/7 availability. The researcher got the
impression that university students are intensely annoyed by waiting for answers, and it is not
associated with any positive feelings. The researcher assumes that, in general, waiting and
pending the reply is seen as a very unpleasant aspect of communication. The students do not
demonstrate any patience, thus could not keep a positive attitude during the waiting. Said
exaggeratedly, the waiting itself is viewed as a personal crime towards them. The assumption
is that this attitude might have been associated with the age of the population sample (20-33
years old), thus belonging to millennial and iGeneration culture.

P1: “The biggest benefit is to get an instant response.”

P2: “Fast response.”
33
P3: “Time-saving.”
R1: “Workaround to get a fast, instant answer. No need to stress over the weekend or days to
wait for the teacher to answer.”
R2: “Get the answers quickly.”
R3: “Get basic information fast and easy.”
R5: “I would choose AI chatbot due to its 24/7 availability. In case of complex questions when
I need detailed answers, I would contact a person.”
R15: “Being able to answer questions about university procedures so I don't have to read long
documentation.”
R17: “Quick search for specific information.”

Theme n.3: Simplicity kills. The researcher specifically asked about the previous experience
with AI chatbots and the results of the experience. In total, 29 related responses have been
linked to this theme. The researcher got the impression that participants do not distinguish
between AI chatbots and rule-based chatbots. The participants mainly mentioned poor language
understanding and characteristics, leading the thesis researcher to assume that they have more
rule-based chatbot experiences than AI chatbot experiences, which confuses them. As the latent
analysis seeks to go beyond what has been said by participant, the researcher has the underlying
understanding that simple solutions do not provide an acceptable level of satisfaction.

R2: “Sometimes useless. I had to change my question to point the chatbot in the right direction.”
R5: “One time I got the info I needed quickly, another time chatbot misunderstood my
question.”
R6: “I have used this in my bank info. It helps but for the very basic things.”
R8: “Negative. I was not provided the information I was looking for and no other types of
sources of information were available.”
R9: “I used chatbots on some e-shop websites. Mostly annoying and not very useful.”
R15: “I would not communicate with current AI because it can only answer some questions.”

Theme n.4: Limited capabilities. These responses represent the participants' perceived abilities
of AI chatbots. In total, 12 related responses have been linked to this theme. The researcher
specifically asked participants about the possible challenges of the usage of AI at a university.
The participants mainly mentioned the lack of capabilities of this technology, lack of university
resources, and the prognosis that the technology might get better in the future. However, given
the assumption (substitution of AI chatbots to rule-based chatbots) from the theme n.3, and
participant's lack of belief in the capabilities of AI chatbot technology might be explained by
their experience with the simpler types of chatbots.

R2: “In the future, once technology develops, the chatbots will understand better.”
R6: “Al chatbots can help until one point, you can use the word you want, and it can provide
info. However, the info is not always enough for the word you are searching for. Chatbot
technology can does not know all the words.”
P3: “If the chatbot would make mistakes or don’t have answers even to general things, that
would only mean that the knowledge has not been put into the system. There must be someone
– some person- to upload the data continuously. Many times, there are no resources for that at
my school.”

Theme n.5: Lack of concerns. The researcher specifically asked if the participants had any
concerns regarding AI chatbots (privacy, security, technology is too complicated, non-human,
etc.). Only a handful of participants had suggested some concerns. The responses show that

34
they lack concerns such as privacy and security. Only handful of participants and respondents
(4 people out of 20) mentioned any worry or slightest concern about security/privacy or non-
human communication. It would be hard to believe that the current students have never heard
of security or privacy issues. Thus, the rationale behind it could be that the consumers blindly
believe in system protection. Alternatively, they might not have experienced any
privacy/security breach that harms them or does not see such data (provided by AI chatbot
without educational intentionality) as sensitive and do not prioritize its protection.

R1: “No.”
R5: “No.”
R8: “No.”
R10: “Yes, any departure from face-to-face action raises concerns and risks.”
R16: “No.”
P1: “Not really, it’s the same with the email.”
P2: “Maybe a bit. Did not think about it.”
P3: “There could be privacy and security issues, but as it will be, I assume under login (course
info), so it would be encrypted.”

5. Discussion
This part is dedicated to discussing the findings from the systematic literature review and the
findings from the qualitative data gathered from the interview and questionnaire survey in
relation to the UTAUT2 model. As the researcher followed the case study, themes were
analyzed for multiple cases; these themes were further compiled with UTAUT2 constructs to
identify findings to answer the thesis research questions. Once the primary themes were
compared to the constructs, the researcher indicated that three of the primary themes correspond
with UTAUT2. However, one primary theme points to a new construct of the predictive model,
and one primary theme does not hold any significance at this moment regarding the
acceptability of AI chatbots without educational intentionality (see Figure 9).

Figure 9. Identified themes and codes compiled with UTAUT2

The first theme (No judgment) is not associated with the UTAUT2 model. However, as most of
the responses have been associated with this theme, the researcher believes it is a new construct
of the predictive model called Nonjudgmental expectancy (NE). It can be defined as the degree
to which an individual perceives that using a system will deprive him or her of personal
judgment and prejudice. The degree to which university students perceive using AI chatbots as
35
an unbiased communication medium makes them not worry about any judgments or biases
raised by university staff. AI chatbots can enable students to retrieve information without the
additional worry of offending anyone or being offended by others. Furthermore, the construct
is potentially education sector-specific, possibly due to the hierarchical structure of the
educational system and cultural view of teachers as an authority. These findings align with
research done by Schrodt, Whitt and Truman (2007) suggesting that teachers have rewarding
and punishing power on students (see chapter 2.5 Characterizations of the university students'
generations).

Furthermore, machine learning allows the AI chatbot to learn from data sets and experience and
make opinions. Therefore, the data must be interpreted carefully before applying machine
learning, and the outcomes must be evaluated (Stachowicz-Stanusch and Amann, 2018). To
ensure that, the algorithm does not use information about the person for harm. Therefore, it
could be argued that this new “Nonjudgmental expectancy” construct is very much suitable for
AI technology research.

Integrating Nonjudgmental expectancy into UTAUT2 will complement the theory ‘s focus on
an individual’s psychological and emotional attitude of being judged, that/and is? not addressed
in the model. It is proposed that Nonjudgmental expectancy directly affects technology use
and/or weakens or limits the strength of the relationship between behavioral intention and
technology use. The proposed Nonjudgmental expectancy construct consists of two
prerequisites: organizational and technology. Technology prerequisite refers to technology
being free of prejudice and biases. The organization prerequisite is defined as how using
innovative technology or product within an organization mitigates social prejudice.

The second theme (Fast support) is associated with Performance expectancy and Effort
expectancy constructs. Performance expectancy is the student’s belief in how AI chatbots will
support them in gathering the needed information. Effort expectancy can be explained as a level
of difficulty operating the AI chatbots. The third theme (Simplicity kills) has been associated
with the Habit construct, particularly with one of the criteriums: individual experience. The
responses expressed the perceived quality of service and prior chatbot experience outside the
education field. The fourth theme (Limited capabilities) is associated with Facilitating
conditions: explained as confidence in AI chatbot functional capabilities. The fifth theme (Lack
of concerns) has not been associated with any UTAUT2 construct (see Figure 10). Due to
minimal responses associated with this theme, it is not seen as a vital behavior influencer.
Furthermore, the thesis author mentioned the general lack of security and privacy concerns in
the literature review gap. Thus, can be assumed that the general lack of concerns is among
scholarly researchers as well as the consumers. However, this might differ in situations outside
of this study, such as AI chatbots handling students' medical information. But for AI chatbots
without educational intentionality, the students do not seem to have security or privacy concerns
to influence the acceptability strongly.

36
 Figure 10. Identified themes compiled with UTAUT2 constructs during data analysis to
identify findings

In contrast to the quantitative research of Acceptance of AI chatbots by Almahri, Bell and Merhi
(2020), this thesis suggests different results (see chapter 1.4 Scope and Limitations). These
authors focused on university students at a particular university department in the United
Kingdom and students who experienced working with the AI chatbot in educational settings
during the experiment. The results indicated that Performance expectancy, Effort expectancy,
and Habit are the main predictors of the student's AI chatbot acceptance. This qualitative thesis
suggests that Performance expectancy, Effort expectancy, and a newly proposed construct
called Nonjudgmental expectancy are the main predictors of intention to use AI chatbots.
Unlike the study by Almahri, Bell and Merhi (2020) this thesis has been applied to multicultural
settings on students who have not been exposed to AI chatbots in education settings before. The
results show that the judgment-free solution and the speed of the solution's response and
availability are the priority regarding acceptability for the consumers.

Nevertheless, it can be assumed that once the AI chatbots are more implemented, the use of AI
chatbots will become a custom and a routine activity. Furthermore, this behavior is associated
with the UTAUT2 constructs called Habit (HA). Thus, this particular factor might have a more
significant effect on student's intentions to use AI chatbots in the future.

The findings are further grouped with the research questions from the first section. The research
questions aim to explore the acceptability factors of university students to use AI chatbots in
higher education and to further point to areas and situations where AI chatbots can act as a
medium in student-university communication.

RQ1: What are the factors of the acceptability of AI chatbots by students in higher
education?

This research question is related to the factors which can influence university students'
intentions to use AI chatbots. Factors are associated with these constructs: Nonjudgmental
expectancy (NE), Performance expectancy (PE) and Effort expectancy (EE), Habit (HA), and
Facilitating condition (FC) constructs. Explicitly with emerged theme n.1 (No judgment), n. 2
(Fast support), theme n.3 (Simplicity kills), and theme n.4 (Limited capabilities).

37
University students favor AI chatbots mainly due to their ability to provide information fast and
their 24/7 availability; however, the usage of AI chatbots has been noted as dependent on the
situation. The students are generally looking for fast information provisioning. This assumption
is supported by Linnes and Metcalf (2017) claiming that this generation often expects an
immediate response (see chapter 2.5 Characterizations of the university students' generations).
Further, students also mentioned and described feelings to be offended when the reply was not
considered "warm". Linnes and Metcalf (2017) explain this behavior as the students of this
generation seek closer personal attention from the teacher than previous generations. In general,
students are not aiming to bypass the human-to-human conversation; however, it would have
to be provided at the same speed as computer-human conversation.

Regarding the Performance expectancy, students believe that AI chatbots can support specific
activities (students asking for informational, educational, and assistive tasks) but only related
to general, simple, and practical issues. Some of the participants mentioned that they would
instead engage a human in case of a more personal or sensitive topic. It might not be surprising,
considering the characteristics of millennials and iGeneration, which is pending a fast
information provisioning and personal relationship with the teacher (Linnes and Metcalf, 2017).
Perceived usefulness, one of the PE criterium, has not been highlighted by students. It is a
degree of improvement of performance while using the new technology. Participants did not
identify that AI chatbots would immediately improve their study performance and grades;
however, they have described that it will indirectly support their studies if they get the needed
information quicker. It can be argued that PE is one of the main predictors; however, the
consumers do not see a direct link between the usage and higher performance. However, the
reviewed literature speaks more optimistically, as researchers strongly believe that AI chatbots
can increase and enhance teachers’ capabilities and students’ abilities (Ashfaque, et al., 2020;
Lee, et al., 2020).

Regarding the Effort expectancy, participants addressed the complexity criterium, which
elaborates on how the technology is easy or difficult to use. The participants found it easy to
use AI chatbots but frustrating not to get the needed answer when requested or not being
understood. It has been noted that aspects of inconvenience, frustration, and plain annoyance
are associated with the outcome of the engagement.

Besides the mentioned constructs, the acceptability factors include the new UTAUT2 construct
called Nonjudgmental expectancy (NE). Especially in situations when the students feel shy to
ask or do not want to approach the teacher directly for various reasons. Also, the student
mentioned that there is no need for titles, politeness, or courtesy when speaking to non-human,
which eases the conversation for them. Most importantly, there is no consequential judgment
of others or punishment or stigma due to the question. Thus, if a student perceives that the use
of AI chatbots for communication with the university will contribute meaningfully to their
emotional and social well-being, they may be favorably disposed to use it. However, to truly
achieve this state, the AI chatbots system has to be unbiased.

Furthermore, exploring how the university students perceived AI chatbots showed that a strong
understanding of Artificial intelligence technology is missing. Regarding the individual
experience factor linked to using the technology, most participants described a more rule-based
chatbot experience and did not distinguish between any types of chatbot in general. Students
pointed mainly to poor language understanding and not being able to spot any self-improvement
over time. As AI chatbots are based on machine learning techniques enabling them to learn and
improve their performance (Adamopoulou and Moussiades, 2020), this leads to the assumption

38
that university students apply one opinion on any type of chatbots. As a result, the consumers
might simply overlook the current improvements in the conversational skills of AI chatbots
(Schuetzler, Grimes and Giboney, 2020).

As perceived by university students, the Facilitating conditions are mainly associated with the
missing knowledge base information that prevents AI chatbots from answering more specific
questions. Facilitating conditions express the user’s confidence in technology and the
organization using it. However, based on the data, the participants do not believe in AI chatbot
capabilities to deal with the more complicated issue. Their opinion is that the technology is
currently very limited. However, some of them have noted that technology development is
moving forward, and AI chatbots will have more extensive capabilities in the future. The
opinion regarding the capabilities seems to be still present among the consumers since the
research done by Rubin, Chen and Thorimbert (2010). It can be argued that even though the
researchers recommended consumers get familiar with the AI chatbots, the accomplishment did
not come true. It is depressing that no activities to change consumers' minds have led to a
positive view of this technology. It means that even though this thesis has been conducted
differently than the study by Rubin, Chen and Thorimbert (2010) and two decades later, there
is still a similarity in how AI chatbots are perceived (see chapter 1.1 Introduction and Research
Setting).

Surprisingly, none of the participants mentioned any possible Social influence (SI), for instance,
a friend’s opinion, which would lead to the higher acceptability of AI chatbots. Even though,
according to Linnes and Metcalf (2017) the representatives of iGeneration heavily rely on other
people's opinions, none of the participants mentioned it as an acceptability factor.

Based on the analysis, some factors significantly affect behavioral intention to use AI chatbot
technology. The captured factors that affect university students’ intention to use AI chatbots
are Performance expectancy (PE), Effort expectancy (EE), Nonjudgmental expectancy (NE),
Habit (HA), and facilitating condition (FC). The interpretation of the study findings indicates
that there are positive and negative factors that influence the university students’ intention to
use AI chatbots without educational intentionality. The factors which positively influence the
AI chatbot acceptability are Performance expectancy (PE), Effort expectancy (EE), and
Nonjudgmental expectancy (NE): providing an unbiased, easy-to-use, fast, and highly available
solution for a simple educational, informative, and assistance task. The factors negatively
influencing acceptability in this study are Habit (HA), focusing on a previous negative
experience, and Facilitating condition (FC), primarily pointing to missing confidence that the
technical infrastructure exists to support such a system. The student's acceptability intentions
could be explained by the age moderator, as in many cases, their beliefs and ideas can link to
characteristics of their generation.

RQ2: What application area should universities address by AI chatbots to efficiently assist
in student-university communication?

Reviewed scholarly papers pointed to numerous tasks which can be delegated to software (see
Figure 4). These experiments confirm and support student’s beliefs (gathered during the
interview and questionnaire survey) about the performance expectation of AI chatbots to
provide simple and general informational, educational, and assistive tasks. The AI chatbot area
of application mentioned in the literature review has been compared with the data about the
most common student-university communication topics. Some of the communication topics
correlate with scholarly experiments. Figure 11 depicts the feasible and possible application of
AI chatbot, thus suggesting areas that could be outsourced from the university staff. The figure
39
combines Figure 4 (Area of application of AI chatbots as communication medium) and Table
1 (Most common student topic engagement).

Figure 11. Suggested application of AI chatbots as a communication medium at a university

The feasible application is addressed by scholarly experiments in the literature review and
suggested by study participants as the most addressed communication topic with a university.
The possible application has been addressed only by scholars but is not mentioned by
participants of this study. The tasks which might be seen as feasible to hand over to AT chatbots
to assist in the student-university communication are: - recap on the course material, study
material recommendation, exam and requirements information, and admission information. In
addition, the possible application extends the communication topic scope to providing FAQ,
graduation plans, courses, and navigation/location information. These findings are supported
by Linnes and Metcalf (2017) on how communication technology is viewed by iGeneration.
Students intend customer-orientated solutions and prefer to move from “older“ communication
platforms to immediate social platforms (see chapter 2.5 Characterizations of the university
students' generations).

However, it is suggested to start applying the AI chatbot with the feasible topic’s application of
student-university communication, which scholars have already addressed experimentally. The
findings also show that student satisfaction is linked to more advanced types of chatbots. Per
the research done by Schuetzler, Grimes and Giboney (2020) with the increasing capabilities
of the devices, the consumers' expectations rise. The authors suggested that today's research
tries to understand the design impact perception (see chapter 2.4.1 Conversational Skill).
Following suggestions by Schuetzler, Grimes and Giboney (2020), this thesis might help to
understand when it is more suitable to apply more sophisticated AI chatbots than the relatively

40
simpler rule-based chatbot. So, suppose a university is deciding between rule-based chatbot and
AI chatbot implementation. In that case, the more advanced technology is more appropriate to
satisfy the consumers.

Furthermore, university students expect advanced functions and precise service, simply based
on the characteristics of their generations. Thus, they do not lower their demands to a basic
solution at their university when they are approached by high definition, high resolution, high
availability, and multiple functions of Smart products in their spare time. Thus, applying a rule-
based chatbot or not suggested communication topic applications might lead to negative
consumer satisfaction, detach the usage, and defer the adoption of AI chatbots.

6. Conclusion
This chapter contains the final remarks of the thesis and includes the overall findings of the
research. The chapter is divided into three parts. Firstly, a summary of the research problem
and the research questions are mentioned, followed by the method of the data collection and
the data analysis. The findings are stated in relation to previously mentioned studies. The second
part points to the contribution that research makes to the current body of research, and the third
part suggests future research.

6.1 Conclusions

Adaptation of AI technology is growing globally in many areas; however, the application in the
educational field is still in its infancy. The recent development of communication technology
has accelerated the communication process and enabled instant communication between
people. Thus, student-university communication also increased. There have been many
experiments regarding AI chatbot's adaptation into higher education; however, the acceptability
of such a solution in multiple locations has not been addressed. This thesis aimed to fill this
gap.

The research focused on factors of acceptability of AI chatbots without education intentionality

which might explain the lack of the implementation of this AI solution in higher education.
UTAUT2 theory have been chosen for the research to understand the intention of university
students to use AI chatbots as a communication medium between them and a university. As a
second scope the study points to areas where AI chatbots might assist the communication, seen
from the student’s point of view.

The qualitative data has been gathered through semi-structured interviews and questionnaire
surveys. Data has been analyzed into the themes and compiled with the Unified Theory of
Acceptance and Use of Technology 2 (UTAUT2) model constructs. The research questions
aimed to determine the factors of AI chatbot's acceptability in higher education and what
communication topic areas are most suitable to handed over to AI chatbots. The thesis findings
have been compared to the study of Rubin, Chen and Thorimbert (2010), confirming the long-
term opinion among the consumers. The behavior predictors has been also compared to the
behavior predictors suggested by Almahri, Bell and Merhi (2020).

As a result: Nonjudgmental expectancy, Performance expectancy, and Effort expectancy have

been concluded as the main predictors of acceptability. These constructs can positively
influence the AI chatbot acceptability by providing an easy-to-use, unbiased, 24/7 available
solution for simple educational, informative, and assistance tasks. Further, fast information
41
provisioning positively impacts students' inner well-being by eliminating negative feelings
associated with waiting. This thesis also suggests that a new construct, Nonjudgmental
expectancy (NE), be incorporated into the UTAUT2 predictive model. Nonjudgmental
expectancy (NE) can positively influence students' inner well-being by eliminating negative
feelings associated with communication, such as being judged or offended by provided
unbiased service. Besides UTAUT2 construct is also well suited for AI technology, considering
machine learning characteristics. The study further points to particular communication topic
areas that are convenient to delegate to software.

Students suggested that most possible application of AI chatbots are in in communication areas
concerning recap of course material, study material recommendation and exam and
requirements information.

6.2 Contribution

This thesis aims to contribute to the stock of knowledge in IS literature, focusing on the
adaptation of AI chatbots in higher education by recording the qualitative meanings of people
who did not use them yet, thus adding new perspectives. The master’s thesis also brings a
unique view on acceptability factors concerning student generation characteristics. The most
significant contribution is suggesting a new UTAUT2 construct suitable for education sector
research and AI research.

This master’s thesis can then be used as a base for the framework for AI chatbot application in
higher education, part of the digital strategy proposal of some universities or policymakers, or
as an inspiration for proof of concept to justify if the concept has practical potential. For
instance, the study could be used when an educational institution decides to implement rule-
based or AI-based chatbots, alternatively, whether the AI designers would like to address
specific consumers' intent, such as not being judged.

6.3 Future Research

Predominantly, future research should involve a larger population sample to gain a more
extensive range of perspectives and experiences. With a larger sample, the researcher could
examine the ideal target group for the pilot adaptation of AI chatbots in educational settings
and perform research if the gender or educational background might influence the perception.
Furthermore, the new suggested construct could be exanimated in other fields to expand the
value of the construct. For instance, to fields with similar characteristics as an educational sector
or applied in research of other solutions using AI. Furthermore, researchers need to investigate
the demand for the non-prejudiced and unbiased capability of the solution.

42
References

Adamopoulou, E. and Moussiades, L., 2020. An Overview of Chatbot Technology. Artificial

Intelligence Applications and Innovations, 584, pp.373–383.

Ågerfalk, P.J., 2020. Artificial intelligence as digital agency. European journal of information
systems, 29(1), pp.1–8.

Agrawal, S., 2020. Chatbots vs. AI. What exactly is the difference. [image online] Available
at: <https://www.netomi.com/conversational-chatbot> [Accessed 04 March 2021].

Almahri, F.A.J., Bell, D. and Merhi, M., 2020. Understanding Student Acceptance and Use of
Chatbots in the United Kingdom Universities: A Structural Equation Modelling Approach. In:
IEEE, 6th International Conference on Information Management (ICIM). London, United
Kingdom, 27-29 March 2020. New York: IEEE.

Almaiah, M., Alamri, M. and Al-Rahmi, W., 2019. Applying the UTAUT Model to Explain the
Students’ Acceptance of Mobile Learning System in Higher Education. IEEE Access, 7,
pp.174673-174686.

Alshenqeeti, H., 2014. Interviewing as a Data Collection Method: A Critical Review. English
Linguistics Research, 3(1), pp.39-45.

Andriessen, J. and Sandberg, J., 1999. Where is education heading and how about AI.
International Journal of Artificial Intelligence in Education, 10(2), pp.130–150.

Ashfaque, M.W., Tharewal, S., Iqhbal, S. and Kayte, C.N., 2020. A Review on Techniques,
Characteristics and approaches of an intelligent tutoring Chatbot system. In: IEEE,
International Conference on Smart Innovations in Design, Environment, Management,
Planning and Computing (ICSIDEMPC). Maharashtra, India, 30-31 October 2020. New York:
IEEE.

Bajracharya, D., 2015. Impact of Teacher-Student Communication on “High-Risk Dropout”

Students. The International Institute for Science, Technology and Education (IISTE), 18(5),
pp.88–100.

Baranyi, M., Nagy, M. and Molontay, R., 2020. Interpretable Deep Learning for University
Dropout Prediction. In: SIGITE, 21st Annual Conference on Information Technology
Education. New York, USA, 7–9 October 2020. New York: ACM.

Bazeley, P. and Jackson, K., 2013. Qualitative data analysis with NVivo. Thousand Oaks: Sage
Publications.

Benbya, H., Davenport, T. H. and Pachidi, S., 2020. Special Issue Editorial: Artificial
Intelligence in Organizations: Current State and Future Opportunities. MIS Quarterly
Executive,19(4), pp. 9-21.

Braun, V. and Clarke, V., 2006. Using thematic analysis in psychology. Qualitative Research
in Psychology, 3, pp.77-101.

43
Castillo-Montoya, M., 2016. Preparing for interview research: The interview protocol
refinement framework. The Qualitative Report, 21(5), pp.811-831.

Chandra, Y.W. and Suyanto, S., 2019. Indonesian chatbot of university admission using a
question answering system based on sequence-to-sequence model. Procedia Computer
Science, 157, pp.367–374.

Choque-Díaz, M., Armas-Aguirre, J. and Shiguihara-Juárez, P., 2018. Cognitive technology

model to enhanced academic support services with chatbots. In: IEEE, XXV International
Conference on Electronics, Electrical Engineering and Computing (INTERCON). Lima, Peru,
8-10 August 2018. New York: IEEE.

Clarke, V. and Braun, V., 2013. Teaching thematic analysis: Overcoming challenges and
developing strategies for effective learning. The Psychologist, 26(2), pp. 120-123.

Creswell, J. W., 2007. Qualitative inquiry and research design: Choosing among five
approaches (2nd ed.). Los Angeles: SAGE.

Creswell, J.W., 2014. Research Design: Qualitative, Quantitative, and Mixed Methods. 4th
ed. Thousand Oaks: Sage.

Creswell, J.W., and Creswell, D. J., 2018. Research design: qualitative, quantitative, and
mixed methods approaches. Fifth edition. ed. Los Angeles: SAGE.

Dale, R., 2016. The return of the chatbots. Natural Language Engineering, 22(5), pp. 811–
817.

Davis, F.D., 1989. Perceived usefulness, perceived ease of use, and user acceptance of
information technology. MIS quarterly, pp.319–340.

Dimitriadis, G., 2020. Evolution in Education: Chatbots. Homo Virtualis, 3(1), pp.47–54.
Elo, S. and Kyngäs, H., 2008. The qualitative content analysis process. Journal of advanced
nursing, 62(1), pp.107–115.

Fleming, M., Riveros, P., Reidsema, C. and Achilles, N., 2018. Streamlining student course
requests using chatbots. Proceeding 29th Australasian Association for Engineering Education
Conference. pp. 207–211.

Følstad, A. and Brandtzæg, P.B., 2017. Chatbots and the new world of HCI. Interactions, [e-
journal] 24(4), p. 38-42, 10.1145/3085558.

Fonte, F.A.M., Nistal, M.L., Rial, J.C.B. and Rodríguez, M.C., 2016. NLAST: A natural
language assistant for students. In: IEEE, Global Engineering Education Conference
(EDUCON). Abu Dhabi, UAE, 10-13 April 2016. New York: IEEE.

Fryer, L.K., Ainley, M., Thompson, A., Gibson, A. and Sherlock, Z., 2017. Stimulating and
sustaining interest in a language course: An experimental comparison of Chatbot and Human
task partners. Computers in human behavior, 75, pp.461–468.

44
Fusch, P. I. and Ness, L. R., 2015. Are we there yet? Data saturation in qualitative research.
The Qualitative Report, 20(9), pp. 1408-1416.

Gharrah, Alaa and Aljaafreh, Ali., 2021. Why students use social networks for education:
Extension of UTAUT2. Journal of Technology and Science Education, 11, p.53

Graneheim, U.H., Lindgren, B.-M. and Lundman, B., 2017. Methodological challenges in
qualitative content analysis: A discussion paper. Nurse education today, 56, pp.29–34.

Hancock, D.R. and Algozzine, B., 2017. Doing case study research: A practical guide for
beginning researchers. Columbia: Teachers College Press Columbia University.

Hien, H.T., Cuong, P.-N., Nam, L.N.H., Nhung, H.L.T.K. and Thang, L.D., 2018. Intelligent
assistants in higher-education environments: the FIT-EBot, a chatbot for administrative and
learning support. In: SoICT’, 9th International Symposium on Information and
Communication Technology. Da Nang City, Vietnam, 6–7 December 2018. New York: ACM.

Huang, C-Y. and Kao, Y-S., 2015. UTAUT2 Based Predictions of Factors Influencing the
Technology Acceptance of Phablets by DNP. Mathematical Problems in Engineering. 2015,
pp. 1-23.

Hussain, S., Sianaki, O. A., and Ababneh, N., 2019. A survey on conversational agents/chatbots
classification and design techniques. Proceedings Of The Workshops of the 33Rd International
Conference On Advanced Information Networking And Applications, pp.946-956.

Jaakkola, H., Henno, H., Lahti, A., Järvinen, J-P. and Mäkelä, J., 2020. Artificial Intelligence
and Education. In: MIPRO, 43rd International Convention on Information, Communication and
Electronic Technology. Opatija, Croatia, 28-02 October 2020. New York: IEEE.

Jansen, H., 2010. The Logic of Qualitative Survey Research and its Position in the Field of
Social Research Methods. Forum : Qualitative Social Research, [e-journal] 11(2),
10.17169/fqs-11.2.1450.

Janssen, A., Passlick, J., Rodríguez Cardona, D. and Breitner, M.H., 2020. Virtual Assistance
in Any Context A Taxonomy of Design Elements for Domain-Specific Chatbots. Business &
information systems engineering, 62(3), pp.211–225.

Joshi, S., Rambola, R.K. and Churi, P., 2021. Evaluating Artificial Intelligence in Education
for Next Generation. Journal of Physics: Conference Series, In: IOP, 2nd International
Conference on Smart and Intelligent Learning for Information Optimization (CONSILIO). Goa,
India, 24-25 October 2020. Bristol: IOP Publishing.

Kengam, J., 2020. ARTIFICIAL INTELLIGENCE IN EDUCATION. [online] Available at:

<https://www.researchgate.net/publication/347448363_ARTIFICIAL_INTELLIGENCE_IN_
EDUCATION > [Accessed 01 February 2021].

Khin, N.N. and Soe, K.M., 2020. Question Answering based University Chatbot using
Sequence to Sequence Model. In: IEEE, 23rd Conference of the Oriental COCOSDA
International Committee for the Coordination and Standardisation of Speech Databases and
Assessment Techniques (O-COCOSDA). Yangon, Myanmar, 5-7 November 2020. New York:
IEEE.
45
Krahe, C., Escamilla-Fajardo, P. and López-Carril, S., 2021. The influence of teacher-student
communication on the importance of physical education. Facta Universitatis, Series: Physical
Education and Sport. [e-journal]19, 10.22190/FUPES200916061E.

Lee, L.-K., Fung, Y.-C., Pun, Y.-W., Wong, K.-K., Yu, M.T.-Y. and Wu, N.-I., 2020. Using a
Multiplatform Chatbot as an Online Tutor in a University Course. In: IEEE, 6th International
Symposium on Educational Technology (ISET). Bangkok, Thailand, 21-23 July 2020. New
York: IEEE.

Liberati A., Altman D.G., Tetzlaff J., Mulrow C., Gotzsche P.C., Ioannidis J.P.A. and Clarke
M., 2009. The PRISMA statement for reporting systematic reviews and meta-analyses of
studies that evaluate health care interventions: Explanation and elaboration. PLoS Medicine,
6(7).

Linnes, C. and Metcalf, B. 2017. iGeneration And Their Acceptance of Technology.

International Journal of Management & Information Systems, 21(2), pp.11-26.

Maedche, A., Legner, C., Benlian, A., Berger, B., Gimpel, H., Hess, T., Hinz, O., Morana, S.
and Söllner, M., 2019. AI-based digital assistants. Business & Information Systems
Engineering, 61(4), pp.535–544.

Malterud, K., 2001. Qualitative research: standards, challenges, and guidelines. The Lancet
(British edition), 358(9280), pp.483–488.

Mallya, S., 2021. 8 Best Chatbot Platform Tools to Build Chatbots for Your Business. [image
online] Available at: <https://www.99signals.com/best-chatbot-platform-tools/> [Accessed 24
March 2021].

Mandernach, B.J. and Holbeck, R., 2016. Teaching Online: Where Do Faculty Spend Their
Time?. Online Journal of Distance Learning Administration, 19(4).

Martin, N., Cojean, S. and Ragot, M., 2020. The Acceptability of AI at Work: Predicting the
Intention to Use relying on UTAUT. MIS quarterly, 27(3),pp. 425-478.

Mekni, M., Baani, Z. and Sulieman, D., 2020. A Smart Virtual Assistant for Students.
Proceedings of the 3rd International Conference on Applications of Intelligent Systems, 15,
pp.1–6.

Merriam, S.B., 2002. Introduction to qualitative research. In: S.B. Merriam and Associates, eds.
2002, Qualitative research in practice: Examples for discussion and analysis. San Francisco:
Jossey-Bass. pp.1-17.

Molnár, G. and Szüts, Z., 2018. The role of chatbots in formal education. In: SISY, 16th
International Symposium on Intelligent Systems and Informatics. Subotica, Serbia, 13-15
September 2018. New York: IEEE.

Myers, M.D., Avison, D., 2002. Qualitative research in information systems: a reader. [e-book]
London: Sage. Available through: LNU Library website <https://ebookcentral-proquest-
com.proxy.lnu.se/lib/linne-ebooks/detail.action?pq-origsite=primo&docID=1138474>
[Accessed 18 Feb. 2021].

46
Peng, C., 2017. Good Record Keeping for Conducting Research Ethically Correct. Research
ethics, pp.1-7. DOI:10.13140/RG.2.2.24463.64160.

Popenici, S.A. and Kerr, S., 2017. Exploring the impact of artificial intelligence on teaching
and learning in higher education. Research and Practice in Technology Enhanced Learning,
12(1), p.22.

Quiroga P, J,. Daradoumis, T. and Puig, J., 2020. Rediscovering the use of chatbots in
education: A systematic literature review. Computer Applications in Engineering Education,
28(6), pp.1549–1565.

Ranoliya, B.R., Raghuwanshi, N. and Singh, S., 2017. Chatbot for university related FAQs.
International Conference on Advances in Computing, Communications and Informatics
(ICACCI). Udupi, India, September 2017. New York: IEEE.

Rieke, D.T., 2018. The relationship between motives for using a Chatbot and satisfaction with
Chatbot characteristics in the Portuguese Millennial population: an exploratory study. Master
in Management. Faculdade De Economia, Universidade Do Porto. [online] Available at:
<https://repositorio-aberto.up.pt/bitstream/10216/116509/2/296743.pdf> [Accessed 08
February 2021].

Rossi, P.G. and Carletti, S., 2011. MAPIT: a pedagogical-relational ITS. Procedia Computer
Science, 3, pp.820–826.

Roberts, R. E., 2020. Qualitative Interview Questions: Guidance for Novice Researchers. The
Qualitative Report, 25(9), pp.3185-3203.

Rubin, V.,Chen, Y. and Thorimbert, L. 2010. Artificially Intelligent Conversational Agents in

Libraries. Library Hi Tech. 28, pp.496-522.

Salkind, N.J., 2010. Encyclopedia of Research Design. Thousand Oaks: SAGE Publications.

Sandu, N. and Gide, E., 2019. Adoption of AI-Chatbots to enhance student learning experience
in higher education in India. 18th International Conference on Information Technology Based
Higher Education and Training (ITHET). Magdeburg, Germany, 26-27 September 2019. New
York: IEEE.

Saris, W.E. and Gallhofer, I.N., 2014. Design, Evaluation, and Analysis of Questionnaires for
Survey Research. 2nd Edition. [e-book] Barcelona:Wiley. Available through:
<https://ebookcentral-proquest-com.proxy.lnu.se/lib/linne-ebooks/detail.action?pq-
origsite=primo&docID=1602919> [Accessed 19 Feb. 2021].

Schuetzler, R.M., Grimes, G.M. and Giboney, J.S., 2020. The impact of chatbot conversational
skill on engagement and perceived humanness. Journal of management information systems,
37(3), pp.875–900.

Shum, H.Y., He, X.D. and Li, D., 2018. From Eliza to XiaoIce: challenges and opportunities
with social chatbots. Frontiers of information technology & electronic engineering, 19(1),
pp.10–26.

47
Shumanov, M. and Lester, J., 2021. Making conversations with chatbots more personalized.
Computers in human behavior.Computers in Human Behavior, 117.

Schrodt, P., Whitt, P.L., Truman, P.D. 2007. Reconsidering the measurement of teacher power
use in the college classroom. Communication Education, 56 (3), 308-332.

Schwartz-Shea, P. and Yanow, D., 2013. Interpretive research design: Concepts and processes.
Critical Policy Studies, 8(1), pp.116-118.

Stachowicz-Stanusch, A. and Amann, W., 2018. Artificial intelligence at universities in Poland.

Organizacja i Zarządzanie: kwartalnik naukowy, [e-journal] 2, pp.65-82. 10.29119/1899-
6116.2018.42.6.

Stake, R.E., 2006. Multiple case study analysis, New York: The Guilford Press.

Smutny, P. and Schreiberova, P., 2020. Chatbots for learning: A review of educational chatbots
for the Facebook Messenger. Computers & Education, 151, pp.1-11.

Taherdoost, H., 2019. Importance of Technology Acceptance Assessment for Successful

Implementation and Development of New Technologies. Global Journal of Engineering
Sciences. 1(3), pp. 1-3.

Taherdoost, H., 2016. Sampling Methods in Research Methodology. How to Choose a

Sampling Technique for Research. International Journal of Academic Research in
Management. 5, pp. 18-27.

The Norwegian National Committee for Research Ethics in Science and Technology, 2016.
Guidelines for Research Ethics in Science and Technology. [pdf] Norway: The Norwegian
National Committee for Research Ethics in Science and Technology. Available at:
<https://www.forskningsetikk.no/ressurser/publikasjoner/guidelines-science-and-
technology/> [Accessed 1 March 2021].

Touimi, Y.B., Hadioui, A., El Faddouli, N. and Bennani, S., 2020. Intelligent Chatbot-LDA
Recommender System. International Journal of Emerging Technologies in Learning (iJET),
15(20), pp.4–20.

Zawacki-Richter, O., Marín, V.I., Bond, M. and Gouverneur, F., 2019. Systematic review of
research on artificial intelligence applications in higher education–where are the educators?.
International Journal of Educational Technology in Higher Education, 16 (1), p.39.

Venkatesh, V., Morris, M., Davis, G., & Davis, F. D. 2003. User Acceptance of Information
Technology: Toward a Unified View. Management Information Systems Quarterly, 27(3),
pp.425–478.

Venkatesh, V., (2021). Adoption and use of AI tools: a research agenda grounded in UTAUT.
Annals of Operations Research. [e-journal], pp.1–12. 10.1007/s10479-020-03918-9.

Venkatesh, V., Thong J. and Xin X., 2012. Consumer Acceptance and Use of Information
Technology: Extending the Unified Theory of Acceptance and Use of Technology. MIS
Quarterly, 36(1), pp.157-78.

48
Wohlin, C., 2014. Guidelines for snowballing in systematic literature studies and a replication
in software engineering. Proceedings of the 18th international conference on evaluation and
assessment in software engineering, 38, pp.1-10.

Yang, S. and Evans, C., 2019. Opportunities and challenges in using AI chatbots in higher
education. 3rd International Conference on Education and E-Learning. Barcelona, Spain,
November 2019. New York: ACM.

Yin, R. K., 2009. Case study research: Design and methods. Thousand Oaks: Sage publications.

49
Acknowledgment
The author would like to thank the fellow student, Olga Terzi, for her contribution and
comments on an earlier version of this pape

50
Appendix A.: Literature map

50
 Appendix B.: PRISMA flow

Records identified through Additional records identified

Identification

database searching through other sources

(n = 5,333) (n = 0)

Records after duplicates removed

(n = 5,251)
Screening

Records screened Records excluded

(n = 82) (n = 36)

Full-text articles assessed Full-text articles excluded,

for eligibility with reasons
(n = 46) (n = 15)
Eligibility

Studies included in
qualitative synthesis
(n = 32)

Studies included in
Included

quantitative synthesis
(meta-analysis)
(n = 0)

51
Appendix C.: Survey questions

SCREENING QUESTIONS - COMMUNICATION WITH UNIVERSITY

1. How often are you engaging teachers or university per month (email, forum, etc.)
2. With what questions are you mostly addressing teachers/ the university office/ university
library?

MAIN QUESTIONS

3.What were the main reasons you would choose AI chatbots for communication with a
university? Are there reasons you would not choose it?
4. What do you think might be AI chatbot the biggest challenge and the biggest benefit in
usage at a university?
5.What would be your expectations of an AI chatbot at your university? What is the most
important feature for you?
6. Do you have any concerns regarding AI chatbots? (privacy, security, too complicated, not
same as human, etc.)
7. If you ever used chatbot outside of the education field, how would you describe your
experience with it? How do you remember it? How did you experience it?
8. Do you think chatbots can be capable of providing you an info service? Or what kind of
service would be acceptable for you to receive from a chatbot?
9. What would make you use an AI chatbot at the university again and again? ( that you will
use it instead of email, regularly) Can a recommendation of an influencer, friend, peer made
you use it?
10. Could you describe in as much detail as possible the state of the communication with your
university? What do you think of current channels, speed of reply, accuracy?

52
Appendix D.: Interview questions
SCREENING QUESTION - COMMUNICATION WITH UNIVERSITY
1. How often are you engaging teachers or university per month (email, forum, etc.)
2. With what questions are you mostly addressing teachers/ the university office/ university
library?

FEELINGS DURING THE COMMUNICATION:

3. Could you describe in as much detail as possible the state of the communication with your
university? What do you think of current channels, speed of reply, accuracy? Walk me
through it.
4. Have you ever come across a not pleasant teacher/university staff response? How was your
emotional reaction?
5. Did ever happen that the teacher/university staff did not reply to your question or you have
to wait more than 5 business days? What happened first, next, and later?
6. I want to understand the situation from your point of view. Are you sometimes shy to ask
questions out loud in class or ask the teacher? (or not at all)
7. Do you ever feel like you bothering the respondent with your question?
8. I want to understand the meaning of your experience. How do you feel when you have to
wait for the answer?

FEELINGS WHEN SPEAKING TO A NOT-HUMAN MACHINE

9. Do you mind speaking with a machine? Do you find it confusing or you don't mind at all?
What do you feel about it?
10. What would you prefer, what is more, pleasant for you, to use a chatbot with a face
(avatar) or classic chat?

OPINIONS ABOUT CHATBOT USAGE IN GENERAL

11.What is your opinion about chatbots? Do you find them helpful? Could you describe it in
as much detail as possible?
12. If you ever used chatbot outside of the education field, how would you describe your
experience with it? How do you remember it? How did you experience it?
13. Do you think chatbots can be capable of providing you a service (info)? And what kind of
service would be acceptable for you to receive from chatbot (outside the basic course and
university info)?

OPINIONS ABOUT CHATBOT USAGE AT UNI:

14. What were the main reasons you would choose AI chatbots for communication with a
university?
15. Are there reasons you would not choose AI chatbots for communication with a university?
16. Do you have any security or privacy concerns? (Or non before I have asked this)
17. Do you think that AI chatbots could be used (have the needed capability) to act as a
communication medium to answer your inquiries between you and the university admissions
department, library, or your teachers?

53
18. What would make you use an AI chatbot at the university again and again? (that you will
use instead of email, regularly) Can a recommendation of an influencer, friend, peer made you
use it?
19. Would you prefer AI chatbots to be used in kiosks at the university or on a web interface
on a university website or both?

EXPECTATIONS - AI CHATBOT FUNCTIONS:

20. What is the most important feature of the AI chatbot for you? Walk me through your
expectations.

54
Appendix E.: Consent form template

Consent form for taking part in Assessment of the role of AI chatbots as communication
assistants in higher education

The purpose of this research is to seek the motivation behind the intention to use AI chatbots
without education intentionality in universities and to suggest the ways that this technology
can assist in student-university communication.

By signing this consent form, you approve that your personal data is processed within the
frame of the thesis described above. You can withdraw your consent at any time by contacting
one of the contact persons below. In that case, your personal data will not be saved or
processed any longer.
The personal data that will be collected from you is the gender/background of the study. Your
personal data will be processed in March-April 2021 and after this, the data will be archived
for 5 years.

You always have the right to request information about what has been registered about you
and to comment on the processing of the data that has been collected by contacting one of the
contact persons below or the higher education institution’s personal data ombudsman on
[email protected]. Complaints that cannot be solved in dialogue with Linnaeus
University can be sent to the Swedish Authority for Privacy Protection.

………… Ostrava, 26th March 2021

Signature City and date

Marta Slepankova
Name in block letters

Contact information:
Student’s name:
Student’s email address:
Supervisor’s name:
Supervisor’s email address

55
Appendix F.: Themes, subthemes, codes

56
Faculty of Technology
SE-391 82 Kalmar | SE-351 95 Växjö
Phone +46 (0)772-28 80 00
[email protected]
Lnu.se/fakulteten-for-teknik