ISTANBUL TECHNICAL UNIVERSITY  GRADUATE SCHOOL

PROPOSING A HYBRID TALENT ACQUISITION METHODOLOGY

TOWARDS TANKER INDUSTRY

M.Sc. THESIS

Muhittin ORHAN

Department of Maritime Transportation Engineering

Maritime Transportation Engineering Programme

JANUARY 2024
 ISTANBUL TECHNICAL UNIVERSITY  GRADUATE SCHOOL

PROPOSING A HYBRID TALENT ACQUISITION METHODOLOGY

TOWARDS TANKER INDUSTRY

M.Sc. THESIS

Muhittin ORHAN

512211008

Department of Maritime Transportation Engineering

Maritime Transportation Engineering Programme

Thesis Advisor: Prof. Dr. Metin CELIK

JANUARY 2024
İSTANBUL TEKNİK ÜNİVERSİTESİ  LİSANSÜSTÜ EĞİTİM ENSTİTÜSÜ

TANKER ENDÜSTRİSİNE YÖNELİK BİR HİBRİT YETENEK KAZANIM

YÖNTEMİ ÖNERİSİ

YÜKSEK LİSANS TEZİ

Muhittin ORHAN

(512211008)

Deniz Ulaştırma Mühendisliği Anabilim Dalı

Deniz Ulaştırma Mühendisliği Programı

Tez Danışmanı: Prof. Dr. Metin ÇELİK

OCAK 2024
Muhittin ORHAN, a M.Sc. student of ITU Graduate School student ID 512211008
successfully defended the thesis entitled “PROPOSING A HYBRID TALENT
ACQUISITION METHODOLOGY TOWARDS TANKER INDUSTRY”, which he
prepared after fulfilling the requirements specified in the associated legislations,
before the jury whose signatures are below.

Thesis Advisor : Prof. Dr. Metin CELIK ..............................

Istanbul Technical University

Jury Members : Assoc. Prof. Kadir CICEK .............................

Istanbul Technical University

Dr. Burcu Ozturk TAC ..............................

Turk Loydu

Date of Submission : 15 December 2023

Date of Defense : 19 January 2024

v
vi
 To my family,

vii
viii
FOREWORD

This thesis has emerged in line with the needs of the maritime industry. In this context,
my advisor, who did not spare his help throughout the thesis, I would like to thank
Prof. Dr. Metin Celik. Also, I would like to thank my family for their support during
this process.

January 2024 Muhittin ORHAN

Research Assistant

ix
x
TABLE OF CONTENTS

Page
FOREWORD ............................................................................................................. ix
TABLE OF CONTENTS .......................................................................................... xi
ABBREVIATIONS ................................................................................................. xiii
LIST OF FIGURES ................................................................................................. xv
SUMMARY ............................................................................................................ xvii
ÖZET………............................................................................................................ xxi
1. INTRODUCTION .................................................................................................. 1
1.1 Talent Acquisition and Management ................................................................. 4
1.2 Assessment and Evaluation ................................................................................ 6
1.3 Key Findings ...................................................................................................... 7
2. LITERATURE REVIEW.................................................................................... 11
2.1 Human Resources Management Studies .......................................................... 11
2.2 Critical Studies on Talent Acquisition ............................................................. 13
2.3 Assessment and Evaluation Approaches .......................................................... 26
2.4 Gaps .................................................................................................................. 28
3. METHODOLOGY............................................................................................... 31
3.1 Evidence Centered Design Framework ............................................................ 31
3.1.1 Domain analysis ........................................................................................ 32
3.1.2 Domain modeling ...................................................................................... 33
3.1.3 Conceptual assessment framework ........................................................... 33
3.1.4 Assessment delivery and implementation ................................................. 34
3.2 Question Generation ......................................................................................... 35
3.2.1 Pre processing ........................................................................................... 39
3.2.2 Sentence selection ..................................................................................... 41
3.2.3 Key selection ............................................................................................. 42
3.2.4 Question formation ................................................................................... 44
3.2.5 Distractor selection ................................................................................... 45
3.2.6 Post processing .......................................................................................... 47
3.3 Bloom’s Taxonomy .......................................................................................... 49
3.4 Hybrid Talent Acqusition Model ..................................................................... 58
4. CASE STUDY AND RESULTS.......................................................................... 63
4.1 Phase 1: Domain Analysis................................................................................ 63
4.2 Phase 2: Domain Modeling .............................................................................. 65
4.3 Phase 3: Conceptual Assessment Framework .................................................. 67
4.4 Phase 4: Assesment Delivery and Implementation .......................................... 69
4.5 Phase 5: Recommendations .............................................................................. 70
5. CONCLUSION..................................................................................................... 71
REFERENCES ......................................................................................................... 75
APPENDICES .......................................................................................................... 91
CURRICULUM VITAE ........................................................................................ 109

xi
xii
ABBREVIATIONS

AE : Assessment Engineering
App : Appendix
CAF : Conceptual Assessment Framework
CBL : Competency Based Learning
CBT : Computerized Based Training
ECD : Evidence Centered Design
HRM : Human Resources Management
IMO : International Maritime Organization
MASS : Maritime Autonomous Surface Ship
STCW : Standards of Training Certification and Watchkeeping

xiii
xiv
LIST OF FIGURES

Page

Figure 3.1 : Hybrid talent acquisition model illustration. ......................................... 62

Figure 4.1 : Illustration of ship type selection .......................................................... 64
Figure 4.2 : Illustration of knowledge resource utilization. ...................................... 65
Figure 4.3 : Visualization decision selected knowledge resource part. .................... 66
Figure 4.4 : Illustration of decision selected knowledge resource chapter ............... 66
Figure 4.5 : Illustration of decision selected knowledge resource theme ................. 67
Figure 4.6 : Visualization conceptual assessment framework taxonomy determination
.............................................................................................................. 68
Figure 4.7 : Visualization conceptual assessment framework choosing question type
.............................................................................................................. 68
Figure 4.8 : Illustration of conceptual assessment framework employing question
generation model .................................................................................. 68
Figure 4.9 : Illustration of applying competency demonstration techniques ............ 70

xv
xvi
 PROPOSING A HYBRID TALENT ACQUISITION METHODOLOGY
TOWARDS TANKER INDUSTRY

SUMMARY

Human resource management has a very important place in the maritime industry, as
in other industries. Recruiting, training, choosing the right candidate and then
onboarding it to the ships; highly important for the maritime industry. In this context,
the importance of talent acquisition in subjects such as artificial intelligence and
machine learning is increasing, especially with the effect of industry 4.0. Today,
human resources management is carried out successfully with the support of artificial
intelligence and machine learning algorithms. In this way, companies can handle the
seafarer pool in a shorter time and thus reduce their costs.
When the STCW revision status, automation ships, industry demands and changing
maritime needs are added to all these, the issue of seafarers' transformation comes to
the fore. For example, engineers who will work on LNG-LPG ships need to know the
requirements of tanker ships, safety management systems, emergency plans and hot
work requirements in order to perform their duties. It is necessary to provide such
important information to seafarers first and then to support the development of
seafarers with future skills. It is necessary to understand whether the seafarers to be
recruited to be trained within the company are suitable for the job.
In this context, basic knowledge should be measured. Considering that it is an
international job, it is seen that the number of candidates is quite high. Therefore,
making the knowledge level part quickly and automatically will benefit both
companies and seafarers. It is thought that there will be a gap in between for the
transition from traditional competencies to subjects such as digital, green and soft
skills. Therefore, while industry needs will increase, seafarers will need to be ready
for it. In this context, first of all, the knowledge base should be prepared for the
seafarers who will be assigned to the ships. Consequently, seafarers will possess the
capability to adapt to the evolving demands of the maritime sector, enhance their own
skills, and so augment the overall worth of organisations and firms.
However, due to reasons such as the number of pages, languages and information of
the source documents to be used in order to do all these, the information is not suitable
for every seafarer, it cannot be provided adequately. Therefore, it seems that it will
take time and very costly for most seafarers to read and understand this and become
proficient. In this context, it is seen that a fast, effective and easily accessible model is
needed to provide basic knowledge to seafarers.

xvii
The provision of this issue for maritime companies and organizations is achieved
through successful human resource management. In this context, in order to carry out
studies, it is necessary to use various approaches, not traditional methods. In this
direction, the importance of hybrid talent acquisition models comes to the fore. When
the literature studies are examined, it is seen that hybrid talent acquisition models are
used effectively in many different industries. With hybrid talent acquisition models, it
provides benefits such as finding talent sources, talent screening, training talented
candidates, planning interviews and interacting; demonstrated by many studies.
In this context, a hybrid talent acquisition model that can be used in the maritime
industry has been proposed in order to respond to the changing needs of the maritime
industry, to ensure that companies can successfully complete their talent acquisition
processes, to recruit the right candidate and to provide effective training. Bloom's
taxonomy, multiple choice question generation model and evidence centered design
approaches were used in the creation of the model. The application of the developed
model was conducted with a Company A.
In the application, first of all, important findings such as the tanker company's
information, vision, mission, investment plans, audit reports and company analysis
were analyzed. Due to the requirements of evidence centered design, all of this carried
out through a five-step process in total.
In the domain analysis, which is the first phase, detailed examinations of the company
were conducted. The International Safety Guide for Oil Tankers and Terminals
(ISGOTT) was chosen as a knowledge resource, considering various reasons such as
the safety management system, human accidents, lack of emergency planning and
insufficient crew knowledge. Then, in the domain modeling, which is the second
phase, information such as the crew information of the company and the views of the
training department were examined in order to be able to choose from ISGOTT
chapters. Thus, the second chapter of ISGOTT was selected and then the knowledge
resource selection was completed with the appropriate parts. After, the third phase, the
conceptual assessment framework, was applied. At this phase, factors such as the
workload of the crew, the training opportunities of the company and the time planning
of the crew were taken into account. In line with the information, Bloom's taxonomy,
evidence centered design layers and multiple choice question generation model were
employed for the conceptual assessment framework. Thus, questions and answers
were prepared from the selected sections. Then, computer based assessment was
chosen within the scope of the fourth phase, the assessment delivery and
implementation. Finally, ten multiple choice questions were presented to the company
in a way that could be adapted to computer based assessment for the fifth step,
suggestions. Ten multiple-choice questions produced with the hybrid talent acquisition
survey and protocol are included in the appendices.
Since the proposed hybrid talent acquisition model is holistic, it is thought that this
thesis can be used not only by tanker companies but also by other organizations and
companies in the maritime industry. Thus, it is thought that this thesis can assist for
seafarers to acquire new generation skills.

xviii
Also, it is thought that this thesis can contribute to academic studies apart from its
contributions to the maritime industry. It seems that it can help and contribute to
academic studies such as question generation models, Bloom's taxonomy, evidence
centered design and talent acquisition. In addition, approaches such as artificial
intelligence, machine learning, deep learning and decision support mechanisms; when
used together with the outputs of the thesis, future studies can contribute to the
maritime industry.

xix
xx
 TANKER ENDÜSTRİSİNE YÖNELİK BİR HİBRİT YETENEK KAZANIM
YÖNTEMİ ÖNERİSİ

ÖZET

İnsan kaynakları yönetimi diğer endüstrilerde olduğu gibi denizcilik endüstrisinde de

oldukça önemli bir yere sahiptir. Doğru adayın işe alınması, eğitilmesi, şirket
bünyesine kazandırılması ve sonrasında de gemilere gönderilmesi; denizcilik
endüstrisi için oldukça önemlidir. Bu kapsamda özellikle endüstri 4.0 etkisiyle birlikte
yapay zeka, makine öğrenmesi gibi konularda yetenek edinimi konusunun önemi
giderek artmaktadır. Günümüzde insan kaynakları yönetimi yapay zekanın ve makine
öğrenmesi algoritmalarının destekleriyle başarılı bir şekilde gerçekleştirilmektedir. Bu
sayede şirketler denizci havuzunu daha kısa sürede elleçleyebilmekte ve böylece
maliyetlerini düşürebilmektedir.
Tüm bunlara STCW revizyon durumu, otomasyon gemiler, endüstri talepleri ve
değişen denizcilik ihtiyaçları eklenince denizcilerin dönüşümleri konusu gündeme
gelmektedir. Örneğin LNG-LPG gemilerinde görev alacak mühendislerin görevlerini
yapabilmeleri için tanker gemilerinin gerekliliklerini, emniyet yönetim sistemlerini,
acil durum planlarını ve sıcak çalışmanın gerekliliklerini bilmesi gerekmektedir. Bu
gibi önemli bilgilerin öncelikle denizcilere kazandırılması ve sonrasında yeni nesil
yetkinliklerle denizcilerin gelişimlerine destek olunması gerekmektedir. Şirket
bünyelerinde yetiştirilmek üzere işe alınacak denizcilerin işe uygun olup olmadığını
anlamak gerekmektedir.
Bu kapsamda temel olarak bilgisinin ölçülmesi gerekmektedir. Özellikle uluslararası
bir iş olduğu düşünülünce aday sayısı oldukça fazla olduğu görülmektedir. Bu yüzden
bilgi düzeyi kısmının hızlı ve otomatik bir şekilde yapılması hem şirketlere hem
denizcilere fayda sağlayacaktır.
Geleneksel yetkinliklerden dijital, yeşil ve yumuşak beceri gibi konulara geçiş için
arada bir boşluğun oluşacağı düşünülmektedir. Bu yüzden bir taraftan endüstri
ihtiyaçları artarken bir taraftan da denizcilerin buna hazır olması gerekecektir. Bu
kapsamda gemilerde görev alacak denizcilere öncelikle bilgi zeminin hazırlanması
gerekmektedir. Böylece denizciler değişen denizcilik endüstrisi ihtiyaçlarına cevap
verebilecek, kendilerini geliştirebilecek ve böylece organizasyon ve şirketlerin katma
değerlerini yükseltebileceklerdir.
Fakat tüm bunların yapılabilmesi için kullanılacak olan kaynak dokümanların sayfa
sayıları, dilleri ve bilgilerin her denizciye uygun olmaması gibi sebeplerden dolayı
yeterli olarak sağlanamamaktadır. Bu yüzden çoğu denizcinin bunu okuyup anlaması
ve yetkin seviyeye gelmesinin zaman alacağı ve oldukça maliyetli olacağı
görülmektedir.
Bu kapsamda temel seviyede denizcilere bilgisinin kazandırılması için hızlı, efektif ve
kolay ulaşılabilir bir modelin ihtiyaç duyulduğu görülmektedir.

xxi
Denizcilik şirketleri ve organizasyonları için bu konun sağlanması ise insan kaynakları
yönetiminin başarılı bir şekilde yapılmasıyla olmaktadır. Bu kapsamda çalışmaların
yapılması için geleneksel metotların değil çeşitli yaklaşımların bir arada
kullanılmasıyla olması gerekmektedir.
Bu doğrultuda hibrit yetenek edinimi modellerinin önemleri gündeme gelmektedir.
Literatür çalışmaları incelendiğinde pek çok farklı endüstride hibrit yetenek edinimi
modellerinin etkin bir şekilde kullanıldığı görülmektedir. Hibrit yetenek edinimi
modelleriyle yetenek kaynaklarını bulma, yetenek tarama, yetenekli adayları
yetiştirme, görüşme planlama ve etkileşim kurma gibi konulara fayda sağladığı; pek
çok çalışmayla gösterilmiştir.
Bu tezin amacı, denizcilik endüstrisinin değişen ihtiyaçlarına cevap verebilmek,
şirketlerin yetenek edinimi süreçlerini başarılı bir şekilde tamamlayabilmelerini
sağlamak, doğru adayı işe almak ve etkin eğitimler vermek konularını desteklemek
için denizcilik özelinde kullanılabilecek hibrit yetenek edinimi modelinin
önerilmesidir. Modelin oluşturulmasında Bloom taksonomisi, çoktan seçmeli soru
üretme modeli ve kanıt merkezli tasarım yaklaşımlarından faydalanılmıştır.
Geliştirilen modelin uygulaması, varsayımsal bir tanker firmasıyla yapılmıştır.
Yapılan uygulamada öncelikle tanker firmasının bilgileri, vizyonu, misyonu, yatırım
planları, denet raporları ve şirket analizi gibi önemli bulguları analiz edilmiştir. Kanıt
merkezli tasarımın gerekliliklerinden dolayı tüm bunlar toplamda beş aşamalı bir
süreçten geçirilmiştir.
İlk aşama olan alan analizinde şirketin detaylı incelemeleri yapılmıştır. Emniyet
yönetim sistemi, insan kazaları, acil durum planlaması eksikliği ve mürettebat bilgi
seviyesinin yeterli düzeyde olmaması gibi çeşitli sebepler göz önüne alındığında
yardımcı kaynak olarak Petrol Tankerleri ve Terminalleri için Uluslararası Güvenlik
Rehberi (ISGOTT) seçilmiştir.
Ardından ikinci aşama olan alan modellemesinde ISGOTT bölümlerinden seçim
yapılabilmesi için şirketin mürettebat bilgileri ve eğitim departmanının görüşleri gibi
bilgiler incelenmiştir. Böylece ISGOTT ikinci bölümü seçilmiş ve sonrasında uygun
olan parçalarla kaynak seçimi tamamlanmıştır.
Ardından üçüncü aşama olan kavramsal değerlendirme çerçevesi uygulanmıştır. Bu
aşamada mürettebatın iş yükü, şirketin eğitim olanakları ve mürettebatın zaman
planlaması gibi unsurlar dikkate alınmıştır.
Bilgiler doğrultusunda kavramsal değerlendirme çerçevesi için Bloom taksonomisi,
kanıt merkezli tasarım katmanları ve çoktan seçmeli soru üretme modeli kullanılmıştır.
Böylece seçilen bölümlerden sorular ve cevaplar hazırlanmıştır. Ardından dördüncü
aşama olan değerlendirme sunumu ve uygulaması kapsamında bilgisayar destekli
değerlendirme seçilmiştir.
Son olarak beşinci aşama olan öneriler için şirkete on adet çoktan seçmeli soru
bilgisayar destekli değerlendirmeye uyarlanabilecek şekilde teslim edilmiştir. Hibrit
yetenek edinimi inceleme ve protokolüyle birlikte üretilen on adet çoktan seçmeli soru,
ekler bölümünde yer almaktadır.
Önerilen hibrit yetenek edinimi modelinin holistik olmasından dolayı bu tezin sadece
tanker firmalarına değil aynı zamanda denizcilik endüstrisindeki diğer organizasyonlar
ve şirketler tarafından da kullanılabileceği düşünülmektedir.

xxii
Böylece otonom gemiler, hibrit yakıtlar ve büyük veri gibi yeni nesil yeteneklerin
denizcilere kazandırılması için de bu tezin bir zemin hazırlayabileceği
düşünülmektedir.
Tezin denizcilik endüstrisine olan katkıları dışında akademik çalışmalara da katkı
sağlayabileceği düşünülmektedir. Soru üretme modelleri, Bloom taksonomisi, kanıt
merkezli tasarım ve yetenek edinimi gibi akademik çalışmalara yardımcı olabileceği
ve katkı sağlayabileceği görülmektedir. Ayrıca yapay zeka, makine öğrenmesi, derin
öğrenme ve karar destek mekanizmaları gibi yaklaşımların; tezin çıktılarıyla birlikte
kullanıldığında gelecek çalışmaların denizcilik endüstrisine katkı sağlayabileceği
düşünülmektedir.

xxiii
xxiv
1. INTRODUCTION

Human resources managers in shipping companies locate the most effective seafarers
for their ship organization, and activities are specified as a whole to assure its
continuation. Despite of technological breakthroughs, human error is responsible for
over 80% of all catastrophes, according to studies. Factors such as crew fatigue caused
by tight schedules, potential understaffing of ships, inadequate management,
inconsistent or low wages, outdated or generally the crew's view on safety at sea is
influenced by variables such as lack of supplies or technology, the company's safety
culture, and its recruiting strategy. These elements are difficult, if not impossible, for
any one crew or party to change or affect. These components are often conveyed, but
sometimes they are thoroughly scrutinised (Berg et al., 2013). The role of human
resources management (HRM) is vital to the overall operation of the organisation.
Seafarer training mostly on technical aspects, although there is a deficiency in training
programmes aimed at improving managerial abilities. Hence, crew resource managers
must provide instruction and training in this domain. As said before, ship mishaps are
largely attributed to seafarers.

Human resource managers' a vital role in reducing ship accidents in marine enterprises.
The concept is crucial for the management of ship safety. The importance of
management will escalate in the future. The younger generation expects equality,
creativity, and above all, a common goal that unites the team. This is because owners
of vessels will need a larger quantity of highly proficient as compared to what is
needed at present. A ship manager is responsible for ensuring the dedication of
seafarers to the company, promoting job satisfaction, and reducing the rate of seafarers
leaving their positions. The marine sector has a higher incidence of employee turnover
compared to other industries. This poses a threat to the management of ship
safety.Shipping firms enhance their efficiency by prioritising the human element,
which requires implementing transparent and intentional human resource management
strategies. These human resource planning should include provisions for both people
on board and those on land. The resource management should give priority to the

1
seafarer's competence and stability of their experience. In order to adapt to the future,
mariners need to cultivate novel skills and aptitudes.

Crew management relies on several key factors: (1) fostering a culture of open
communication and shared responsibility between the ship's crew and onshore staff,
with a focus on constructive feedback; (2) implementing a robust and reputable
monitoring system for organisational oversight; (3) safeguarding the crew, as well as
the vessel and its surroundings, from maritime hazards; and (4) ensuring the well-being
of the crew by providing adequate provisions of water and nutritious food (Anastasiou,
2017). The use of crew supervision,independent vessel management is already
prevalent on a global scale. A ship management business may be shown as an entity
that assumes accountability and responsibility for tasks delegated by others, or as an
entity that executes the duties of the executives, irrespective of whether the individual
in question has authority overs (Dickie, 2014). The ship and crew executives are
responsible for various tasks including managing the recruitment, training, and
performance improvement of the seafarers. They also ensure compliance with
mandatory and global regulations and maintain a good understanding of
costumers. (Anastasiou, 2017). The presence of subpar vessels and inadequately
skilled sailors on such ships is a significant contemporary concern. Independent ship
management and the lack of convenient flags are the primary causes of diminishing
standards in the marine fields. The MLC 2006 is very effective in mitigating the
prevalence of substandard vessels and their associated entities.

Industry 4.0, which emerged in the early 21st century, is principally characterised by
the convergence of Cyber-Physical Systems (CPS), innovatives to create advanced
intelligent systems. These systems are now used across several industries and are
integral to our everyday lives, such as in autonomous vehicles, automated train
systems, and drone deliveries. (Bai et al., 2020; Krupitzer et al., 2020; Turan et al.,
2016; Vagia, Transeth, and Fjerdingen, 2016; Wahlström et al., 2015). The
transformation of work has also resulted in the emergence of a distinct area of study
focused on understanding human-automation interactions in the workplace (Benitez,
Ayala and Frank, 2020; Onnasch, 2015; Parasuraman and Riley, 1997). While
automation systems have brought about significant safety, environmental, and
economic advantages, they have also given rise to a number of issues (Gualtieri et al.,
2020; Munim, 2019). Due to their human construction and operation, these systems

2
are not immune to imperfections and may experience periodic failures (Bruder and
Hasse, 2020; Endsley, 2016). According to empirical studies, the influence of these
technologies on human performance might be negative, depending on the level of
autonomy (Lee et al. 2020). The need of providing adequate training and instruction
to workers who interact with automation systems has been widely recognised as
essential in reducing mistakes and minimising the negative impact on performance
(Cárdenas, Shin, and Kim, 2020; de Visser, Pak, and Shaw, 2018; Emad, 2010;
Endsley, 2016; Rondon and Fontes, 2017). The onset of the latest industrial revolution
occurred around ten years ago and continues to profoundly reshape every industrial
domain and society as a whole. The impact it had on the shipping sector led to the
creation of the phrase Shipping 4.0, which refers to the significant technological
advancements that have fundamentally transformed the enterprise. Within the context
of Shipping 4.0, there has been a growing focus on automation in recent years. This is
mostly due to the need for increased efficiency and the need to reduce congestion on
routes and minimise transportation emissions. Given the completion of many ship
automation projects and ongoing ones, it is anticipated that there will be an increase in
the installation of automated equipment on ships. Additionally, it is envisaged that
autonomous vessels will be operated from land-based locations in the near future. The
primary emphasis has been on technological breakthroughs that will signal this shift
towards more automated and possibly autonomous vessels. However, there has been a
lack of focus on teaching the sailors who will operate these upcoming vessels.
However, empirical evidence from various fields such as air travel, power generation,
extraction, and broad human factor literature has highlighted the significance of
providing adequate training to users of automation technology. This training is crucial
in preventing and reducing the impact of accidents that are related to automation. Thus,
it is crucial to ascertain the novel proficiencies and capabilities that forthcoming
seafarers would need to effectively operate extensively automated vessels either as
passengers or from onshore operation centres. Conventional shipping will be
substituted by a marine sector that incorporates computer and software support as part
of the sector 4.0 process. Consequently, Industry 4.0 will bring about many alterations
in the job dynamics within the marine sector. This occurrence in the marine sector is
referred known as marine 4.0. The advent of the next industrial revolution will result
in progress in several facets of workplace connections which includes flexible work
arrangements, enduring employment, job contentment, and motivating strategies. This

3
paper examines the potential shifts in the dynamics between sailors and ship owners
as the clever industrial revolution advances.

The reduction in crew size on super carrier ships, from 40-50 to about 20-25, has
generated significant attention due to advancements in technology (Ljung, 2010). The
qualifications and training necessary for the immediate, intermediate, and long-term
prospects of the marine business will vary from the current requirements. Another
potential obstacle might occur due to the shipping industry facing competition from
other industries in hiring a highly trained workforce. However, this situation also
presents a chance to showcase that the skills needed in the marine sector might be
advantageous in other fields as well, so enabling individuals to have more flexibility
in their career paths.

The perception that generations prior to him are reluctant to acquire knowledge of new
technologies as a result of inadequate communication across different industrial
sectors. Companies in the future will choose the following generation of mariners, as
shown by this.

1.1 Talent Acquisition and Management

Regarding surrounding the administration of training, the necessary preparations that

maritime institutions should undertake, and the future skills required by autonomous
ship operators in the absence of a well-established independent ship technology,
regulatory framework, and training syllabus for future seafarers. Given these
uncertainties, participants anticipate a transition in the sector from specialised to
generic training. Specifically, the first training will be offered by technology
companies or vendors. Workshops and seminars may be offered to give instruction on
emerging versions of autonomous technology. Once the technology is implemented,
the STCW Convention will generate course syllabi that maritime institutions may
readily accept and administer. Maritime institutions acknowledge the need for change,
but their ability to prepare is hindered by uncertainties over the specific facilities and
skills necessary for teaching future seafarers. Consequently, they can only make
approximations about the skills that could be necessary in the future. This is predicated
on the assumption that the precise abilities will be known only when autonomous ship
technology advances.

4
As autonomous ship technology advances and stabilizes, the skills necessary by
seafarers to operate autonomous ships will become clear. Although unmanned ship
technology is now in its nascent phase, it is anticipated that conventional seafaring
training will continue to be necessary for the foreseeable future. Once autonomous
shipping technology reaches a more advanced stage, the necessary rules and guidelines
will be established, together with a comprehensive training programme, to ensure the
safe and efficient functioning of these systems.

Numerous studies (Alop 2019; Deling et al. 2020; Edler & Infante 2019; Pazouki et
al. 2018; Streng & Kuipers 2020) have emphasised the need of education and training
in response to the increasing use of automation in this sector. Current
investigations about unmanned vessels indicate that these technologies will have a
significant role in the future of marine transportation. This is supported by studies
conducted by Bertram (2020), Munim (2019), and Tanakitkorn (2019). Upon the
commencement of these vessels' operations, it will be necessary to include novel forms
of instruction into the seafarer training curriculum. Gaining insight into the viewpoints
of stakeholders in the maritime industry regarding the necessary skills for seafarers to
operate these new ships is crucial. This understanding will allow governing bodies in
the maritime industry to develop future training programmes and regulations that
facilitate the secure and safe operation of these vessels. The present responsibilities of
employee relations departments include several tasks, including as motivation, career
planning, work satisfaction, fostering and internal communication. The importance of
human resources in the maritime industry has increased as a result of recent
advancements. The growing importance of the human resources in the maritime
industry is apparent. Technological progress has led to an innovative management
paradigm in the maritime industry. The handling of human resources has become the
dominant management strategy in the marine industry, particularly for sailors. In the
era of Industry 4.0, the acquisition and management of talent are becoming more
crucial. Due to the growing flexibility in the workforce, there will be a change in
demand for sailors who possess the ability to effectively use and develop advanced
smart technologies. The retention of these skilled seafarers will depend on the extent
to which the notion of human resources management can be put into practice (Muslu,
2020).The fast advancement of technology has had an impact on transportation
companies. Seafarers should improve a variety of skills. Ship operators are burdened

5
with extra responsibility due to this dilemma. The vessel operating method, the
manager does not have a distinct role inside the organisation, but rather aligns
themselves with every aspect of the association's activities. An organisation might
consist of many divisions, such as specialised, operational, contracting, and HR. Each
division has executive authority and operates with its own internal administrative
framework. Conversely, crew training is a very important function of HRM. To ensure
efficient ship operations, ship operators should give priority to crew training. Seafarers
should be equipped with the requisite skills and knowledge provided by Industry 4.0,
together with a comprehension of lifelong learning. Job analysis is a crucial
responsibility carried out by the HRM department. These exams include explicit work
standards and detailed task outlines. There is a need to do a fresh job analysis in marine
organisations.

Contemporary vessels are progressively becoming equipped with advanced computer

systems, making them more susceptible to control frameworks that rely on
programming. Advanced programming and simulation skills will lead to the control of
more intricate systems through software. Additionally, continuous evaluation options
will be available, along with recommendations for corrective actions by the team,
providing decision support for supply chain management. The growth of digitalization
and the availability of excellent, in autonomous and remotely controlled operations.

1.2 Assessment and Evaluation

Assessment and evaluation are needed for the effective realization of talent acquisition
and management processes. In this direction, especially with technological
developments, creative, modern and artificial intelligence supported assessment and
evaluation tools are used. The terms "assess" and "evaluate" are sufficiently
interconnected to be utilized in reciprocal definitions. Nevertheless, disparities exist in
the procedures of educational assessment and evaluation. In the process of evaluating
students, educators collect, condense, and analyze information to ascertain which
approaches to employ in order to augment the educational encounter. The educators
evaluate the preparedness of students for learning, their favored approaches to
learning, their previous encounters with the subject matter, and any hindrances to their
learning. Assessment sources can be categorized into two types: subjective, such as
self-assessment checklists, and objective, such as pretests. Assessment can be either

6
formal, such as through structured interviews, or informal, such as through questioning
during lectures.

Assessment enables the instructor to comprehend the cognitive, psychomotor, and

affective learning requirements of the student, thereby facilitating the identification of
subsequent educational measures. In the process of assessing students, educators
collect, condense, and analyze information to ascertain the extent of the student's
comprehension of the subject matter and the efficacy of the pedagogical approaches
employed. The assessment of students encompasses an evaluation of their
comprehension of novel concepts, proficiency in executing specific skills, and the
development of their ethical principles. Evaluation is a structured procedure that takes
place at predetermined intervals throughout the duration of the course and curriculum.
The establishment of evaluation criteria is a pre-requisite for the execution of
educational activities, and is a collaborative process involving all instructors. It is
imperative that students are provided with prior knowledge regarding the timing of
assessments and the specific criteria that will be utilized for evaluation purposes. By
means of assessment, the educator ascertains the efficacy of the pedagogical
endeavors. The evaluation process furnishes the educator with significant insights to
direct the provision of feedback to the learner.

1.3 Key Findings

1. The field of human resources management is expected to continue expanding.

With the advent of industry 4.0, it is evident that human resources management
would be enhanced via the use of automation and artificial intelligence,
resulting in improved efficiency, speed, and precision. Furthermore, it is
believed that there will be an increased need for human resources management
in the marine industry, as it adapts to evolving talent requirements and
expectations, similar to other sectors.
2. The advancements in human resources management in the marine sector
indicate the need for a more comprehensive, inventive, and contemporary
approach to talent acquisition, in addition to the old methods.
3. Within the scope of the greater need for human resources management, there
is a need to develop assessments to be made in interviews, trainings and
onboarding processes of candidates. In this context, it is thought that

7
 assessments should be used together with various learning methods as called
hybrid methods. In addition, since the maritime industry working areas are very
wide and international, it is seen that making the assessments correctly can help
in matters such as safety, transition of seafarers and diversity.
4. Although there is a need for the development of assessments, it is seen that
there is a need for human-independent, automatic, efficient and holistic
methods. Therefore, it has been seen that organizations operating in the
maritime industry, training centers, ship management companies and human
resources department will need the hybrid talent acquisiton model.
5. It is seen that tanker industry constitute a large part of the maritime domain.
Considering the developments in issues such as environmental pollution,
emissions and automation ships, it has been seen that an effective talent
acquisition will be needed from the smart recruitment of tanker companies to
personnel training. In this context, there will be a need for efficient use of
hybrid talent acquisition models.

The objective of this thesis is to provide a hybrid talent acquisition strategy that can
effectively adjust to the changing demands of the marine sector. The suggested
approach seeks to streamline the talent acquisition process, enhance enterprises' human
resource management endeavours, support the transfer of seafarers, and contribute to
the creation of training modules. Within this particular context, a combination of
methodological approaches were employed in together, consisting of Evidence
Centered Design (ECD), Bloom's Taxonomy, and the Multiple Choice Question
(MCQ) Generation. Subsequently, the model mentioned above was implemented in a
Company A, and a comprehensive presentation of the outcomes was provided. The
research has demonstrated that the model is comprehensive, relevant, and amenable to
automation and artificial intelligence.

The organization of the thesis is as follows. The first chapter covers the introduction
part of the thesis and in this chapter key findings are given along with various fields
of study. In the second chapter, literature studies of talent acquisition, human resources
management and assessment methods will be mentioned. Then, the methods used will
be explained in the third section. In this context, ECD, Bloom's Taxonomy and MCQ
generation model will be examined. Next, the proposed hybrid talent acquisition model

8
will be demonstrated. In the fourth section, hybrid talent acquisition survey and
protocol will be applied to Company A and the results will be discussed. In the fifth
chapter, the outputs, results, limitations and contributions of the thesis will be
mentioned.

9
10
2. LITERATURE REVIEW

In this section, important studies in the field of human resources management will be
mentioned. When the existing literature is examined, various studies on human
resources management, talent acquisition, hybrid talent acquisition models, the
importance of new generation talents in maritime, decentralized employment
platforms, learning methods and question generation methods have been identified.
Studies in various industries will be examined, and then maritime-specific findings
will be presented.

2.1 Human Resources Management Studies

In this article (Meyer and Hammerschmid, 2010) researchers points that how HR
decision-making procedures differ in Europe: while decentralization has been a
common aim of modernization programs, they nevertheless discover a high degree of
centralization. Furthermore, they discover that decentralized decision making typically
coexists with a larger degree of shared decision making. Furthermore, they investigate
the impact of various cultural and institutional aspects in order to address the issue of
embeddedness in greater depth. Their findings indicate that national culture,
administrative traditions, and institutional arrangements all play an essential role in
explaining the structure of HR decision-making throughout Europe. In another study
examined, it is stated that communication technologies and human resource
management are very important for companies. The study includes an extensive
literature review by examining various articles. Information technology has had a
significant impact on human resources operations and practices in recent years.

However, little research has been conducted to analyze its efficacy, and the majority
of current studies have not investigated the extent to which these new methods assist
firms to achieve their HR goals of recruiting, motivating, and keeping personnel. One
factor contributing to this is the presence of several constraints in existing systems,
such as the use of unidirectional communication methods, the lack of personalisation
and engagement, the limited opportunity for human connection, and the frequent

11
establishment of an artificial disconnect between people and organisations. The
primary objectives of this paper are to assess the present impact of technology on HR
operations, evaluate the available research on the topic, and investigate the advantages
and possible disadvantages of using these technologies. In addition, Stone et al. (2015)
provide suggestions for future study and practical use of technology to streamline HR
procedures.

This research analyses a total of 5665 data points collected from 32 distinct nations.
The emergence of HR management has been followed by claims indicating that these
technological advancements (referred to as electronic human resource management
technology or e-HRM) would enhance the strategic nature of HRM in organisations
(Orhan and Celik, 2022). Nevertheless, the existing literature on the role of technology
in organisations presents arguments that question such a forecast. The researchers
use theoretical perspectives in this domain, using a large survey dataset including 5665
firms from 32 distinct nations. The researchers find and prove that there is a direct and
reciprocal relationship between strategic HR engagement and greater e-HRM
capabilities. The authors Marler and Parry (2016) address the potential consequences
of these discoveries on theory, practice, and future research. Another study examined
was carried out in the field of human resource information system. In this study, it has
been determined what human resources management will need in the future and what
the gaps will be in various industries. In addition, the advantages and effects of using
technology in human resources management are mentioned. They analyze the
professional and academic evolution of the human resource information systems
(HRIS) area in this study to assess its success and recommend future directions for
research (Orhan and Celik, 2022). To that end, they investigate the relationship
between the progress of technology and the HR sector over four significant
technological eras: mainframe, client server, ERP and Web-based systems, and cloud-
based systems. They examine how HR practices and needs generated the need for these
systems in each age, as well as how these technologies allowed the HR industry to
grow. Furthermore, they trace the HRIS subfield and its relationship to technical
evolutions in the HR area. Surprisingly, they discovered that most of the study on the
use of technology to assist HR has just emerged in the previous 15-20 years as a result
of the usage of the Web as a medium for providing HRIS. They end by outlining how

12
researchers in information systems and human resources may collaborate to develop
HRIS (Johnson et al., 2016).

2.2 Critical Studies on Talent Acquisition

When the current literature studies are reviewed, it has been determined that there are
various studies in the field of talent acquisition. In these studies, the importance of
identifying the talent, bringing it to the industry and the gains that will be needed in
the future are mentioned. For example, in one study, the importance of artificial
intelligence technologies in human resource management was mentioned. In this
study, the opinions of various experts were taken and a hybrid model proposal was
made.

Artificial intelligence models have found use in the field of human resources. The
opinions of job reviewers on internet discussion forums have been collected in this
article. An comprehensive literature study was conducted to determine the relative
impact of variables. LDA Topic modeling using PCA on unstructured text data has
been investigated. Second, the Li-Hu method of sentiment analysis was used to
determine job searchers' satisfaction with job sites. The suggested strategy, dubbed
"Hybrid AI Talent Acquisition Model," takes a fresh approach to simplifying jobseeker
perceptions of online venues (Jas et al., 2022). In another study examined, information
modeling studies in different industries, especially engineering and construction, were
mentioned. In the study, findings on education, hiring and talent management were
determined. Building Information Modeling, or BIM, is now the standard in most
architectural, engineering, and construction (AEC) firms, but a shortage of competent
BIM expertise is preventing its widespread implementation. This study revisited two
previous research projects completed in 2013 to provide a modify on the modifications
in BIM educational approaches and work practices that have accompanied the
worldwide expansion in the adoption of BIM over the years, as well as to answer the
research questions: The way does colleges and universities prepare learners for BIM-
related place of employment competency, and what did organizations do different
ways in hiring managers and staffing to respond to their clients' increasing
requirements for BIM services. To gather feedback from practitioners and educators,
a survey questionnaire including questions from the two research was sent. The
findings revealed that BIM integration in college courses and deployment in AEC

13
firms will continue to develop. Since 2013, many trends and patterns in BIM education
and hiring methods have been addressed. This study shows enhanced commercial
training in BIM and university BIM education as a result of increasing collaboration
between academics and industry. This research will assist institutions in determining
educational objectives for developing BIM talent and will urge businesses to
reconsider their BIM talent acquisition and recruiting tactics (Elias et al., 2022).

In another example (Walford and Scott-Jackson, 2018), it is seen that it is done in the
field of improving the talent acquisition process and what factors it depends on. In this
study, it has been analyzed which parameters employers and job candidates have
expectations. Data analysis reveals a considerable improvement in all major indicators
connected to the talent acquisition process following the deployment of the "Talent
Rising" strategy. A recruiter's and the talent acquisition team's function has shifted
dramatically, and they are now an inside strategic partner with aligned interests, an
adviser to the organization. With a growth in millennial prospects entering the
workforce, social media will become increasingly crucial in talent branding and
recruitment; the digital by default generation is already here (Orhan and Celik, 2022).

In another article, the effects of increasing technology with globalization on human

resource management were investigated. In this study, it is mentioned how the
recruitment processes are in the current world and how they can be improved
afterwards. It is also mentioned how the recruitment process can be implemented more
sustainable and technologically. This article highlights how growing globalization,
changes in the technological environment, and changes in the way people run
businesses have raised the demand for skilled workforce who can provide the firm with
long-term competitive advantage. As a result, firms, particularly those in the IT/ITES
industry, are devoting significant attention, money, and resources to developing their
talent acquisition strategy. Recruitment has always been a reactive process, with firms
looking for individuals only when there is a vacancy. This article seeks to present the
notion of sustainable talent acquisition in order to maintain a consistent supply of talent
to fulfill an organization's future demands (Orhan and Celik, 2022). The authors
attempted to incorporate the concept of three sustainability pillars - environmental,
social, and economic - into the recruiting landscape and to develop an action plan to
speed up the hiring process and enhance cost efficiency. The authors believe that
environmental sustainability can be attained by constantly gauging talent demand and

14
ensuring consistent supply, social sustainability can be attained by engaging a talent
pool to build long-term relationships, and economic sustainability can be attained by
effectively using data to ensure cost-effective hiring. Using the example of a research
undertaken to optimize the recruiting funnel at an ITES business, the authors
demonstrated how analytics may assist in achieving the desired results (Mohapatra and
Sahu, 2018).

In another study examined, the application areas of smart systems within the scope of
talent acquisition were mentioned. In the study, a model proposal has been made that
can help articles and projects in this field (Fayoumi and Loucopoulos, 2022). Another
article explores how potential job seekers look at criteria to find a more suitable job.
In this study, the importance of social media and technology in human resource
management is mentioned. The talent acquisition process comprises intricate
interactions between potential job seekers and the organizations that want to hire them.
Although both sides are working toward the same goal, the rise of social media
platforms and a workforce competent at using them presents new obstacles for
businesses. Companies are growing conscious that they need to recruit talent
proactively rather than depending on traditional job advertising, as talent managers
recognize shifting views of their potential workforce. Companies may reach out to
today's job searchers using social media tactics that include new sorts of interaction
such as direct contact and personalization. Based on the Elaboration Likelihood Model
and the understand-deliver-measure cycle, this article analyzes the potential and
limitations of using social media throughout the recruiting process. They provide two
case studies, one with a major business and one with a mid-sized firm, to demonstrate
how organizations may use social media to develop a value proposition for job
searchers and personalize content to improve talent acquisition (Phillips-Wren et al.,
2016).

The implementation of novel pedagogical approaches frequently broadens the scope

of anticipated student attainment, necessitating a concomitant reevaluation of the
methods employed to gather substantiating proof that verifies the fulfilment of these
expectations by both the educators and learners. The conventional evaluation approach
employed in a majority of introductory science courses at the college level is the
employment of multiple-choice (fixed response) exams. When utilized optimally, this
strategy can furnish insights into the students' comprehension and retention of

15
fundamental concepts. Disregarding the inherent challenge of formulating a multiple-
choice assessment that effectively measures comprehensive comprehension of course
material, its constraints become particularly salient when educational goals encompass
both cognitive and practical competencies that students are expected to acquire
through their coursework. In order to enhance students' proficiency in conducting
guided laboratory investigations, constructing logical arguments, and expressing their
ideas, alternative assessment methods such as papers, practical exams, problem-
solving demonstrations, model building, debates, and oral presentations should be
utilized as benchmarks of progress and/or in the grading process. In the context of
science learning, what occurs when students lack experience in responding to
performance prompts and faculty lack proficiency in conveying their expectations for
a superior performance. The multiple-choice exam's familiar terrain can create a
deceptive sense of clarity and objectivity in both students and instructors regarding the
evaluation criteria. This can result in other assessment strategies appearing to be
subjective, unreliable, and occasionally unjust in comparison. In a hypothetical
situation where conventional exams are replaced with alternative assessment methods,
it is possible for students to perceive the presence of an implicit or covert curriculum,
which may only be discernible to the instructor of the course (Allen and Tanner, 2006).

When the literature studies are examined, the effect of technology on human resources
management is seen. Especially with the effect of globalization, it is seen that the
majority of the recruitment, training and job placement processes of the candidates
will be done with the help of technology. In this case, the importance of learning and
assessment is once again understood. When the literature studies in this field are
examined, various learning methods have been identified and it will be mentioned that
the assessment methods are examined to complete them. The learning outcomes
(knowledge, abilities, and competencies) that learners must accomplish are at the
center of the learning process in outcome-based learning. To help learners in
effectively achieving the desired learning objectives at the conclusion of the learning
experience, all educational activities and materials must be aligned to the intended
learning outcomes of a learning module or course.

Outcome-based assessment entails aligning the assessment process with the learning
outcomes. This implies it should help learners advance (formative assessment) and
confirm accomplishment of the targeted learning goals at the conclusion of the process.

16
(summative assessment). It also implies that the evaluation procedure should be
tailored to the type of results being evaluated. This study provides an overview of the
current state of outcome-based learning in Europe and suggests a unified conceptual
model for outcome-based assessment, forming a theoretical framework for the
integration of important concepts such as learning outcomes, assessment, and units of
learning. Finally, an application case is provided to demonstrate how the model may
be used (Crespo et al., 2010). Over the last decade, there has been a change in science,
technology, engineering, and math education toward a competency-based approach,
particularly in engineering. Competency-based learning (CBL) is a student-centered,
outcome-based method of education in which students proceed to more advanced tasks
after mastering the needed information and skills.

Many studies on CBL application in engineering higher education have been

published; however, the literature lacks a comprehensive review that summarizes
earlier work to guide both current research and future implementation. The goal of this
study (Henri et al., 2017) is to synthesize prior research and identify shortcomings in
based on competencies engineer educational research. It highlights the many ways to
CBL implementation, the impact of the technique on student results, instruments to
improve its efficacy, and assessment methodologies. There are also comments and
recommendations for further research.

In addition, when the existing literature is examined, it has been determined that there
are articles in the field of learning techniques suitable for both instructors, students and
candidates. In these articles, the deficiencies of the field in which the studies will be
carried out are determined in general. Then, in line with the opinions of various
experts, it is mentioned which skills will come to the fore and how these will be
provided to the candidates. What the success criteria are, the expectations of the
European Union, situation analysis, talent identification and encouragement to learn
are mentioned in detail (Tirziu et al., 2015; Amiti, 2020; Moreno-Guerrero et al., 2020;
Singh et al., 2021; Prasetyo et al., 2021; Damuri et al., 2021; Maatuk et al., 2022).

In this study (Brookhart, 2018) authors conducted a comprehensive review of scholarly

literature pertaining to rubrics in the context of higher education, spanning the years
2005 to 2017. To date, the predominant focus of rubric research in higher education
has been on analytic rubrics, which evaluate each criterion independently. Descriptive
rubrics, featuring four or five performance levels, have been commonly utilized.

17
Ultimately, the study results did not reveal any correlation between the type or quality
of rubric and the outcomes observed. The literature indicates that the implementation
of rubrics has yielded favorable results across all studies.

The field of education is characterized by lively discussions regarding the optimal

methodologies for enhancing students' academic achievement and progress. Currently,
there is a global trend among educators to endorse the implementation of rubrics as a
pedagogical instrument, emphasizing the significant benefits that rubrics can offer
within the context of teaching and learning. Rubrics are an effective assessment
instrument that can assist educators in evaluating the academic performance of
students in a more uniform, dependable, and impartial manner. A proficiently crafted
rubric can assist students in recognizing their areas of proficiency and deficiency, and
foster a more impartial evaluation of their work's. Despite some existing research on
the impact of rubrics on student performance, the body of literature on this topic
remains in its nascent stages. This paper (Chowdhury, 2019) aims to examine the
concept of rubrics, the various classifications of rubrics that can be employed in
educational settings, and the procedures involved in the development of a rubric. The
present discourse will delve into the utilizations of rubrics in pedagogy, and its
potential to enhance student learning outcomes and furnish more efficacious
evaluations of student performance.

Assessments have become increasingly prevalent in various disciplines within the

higher education sector. Generally, educators who serve as primary evaluators during
the grading procedure are also the ones responsible for creating rubrics. In
contemporary times, there has been a growing concern regarding the conventional
teacher-centered approach to rubric construction. Consequently, some scholars have
initiated an investigation into an alternative approach that involves collaborative
efforts between students and teachers in the co-creation of assessment rubrics. The
present study (Kilgour et al., 2020) delineates the methodologies utilized in a research
endeavor that explored the collaborative development of rubrics across six distinct
settings. The objective of the project was to foster student engagement by facilitating
collaboration between students and teachers in the co-construction of rubrics that are
jointly owned by both parties. The study employed a mixed methods methodology to
investigate the efficacy of the aforementioned strategy. Data was collected from both

18
teacher-participants and student-participants through the use of questionnaires,
interviews, and focus groups to gain insight into their experiences within the study.

In another study discusses the inclination of multiple educators towards scrutinizing

the assessment tools and evaluation procedures employed in experimental Chemical
Engineering courses. Experimentation in Chemical Engineering, and Process Scale-
up. Investigation has put forth a methodology for the purpose of devising, constructing,
and implementing rubrics that are grounded in the principles of competency and
learning outcomes. The possibility of implementing comprehensive rubrics for gradual
attainment of competence and formative evaluation over academic years was made
feasible due to the collaborative efforts of the three teaching teams (Tobajas et al.,
2019).

A thorough examination of the literature was conducted on the subjects of rubrics,

educational assessment, authentic assessment, and other relevant topics. This literature
review examines various rubric types and fundamental components necessary for
constructing a comprehensive rubric that enhances classroom efficacy. The purpose of
this review (Nkhoma et al., 2020) is to assist researchers, students, and educators who
are inexperienced in rubric design and implementation. This an occasion for proficient
rubric designers and users to obtain validation from scholarly sources regarding
effective rubric implementation strategies. This project may prove beneficial to
researchers who are engaged in rubric-related studies. Rubrics serve as both an
evaluative mechanism for students during high-stakes examinations and an
educational aid that facilitates learners in selecting suitable learning methods.
Additionally, they assist educators in designing effective instructional strategies, while
also enhancing the reliability and validity of assessments.

The aim of this research (Cheng and Chan, 2019) was to evaluate the efficacy of
incorporating rubrics in reflective writing through an analysis of its impact on the
quality of students' work and the perspectives of both students and instructors. A group
of 134 undergraduate students participated in a one-hour workshop focused on
reflective writing. The study randomly assigned undergraduate students to three
groups, each led by a different instructor. The first experimental group received rubrics
accompanied by detailed explanations, while the second experimental group received
rubrics without any explanation, thereby simulating a self-learning environment in a
university setting.

19
In another study researchers evaluated the efficacy of incorporating rubrics in
reflective writing through an analysis of its impact on the quality of students' work and
the perspectives of both students and instructors. A group of 134 undergraduate
students participated in a one-hour workshop focused on reflective writing. The study
randomly assigned undergraduate students to three groups, each led by a different
instructor. The first experimental group received rubrics accompanied by detailed
explanations, while the second experimental group received rubrics without any
explanation, thereby simulating a self-learning environment in a university setting.
The third group served as the control and did not receive any intervention. Three
instructors conducted measurements of reflection levels in the written works of
students. The instructors evaluated the reflective writings based on a common rubric
provided to the undergraduates. The findings of the study indicated that the
experimental groups demonstrated a statistically significant increase in the degree of
reflection observed in the reflective writings of students, as compared to the control
group. Nevertheless, there were no statistically significant distinctions observed
between the two experimental cohorts (Chan and Ho, 2019).

In this study (Reynders et al., 2020), STEM undergraduate degree program frequently
list process skills, which include important thought and data thinking about, as
expected learning outcomes. However, instructors frequently neglect to explicitly
evaluate these skills within their courses. Rubrics were formulated to evaluate process
skills in STEM undergraduates' written work as a component of the Enhancing
Learning by Improving Process Skills in STEM (ELIPSS) Project. The rubrics were
deployed across various fields of science and technology size of classes, sure levels,
and institutional types to ensure their feasibility for routine classroom application.
According to surveys conducted by instructors, it was found that rubrics were effective
in facilitating the evaluation of students' written assignments across various STEM
learning contexts. The rubrics were found to be useful by graduate teaching assistants
in assessing student work and in providing clarity regarding the instructor's assessment
expectations. According to the students' feedback, they were able to comprehend the
substance provided by the assessments and effectively utilize the feedback received
from them to enhance their forthcoming performance

When the studies specific to maritime are examined, it has been determined that there
are studies in various fields such as talent acquisition, learning and future skill. In these

20
studies, the effects of technological advances on the maritime industry are examined.
In addition, it was emphasized that not only today's talents, but also various talents
such as time management, software and leadership will gain importance in the coming
years. The SKILLSEA initiative intends to equip Europe's maritime professionals with
critical digital, green, and soft management skills for a rapidly changing maritime labor
market. It aims not only to develop a long-term skills plan, but also to grow the number
of these specialists, so improving the safety and efficiency of this critical sector.

The maritime industry is being transformed by technology and digitization. As smart

ships enter service, a new generation of competent, highly skilled maritime personnel
is required. Europe is a long-standing worldwide source of maritime knowledge, and
the four-year SKILLSEA project aims to ensure that the region's marine professionals
have crucial digital, green, and soft management skills for the fast changing maritime
labor market. It aims not only to develop a long-term skills plan for European maritime
professionals, but also to expand the number of these experts, so improving the safety
and efficiency of this critical industry. The European Community Shipowners'
Associations (ECSA) and the European Transport Workers' Federation (ETF)
developed the future-proof project, which is made up of a consortium of national
maritime authorities, shipping companies, shipowners' associations, maritime trade
unions, and maritime education providers from 16 European countries. Among the key
goals and objectives are: analyzing the impact of technical changes on the industry's
talent needs. An even greater connection between the industry's skill requirements and
marine professionals' education and training.

Overcoming hurdles to marine professionals' movement. Improving collaboration and

synergy among educational institutions, marine authorities, and industry. Ensure that
Europe continues to have world-class access to maritime skills and expertise for
increased competitiveness. In addition, findings in the form of Work Package were
published within the scope of this project. For instance Work package 1 will undertake
an analysis of present and future skill needs in the maritime shipping sector. In
addition, a technique for anticipating long-term skill demands will be established. A
method will be established to keep track of the current condition of play in terms of
identifying skills needs in the short, medium, and long term, allowing for early
adaption of educational programs. Throughout WP1, the need for horizontal skills to
enable mobility, particularly from sea to land, but also between specialties and

21
business categories, will be studied alongside the unique skills of maritime
professionals. (e.g. from tanker to cruise or from insurer to broker).

Gaps and mishmashes in the industry will be highlighted, as will the possible influence
on present professional profiles or the development of new vocational profiles. The
requirement for upscaling of ICT competencies at MET academies will be a special
focus throughout the GAP analysis process, as this will be vital during implementation
and future-proofing of skills (SkillSea, 2022). This research investigates how
leadership and skills influence long-term success in the marine sector, as influenced
by organizational culture.

This empirical research is grounded on Service Dominant Logic (SDL) and is bolstered
by a comprehensive survey of 308 shipping enterprises undertaken in 2018. The
research identified four distinct leadership styles: authoritarian, stakeholder, formal,
and growth. It also identified four talent categories and their corresponding
management strategies. Additionally, two types of corporate culture were identified:
formal and learner. Lastly, the analysis identified seven characteristics related to
sustainability. Environmental resources, environmental effect, compliance with
environmental law, social sustainability, and economic sustainability. Leadership has
a more significant impact on the achievement of sustainability goals compared to skill.
The talent-sustainability relationship is moderated by the culture of the organization,
but not by leadership-sustainability connection (Pantouvakis and Vlachos, 2020). A
robust seafaring community is required to support maritime cargo transit. With greater
automation and digitization of vessels caused by Industry 4.0, sailors' responsibilities
will shift, and new skills will be required.

However, there has not been enough study to determine the specific skills required by
seafarers and the way MET for seafarers should be created. The STCW, which serves
as the foundation for MET for seafarers, has been deemed as inadequate in its current
form and unlikely to fulfill future requirements. The gap will expand more as ships
become more modern and automated. This study seeks to bridge the gap between
present seafarers' abilities, which are mostly based on STCW standards, and the new
skills necessary. Interviews and surveys were undertaken to better understand the skills
necessary for seafarers from various stakeholders, efforts for such skill development,
and the maritime industry's readiness to prepare seafarers for the future. According to
the survey, sailors will need three types of skills: digitalization and technological

22
advances, soft abilities, and technical expertise. To operate sophisticated ships safely,
seafarers are going to be trained.

There is also a dearth of understanding about Industry 4.0 among sailors and present
seafaring students. The report also demonstrates the skills mismatch between what the
marine sector requires and what seamen have. The conclusion is that the marine sector
is not future-ready. Raising awareness of Industry 4.0, enhancing collaboration among
marine stakeholders, altering the maritime training environment to be more engaging
and immersive, and bringing new courses to the MET curriculum are among the
proposals. As a result, this study is crucial for tackling the requirement for talent
conversion in the maritime industry in order to guarantee that seafarers are equipped
and trained in future competences (Ow Yong, 2022). According to the Sea Transport
ITM, one of Singapore's primary initiatives is to recruit, retain, and retrain people in
order to establish a future-ready marine workforce. According to existing literature,
the many talent initiatives launched by various government entities as a reaction to the
ITM and the industry's overall requirement to recruit, retain, and upskill talent can only
be gathered. As a result, research gaps in Singapore's marine talent strategy include
the present health of the maritime industry and if maritime enterprises are making
comparable efforts to recruit, retain, and upskill workers.

Surveys and interviews were conducted to acquire quantitative and qualitative data
about the industry's present situation of recruiting, keeping, and retraining personnel,
as well as other realistic challenges that were neglected. Primary research studies
demonstrated that the marine industry's total push factor is greater than its pull factor.
Furthermore, the government typically makes more effort than firms to lure people
into the field, yet there are frequently restrictions. Other issues discovered include
word of mouth as a double-edged sword in attracting ability, inadequate dedication by
organizations in maintaining talent, little assistance provided by businesses in
retraining talent, and people's lack of awareness of the Maritime Port Authority (MPA)
joint funding plans to use in their training.

The most difficult component of establishing a future-ready marine workforce is

aligning maritime enterprises with the goal of marine Singapore, as certain businesses
may have a narrow view. Companies can also choose how much they want to
contribute to Singapore's future-ready marine workforce. As a result, the report
recommends the formation of a marine organization to give more direct assistance to

23
all maritime stakeholders, as well as a multinational synergistic model that reminds
the sector of the interconnectedness of the three core maritime goals - sustainability,
digitalization, and talent. Overall, maritime companies have to and will need to create
greater value (Ang, 2023).

A new talent training mode suited for maritime safety engineering specialty was built
under the concept of innovation and entrepreneurship, and the quality of mass
invention activities between teachers and students was evaluated in comparison to the
conventional training mode. In addition, the quality of instructors' and students'
innovative activity was evaluated in comparison to the standard training paradigm. The
findings indicate that the idea of talent training for innovation and entrepreneurship
education meets the internal criteria of the building of distinctive disciplines in the new
era. Improved teaching style of marine safety engineering satisfies current industrial
demands and corresponds to future societal growth (Kan et al., 2021). Shipping firms
must rely on competence frameworks to pick the proper people since the shipping
sector continues to be the backbone of commerce by functioning as a key method of
transport in transporting big cargo volumes around the globe.

Though there is literature that contributes to the marine sector by providing

competence frameworks for various shipping professions, there is limited attention on
commercial jobs in the industry, notably shipbrokers. As such, the purpose of this
research article is to fill the vacuum left by a lack of research in this field by defining
and categorizing the skills of shipbrokers and organizing them into a hierarchical
model. Instead, the major and sub-competencies were established and extrapolated
from a survey of general broker literature, then contextualized into the function of
shipbrokers in the maritime industry. Financial, management, organizational,
customer service, and digital abilities were recognized as the five primary
competencies, with 15 related sub-competencies.

A survey questionnaire was created and evaluated before being distributed to 35

Singapore shipbroking businesses (Teo, 2021). This research adds to the social pillar
of long-term supply chain management. It accomplishes this by researching how
female maritime executives handle work-family conflict, therefore serving as
institutional entrepreneurs in the creation of a work-family balancing logic. The marine
industry is a male-dominated supply chain management environment with a talent gap
due to a dearth of female CEOs. Work-family balance concerns hinder women from

24
staying in the workforce, which is one cause of this problem (Vo et al., 2022). HRTech
studies have been conducted in the marine industry. Safa et al., (2018) studied the
significance of talent by discussing the characteristics and skills necessary When
overseeing a port, it is essential to consider the necessary qualities for effectively
operating these modern enterprises. They exemplified their points by examining
specific cases at the Texas The port, the need of recruiting skilled individuals to the
organisation, and the function of HR in port management.

The importance of HR was also discussed in this context. The discussion revolved on
strategies for recruiting skilled individuals in the maritime sector, the importance of
the human resources market, the growing need for labour, performance evaluation, and
salary administration. The need of enhancing recruitment procedures, in addition to
the necessity for digitalization and additional functionalities, were highlighted. The
topics of employee competitiveness, cadet programme, and leadership were all
addressed. The reference is from Barsan et al. (2016). Marine organisations have
crucial challenges related to cross-cultural competence, variety, performance
management, and engagement. As a consequence, the requirement of strengthening
the tasks expected of human resources professionals, adding issues connected to
diversity to corporate rules, and increasing difference more are all discussed in order
to enhance company performance and raise international marine to higher heights.
Furthermore, the necessity and significance of adopting HR technology to digitally
execute these and comparable operations was underlined (Progoulaki and Theotokas,
2016).

Upon reviewing the research pertaining to learning techniques, it is evident that the
literature has several publications covering a wide range of issues. The goal of
imitation learning methods is to accurately reproduce human behaviour in a particular
activity. The pedagogical method of teaching has been a well-established idea, but has
recently gained more attention due to developments in computer and sensor methods,
as well as an increasing need for advanced applications. The pedagogical approach of
acquiring knowledge through imitation has been increasingly embraced due to its
efficacy in imparting proficiency in intricate tasks with limited requirement for expert
knowledge of said tasks. The possibility exists that generic imitation learning
techniques could mitigate the challenge of instructing a task by solely presenting
demonstrations, thereby eliminating the requirement for explicit programming or

25
creating task-specific reward functions. Contemporary sensors possess the capacity to
swiftly amass and transmit copious amounts of data, while processors endowed with
formidable computational capabilities enable prompt processing that effectively
correlates the sensory data with corresponding actions. The aforementioned scenario
presents a plethora of potential artificial intelligence (AI) applications that necessitate
instantaneous perception and response, including but not limited to humanoid robots,
autonomous vehicles, human-computer interaction, and computer gaming. Specialized
algorithms are required to learn models effectively and reliably when using imitation
learning due to the unique challenges it presents. This article provides an overview of
various imitation learning techniques and outlines potential design choices at various
stages of the learning process.

In another study provides an introduction to the background and motivation of the

field, while also emphasizing the challenges that are unique to the imitation problem.
The categorization and review of techniques utilized in the creation and assessment of
imitation learning tasks are presented. Emphasis is placed on the acquisition of
knowledge pertaining to learning techniques in the fields of robotics and gaming, as
these areas are extensively discussed in scholarly literature and offer a diverse range
of challenges and approaches. The present study provides a comprehensive analysis of
the integration of various imitation learning techniques from diverse sources and
modalities, along with the incorporation of additional motion learning methods to
augment the process of imitation. Additionally, the potential impact on industry is
examined, significant applications are presented, and current as well as future research
directions are emphasized (Hussein et al., 2017).

2.3 Assessment and Evaluation Approaches

Design patterns are utilized as instruments to facilitate the creation of tasks within the
framework of an evidence-centered approach to assessment design (ECD). The present
chapter provides an overview of the fundamental principles of ECD, with a particular
emphasis on evidentiary reasoning. This text delineates the characteristics of design
patterns and elucidates their respective functions in the development of tasks
pertaining to sound assessment arguments (Mislevy et al., 2017). The ECD
methodology pertains to the development of educational assessments that are
constructed based on evidentiary arguments. The present manuscript offers an

26
overview of the fundamental concepts of ECD along with the lexicon and frameworks
that have been devised to operationalize this paradigm. The text outlines the
Conceptual Assessment Framework (CAF) and the Four-process Delivery
Architecture, which are high-level models utilized in the development of assessment
delivery systems. The study places particular emphasis on the significance of
probability-based reasoning in the process of gathering evidence from various task
performances. This is carried out with regards to the formation of beliefs about
unobservable variables that define the knowledge, skills, and/or abilities of students.
Psychometric models, such as item response theory (IRT) and latent class models, have
conventionally been linked to the aforementioned function. In order to establish a
cohesive framework for the expansion of probability-based reasoning in assessment
applications, it is necessary to adopt a more comprehensive approach utilizing
graphical models. This concise exposition on evidence-centered design offers the
reader an understanding of the role and placement of graphical models within the
broader scope of educational and psychological evaluation. To establish a clear
understanding, a straightforward illustration is employed, drawing upon prominent
standardized assessment (Mislevy et al., 2003). Achievement level descriptors (ALDs)
play a crucial role in numerous extensive assessment program by conveying the
significance of assessment scores and facilitating the interpretation of examinees'
knowledge and abilities based on their test performance. Examinees are frequently
categorized into performance groups based on their test results.

Creating a positive work environment, enhancing the recruitment process to hire

suitable candidates for all roles, decreasing employee turnover, enhancing diversity
and inclusion in local and global offices, improving performance evaluation systems
and measurements, ensuring the contentment of high-achieving employees,
establishing a sustainable pool of talent for future leadership positions, and effectively
incorporating women.The ALDs are used to communicate the meanings of the
outcome categories. In addition to providing proof of validity for score conclusions,
ALDs also have a vital role in determining the cut results on the evaluation continuum,
which determine how test-takers' scores are classified into performance groups. The
method of establishing standards is used to calculate these threshold scores. The
purpose of this academic paper is to provide a methodical strategy the interpretation
and use of scores via the development of assessment delivery systems. This paper

27
further examines the capacity standard formulation process (Plake et al., 2010). . Due
to the rapid advancement of computational intelligence, it is now possible to use
intelligent methods to enhance ship detection outcomes in marine applications. Our
objective in this study is to create an advanced convolutional neural network that can
effectively boost ship identification in various weather circumstances. The network's
learning and representation capabilities are enhanced by the modification of anchor
box sizes, estimation of localization uncertainty for bounding boxes, implementation
of sensitive non-maximum suppression, and reconstruction of a mixed loss function.
The identification and evaluation of the probability and impact of maritime incidents
has been a central area of investigation in the maritime industry. The prevailing
techniques employed for evaluating maritime hazards have been limited to a handful
of approaches, each of which has acknowledged deficiencies. A detailed review of the
academic literature relevant to the employment of supervised machine learning as well
as large-scale data mining for safe environment in the marine sector has been done, in
light of the increased attention that this discipline has attracted in other domains. The
paper offers a comprehensive analysis of accident probability forecasts, accident
severity assessment, danger of ship collisions. The purpose is to examine and
differentiate the goals, methods, data sources, and features of these research in order
to get a more full comprehension of their practical usefulness and comparative
benefits. The essay emphasises many notable obstacles that are categorised under these
categories.

These include issues related to the accessibility and comprehensiveness of datasets, as

well as methodological obstacles pertaining to transparency, model construction, and
the assessment of outcomes. Although the scope of this study is primarily centered on
the maritime industry, the outcomes and conclusions drawn from this research can be
applied to other areas of transportation as well. The present study underscores the
innovative uses of these techniques in enhancing maritime safety, while also
identifying significant obstacles that necessitate additional investigation to fortify this
methodological framework (Rawson and Brito, 2023).

2.4 Gaps

When the existing literature studies, projects and industry activities are examined,
some gaps have been identified. Important gaps are listed as follows:

28
1. When academic studies and projects are examined, it is seen that artificial
intelligence and automation are used effectively in human resources
management processes in sectors such as health, tourism and automotive.
However, it has been observed that human resources management processes in
the maritime industry are not yet sufficient in this context.
2. There will be a need to improve human resources management processes in the
maritime industry, taking into account the objectives of IMO for the coming
years, the reviews and recommendations of classification societies, the
updating of STCW in the coming years, automation ships and changing needs.
When the studies conducted in this context are examined, it has been seen that
there are deficiencies in determining more detailed, domain-specific and
specific methods.
3. Considering the developments in areas such as digital, green, soft and technical
skills, it has been determined that there may be a manpower deficit in the
maritime industry in the coming years. When the projects and studies carried
out in this direction are examined, it is seen that there are suggestions,
researches and findings related to this field. However, it has been observed that
insufficient attention has been paid to the issues of talent acquisition, which
includes the correct selection, training and recruitment of seafarers by
companies.
4. Considering the developments such as SIRE 2.0, OCIMF, BIMCO and IMO's
expectations, it is seen that one of the biggest changes in the maritime industry
will be in tanker industry and tanker management companies. Considering the
literature studies, it has been seen that there are various findings in areas such
as risk assessment and safety. In addition, considering the transition of tanker
ships in the coming years and the needs of tanker companies, it has been seen
that a modern, artificial intelligence-supported and holistic model is needed for
the successful completion of the stages of talent acquisition such as recruit,
train and onboarding. In this context, the importance of hybrid talent
acquisition models is understood.
5. The aim of this thesis is to propose a hybrid talent acquisiton model that can
respond to the changing maritime industry needs, ensure the successful
completion of the talent acquisition process, help the human resources
management of companies, support the transition of seafarers and play a role

29
in the design of training modules. In this context, three methodological
approaches were used together, namely ECD, Bloom's taxonomy and MCQ
generation model. Then, this model was applied to the Company A and the
results were shown in detail. In this way, it has been shown that the model is
holistic, applicable, and can be supported by automation and artificial
intelligence. The aspects of the thesis that respond to academic studies and
industry needs, as well as limitations and suggestions for future studies are
examined in the Results section.

30
3. METHODOLOGY

In this section, the methods used in the thesis will be discussed. In the study, Bloom's
Taxonomy, Evidence Centered Design Framework and Question Generation
approaches were used. In this section, their details will be explained. In addition, the
proposed hybrid talent acquisition model will be examined.

3.1 Evidence Centered Design Framework

In this section, the background of the proposed method will be discussed. First of all,
all the steps of Evidence Centered Design Framework will be examined.

The process of creating, developing, and utilizing tests is categorized into four distinct
groups of activities referred to as "layers" in ECD. The four stages of the assessment
process are: 1) Domain Analysis, 2) Domain Modeling, 3) Conceptual Assessment
Framework, and 4) Assessment Implementation and Delivery.

Although the discourse surrounding the distinct layers may suggest their separation, it
is important to note that their boundaries are permeable. Constraints are a ubiquitous
presence in the practical realm, imposing limitations on the scope and methodology of
testing. Anticipating and accounting for the effects of constraints in test design is
feasible in certain cases. As an illustration, a customer may stipulate that the
examination must be conducted within a timeframe of one hour or less. The imposition
of temporal limitations will inevitably curtail the scope of assignments that can be
administered, thereby restricting the quantity of empirical data that can be amassed.
Consequently, this will constrain the number of assertions that can be posited. Often,
nevertheless, the consequences of the power source constraints are somewhat of clear
at the very start of the process. In the practical application of ECD, there is a significant
amount of interlayer movement. In operational settings, it is common for multiple
layers of work to be carried out simultaneously. Therefore, arguments regarding which
layer must contain a specifics aspect of test expansion are pointless. The ECD is
characterized by iteration. It is possible that complications arising in a subsequent

31
stage could necessitate revisiting a preceding stage. As an illustration, the recurrence
of unsuccessful endeavors to formulate assignments for procuring the requisite
substantiation to corroborate specific contentions may indicate that the evidence is
unattainable within the limitations prescribed by the assessment scheme (Mislevy and
Yin, 2013; Snow et al., 2010; Clarke-Midura et al., 2021).

3.1.1 Domain analysis

The process of domain analysis and domain modelling involves identifying and
categorizing the primary content or experiential domains that require evaluation, as
well as establishing connections between them. The content refers to the body of
knowledge or subject matter that is relevant to the educational enterprise, as well as
the various ways in which individuals utilize it. The types of scenarios in which it is
employed. What are the defining characteristics of mathematics, troubleshooting, and
teamwork within the current context? What is currently understood regarding the
progressions of cognitive processes or the development of expertise, as well as the
recurring patterns of mistakes and prevalent mental representations? In the present
context, which elements are pertinent, including knowledge, skills, objectives,
resources, facilitators, representations, and potential hindrances? What are the
mechanisms by which individuals engage with the physical environment, conceptual
frameworks, and other agents to achieve specific objectives? According to which
criteria are these endeavors evaluated? In order to address such inquiries, designers
perform a domain analysis. The domain is evaluated from multiple angles, including
cognitive research, existing curricula, professional standards, ethnographic research,
expert opinions, testing practice, test objectives, and the diverse demands, resources,
and limitations that may affect the proposed evaluation. Analyses conducted within
the aforementioned perspectives have demonstrated a growing recognition of the
significance of individuals' engagement with the technological milieu, as previously
highlighted in the introductory section. The emergence of novel modes for the
depiction and transmission of existing information through knowledge representations
(KRs) is an ongoing consequence of technological advancements. In addition, the
advent of technology results in the development of novel and frequently intricate
instruments and settings that individuals must familiarize themselves with, to
effectively generate and modify data. Consequently, this leads to the types of
competencies that require evaluation. The capacity to create and resolve issues in

32
computer networks is a quintessential illustration of a significant field that has emerged
in the recent past.

3.1.2 Domain modeling

Within the context of domain modelling, designers utilize domain analyses to arrange
information in order to depict the connections between various capabilities, observable
actions, products, and circumstances that are utilized to elicit them. These elements
are integral components of assessment arguments. Visual and organizational
depictions, such as graphs, tables, and diagrams, are created to communicate these
connections. In addition, prototypes can serve the purpose of validating assumptions
or refining concepts. Various forms of representation have been employed in the
implementation of ECD. These include the utilization of claims and evidence
worksheets, Toulmin diagrams for the assessment of arguments, and design patterns
for the construction of assessment arguments pertaining to specific capabilities, such
as design under constraints and model-based reasoning (Mislevy et al., 2017). The
process of domain modelling involves an initial exploration of the pertinent real-world
domain, followed by a deliberate selection of specific domain elements that can be
utilized to construct a persuasive argument. The standard structure of the argument for
assessment is formulated as follows: "If (X), then (Y) because (Z)." X refers to either
the manifestation of test taker behavior or the resultant outcome of such behavior. The
data on which a claim is based consists of pertinent aspects of the behavior or product.
The letter Y denotes a statement made by the examinee regarding their possession or
absence of a particular KSA or a group of related KSAs. The letter Z represents the
justification provided by the examinee to support their claim, which explains how the
exhibited behavior or produced outcome serves as evidence of the possession or lack
of the aforementioned KSA (Zieky, 2014; Mislevy and Haertal, 2006).

3.1.3 Conceptual assessment framework

The activities of domain analysis and domain modelling are fundamental components
that contribute to what has conventionally been regarded as an assessment activity, as
explicated in ECD through a set of abstract models referred to as the Conceptual
Assessment Framework (CAF). The CAF serves as a framework wherein domain
information is integrated with specific objectives, limitations, and reasoning to
formulate a plan for conducting an evaluation. The process of designing an assessment

33
can be conceptualized as a sequence of inquiries, including but not limited to: "What
is the object of measurement?", "What is the preferred method of organizing the data
collection process?", and "What are the theoretical connections between empirical
evidence and theoretical deductions?" In contrast to domain modelling, which
considers these inquiries as interconnected components of an evaluation argument, the
CAF formulates responses as specifications for the mechanisms that implement the
assessment. The CAF consists of various models that encompass objects,
specifications, and processes. The framework for the operational components of work
is provided by objects and specifications, which encompass the joint creation of
evaluations, tasks, and mathematical models, the execution and maintenance of the
assessment, and the analysis of data received from the field. The instantiation of these
entities and their synchronization in accordance with the four-step distribution
mechanism (as elaborated in a forthcoming segment) animates the evaluation process.
The domain modelling approach places significant emphasis on the interrelatedness of
various facets of individuals' abilities, behaviors, and circumstances. In contrast, the
CAF framework leverages the distinctiveness of the entities employed to instantiate
an evaluation. This holds significance in light of contemporary technology, as the
representations and their constituents can be digitized. Subsequently, they become
susceptible to aided and mechanized techniques for the production, alteration,
functioning, and integration (Rupp et al., 2010; Riconscente et al., 2016).

3.1.4 Assessment delivery and implementation

Assessment delivery and implementation system that adheres to the Four Process
architecture recommended in the ECD framework as outlined by Almond, Steinberg,
and Mislevy (2002). A standardized assessment in the form of a multiple-choice test
may necessitate only one iteration; conversely, a task based on simulation may demand
numerous interactions among the processes during the execution; and a sophisticated
tutoring system may transition to instructional or practice models. The CAF models
have been utilized to instantiate the assessment contention in domain analysis by
specifying the form of logic, processes, and messages. Almond, Steinberg, and
Mislevy (2002) explicate the relationships among the CAF examples and the execution
system processes. Thus, it can be observed that the typically concealed function of the
machinery components in an evaluation is instrumental in the process of reasoning.
The process of selecting an activity or task is centered on the options that will be made

34
available to the individual being assessed. The mechanism for determining the
subsequent question in an examination may range from a basic directive to display the
following query to a sophisticated mathematical framework that considers the
examinee's cognitive and attitudinal conditions. The subsequent stage in the procedure
is referred to as presentation, encompassing the dissemination of knowledge to the
examinee and the retrieval of knowledge from the examinee. The outcome of the
process of delivering a presentation is a final product that necessitates assessment and
grading based on criteria derived from the assessment of a sub model within the CAF.
The third process in question is commonly referred to as reaction processing or
evidence identification, which is one of the four processes involved.

3.2 Question Generation

Several of the numerous difficulties for natural language comprehending and

interfaces are question generation (QG) and question answering (QA). Because
humans require thoughtful inquiries, the possible advantages offered by automated QG
systems may help them meet useful inquiry needs. They investigate an automatic
phrase-to-Question generation job in this study, where given a phrase, the Question
Generation system creates a series of questions for which the sentence includes,
suggests, or requires responses. To help with the question generating process, they use
a syntactic parser to generate basic phrases from complicated input sentences.

On each of these sentences, an entity with a name classifier and an element speech risk
and operational risk are used to encode the necessary information. They categories
phrases based on their subject, verb, object, and preposition to determine the sort of
queries that can be created. For our experimentation and assessment, they employ the
TREC-2007 (Question Answering Track) dataset (Ali and Hasan, 2010). This study
outlines the methodology for utilizing neural networks to create questions from
provided passages. The research involves the automated retrieval and processing of a
substantial quantity of question-answer pairs from a Community-QA website, which
are subsequently employed as training data. The paper's contribution is twofold:

In another study introduces two distinct approaches for question generation. The first
approach is based on retrieval and utilizes a convolution neural network (CNN), while
the second approach is based on generation and employs a recurrent neural network
(RNN). Additionally, the study demonstrates the potential of utilizing the generated

35
questions to enhance the performance of current question answering systems. The
present study assesses the efficacy of our question generation approach in the context
of the answer sentence selection task, utilizing three established benchmark datasets,
namely SQuAD, MS MARCO, and WikiQA. Empirical findings indicate that the
incorporation of produced questions as an additional cue can lead to a noteworthy
enhancement in the quality of question-answering (Duan et al., 2017).

The utilization of exam-style questions is a crucial pedagogical method that serves

diverse objectives. However, the creation of questions manually is a multifaceted
undertaking that necessitates adequate training, expertise, and resources.
Consequently, the utilization of educational activities such as the provision of practice
questions and innovative approaches such as adaptive testing, which necessitate a
substantial reservoir of questions, is impeded and delayed. Automatic question
generation (AQG) techniques were introduced to address the need for a continuous
supply of new questions and to reduce the expenses associated with manual
construction of questions. Paper serves as an extension of a prior literature review on
Automatic Question Generation, encompassing publications up until the end of 2014.
The corpus comprises 93 scholarly articles published from 2015 to early 2019, which
focus on the automated production of questions for pedagogical objectives. The
objective of this review is to present a comprehensive analysis of the AQG community
and its undertakings, outline the present tendencies and advancements in AQG,
elucidate the modifications that have occurred in this field in recent times, and propose
potential avenues for enhancement and future prospects for AQG. Consistent with
prior research, the existing literature exhibits a limited emphasis on the development
of controlled difficulty levels for question generation, enhancement of question forms
and structures, automation of template construction, refinement of presentation, and
production of feedback. The results of this study (Kurdi et al., 2020) indicate that there
is a requirement for enhancing the reporting of experiments, standardizing the
assessment metrics, and exploring alternative evaluation techniques that are more
practical. Recent studies in the field of Neural Question Generation (NQG) have begun
to incorporate a wider range of inputs and generate questions that demand greater
levels of cognitive processing. The aforementioned trends suggest that NQG serves as
an indicator for the progress of NLP, particularly in relation to the manner in which
human intelligence encompasses the abilities of inquisitiveness and assimilation.

36
The authors (Pan et al., 2019) provide a thorough analysis of neural question
generation, encompassing the datasets, techniques, and assessment criteria. The
authors expound upon the emerging trend observed in NQG with regards to the
learning paradigms, input modalities, and cognitive levels that are being taken into
account. The conclusion of our discourse involves highlighting the possible avenues
for future exploration The authors tackle the task of developing an automated system
for generating questions based on written texts, with the aim of facilitating educational
practice and assessment. The methodology employed involves the generation of a
surplus of inquiries, which are subsequently evaluated and prioritized. Manually
crafted rules are utilized to execute a series of syntactic transformations that are
applicable to a wide range of contexts, such as subject-auxiliary inversion, in order to
convert declarative sentences into interrogative ones.

The aforementioned inquiries are subsequently prioritized through implementation of

a logistic regression model that has been educated on a limited and customized dataset
containing classified results generated by our system. Empirical findings indicate that
the implementation of a ranking system results in a significant increase in the
proportion of questions deemed acceptable by annotators. Specifically, the percentage
of acceptable questions rises from 27% to 52% when considering only the top 20% of
ranked questions (Heilman and Smith, 2010). The REAP system offers users
personalized reading materials that are tailored to their respective reading proficiency
levels. In order to identify suitable texts, it is necessary to evaluate the user's
proficiency in vocabulary. The present study outlines a methodology for the automated
production of inquiries aimed at evaluating one's knowledge of lexicon. Historically,
these evaluations have been completed manually by hand. Six types of vocabulary
questions are generated utilizing data from WordNet. Various formats are available
for these assessments, such as word bank and multiple-choice.

When another study examined, the study provide empirical evidence indicating that
the employment of automatically-generated questions can serve as a reliable indicator
of one's vocabulary proficiency, which is highly associated with their performance on
human-written questions that are developed independently. Furthermore, the robust
associations observed with standardized assessments of lexicon proficiency lend
credence to their methodology for the automated evaluation of lexical knowledge. In
recent times, scholars from various fields have demonstrated a shared inclination

37
towards the development of automated question generation techniques for pedagogical
objectives. This paper (Le et al., 2014) provides an overview of the current state of
the art in the development of educational applications for question generation. The
authors arrive at the conclusion that despite the existence of a wide range of automatic
question generation techniques, only a limited number of educational systems that
utilize such techniques have been created and implemented in actual classroom
environments. The authors additionally suggest potential avenues for research
pertaining to the implementation of question technology within computer-supported
educational systems The objective of automatic question generation is to produce
inquiries based on a textual excerpt, wherein the resulting questions can be resolved
by specific sub-sections of the provided passage. Conventional techniques primarily
rely on inflexible heuristic principles for converting a sentence into associated
interrogative statements. This study aims to utilize the neural encoder-decoder model
for the purpose of generating a range of significant and varied questions based on
natural language sentences.

The encoder processes the input text and answer location to create an input
representation that is sensitive to the answer, and this is then supplied to the decoder
to produce a question that is centered on the answer. A preliminary investigation was
carried out on the generation of neural questions from text utilizing the SQuAD
dataset. The outcomes of the experiment indicate that the approach employed can
generate questions that are both diverse and fluent (Zhou et al., 2018). Visual Question
Answering (VQA) and Visual Question Generation (VQG) are currently popular areas
of research in the field of computer vision. However, despite their inherent
complementary nature, they are often studied independently of each other. The authors
of this scholarly article present a novel approach, namely the Invertible Question
Answering Network (iQAN), which is an end-to-end unified model. The iQAN is
designed to enhance the performance of VQA by introducing question generation as a
dual task of question answering (Chan and Fan, 2019).

The system generates questions that are semantically grounded and purposeful based
on its partial understanding of the sentence meaning. The generation results are
guaranteed to be grammatically correct with the assistance of comprehensive linguistic
grammars. Moreover, a specialized ranking model is employed to further refine the
linguistic realizations derived from the general-purpose generation model for the

38
purpose of our question generation task. The assessment outcomes derived from
QGSTEC2010 exhibit encouraging potential of the suggested methodology (Yao et
al., 2012).

The focus of this study (Chali and Hasan, 2015) pertains to the automated production
of a comprehensive set of inquiries based on a given subject matter. The authors posit
that individual topics are linked to a corpus of literature that contains valuable insights
pertaining to the subject matter. Subsequently, inquiries are produced through the
utilization of the named entity data and the predicate argument frameworks of the
sentences contained within the corpus of written material. The significance of the
produced inquiries is evaluated through the utilization of Latent Dirichlet Allocation,
which involves the identification of subtopics that are closely associated with the
primary topic within the provided text corpus.

3.2.1 Pre processing

The researchers have executed multiple pre-processing procedures to facilitate the

automated production of multiple-choice questions from a given text. Below, it is
provided a summary of the techniques.

Text normalization: Text normalization involves converting the input text to the
desired format and eliminating extraneous content from the text. The necessity for pre-
processing is highly dependent on the specific domain and applications being
considered. Consequently, the methodology to be utilized in this context is contingent
upon the requirement of a specific undertaking. Diverse forms of text normalization
and sentence normalization have been employed in studies (Bednarik and Kovacs,
2012; Heilman, 2011; Heilman and Smith, 2010).

Structural analysis: The initial phase of structural analysis involves the identification
of pertinent tags such as chapters, sections, paragraphs, and other relevant elements
within the text. Certain authors utilize structural information to aid in the generation
of multiple-choice questions. Furthermore, given that the multiple choice questions
pertain solely to textual material, it is imperative to eliminate any non-textual content
(Coniam, 1997; Mitkov et al., 2006; Chen et al., 2006; Sumita et al., 2005; Hoshino
and Nakagawa, 2007; Belkin and Croft, 1992; Pino et al., 2008; Liu et al., 2005;
Hoshino and Nakagawa, 2005; Curto et al., 2010).

39
Sentence simplification and reduction: Each multiple-choice question should consist
of a single sentence that presents a single disputable piece of information. A lengthy
sentence may potentially include specific indicators that aid in the determination of the
accurate solution. The process of simplifying sentences involves transforming intricate
and combined sentences into basic ones. The aforementioned task is predominantly
accomplished by means of utilizing the external system or via parse structure analysis.
Sentence simplification has been implemented in diverse systems (Aldabe et al., 2007;
Afzal and Mitkov, 2014; Majumder and Saha, 2015; Pino et al., 2008; Das et al., 2018;
Heilman and Eskenazi, 2007; Das and Majumder, 2017).

Lexical analysis: The process of lexical analysis entails the segmentation of the textual
content of a document into a sequence of alphanumeric characters, symbols, and
numerical values. Furthermore, inflections may have an impact on different specific
components of the task. Consequently, various systems conduct stemming, which
involves identifying the fundamental form of a word (Bednarik and Kovacs, 2012;
Heilman, 2011; Pabitha et al., 2014).

Statistical analysis: Various modules employed word-level analysis techniques such

as frequency of words, the n-gram frequency, frequency of terms * inverse document
frequency, and relationship data analysis in multiple systems. Nevertheless, a
considerable number of these articles failed to explicitly acknowledge the inclusion of
said step as a pre-processing measure (Wu et al., 2008; Awad and Dahab, 2014;
Agarwal and Mannem, 2011; Singh Bhatia et al., 2013; Liu et al., 2005).

Syntactic analysis: The utilization of diverse levels of syntactic analysis has been
observed in numerous systems. These levels include parts-of-speech tagging, which
involves assigning parts-of-speech to individual words, named entity recognition,
which involves identifying names of predetermined categories from a given text, and
syntactic parsing, which involves generating a parse structure of the sentence (Chen et
al., 2006; Karamanis and Mitkov, 2006; Aldabe et al., 2006; Majumder and Saha,
2014).

The pre-processing methodology necessary for the development of a multiple-choice

question system is contingent upon the characteristics of the input text and the
requirements of the ensuing modules. In the event that the input text is obtained via
web crawling, it is possible that the document may contain extraneous content and

40
superfluous noise. Subsequent to data acquisition, the process of text cleaning assumes
a crucial role. It has been observed that systems utilizing Wikipedia documents as input
frequently require a step for simplifying sentences in order to manage the lengthy and
intricate sentence structures. Several authors conducted statistical analysis and utilized
fundamental natural language processing techniques, although they did not explicitly
specify these as a pre-processing measure. Nevertheless, it is deemed appropriate to
include these factors as pre-processing steps in order to prevent any potential confusion
for the readers.

The educational material incorporates various mathematical elements such as

expressions, equations, visual aids, tabular data, and itemized lists. The extant
literature on multiple-choice questions has largely neglected the aforementioned
information and instead prioritized analysis of textual components. In the pre-
processing stage, it is common for such systems to perform text extraction by
eliminating non-textual components. A Multiple Choice Question generation system
must possess the ability to effectively manage the information that is incorporated
within the text. Subsequent studies pertaining to the generation of multiple-choice
questions thought to priorities the development of MCQs derived from a blended
educational text (Ch and Saha, 2018).

3.2.2 Sentence selection

The process of choosing the most appropriate sentences for a piece of writing is an
important aspect of effective communication. It is not possible for every sentence in a
text to produce a legitimate question. Only sentences that contain potentially
inaccurate information are suitable for generating multiple-choice questions. Hence,
the process of sentence selection assumes a significant role in the task of generating
multiple-choice questions automatically. Various methodologies have been employed
in scholarly works to choose significant and doubtful sentences from a given text. The
following approaches are summarized below.

Summarization: Kurtasov (2010) utilized a methodology centered on summarization

to facilitate the process of selecting sentences. Shah et al. (1990) employed an
extractive summarization technique called MEAD (McDonald, 1997) to select
sentences for their summaries.

41
Machine learning: Multiple machine learning algorithms have been employed for the
task. Several classification algorithms, including Naive Bayes, Support Vector
Machine, Ranking Voted Perceptron, Neural Network, and Counter Propagation
Network-based classification, have been developed and utilized in various applications
(Goto et al., 2009; Goto et al., 2010; Correia et al., 2012; Bednarik and Kovacs, 2012;
Hoshino and Nakagawa, 2005).

Semantic information: The use of semantic information contained within the text has
been employed as a criterion for the selection of sentences. The process of selecting
sentences involves the utilization of feature extraction techniques such as semantic
role labelling and named entity information. In addition, diverse forms of semantic
processing are employed, such as coreference resolution, paraphrase detection, and
identification of a correlation between concepts (Fattoh, 2014; Gildea and Jurafsky,
2002; Lee et al., 2011; Lamba and Govilkar, 2017).

Parts-of-speech information: Aldabe, Maritxalar, and Mitkov (2009) conducted an

analysis of verb and verb form patterns, as well as sentence length, in order to inform
their sentence selection process. Meanwhile, Lin, Sung, and Chen (2007) focused their
analysis on the occurrence of adjective-noun pairs.

Occurrence of a particular word: Several studies have employed an occurrence-based

filtering method, which examines the presence of a specific word or n-gram within a
sentence (Brown et al., 2005; Correia et al., 2012; Liu et al., 2005; Lin et al., 2007).

Sentence length: This refers to the quantity of words present within a given sentence.
A sentence of limited length may not possess an adequate amount of information to
facilitate the generation of questions. A lengthy sentence may present challenges in
the task due to the presence of multiple facts and relationships. The sentence length is
employed sentence choosing requirements in several articles (Correia et al., 2012;
Agarwal and Mannem, 2011; Zavala and Mendoza, 2018).

3.2.3 Key selection

It is evident that not all words within an informative sentence can function as the
primary keyword. Hence, the process of key selection is a crucial stage that determines
the specific word or phrase. In the specified sentence that is going to be omitted. The
present discourse pertains to the methodologies employed in the literature.

42
Frequency count: The utilization of frequency count of the term "has" has been
employed as a selection criterion in various multiple-choice question systems, as
evidenced by references (Coniam, 1997; Mitkov et al., 2006; Agarwal and Mannem;
2011; Shei, 2001; Mitkov, 2003). Merely relying on the frequency count alone is
insufficient. Therefore, supplementary information is utilized in conjunction with the
frequency count to arrive at the ultimate selection.

Machine learning: Several studies have employed machine learning techniques for the
purpose of key selection. Hoshino and Nakagawa (2005) employed machine learning
methodologies to produce keywords in the form of verbs, partial idioms, or adverbs.
The authors Goto et al. (2009, 2010) employed a conditional random field classifier in
their research to estimate the key. This information can be found in references.

Pattern matching: Pattern matching is an additional methodology utilized for the

purpose of selecting keywords. Chen, Liou, and Chang (2015) conducted an analysis
in which they extracted sentences that exhibited comparable structural features. Their
focus was on identifying common patterns, with a primary emphasis on the verbs used
within the sentences. The identified patterns facilitated the recognition of the essential
terms. In their study, Gates et al. (2010) utilized syntactic patterns that were derived
from the parse structure of sentences in order to identify key elements.

Parts-of-speech information: It has been observed that certain parts-of-speech or parse

categories exhibit dominance as potential keywords within specific domains or
applications. In a study conducted by Sumita, Sugaya, and Yamamoto (2005), verbs
were identified as the primary keywords. Lin, Sung, and Chen (2007) determined that
sense association among adjectives was the correct answer, while Lee and Seneff
(2007) selected a preposition as the key in their research. Aldabe et al. (2011), Agarwal
and Mannem (2017), Mitkov, Ha, and Karamanis (2003), Chen, Liou, and Chang
(2009), and Gates et al. (2015) have employed part-of-speech or parse information in
their key selection methodologies.

As exemplified in the aforementioned sentence, the appropriateness of a given method

for the purpose of key selection is contingent upon both the specific domain and
application in question. In the initial stages, the systems predominantly depended on
fundamental statistical and syntactic data. Nevertheless, contemporary practice
involves the utilization of semantic data or artificial intelligence techniques.

43
The process of formulating a question from a sentence heavily relies on the
information contained within the sentence and the semantic connections between the
entities involved. It is believed that approaches utilizing semantic level analysis are
widely applicable across various domains due to their generic nature.

3.2.4 Question formation

Following the process of key selection, the subsequent step involves the conversion of
a declarative sentence into an interrogative form. Nevertheless, it has been observed
in the academic literature that this particular step has been neglected in numerous
multiple-choice question systems. In the event that the transformation is not executed,
the sentence will persist in its initial structure, and a blank space will substitute the
key.

Consequently, the question transforms into a type that requires completion with
alternative options. Several works were discovered that contain the transformation
from a declarative sentence to an interrogative sentence. The prevailing approach in
this context is centered around methods that rely on rules or patterns. The methodology
for establishing rules differs across various systems. The following information is
presented in a condensed form.

Semantic based transformation: Yao and Zhang (2010) presented a methodology

grounded in semantics, which involves converting declarative sentences into
interrogative structures.

Semantic based information: Lindberg and colleagues (2013) devised a framework

that employs a template-based approach and utilizes semantic role labelling to detect
patterns for the purpose of generating questions. Mazidi and Nielsen (2014) employed
a framework based on semantic role labelling for the purpose of generating questions.

Syntactic transformation: Heilman (2011) identified answer phrases and conducted

syntactic transformations to generate questions. The primary procedures undertaken
involved identifying unalterable phrases, producing potential interrogative phrases
through a methodology based on rules, breaking down the principal verb, inverting the
subject-auxiliary, eliminating responses, and incorporating question phrases.

Dependency based patterns: The utilization of dependency-based patterns for the

purpose of question formation was employed by Afzal and Mitkov (2014). The

44
primary verb and the interrogative segment were determined from the sentence's
dependency tree. The extraneous segments were eliminated and a suitable wh-word
was chosen to convert the statement into an interrogative form.

Appropriate wh-word selection: Numerous studies have endeavored to ascertain the

suitable wh-word and construct the corresponding inquiry. As an illustration, the term
'what' pertains to an object, 'who' pertains to individuals, 'where' pertains to a specific
place, 'when' pertains to a particular time, and so forth. The authors Majumder and
Saha (2015) conducted an analysis of the parse structure of the sentence in question in
order to determine the crucial elements and their placement within the sentence.
Subsequently, the appropriate wh-word was determined and utilized to construct the
inquiry in accordance with established guidelines.

Knowledge in sentence: Pabitha and colleagues (2014) employed a methodology based

on rules. The regulations were established according to the particular type of
knowledge that was conveyed within the sentence. A variety of knowledge labels were
utilized. This text outlines various elements that are commonly found in academic
writing, including concepts, definitions, examples, calculations, procedures, and
results.

The generation of questions from a given text is a research area that has received
significant attention in recent years. Numerous research efforts have been dedicated to
this field. The aim of said domain is to produce an inquiry based on a given textual
input. In multiple-choice questions, an initial informative sentence is selected,
followed by the identification of the key, and ultimately, the conversion of the sentence
into a question format centered around the key. The process of forming a question
involves transforming an assertive sentence into an interrogative sentence, with the
key serving as the focal point. When prioritizing multiple-choice questions that have
predetermined sentences and keys, utilizing rule-based or pattern-based techniques can
be a straightforward and effective approach.

3.2.5 Distractor selection

The inclusion of distractors holds significant value in the process of generating

multiple-choice questions. The efficacy of a multiple-choice question is significantly
influenced by the calibre of the distractors. If the distractors fail to sufficiently confuse
the examinee, then they are more likely to select the correct answer with ease.

45
Consequently, the overall quality and usability of the multiple-choice questions
diminish. Various methodologies have been employed for the purpose of generating
distractors in scholarly works. The following information is presented in a condensed
form.

Semantic analysis: The utilization of semantic analysis is a prevalent method in

identifying distractors, which are concepts that possess semantic proximity to the keys.
Aldabe and Maritxalar (2010) employed Latent Semantic Analysis as a means of
producing distractors. The process of generating distractors was carried out by Pino,
Heilman, and Eskenazi (2008) through the computation of semantic similarity between
two words, utilising the Patwardhan and Pedersen's (2006) methodology. Aldabe,
Maritxalar, and Mitkov (2013) conducted a study in which they calculated the
semantic similarity among verbs based on distributional data. The selection of
distractors was carried out by Kumar, Banchs, and D'Haro (2015) through the
utilization of semantic similarity computation.

Pattern matching: The utilization of pattern matching is an alternative method.

Patterns are established by utilizing diverse forms of data, such as parse information
and the key. Subsequently, the aforementioned patterns are employed to extract the
distractor. Hoshino and Nakagawa (2019), Aldabe, Maritxalar, and Martinez (2018),
Chen, Liou, and Chang (2015), and Goto et al. (2016, 2017) employed a pattern-based
approach in the creation of distractors.

Domain ontology: Papasalouros, Kanaris, and Kotis (2008) utilized the Web Ontology
Language (2009) in their pursuit of identifying distractors.

Frequency: The frequency of words can serve as an additional indicator. It is important

to ensure that both the distractors and the key exhibit comparable frequency counts.
Frequency is a parameter utilized in various systems that employ a combination of
parts-of-speech, frequency, and other types of information (Lee and Seneff, 2007;
Belkin and Croft, 1992; Agarwal and Mannem, 2011; Hoshino and Nakagawa, 2007).

Parts-of-speech information: The distractor bears semantic proximity to the key.

Consequently, it is recommended that the key and distractors be classified under the
same parts-of-speech category. This observation has been utilized as a valuable
indicator in academic research (Goto et al., 2009; Goto et al., 2010; Brown et al., 2005;
Correia et al., 2012).

46
Based on the analysis of relevant literature, it can be inferred that the selection of an
appropriate method for generating distractors is contingent upon the specific
application and nature of the distractors in question. Distractors that are based on basic
syntactic or statistical information, such as frequency and parts-of-speech data, have
demonstrated effectiveness in various applications, including but not limited to second
language acquisition, vocabulary acquisition, and grammar assessment. Multiple
choice questions from various fields such as biomedical, history, and school-level
subjects often necessitate a profound level of statistical or semantic proximity. The
inclusion of numerical distractors is deemed indispensable in multiple choice
questions pertaining to the fields of science and mathematics. In certain fields such as
sports, entertainment, and tourism, named entities hold prominence as either the
primary focus or a potential source of distraction.

In order to produce appealing distractors, it is necessary to calculate the similarity

between the names. The majority of current systems produce uncomplicated
distractors. However, various types of distractors may be present in actual multiple-
choice questions. As an illustration, the named entity distractor, numerical values, and
distractor comprising multiple words are noteworthy factors to consider. The
production of intricate distractors poses a significant challenge, as the methods
employed for generating basic distractors may prove inadequate in this context. An
effective multiple-choice question generation system should be capable of managing
all forms of distractors.

Nonetheless, the existing body of literature lacks sufficient methodologies for

producing a diverse range of distractors. One potential avenue for generating complex
distractors could involve employing deep semantic analysis or utilizing neural
embedding techniques.

3.2.6 Post processing

The post-processing stage is the ultimate phase that endeavors to enhance the calibre
of the multiple-choice questions generated by the system. There exist diverse
categories of errors that may be present in multiple-choice questions generated by a
system. The aforementioned issues encompass punctuation errors, inappropriate usage
of question words, excessively long stems, presence of discourse words in the

47
question, discrepancies in number agreement, and inadequate quality of distractors. It
is imperative for the system to reduce these errors in the ultimate output.

The post-processing phase aims to correct any errors that may have occurred, failing
which, it eliminates any inaccurate queries. The literature has predominantly employed
three distinct forms of post-processing. The three techniques employed are post-
editing of malformed questions, prioritization of questions, and removal of
inappropriate questions.

Question ranking: Ranking is an alternative method. The utilization of a heuristic-

based ranking module was implemented by Mannem, Prasad, and Joshi (2010). The
study identified certain heuristics, such as the distance of the predicate used to generate
the question and the frequency of pronoun usage. Heilman and Smith (2010) utilized
a ranking system to evaluate the questions generated, which was implemented as a
post-processing measure. A subset of the annotated dataset was partitioned and utilized
for the purpose of constructing the discriminative inquiry ranking algorithm. The
acceptability of questions was assessed based on the predictions generated by a logistic
regression model. The questions were subsequently ranked according to this
evaluation. The system developed by Effenberger (2015) has integrated a module for
grading exercises. The evaluator undertook an assessment to allocate grades based on
the level of complexity, pertinence to the article, and syntactic accuracy.

Question filtering: A filter was developed by Aldabe et al., (2006), Aldabe, Maritxalar,
and Martinez (2007) with the purpose of eliminating improperly constructed questions.
The evaluator primarily assessed the calibre of the distractors, such as those that were
identical but varied in wording yet conveyed the same meaning, or those that were
inflections of one another. In cases where the necessary quantity of effective
distractors was not generated for a given question, the question was deemed
unsatisfactory and subsequently excluded. The implementation of distractor filtering
was also employed by Gates et al. (2011) in their system. The filtering mechanism
utilized by the individual was predicated on factors such as length, phrase
classification, and semantic proximity. Sumita, Sugaya, and Yamamoto (2005) have
proposed a filter methodology that relies on item information to decrease the size of a
test through the selection of efficient items. The researchers utilized various metrics
such as the proficiency level of the test taker, discrimination, and difficulty to measure
item information. The items that were chosen are those that have the highest potential

48
to contribute to the test's information. Heilman (2011) incorporated a filter into the
post-processing phase. The filtering process excluded inquiries that contained noun
phrases comprised exclusively of determiners (such as "those"), noun phrases that
contained unresolved pronouns, and those that exceeded a length of 30 tokens.

Question post-editing: A series of manual post-editing procedures were conducted by

Mitkov, Ha, and Karamanis (2006). Initially, a classification was conducted to
ascertain the extent of post-editing that was deemed necessary, which was categorized
as minor, fair, or major. The minor revision entails a restricted amount of post-editing
of the item, which primarily involves rectifying spelling and punctuation errors. The
process of fair revision involves the reorganization of words, potential insertion or
deletion of certain words, and substitution of a single distractor at maximum. The
process of major revision entails significant modification of the question prompt and
substitution of a minimum of two answer options. Following the classification process,
a post-editing environment was established. Subsequently, a group of human
specialists were enlisted to revise and refine the inquiries. Heilman (2011)
implemented several post-editing procedures, including the conversion of terminal
periods to question marks and the elimination of superfluous whitespace characters.

3.3 Bloom’s Taxonomy

A comprehensive exploration of the internet will reveal conclusive proof that Bloom's
Taxonomy has been implemented in diverse contexts. The present findings encompass
a wide range of applications, as evidenced by various articles and websites that cover
topics ranging from corrosion education to medical readiness.

Bloom's Taxonomy can be a useful tool for instructors seeking to guide a group of
students through a structured learning process in most situations. However, it is
noteworthy that the educational context, spanning from kindergarten to graduate level,
continues to be the most frequently employed utilization (Forehand, 2005).

The taxonomy of educational objectives serves as a systematic structure for

categorizing declarations that outline the anticipated or intended knowledge
acquisition of students consequent to pedagogical guidance. The framework was
initially developed with the aim of streamlining the process of sharing test items

49
between faculty members across different academic institutions. The ultimate goal was
to establish a repository of items that would assess identical educational objectives.

The concept was first introduced by Benjamin S. Bloom, who held the position of
Associate Director of the Board of Examinations at the University of Chicago. The
intention behind this initiative was to streamline the process of creating yearly
comprehensive exams and alleviate the associated workload. In order to facilitate his
endeavor, he recruited a cohort of metrology experts hailing from various regions of
the United States, a significant proportion of whom encountered the identical
predicament on multiple occasions. Bloom perceived the initial Taxonomy as having
a broader scope than merely serving as an assessment instrument. The individual held
the belief that it could function as a universal mode of communication concerning
educational objectives, thereby facilitating effective communication across
individuals, subject areas, and grade levels. Additionally, it could serve as a foundation
for determining the specific interpretation of broad educational objectives, such as
those currently present in national, state, and local standards, for a given course or
curriculum.

It could also be utilized as a tool for assessing the consistency between educational
objectives, activities, and assessments within a unit, course, or curriculum. Lastly, it
could provide a comprehensive overview of the full range of educational possibilities,
against which the limited scope of any particular educational course or curriculum
could be compared. The aforementioned group convened biannually from 1949
onwards to evaluate advancements, effect modifications, and strategize future actions.
The initial Taxonomy presented meticulously crafted explanations for each of the six
primary classifications within the cognitive realm. The categories were Knowledge,
Comprehension, Application, Analysis, Synthesis, and Evaluation. All of the
categories, except for Application, were subdivided into smaller categories. The
categorization was arranged in ascending order of complexity and in a progression
from tangible to intangible. Moreover, it was postulated that the initial Taxonomy
embodied a progressive hierarchy, wherein proficiency in each elementary
classification was a precondition for proficiency in the succeeding more intricate
classification. Upon its introduction, the term "taxonomy" was not commonly
recognized as a pedagogical term.

50
The original Taxonomy received little attention initially due to a lack of
comprehension among potential users. As the framework's potential was recognized
by readers, it gained widespread recognition and was frequently referenced. This led
to its translation into 22 different languages. The original Taxonomy has commonly
been utilized to categories educational objectives and examination items with the
intention of demonstrating the comprehensiveness, or insufficiency thereof, of said
objectives and items across the range of classifications. The analyses have consistently
demonstrated a predominant focus on objectives that pertain to the recognition or recall
of information, which are categorized as Knowledge objectives.

Objectives pertaining to the comprehension and application of knowledge, falling

under the categories ranging from comprehension to synthesis, are typically regarded
as the foremost educational goals. Consequently, these analyses have frequently
served as a foundation for transitioning curricula and assessments towards goals that
would be categorized within the more intricate classifications (Krathwohl, 2002).

Typically, instructional objectives that delineate the desired learning outcomes are
expressed in relation to a specific subject matter content. The second component of the
agreement involves a delineation of the actions or processes that will be applied to the
aforementioned content. Statements of objectives generally comprise a subject matter
content in the form of a noun or noun phrase, and one or more cognitive processes
expressed through a verb or verb phrase. One possible objective to consider is as
follows:

The student is expected to retain knowledge of the fundamental economic principle of

supply and demand. The phrase "The student shall be able to" (or a similar variation
such as "The learner will") is a ubiquitous component of all educational objectives.
This is because an objective serves to specify the intended learning outcomes for
students. In academic contexts, objectives are frequently formulated without the
inclusion of the phrase "The student shall be able to." Instead, the objectives are stated
in a concise manner, highlighting the distinct aspect of the task (e.g., "Recall the
principles of supply and demand in economics."). The given form elucidates that the
noun phrase being referred to is the "law of supply and demand" and the verb in
question is "remember."

51
The initial Taxonomy incorporated both nominal and verbal aspects within the
Knowledge classification. The extensive subcategories of Knowledge specified the
noun or subject matter aspect. The definition of Knowledge encompassed the aspect
of verb usage, whereby the student was required to demonstrate the ability to recall or
recognize knowledge. The framework was rendered unidimensional by this process,
albeit at the expense of a Knowledge category that exhibited duality and therefore
deviated from the remaining Taxonomic categories. The revised Taxonomy addressed
the aforementioned anomaly by separating the noun and verb into distinct dimensions.
The Knowledge dimension is based on the noun, while the Cognitive Process
dimension is based on the verb.

Similar to the initial version, the revised Taxonomy's knowledge domains transcend
disciplinary boundaries. The recently introduced Knowledge dimension comprises
four primary categories, as opposed to the previous three. Three of them encompass
the essence of the subcategories of Knowledge as outlined in the initial framework.
The framework underwent reorganization to incorporate the terminology and
distinctions of cognitive psychology that have emerged since its initial development.
The original scheme did not widely recognize the fourth and novel category of
Metacognitive Knowledge, which now serves to provide a distinction. Metacognitive
knowledge pertains to cognition in a broad sense, and encompasses an individual's
awareness and understanding of their own cognitive processes (Pintrich, this
publication). As scholars persist in showcasing the relevance of students' awareness of
their metacognitive processes, it becomes increasingly crucial for them to utilize this
knowledge to effectively modify their cognitive strategies and behaviors (Krathwohl,
2002).

The initial quantity of categories, which was six, was preserved, albeit with significant
modifications. Three categories underwent a renaming process, while the order of two
was altered. Furthermore, the retained category names were modified to conform to
their corresponding verb form, which aligns with their usage in objectives. The initial
aspect of the Knowledge classification pertaining to verbs was retained as the foremost
among the six principal categories, albeit with a new nomenclature of Remember. The
renaming of comprehension occurred due to the utilization of category labels that align
with the language commonly employed by educators when discussing their
professional responsibilities. The omission of the term "understand" in the objectives

52
was a common critique of the initial Taxonomy, given its frequent usage. The initial
group contemplated utilizing the concept, however, subsequent analysis revealed that
educators' desire for students to "truly" comprehend material encompasses a range of
cognitive processes from basic comprehension to higher-order synthesis. However,
according to the revising authors, there appeared to be a prevalent usage where the
term "understand" was commonly used as a synonym for "comprehend." The second
category originally known as Comprehension underwent a name change and is now
referred to as Understand. This modification was made evident by the fourth edition
(Krathwohl, 2002).

The verbs Apply, Analyze, and Evaluate were utilized in lieu of the corresponding
nouns Application, Analysis, and Evaluation. The process of Synthesis underwent a
reordering with Evaluation and was subsequently rebranded as Create. The initial
subcategories were substituted with gerunds and denoted as "cognitive processes." In
contrast to the original Taxonomy, which placed greater emphasis on the six primary
categories than on the subcategories, the revised Taxonomy prioritizes the 19 distinct
cognitive processes that fall within the six cognitive process categories. The six
primary categories of revision can be best understood through the explanations
provided for each of the distinct cognitive processes involved. The breadth and depth
of each category are characterized by these processes in conjunction. Similar to the
original Taxonomy, the revised version is structured as a hierarchy whereby the six
primary categories within the Cognitive Process dimension are perceived to vary in
their level of complexity.

Specifically, the category of remember is considered less intricate than understand,

which in turn is less intricate than apply, and so forth. The revised version of the text
places greater emphasis on teacher usage, resulting in a relaxation of the strict
hierarchy requirement. As a consequence, the categories are now allowed to overlap
with one another. The category of "Understand" is most evidently exemplified by this
case. Due to the significant expansion of its scope in comparison to the original
framework, certain cognitive processes that are linked to the concept of
comprehension, such as explaining, exhibit a higher degree of cognitive complexity
than at least one of the cognitive processes that are associated with the concept of
application, such as executing. In the event that an individual were to identify the
central point of each of the six principal classifications on a complexity assessment

53
scale, it is probable that a gradient ranging from uncomplicated to intricate would
emerge. The Cognitive Process dimension can be considered a hierarchical structure,
which is likely to receive substantial support, similar to the original Taxonomy.

The Taxonomy of Educational Objectives is a framework utilized to categories

educational aspirations, aims, and, more recently, benchmarks. The utilization of an
organizational framework facilitates a shared comprehension of objectives that are
categorized within its structure, ultimately improving communication. The initial
Taxonomy comprised six distinct categories, each of which had numerous
subcategories. The skills and abilities were organized in a hierarchical framework
where each subsequent level required the mastery of the preceding one. The primary
focus of the initial Taxonomy publication was on evaluating learning outcomes, and it
included numerous illustrations of examination questions (primarily in the form of
multiple-choice) for each classification. The revised Taxonomy we have developed is
a framework that comprises two dimensions, namely Knowledge and Cognitive
Processes. The former category bears a greater resemblance to the subcategories that
are present within the primary Knowledge category.

The former category bears a resemblance to the six classifications of the initial
Taxonomy, wherein the Knowledge category is denoted as Remember, the
Comprehension category is referred to as Understand, and Synthesis has been given a
new name. The initial step involved the establishment of the highest tier category,
while the subsequent categories were modified to their corresponding verb forms,
namely: Application, Analysis, and Evaluation. The items are organized in a
hierarchical manner, albeit with less rigidity compared to the original Taxonomy. The
Taxonomy Table is a valuable tool that is formed by the integration of the Knowledge
and Cognitive Process dimensions.

The utilization of a Table for the categorization of objectives, activities, and

assessments offers a lucid, succinct, and graphical depiction of a specific course or
unit. Upon completion, the entries within the Taxonomy Table may be utilized to
analyses the degree of emphasis, alignment with the curriculum, and educational
opportunities that were overlooked. Drawing from this analysis, educators can make
informed decisions regarding the areas and methods for enhancing curriculum design
and instructional delivery (Krathwohl, 2002).

54
In Remembering, this aspect of the taxonomy implies the act of accessing information.
This constitutes a crucial component in light of the expansion of knowledge and
information. The present document outlines the digital augmentations and their
corresponding elucidations. Bullet pointing is a digital format that is analogous to
listing. Emphasizing important information through highlighting is a fundamental
component of numerous productivity software packages. This approach of identifying
and highlighting significant words and phrases is a memory retention technique that is
often recommended to students. The act of bookmarking or favoriting involves the
selection and saving of web pages, resources, and files by students for future reference.
The user is engaging in the process of knowledge acquisition by providing comments
on the pages. This practice is comparable to jotting down annotations on printed
materials, but holds greater potential as it allows for linking and indexing capabilities.
The act of subscribing involves advancing bookmarking and simplified reading to a
higher level. The mere act of subscribing to a feed does not necessarily indicate or
foster comprehension. However, frequently engaging in the process of reading and
revisiting subscribed feeds can result in heightened levels of understanding (Churches,
2008).

Evaluating, the present text enumerates and describes the digital augmentations. The
practice of providing constructive criticism and engaging in reflective thinking is
frequently supported through the utilization of blogs and video blogs. When engaging
in online discussions, students are expected to critically assess the content presented
and respond accordingly within the appropriate context. The act of posting refers to
the act of leaving comments on blogs, discussion boards, or threaded discussions.
These components are becoming more prevalent in the daily routines of students.
Effective contributions, akin to thoughtful remarks, are not mere concise responses but
rather are methodically crafted to assess the subject or idea at hand. Moderation
involves a rigorous assessment process, wherein the moderator is expected to evaluate
a posted content or comment from diverse angles, gauging its merit, significance, and
suitability.

Creating, the present document outlines the digital augmentations and their
corresponding elucidations. Programming is a common practice among students,
where they engage in developing their own applications, programming macros, and
creating games or multimedia applications within structured environments to meet

55
their specific needs and objectives. The proliferation of multimedia and multimedia
editing tools has led to the rise of various forms of media production such as filming,
animating, videocasting, podcasting, mixing, and remixing. It is common for students
to engage in the capture, creation, mixing, and remixing of content in order to generate
distinct and original products. The process of directing or producing a product,
performance, or production is a highly creative endeavor.

The Taxonomy, as viewed from the DTL perspective, presents several conceptual
issues. It embodies various misconceptions that are deeply ingrained and widely
disseminated regarding the functioning of the human mind, the process of student
learning, and consequently, the pedagogical approach that educators should adopt.
This shift in perspective can significantly alter the conversation surrounding
educational objectives, learning, and pedagogy (Arievitch, 2020).

The present analysis (Sobral, 2021). compiles and examines research conducted in the
area of instructing fundamental programming skills through the lens of Bloom's
taxonomy. Impact on creative thinking abilities and scientific literacy, the variable that
is most strongly influenced is creative thinking skill, as evidenced by a noteworthy rise
in the mean value. Learning activities that are grounded in Bloom's Taxonomy serve
as instructional guidelines for fostering creative thinking abilities and scientific
literacy (Pujawan et al., 2022). The utilization of Bloom's cognitive learning taxonomy
is prevalent in the field of medical education. The levels of action forms are described
in the revised taxonomy published by David Kathwohl (1956) and his group as
follows: Remember, Understand, Apply, Analyze, Evaluate, and Innovate.

Taxonomy has frequently been employed to develop and organize educational aims
and outcomes. Nonetheless, the majority of applications take place at the level of
course or program curriculum, as initially expounded. The implementation of the
taxonomy at the individual level of educational activities was restricted due to various
factors. The objective of this article is to encourage the utilization of taxonomy in
routine clinical instruction by presenting practical clinical illustrations in an
uncomplicated manner (Tuma and Nassar, 2021).

The paper emphasizes the significance of employing methodical methods in courses

using blended learning in order to enhance their creativity and appeal to the
contemporary cohort of distance learning customers. Additional research is required

56
to validate the findings of this study due to the limited number of participants (Pikhart
and Klimova, 2019). Nonetheless, the outcomes of our research prompt inquiries
regarding the substantiation that supports a prevalent methodology for instructing and
evaluating in higher education (HE) in the United Kingdom, as well as education more
broadly (Newton et al., 2020). The queries were classified into six distinct categories
of Bloom's taxonomy, namely knowledge, comprehension, application, analysis,
synthesis, and evaluation. The results indicate that the questioning techniques
employed by teachers primarily focus on convergent, knowledge-based, and
comprehension-based approaches. Educators seldom pose inquiries at the levels of
analysis, synthesis, and evaluation. It is suggested that there is a need for an increased
frequency of higher-order thought-provoking inquiries in educational settings. It is
recognized that encouraging students to employ advanced cognitive abilities is crucial.
However, this objective can only be attained if educators are proficient in utilizing a
categorization system like Bloom's taxonomy (Bibi et al., 2020).

The knowledge components comprised a lecture delivered by an expert on

multidisciplinary design for engineering and the variances between disciplines. The
cognitive process entailed the creation of design tasks and their arrangement within a
comprehensive design methodology. The process of altering an example development
challenge entailed utilizing Bloom's Taxonomy action verbs to generate instructive
measures that direct students' cognitive processes during the problem-solving process.
The findings indicate a significant correlation between design for engineering activity
as well as intellectual mental activity among students as well as instructors. The new
approach was piloted by course instructors who provided positive feedback, stating
that it was a valuable tool for introducing trans disciplinarity and promoting students'
reflection on their approach to pondering. Furthermore, the utilization of Bloom's
Taxonomy in design-related predicaments proposed prospective enhancements in the
academic achievements of pupils in design assignments. The findings of this research
hold relevance in the context of the advancement of engineering design curricula and
interdisciplinary initiatives (Sharunova et al., 2022). The primary professional
objective of many pharmacy academicians is career advancement, specifically through
promotion. However, the field of pharmacy education pertaining to this objective is
often perceived as unclear at best and daunting at worst. The Revised Bloom's
Taxonomy, which is commonly employed by educators to facilitate the attainment of

57
a desired level of knowledge and skill demonstration among learners, can likewise be
utilized to steer faculty members towards a successful promotion trajectory. The RBT
is categorized into two primary dimensions.

The dimension of cognitive process delineates a set of verbs that denote distinct
actions, such as remembering, comprehending, applying, analyzing, evaluating, and
creating. These actions exhibit a progressive increase in complexity and cognitive
demand. The dimension of knowledge enables individuals to advance from tangible to
increasingly theoretical reasoning within a framework characterized by factual,
conceptual, procedural, and metacognitive knowledge. When formulating objectives
for an activity or curriculum, the cognitive process and knowledge dimensions are
utilized in conjunction, and the full range of verbs within the cognitive process
dimension can be applied to any domain within the knowledge dimension (Waite et
al., 2020).

3.4 Hybrid Talent Acqusition Model

Talent acquisition is a recruitment strategy that prioritises the identification, attraction,

selection, development, and retention of high-performing individuals within an
organisation. Although there may exist a sufficient reservoir of potential employees to
select from, the task of staffing vacant positions with exceptional personnel is a
challenging undertaking. Unless an organisation belongs to the top-tiered category
within its respective industry, it can be exceedingly challenging to vie for the most
exceptional candidates in the job market. In order to acquire the most qualified
personnel, it is imperative to devise a strategic plan for recruitment, whether through
internal or external means. This does not necessarily imply that providing higher
compensation to potential employees is the sole means of attracting them to join your
company. In this context, hybrid models are being studied to develop talent acquisition
strategies. In this section, the hybrid talent acquisition process, which forms the basis
of the study, will be explained. A hybrid talent acquisition is commonly composed of
nine distinct stages.

An analysis of organisational needs. The process of conducting an organisational needs

analysis serves as the fundamental basis for formulating an effective recruitment
strategy and making informed decisions regarding personnel selection. In this process,
the mission, vision, goals, and values of the organisation are transformed into a set of

58
fundamental competencies and values. These aid in identifying the desired profile and
shaping the selection criteria to assess the compatibility of the individual with the
organisation and position. Conducting a needs analysis, commonly known as a skill-
gap analysis, may have already been completed and integrated into established human
resources protocols.

Authorization of the employment request. This marks the inception of the recruitment
procedure. The formal procedure for requesting a hire in an organisation is through a
job requisition, which is initiated when there is a need for additional personnel. The
job requisition frequently manifests as a written record that elucidates: The rationale
for the necessity of the position is unclear, whether the position is novel or pre-existing.
The nomenclature of the department, the hiring manager, and the department name are
the key components of the subject matter, job responsibilities and compensation and
financial planning etc.

Recruitment of new personnel. The vacancy intake is widely considered as the crucial
step in the hybrid talent acquisition. During the process of hiring, a comprehensive job
analysis is carried out to gather all pertinent data necessary for making a sound
recruitment decision.

Determining selection criteria & methods. Based on the vacancy intake, the selection
criteria and method are determined. There are many possible selection methods,
including a general mental abilit test, work sample, structured or unstructured
interviews, and so on.

The topic of interest is the concepts of searching and attraction. Subsequently, the
process of searching for a suitable candidate ensues. The aforementioned phase is
widely regarded as the most conspicuous and acknowledged aspect of the talent
acquisition procedure. Broadly speaking, individuals seeking a particular position can
be categorised into two groups: those who are actively pursuing the opportunity and
those who are not. Passive candidates exhibit a lower level of job-seeking activity and
are less likely to frequently monitor your organization's careers page, whereas active
candidates demonstrate a higher level of job-seeking activity. In order to generate
active interest in one's job, it is imperative to employ targeted triggers that cater to the
diverse groups of potential candidates. Hence, various methods may be utilised to

59
attract these distinct cohorts. The recruitment channels that exhibit the highest efficacy
are contingent upon the candidate persona that was previously alluded to.

Implement selection techniques. Upon submission of the application by the candidate,

selection methods are implemented. The recruitment process encompasses various
stages, such as pre-selection, conventional interviews, as well as background and credit
verifications, among others.

The process of making a decision regarding the selection of a candidate for a job
position. Upon selection of a candidate, an employment offer is extended. This phase
may entail iterative discussions and bargaining prior to the acceptance of the proposal.
The offer acceptance rate, a widely recognised recruitment metric, is the primary
measure of success for this phase. It is calculated by dividing the number of accepted
offers by the total number of offers extended.

Onboarding. The subsequent stage in the process of acquiring talent involves the
concrete onboarding process. A highly effective resource for job applicants is the
implementation of a strategic 30-60-90 day plan. The definition of this concept is
contingent upon the quantity of available positions, and its utility can be observed in
two distinct manners. Initially, it is advantageous for the manager to articulate their
expectations with clarity. Secondly, it provides the candidate with a clear
understanding of the expectations placed upon them, thereby organising their
onboarding procedure. Frequently, there exists a lack of clarity regarding the
expectations of both the supervisor and the candidate.

Evaluation. The assessment phase constitutes the ultimate step in the talent
procurement procedure. The candidate is queried regarding the congruence between
their expectations and the job and organisation at hand. This presents a valuable
opportunity to reconnect with the candidate. It is imperative to address instances of
incongruous expectations, not only with the candidate in question but also in a
systematic manner if it occurs frequently.

The advantages of hybrid talent acquisition models can be listed as follows. The
following are some of the key objectives that organisations strive to achieve in order
to enhance their overall performance, enhanced employee performance, Creating a
positive work environment, enhancing the recruitment process to hire suitable
candidates for all roles, decreasing employee turnover, enhancing diversity and

60
inclusion in local and global offices, improving performance evaluation systems and
measurements, ensuring the contentment of high-achieving employees, establishing a
sustainable pool of talent for future leadership positions, and effectively incorporating
women.

The hybrid talent acquisition model consists of 5 phases in total as can be seen in
Figure 3.1. In this context, domain analysis should be evaluated first. In the domain
analysis phase, interviews are carried out with companies and organization; targets and
deficiencies are determined. In the light of the information obtained, the knowledge
resource to be evaluated is determined. The second phase of the model is domain
modeling. At this stage, specific topics are determined from the selected knowledge
resource. Thus, it is ensured that companies and organizations gain knowledge
resources. Then the third phase, the conceptual assessment framework, is completed.
At this phase, determine taxonomy, choose question type and employ question
generation model are carried out. In order to do all this, the visions, missions,
departments, opportunities and levels of the companies are determined. Then, the
fourth phase, assessment delivery and implementation, is applied. In order to
successfully submit the assessments created in this phase, the apply competency
demonstration technique stage is performed. At the end of all this, recommendations
are carried out according to the information, reviews and analysis obtained.

61
Figure 3.1: A hybrid talent acquisition model illustration.

62
4. CASE STUDY AND RESULTS

In this section, hybrid talent acquisition model will be applied. For this, assumptions
will be conducted for a tanker company. The company has a fleet of 12 tanker vessels,
which were constructed between the years 2002 and 2016. There are four vessels
classified as Oil Chemical Tankers and eight classified as Oil Tankers. The
Deadweight Tonnages (DWTs) of the vessels exhibit a range of 38,000 to 180,000.
Typically, shipping companies opt for the West Asia route, specifically the Strait of
Hormuz. Within this particular context, vessels transport oil to Eastern and Asian
nations by traversing through the Arabian Sea, Indian Ocean, East African coast, Cape
of Good Hope, Atlantic Ocean, and Western European territories.

During the meeting with the company, a range of documents were reviewed. These
included the crew information, crew matrix, fleet details, investment plan for the
upcoming decade, revisions, as well as the company's visions, missions, and shipowner
expectations. Furthermore, an examination was conducted on the PSC and FSC
inspections of the vessels within the past five years. The reports have identified
inadequacies in emergency management, human factors, and safety management
systems on company vessels. Finally, in order to authenticate the analyses and
determinations, validation was conducted through the utilisation of reports,
inspections, and expectations from classification societies (see Appendix A for hybrid
talent acquisition implementation protocol).

4.1 Phase 1: Domain Analysis

In order to conduct a comprehensive analysis of the company, the reports, documents,

interviews, and opinions gathered throughout this procedure will be examined. In this
respect, the company's fundamental requirements will be determined, followed by the
selection of suitable knowledge resources. Examine human factor accidents and

63
reports analyse best management practises, safety management system, ISPS Code
protocol, the transition procedure.

As a result of the interview with the company, it has been determined that there are
several inconsistencies with the examined reports, particularly regarding best
management practises, human factor accidents and reports, and the safety management
system. It has been observed that the company has been unable to provide adequate
training on these matters to its personnel. Particularly within the scope of safety
management system and ISPS Code protocol, human factor incidents and reports. The
personnel recruited by the company is onboarding the ships without receiving adequate
training, causing numerous issues for the company. In addition, it was revealed as a
result of the investigations that the company would purchase LNG and LPG vessels
following the sale of the some Oil Tankers. It was determined at this time that the
personnel with knowledge of green skills, digitalization, and carbon capture did not
work for the company. Therefore, it is anticipated that deficiencies will exist in the
company's transition process in the forthcoming period. The personnel should now be
briefed on the rules and regulations of the company's transition procedure.

It was observed that they have tanker ships in their fleet and that they will add LNG-
LGP vessels in the near future. Then, it was determined that the company lacks
adequate training programmes on topics like safety management system and best
management practises. In addition, the recurrence of accidents caused by human
factors, the absence of route regulations, and information about the forthcoming
transition process were taken into account. In light of these factors, ISGOTT was
designated as the company's knowledge resource (see Figure 4.1 and 4.2).

64
 Figure 4.1: Illustration of ship type selection.

Figure 4.2: Illustration of knowledge resource utilization.

4.2 Phase 2: Domain Modeling

Detailed examinations are required in this phase so that the chosen knowledge resource
can be incorporated into the organisation. For the successful completion of the Domain
Modeling phase, the following checks must be performed, followed by an ISGOTT
evaluation of chapter and section selections. Examine emergency management;
analyse PSC and FSC inspections from the previous five years; investigate and report
accidents and near-misses; examine superintendent recommendations and strategies.

In the company's PSC and FSC inspections over the past five years, a number of
findings were identified as a consequence of the examinations. For instance, following
the crossing of the Arabian Sea in one of the Oil Chemical Tanker ships with 143,500
DWT, the port state controller observed deficiencies in the safety management system
documents. PSC has determined that the officers of watch on the same ship did not
prepare the hot work permits in a timely manner. On another vessel, it was observed
that the tanker emergency plan could not be prepared on time. In addition, it has been
reported that after joining the ships, the newly recruited personnel caused accidents
and near misses. Finally, the findings were compared to the recommendations and
strategies of the superintendent.

ISGOTT Part 2, Chapter 9 (Safety Management Systems, Work planning and permit
to work systems, Hot work, Electric welding equipment, Other hazardous tasks,
Emergency management, Managing simultaneous operations, Hazards on ships with

65
exposed transverse frames, and Repairs at a facility other than a shipyard) was selected
as the part, chapter, or section of the knowledge resource based on the observations
and alternatives (see Figure 4.3, 4.4 and 4.5).

Figure 4.3: Visualization of decision selected knowledge resource part.

Figure 4.4: Illustration of decision selected knowledge resource chapter.

66
 Figure 4.5: Illustration of decision selected knowledge resource theme.

4.3 Phase 3: Conceptual Assessment Framework

The results of the domain analysis and domain modelling segments should be
evaluated in this phase. In this phase, it should also be investigated the company's
facilities, personnel details, and information in order to prepare the assessment. This
requires the completion of the three elements listed below: Determine
taxonomy, choose question type and employ question generation model.

As a result of the company interviews, it was determined that a training programme

was required for ISGOTT Part 2, Chapter 9 topics such as safety management system,
work planning and permit to work systems, hot work, and emergency management. In
this context, it was discussed that the personnel would have access to the computer
while on vacation and on board. Prior to preparing the assessments in this context,
taxonomy determination was performed so that each crew could comprehend the
ISGOTT chapters and sections from beginning to last chapter. Then, the choosing of
question type for the assessments was initiated. Finally, a question generation model
was employed so that the human resources management can create assessments easily,
swiftly, and at a lower cost.

67
Bloom's taxonomy from taxonomy, multiple choice questions from question types, and
the MCQ generation model from question generation models were selected after
considering the brief, observations, and options (see Figure 4.6, 4.7 and 4.8).

Figure 4.6: Visualization conceptual assessment framework taxonomy

determination.

Figure 4.7: Visualization conceptual assessment framework choosing question type.

68
 Figure 4.8: Illustration of conceptual assessment framework employing question
generation model.

4.4 Phase 4: Assesment Delivery and Implementation

To ensure the success of the hybrid talent acquisition model, the fourth phase
assessments must be presented to the personnel in an appropriate manner. Interviews
with the company's HSEQ, training, and human resources departments are necessary
at this phase. In accordance with the gathered data, it is necessary to apply a
competency demonstration technique.

As a result of the company interviews, it was determined that the crew's burden was
quite heavy. It is recognised that traditional assessment delivery methods cannot be
selected in this context. In accordance with the information acquired from the HSEQ,
training, and human resources divisions, it was also determined that company
personnel required a rapid, efficient, and automated model. Consequently, the
appropriate techniques for demonstrating competency were implemented.

In accordance with the obtained information, it has been determined that there is no
approved simulator within the organisation. In addition, given the crew's burden and
the ship's duties, it was deemed inappropriate to use competency demonstration
techniques such as reports, direct observation, classroom assessment, and individual
and group assignments. In this instance, computer-based assessment (see Figure 4.9)
was selected because it is accessible from anywhere and is more conducive to
automation and artificial intelligence.

69
 Figure 4.9: Illustration of applying competency demonstration techniques.

4.5 Phase 5: Recommendations

As a consequence of the interviews and examinations, ten multiple-choice questions

in computer-based assessment format were presented to the company (see Appendix
B).

70
5. CONCLUSION

Human resources management has a very important place in the maritime industry, as
in other industries. Especially with the effect of industry 4.0, the importance of talent
acquisition and talent management is increasing in subjects such as artificial
intelligence and machine learning. Revision status of STCW, automation ships,
industry demands and changing maritime needs are added to all these, the subject of
transition for seafarers comes to the fore. For maritime companies and organizations,
this issue is provided by human resources management. In this context, not traditional
assessments for carrying out studies; competency-based assessments should be carried
out. For competency-based assessment and evaluation, knowledge, understanding and
proficiency subjects must be completed.

In this context, the first step is knowledge. For example, engineers who will work on
LNG-LPG ships need knowledge before they pass on understanding and proficiency
in order to perform their duties. It is necessary to understand whether the seafarers to
be recruited or trained within the company are suitable for the job. In this context, it is
necessary to measure the basic knowledge. Considering that it is an international
industry, it is seen that the number of candidates is quite high. Therefore, a quick and
automatic knowledge acquisition part will benefit both companies and seafarers.

It is thought that there will be a gap in between for the transition from traditional
competencies to subjects such as digital, green and soft skills. Therefore, while
industry needs will increase, seafarers will need to be ready for it. In this context, it is
necessary to transfer knowledge to seafarers at a basic level. The pages of knowledge
resources contain quite a lot of information. Therefore, it seems that it will take time
and very costly for most seafarers to read and understand this and become proficient.
In this context, it is seen that there is a need for a fast, effective and easily accessible
assessment and evaluation tool in order to provide seafarers with basic knowledge. In
addition, in order to use, expand and evaluate knowledge resources effectively, this
information should be easily transferred to seafarers. In this context, there is a need for
an assessment and evaluation tool that includes knowledge level.

71
Considering these, it is thought that the importance of assessment and evaluation tools
will increase even more in the coming years.

The aim of this thesis is to contribute to the maritime industry and transition for
seafarers with developing and proposing the hybrid talent acquisition model. In this
thesis, a hybrid model has been developed that can assist the stages of talent acquisition
such as interview, recruit and onboarding. Bloom's taxonomy, MCQ generation model
and ECD framework were used in the development of the model. Thus, the developed
hybrid talent acquisition model has been ensured to be holistic. The application of the
developed model was completed by making assumptions for a tanker company.
Findings in this context were examined in the Results section. In addition, the hybrid
talent acquisition survey & implementation protocol and the ten multiple choice
questions produced are included in the Appendices section.

Since the proposed model is a holistic approach, it can also be used to gain knowledge
for seafarers who will be assigned to different types of ships such as bulk carrier,
passenger and LNG-LPG. In this context, the proposed model is thought to lay the
groundwork for new generation talents. In order to acquire new generation talent in
subjects such as shore based, autonomous ships, hybrid fuels and data flow process,
the basic level of knowledge of these subjects will be transferred to seafarers. In this
way, companies will be able to get to know the seafarers they will recruit more easily,
and thus, they will have the opportunity to invest in the right seafarer.

Seafarers will also be able to develop themselves in their profession as they can access
CBAs from anywhere. In this context, it is thought that the issue of transition for
seafarers can be better resolved.

In this way, human resources managers in tanker companies will be able to handle the
seafarer pool, and they will be able to invest in simulators or onboard training for the
development of successful candidates in understanding and proficiency.

It is thought that this thesis will contribute to the literature studies on bloom's
taxonomy, question generation and evidence centered design. When the existing
literature is examined, it is seen that there are many studies and projects on these issues
in various industries. However, in line with the maritime-specific studies, it has been
determined that the studies on talent acquisition are not sufficient for the changing
industry needs and transition process. It is thought that the model developed in this

72
context will help academic studies for many different ship types and maritime
organizations. It is seen that academic studies can be carried out more
comprehensively with models on subjects such as decision support models, machine
learning and artificial intelligence.

While developing the model, some difficulties were encountered on various issues.
For example, the keywords and definitions of Bloom's taxonomy were used when text
reduction was made for ISGOTT Chapters. This process took a lot of time as it was
conducted manually, not automatically. Likewise, since ISGOTT sections consist of
many different topics, it was not possible to ask questions from each section.
Therefore, a certain section was selected and questions were produced in this context.

In addition, it has been determined that some of the topics in the ISGOTT sections are
not suitable for the levels in Bloom's taxonomy. Therefore, in some subjects, questions
can be asked from the understand level, while in some subjects evaluate or apply levels
are preferred.

Considering all these, it is seen that topics such as artificial intelligence, machine
learning, text mining and deep learning can be supported with hybrid talent acquisition
model for future studies. For example, using text mining, the text reduction process
can be conducted more effectively and accurately. In addition, the generated questions
can be multiplied by machine learning. In this way, not only Chapter 9, but also all
chapters of ISGOTT can be effectively adapted to CBA. Competency demonstration
techniques such as simulators and onboard training can be used in future studies. Thus,
especially the apply and evaluate levels of Bloom's taxonomy; can be presented to
seafarers more effectively.

73
74
REFERENCES

Afzal, N., & Mitkov, R. (2014). Automatic generation of multiple choice questions
using dependency-based semantic relations. Soft Computing, 18, 1269-
1281.
Agarwal, M. (2012). Cloze and open cloze question generation systems and their
evaluation guidelines. International Institute of Information
Technology, Hyderabad.
Agarwal, M., & Mannem, P. (2011). Automatic gap-fill question generation from
text books. In Proceedings of the sixth workshop on innovative use of
NLP for building educational applications (pp. 56-64).
Aldabe, I., & Maritxalar, M. (2010). Automatic distractor generation for domain
specific texts. In Advances in Natural Language Processing: 7th
International Conference on NLP, IceTAL 2010, Reykjavik, Iceland,
August 16-18, 2010 7 (pp. 27-38). Springer Berlin Heidelberg.
Aldabe, I., De Lacalle, M. L., Maritxalar, M., Martinez, E., & Uria, L. (2006).
Arikiturri: an automatic question generator based on corpora and nlp
techniques. In Intelligent Tutoring Systems: 8th International
Conference, ITS 2006, Jhongli, Taiwan, June 26-30, 2006. Proceedings
8 (pp. 584-594). Springer Berlin Heidelberg.
Aldabe, I., Maritxalar, M., & Martinez, E. (2007). Evaluating and improving the
distractor-generating heuristics. In Workshop on NLP for Educational
Resources. In conjuction with RANLP07 (pp. 7-13).
Aldabe, I., Maritxalar, M., & Mitkov, R. (2009). A study on the automatic selection
of candidate sentences distractors. In Artificial Intelligence in
Education (pp. 656-658). IOS press.
Ali, H., Chali, Y., & Hasan, S. A. (2010). Automatic question generation from
sentences. In Actes de la 17e conférence sur le Traitement Automatique
des Langues Naturelles. Articles courts (pp. 213-218).
Allen, D., & Tanner, K. (2006). Rubrics: Tools for making learning goals and
evaluation criteria explicit for both teachers and learners. CBE—Life
Sciences Education, 5(3), 197-203.
Almond, R., Steinberg, L., & Mislevy, R. (2002). Enhancing the design and delivery
of assessment systems: A four-process architecture. The Journal of
Technology, Learning and Assessment, 1(5).
Alop, A. (2019). The challenges of the digital technology era for maritime education
and training. In 2019 European Navigation Conference (ENC) (pp. 1-
5). IEEE.
Amiti, F. (2020). Synchronous and asynchronous E-learning. European Journal of
Open Education and E-Learning Studies, 5(2).

75
Anastasiou, J. (2017). Shipping operations management. In I. D. Visvikis & P. M.
Panayides (Eds.), WMU Studies in Maritime Affairs 4. Springer
International Publishing AG. doi: 10.1007/978-3-319-62365-8_4
Andrade, H. G. (2000). Using rubrics to promote thinking and learning. Educational
leadership, 57(5), 13-19.
Ang, Z. X. (2023). Strategies for Singapore maritime industry in an uncertain global
environment-Part C: Talent.
Antoniou, G., & Harmelen, F. V. (2009). Web ontology language: Owl. Handbook
on ontologies, 91-110.
Arievitch, I. M. (2020). Reprint of: The vision of Developmental Teaching and
Learning and Bloom's Taxonomy of educational objectives. Learning,
culture and social interaction, 27, 100473.
Arroyo, I., Cooper, D. G., Burleson, W., & Woolf, B. P. (2010). Bayesian networks
and linear regression models of students’ goals, moods, and emotions.
Handbook of educational data mining, 323-338.
Awad, A. E., & Dahab, M. Y. (2014). Automatic generation of question bank based
on pre-defined templates. International Journal of Innovations &
Advancement in Computer Science IJIACS ISSN, 2347-8616.
Bai, C., Dallasega, P., Orzes, G., & Sarkis, J. (2020). Industry 4.0 technologies
assessment: A sustainability perspective. International journal of
production economics, 229, 107776.
Barari, N., RezaeiZadeh, M., Khorasani, A., & Alami, F. (2022). Designing and
validating educational standards for E-teaching in virtual learning
environments (VLEs), based on revised Bloom’s taxonomy. Interactive
Learning Environments, 30(9), 1640-1652.
Barsan, E., Surugiu, F., & Dragomir, C. (2016). Factors of human resources
competitiveness in maritime transport. In Human Resources and Crew
Resource Management (pp. 35-38). CRC Press.
Batrinca, G. H. I. O. R. G. H. E., & Raicu, G. A. B. R. I. E. L. (2010).
Considerations about effectiveness and limits of computer based
training in maritime industry. In Proceedings of the 3rd International
Conference on Maritime and Naval Science and Engineering, Romania
(pp. 15-20).
Bechard, S., Clark, A., Swinburne Romine, R., Karvonen, M., Kingston, N., &
Erickson, K. (2019). Use of evidence-centered design to develop
learning maps-based assessments. International Journal of Testing,
19(2), 188-205.
Bednarik, L., & Kovacs, L. (2012). Implementation and assessment of the automatic
question generation module. In 2012 IEEE 3rd international conference
on cognitive infocommunications (CogInfoCom) (pp. 687-690). IEEE.
Behrens, J. T., Frezzo, D., Mislevy, R., Kroopnick, M., & Wise, D. (2017).
Structural, functional, and semiotic symmetries in simulation-based
games and assessments. In Assessment of problem solving using
simulations (pp. 59-80). Routledge.

76
Bejar, I. (2010). Can speech technology improve assessment and learning? New
capabilities may facilitate assessment innovations. RD Connections
Report No. RDC-15.
Belkin, N. J., & Croft, W. B. (1992). Information filtering and information retrieval:
Two sides of the same coin?. Communications of the ACM, 35(12), 29-
38.
Benitez, G. B., Ayala, N. F., & Frank, A. G. (2020). Industry 4.0 innovation
ecosystems: An evolutionary perspective on value cocreation.
International Journal of Production Economics, 228, 107735.
Berg, N., Storgård, J., & Lappalainen, J. (2013). The impact of ship crews on
maritime safety. Publications of the Centre for Maritime Studies,
University of Turku A, 64, 1-48.
Bernadin, S., & Udhan, T. (2014). Developing a Task Model for Evidence-Centered
Assessment Design in a Speech Signal Processing Course.
Bertram, V. (2020). Technology trends for ships and shipping of tomorrow. Maritime
Technology and Research, 2(1), 1-18.
Bibi, W., Butt, M. N., & Reba, A. (2020). Relating teachers' questioning techniques
with students' learning within the context of Bloom's taxonomy. FWU
Journal of Social Sciences, 14(1), 111-119.
Brookhart, S. M. (2018). Appropriate criteria: key to effective rubrics. In Frontiers in
Education (Vol. 3, p. 22). Frontiers Media SA.
Brown, J., Frishkoff, G., & Eskenazi, M. (2005). Automatic question generation for
vocabulary assessment. In Proceedings of Human Language
Technology Conference and Conference on Empirical Methods in
Natural Language Processing (pp. 819-826).
Bruder, C., & Hasse, C. (2020). What the eyes reveal: Investigating the detection of
automation failures. Applied ergonomics, 82, 102967.
Cárdenas, J. F. S., Shin, J. G., & Kim, S. H. (2020). A few critical human factors
for developing sustainable autonomous driving technology.
Sustainability, 12(7), 3030.
Ch, D. R., & Saha, S. K. (2018). Automatic multiple choice question generation from
text: A survey. IEEE Transactions on Learning Technologies, 13(1),
14-25.
Chali, Y., & Hasan, S. A. (2015). Towards topic-to-question generation.
Computational Linguistics, 41(1), 1-20.
Chan, Y. H., & Fan, Y. C. (2019). A recurrent BERT-based model for question
generation. In Proceedings of the 2nd workshop on machine reading for
question answering (pp. 154-162).
Chan, Z., & Ho, S. (2019). Good and bad practices in rubrics: the perspectives of
students and educators. Assessment & Evaluation in Higher Education,
44(4), 533-545.
Chawla, C. P. (2015). Can company strategies enhance competence? World Maritime
Day IMO. Retrieved from
http://www.imo.org/en/About/Events/WorldMaritimeDay/Documents/

77
 2015%20WMD/Captain%20Pradeep%20Chawla,%20Managing%20
Director,%20QHSE%20and%20Training,%20Anglo%20Eastern%20
Group.pdf (Data Retrieved 01.02.2023).
Chen, C. Y., Liou, H. C., & Chang, J. S. (2006). Fast–an automatic generation
system for grammar tests. In Proceedings of the COLING/ACL 2006
Interactive Presentation Sessions (pp. 1-4).
Cheng, M. W., & Chan, C. K. (2019). An experimental test: Using rubrics for
reflective writing to develop reflection. Studies in Educational
Evaluation, 61, 176-182.
Chowdhury, F. (2019). Application of rubrics in the classroom: A vital tool for
improvement in assessment, feedback and learning. International
education studies, 12(1), 61-68.
Churches, A. (2008). Bloom's taxonomy blooms digitally. Tech & Learning, 1, 1-6.
Coniam, D. (1997). A preliminary inquiry into using corpus word frequency data in
the automatic generation of English language cloze tests. Calico
Journal, 15-33.
Correia, R., Baptista, J., Eskenazi, M., & Mamede, N. (2012). Automatic
generation of cloze question stems. In Computational Processing of the
Portuguese Language: 10th International Conference, PROPOR 2012,
Coimbra, Portugal, April 17-20, 2012. Proceedings 10 (pp. 168-178).
Springer Berlin Heidelberg.
Coughlin, P. A., & Featherstone, C. R. (2017). How to write a high quality multiple
choice question (MCQ): a guide for clinicians. European Journal of
Vascular and Endovascular Surgery, 54(5), 654-658.
Clarke-Midura, J., Silvis, D., Shumway, J. F., Lee, V. R., & Kozlowski, J. S.
(2021). Developing a kindergarten computational thinking assessment
using evidence-centered design: the case of algorithmic thinking.
Computer Science Education, 31(2), 117-140.
Crespo, R. M., Najjar, J., Derntl, M., Leony, D., Neumann, S., Oberhuemer, P.,
... & Kloos, C. D. (2010). Aligning assessment with learning outcomes
in outcome-based education. In IEEE EDUCON 2010 Conference (pp.
1239-1246). IEEE.
Crew Connect Global. (2017). The future of crewing shipping’s challenges and
opportunities an industry review paper. Crew Connect Global 2017,
KNect365 Maritime.
Crompton, H., Burke, D., & Lin, Y. C. (2019). Mobile learning and student
cognition: A systematic review of PK‐12 research using Bloom’s
Taxonomy. British Journal of Educational Technology, 50(2), 684-701.
Curto, S. D. S. L. (2010). Automatic generation of multiple-choice tests. Unpublished
master’s thesis). Universida de Técnica de Lisboa, Portugal.
Cwilewicz, R., Tomczak, L., & Pudłowski, Z. (2003). The development and
application of computer-based training programs in maritime
engineering education. Global J. of Engng. Educ, 7(2), 209-218.

78
Damuri, A., Isnain, N., Rahmatika, R., Priyatama, A., Chandra, Y. I., & Putra,
A. S. (2021). E-Learning Proposal System in Public Secondary School
Learning. International Journal of Educational Research and Social
Sciences (IJERSC), 2(2), 270-275.
Das, B., & Majumder, M. (2017). Factual open cloze question generation for
assessment of learner’s knowledge. International Journal of
Educational Technology in Higher Education, 14, 1-12.
Das, B., Majumder, M., & Phadikar, S. (2018). A novel system for generating
simple sentences from complex and compound sentences. International
Journal of Modern Education and Computer Science, 12(1), 57.
De Visser, E. J., Pak, R., & Shaw, T. H. (2018). From ‘automation’to ‘autonomy’:
the importance of trust repair in human–machine interaction.
Ergonomics, 61(10), 1409-1427.
Deling, W., Dongkui, W., Changhai, H., & Changyue, W. (2020). Marine
Autonomous Surface Ship–a Great Challenge to Maritime Education
and Training. American Journal of Water Science and Engineering,
6(1), 10-16.
Dewan, M. H., & Godina, R. (2023). Effective Training of Seafarers on Energy
Efficient Operations of Ships in the Maritime Industry. Procedia
Computer Science, 217, 1688-1698.
DiCerbo, K. E. (2007). Knowledge structures of entering computer networking
students and their instructors. Journal of Information Technology
Education: Research, 6(1), 263-277.
DiCerbo, K. E. (2009). Communicating with instructors about complex data analysis.
In annual meeting of the American Educational Research Association,
San Diego, CA.
DiCerbo, K. E., & Behrens, J. T. (2012). Implications of the digital ocean on current
and future assessment. Computers and their impact on state assessment:
Recent history and predictions for the future, 1(1), 273-306.
Dickie, J. W. (2014). Reeds 21st century ship management. London: Adlard Coles
Nautical. ISBN 978-1-4729-0068-5.
Digitalization of Shipping, DNV-GL. (2018). Retrieved from
https://to2025.dnvgl.com/shipping/digitalization/ (Data retrieved
01.02.2023).
Duan, N., Tang, D., Chen, P., & Zhou, M. (2017). Question generation for question
answering. In Proceedings of the 2017 conference on empirical
methods in natural language processing (pp. 866-874).
Edirisinghe, L., Siriwardena, S., & Ranwala, L. (2022). Significance of Blooms
Taxonomy in Tartary Education A Case Study in the Maritime Industry.
Sustainable Development Research, 4(1), p12-p12.
Edler, J., & Infante, V. (2019). Maritime and other key transport issues for the future–
education and training in the context of lifelong learning. Transactions
on Maritime Science, 8(01), 84-98.

79
Effenberger, T. (2015). Automatic question generation and adaptive practice
(Doctoral dissertation, Masarykova univerzita, Fakulta informatiky).
Elias, R., Issa, R. R., & Wu, W. (2022). Progress on Building Information Modeling
Education and Talent Acquisition. International Journal of Construction
Education and Research, 1-20.
Emad, G. R. (2010). Introduction of technology into workplace and the need for
change in pedagogy. Procedia-Social and Behavioral Sciences, 2(2),
875-879.
Endsley, M. R. (2016). Designing for situation awareness: An approach to user-
centered design. CRC press.
Fattoh, I. E. (2014). Automatic multiple choice question generation system for
semantic attributes using string similarity measures. Computer
Engineering and Intelligent Systems, 5(8), 66-73.
Fayoumi, A., & Loucopoulos, P. (2022). Bridging the Strategy Execution Gap of
Designing Intelligent Talent Acquisition Systems Using Enterprise
Modelling and Simulation. Enterprise Information Systems, 1-36.
Forehand, M. (2005). Bloom's taxonomy: Original and revised. Emerging
perspectives on learning, teaching, and technology, 8, 41-44.
Frezzo, D. C., Behrens, J. T., Mislevy, R. J., West, P., & DiCerbo, K. E. (2009).
Psychometric and evidentiary approaches to simulation assessment in
Packet Tracer software. In 2009 Fifth International Conference on
Networking and Services (pp. 555-560). IEEE.
Gates, D. (2011). G. Aist, J. Mostow, M. Mckeown, and J. Bey. How to Generate
Cloze Questions from Definitions: a Syntactic Approach. In
Proceedings of the AAAI Symposium on Question Generation.
Ge, Y., Ge, Y., Liu, X., Li, D., Shan, Y., Qie, X., & Luo, P. (2022). Bridging video-
text retrieval with multiple choice questions. In Proceedings of the
IEEE/CVF Conference on Computer Vision and Pattern Recognition
(pp. 16167-16176).
Gibson, P. (2017). Talent management and the cruise industry. In Cruise ship tourism
(pp. 161-176). Wallingford UK: CABI.
Gildea, D., & Jurafsky, D. (2002). Automatic labeling of semantic roles.
Computational linguistics, 28(3), 245-288.
Goto, T., Kojiri, T., Watanabe, T., Iwata, T., & Yamada, T. (2009). An automatic
generation of multiple-choice cloze questions based on statistical
learning. In Proceedings of the 17th International Conference on
Computers in Education (pp. 415-422). Hong Kong: Asia-Pacific
Society for Computers in Education.
Goto, T., Kojiri, T., Watanabe, T., Iwata, T., & Yamada, T. (2010). Automatic
generation system of multiple-choice cloze questions and its evaluation.
Knowledge Management & E-Learning, 2(3), 210.
Gualtieri, L., Palomba, I., Wehrle, E. J., & Vidoni, R. (2020). The opportunities
and challenges of SME manufacturing automation: safety and

80
 ergonomics in human–robot collaboration. Industry 4.0 for SMEs:
Challenges, opportunities and requirements, 105-144.
Hanzu-Pazara, R., & Arsenie, P. (2010). New challenges in the maritime academics.
In Latest trends on engineering education proceedings of 7th WSEAS
International Conference on Education and Educational Technologies
(pp. 299-304).
Haertel, G., Wentland, E., Yarnall, L., & Mislevy, R. J. (2012). Evidence-centered
design in assessment development. In Handbook on Measurement,
Assessment, and Evaluation in Higher Education (pp. 277-296).
Routledge.
Heilman, M. (2011). Automatic factual question generation from text (Doctoral
dissertation, Carnegie Mellon University).
Heilman, M., & Eskenazi, M. (2007). Application of automatic thesaurus extraction
for computer generation of vocabulary questions. In Workshop on
Speech and Language Technology in Education.
Heilman, M., & Smith, N. A. (2010). Good question! statistical ranking for question
generation. In Human Language Technologies: The 2010 Annual
Conference of the North American Chapter of the Association for
Computational Linguistics (pp. 609-617).
Helal, H. M. (2002). Computer based training: a global survey of current
developments and its application to maritime education and training.
Henri, M., Johnson, M. D., & Nepal, B. (2017). A review of competency‐based
learning: Tools, assessments, and recommendations. Journal of
engineering education, 106(4), 607-638.
Hoshino, A., & Nakagawa, H. (2005). A real-time multiple-choice question
generation for language testing: a preliminary study. In Proceedings of
the second workshop on Building Educational Applications Using NLP
(pp. 17-20).
Hoshino, A., & Nakagawa, H. (2007). Assisting cloze test making with a web
application. In Society for Information Technology & Teacher
Education International Conference (pp. 2807-2814). Association for
the Advancement of Computing in Education (AACE).
Huff, K., Steinberg, L., & Matts, T. (2010). The promises and challenges of
implementing evidence-centered design in large-scale assessment.
Applied Measurement in Education, 23(4), 310-324.
Hussein, A., Gaber, M. M., Elyan, E., & Jayne, C. (2017). Imitation learning: A
survey of learning methods. ACM Computing Surveys (CSUR), 50(2),
1-35.
International Safety Guide for Oil Tankers and Terminals (ISGOTT 6), OCIMF.
(2020). Retrieved from
https://www.ocimf.org/publications/books/international-safety-guide-
for-tankers-and-terminals-1 (Data retrieved 05.05.2023).
Jas, D., Antony, A. C., Saxena, A., Sharma, M., & Gupta, S. (2022). Hybrid AI
Talent Acquisition Model: An Opinion Mining and Topic based

81
 approach. In 2022 2nd International Conference on Intelligent
Technologies (CONIT) (pp. 1-5). IEEE.
Johnson, R. D., Lukaszewski, K. M., & Stone, D. L. (2016). The evolution of the
field of human resource information systems: Co-evolution of
technology and HR processes. Communications of the Association for
Information Systems, 38(1), 28.
Junker, B. W. (2011). The role of nonparametric analysis in assessment modeling:
then and now. In Looking back: proceedings of a conference in honor
of Paul W. Holland (pp. 67-85). Springer New York.
Kan, W. A. N. G., Yu, J. I. A. O., Jiahao, L. I. U., & Yujian, W. U. (2021). Research
on talent training model of maritime safety professional based on
innovation and entrepreneurship education idea. China Safety Science
Journal, 31(5), 138.
Karamanis, N., & Mitkov, R. (2006). Generating multiple-choice test items from
medical text: A pilot study. In Proceedings of the fourth international
natural language generation conference (pp. 111-113).
Kilgour, P., Northcote, M., Williams, A., & Kilgour, A. (2020). A plan for the co-
construction and collaborative use of rubrics for student learning.
Assessment & Evaluation in Higher Education, 45(1), 140-153.
Krathwohl, D. R. (2002). A revision of Bloom's taxonomy: An overview. Theory into
practice, 41(4), 212-218.
Krupitzer, C., Wagenhals, T., Züfle, M., Lesch, V., Schäfer, D., Mozaffarin, A.,
... & Kounev, S. (2020). A survey on predictive maintenance for
industry 4.0. arXiv preprint arXiv:2002.08224.
Kumar, G., Banchs, R. E., & D'Haro, L. F. (2015). Automatic fill-the-blank
question generator for student self-assessment. In 2015 IEEE Frontiers
in Education Conference (FIE) (pp. 1-3). IEEE.
Kurdi, G., Leo, J., Parsia, B., Sattler, U., & Al-Emari, S. (2020). A systematic
review of automatic question generation for educational purposes.
International Journal of Artificial Intelligence in Education, 30, 121-
204.
Lamba, H., & Govilkar, S. (2017). A survey on plagiarism detection techniques for
indian regional languages. Int. J. Comput. Appl, 975, 8887.
Landauer, T. K., Foltz, P. W., & Laham, D. (1998). An introduction to latent
semantic analysis. Discourse processes, 25(2-3), 259-284.
Le, N. T., Kojiri, T., & Pinkwart, N. (2014). Automatic question generation for
educational applications–the state of art. In Advanced Computational
Methods for Knowledge Engineering: Proceedings of the 2nd
International Conference on Computer Science, Applied Mathematics
and Applications (ICCSAMA 2014) (pp. 325-338). Springer
International Publishing.
Lee, H., Peirsman, Y., Chang, A., Chambers, N., Surdeanu, M., & Jurafsky, D.
(2011). Stanford’s multi-pass sieve coreference resolution system at the
conll-2011 shared task. In Proceedings of the fifteenth conference on
computational natural language learning: Shared task (pp. 28-34).
82
Lee, J., & Seneff, S. (2007). Automatic generation of cloze items for prepositions. In
Eighth Annual Conference of the International Speech Communication
Association.
Li, Y., Duan, N., Zhou, B., Chu, X., Ouyang, W., Wang, X., & Zhou, M. (2018).
Visual question generation as dual task of visual question answering. In
Proceedings of the IEEE conference on computer vision and pattern
recognition (pp. 6116-6124).
Lin, Y. C., Sung, L. C., & Chen, M. C. (2007). An automatic multiple-choice
question generation scheme for english adjective understanding. In
Workshop on Modeling, Management and Generation of
Problems/Questions in eLearning, the 15th International Conference on
Computers in Education (ICCE 2007) (pp. 137-142).
Lindberg, D., Popowich, F., Nesbit, J., & Winne, P. (2013). Generating natural
language questions to support learning on-line. In Proceedings of the
14th European Workshop on Natural Language Generation (pp. 105-
114).
Liu, C. L., Wang, C. H., Gao, Z. M., & Huang, S. M. (2005). Applications of lexical
information for algorithmically composing multiple-choice cloze items.
In Proceedings of the second workshop on Building Educational
Applications Using NLP (pp. 1-8).
Liu, M., & Haertel, G. (2011). Design patterns: A tool to support assessment task
authoring. Large-Scale Assessment Technical Report, 11.
Liu, R. W., Yuan, W., Chen, X., & Lu, Y. (2021). An enhanced CNN-enabled
learning method for promoting ship detection in maritime surveillance
system. Ocean Engineering, 235, 109435.
Ljung, M. (2010). Function based manning and aspects of flexibility. WMU Journal
of Maritime Affairs, 9(1), 121–133.
Luecht, R. M. (2008). Assessment engineering in test design, development, and
scoring. Presentation at the East Coast Organization of Language
Testers (ECOLT) Conference.
Maatuk, A. M., Elberkawi, E. K., Aljawarneh, S., Rashaideh, H., & Alharbi, H.
(2022). The COVID-19 pandemic and E-learning: challenges and
opportunities from the perspective of students and instructors. Journal
of computing in higher education, 34(1), 21-38.
Majumder, M., & Saha, S. K. (2014). Automatic selection of informative sentences:
The sentences that can generate multiple choice questions. Knowledge
Management & E-Learning, 6(4), 377.
Mannem, P., Prasad, R., & Joshi, A. (2010). Question generation from paragraphs
at UPenn: QGSTEC system description. In Proceedings of QG2010:
The Third Workshop on Question Generation (pp. 84-91).
Marler, J. H., & Parry, E. (2016). Human resource management, strategic
involvement and e-HRM technology. The International Journal of
Human Resource Management, 27(19), 2233-2253.
Mazidi, K., & Nielsen, R. (2014). Linguistic considerations in automatic question
generation. In Proceedings of the 52nd Annual Meeting of the
83
 Association for Computational Linguistics (Volume 2: Short Papers)
(pp. 321-326).
McCoubrie, P. (2004). Improving the fairness of multiple-choice questions: a
literature review. Medical teacher, 26(8), 709-712.
Messick, S. (1994). The interplay of evidence and consequences in the validation of
performance assessments. Educational Researcher, 23(2), 13-23.
Meyer, R. E., & Hammerschmid, G. (2010). The degree of decentralization and
individual decision making in central government human resource
management: A European comparative perspective. Public
Administration, 88(2), 455-478.
Mikolov, T., Chen, K., Corrado, G., & Dean, J. (2013). Efficient estimation of word
representations in vector space. arXiv preprint arXiv:1301.3781.
Mislevy, R. J. & Haertel, G. D. (2006). Implications of evidence-centered design for
educational testing. Educational Measurement: Issues & Practice,
25(4), 6-20.
Mislevy, R. J. (2013). Evidence-centered design for simulation-based assessment.
Military medicine, 178(suppl_10), 107-114.
Mislevy, R. J., & Yin, C. (2013). Evidence-centered design in language testing. In
The Routledge handbook of language testing (pp. 222-236). Routledge.
Mislevy, R. J. (2011). Evidence-Centered Design for Simulation-Based Assessment.
CRESST Report 800. National Center for Research on Evaluation,
Standards, and Student Testing (CRESST).
Mislevy, R. J. & Riconscente, M. M. (2006). Evidence-centered assessment design:
Layers, concepts, and terminology. In S. Downing & T. Haladyna
(Eds.), Handbook of Test Development (pp. 61-90). Mahwah, NJ:
Erlbaum.
Mislevy, R. J., Almond, R. G., & Lukas, J. F. (2003). A brief introduction to
evidence‐centered design. ETS Research Report Series, 2003(1), i-29.
Mislevy, R. J., Behrens, J. T., Dicerbo, K. E., & Levy, R. (2012). Design and
discovery in educational assessment: Evidence-centered design,
psychometrics, and educational data mining. Journal of educational
data mining, 4(1), 11-48.
Mislevy, R. J., Haertel, G., Riconscente, M., Wise Rutstein, D., Ziker, C., Mislevy,
R. J., ... & Ziker, C. (2017). Evidence-centered assessment design.
Assessing Model-Based Reasoning using Evidence-Centered Design:
A Suite of Research-Based Design Patterns, 19-24.
Mislevy, R. J., Steinberg, L. S., & Almond, R. G. (2013). On the roles of task model
variables in assessment design. In Item generation for test development
(pp. 129-160). Routledge.
Mitkov, R. (2003). Computer-aided generation of multiple-choice tests. In
Proceedings of the HLT-NAACL 03 workshop on Building educational
applications using natural language processing (pp. 17-22).

84
Mitkov, R., Le An, H., & Karamanis, N. (2006). A computer-aided environment for
generating multiple-choice test items. Natural language engineering,
12(2), 177-194.
Mohammed, M., & Omar, N. (2020). Question classification based on Bloom’s
taxonomy cognitive domain using modified TF-IDF and word2vec.
PloS one, 15(3), e0230442.
Mohapatra, M., & Sahu, P. (2018). Building a sustainable talent acquisition model
in a dynamic business environment. International Journal of Human
Capital and Information Technology Professionals (IJHCITP), 9(3), 42-
52.
Moreno-Guerrero, A. J., Aznar-Díaz, I., Cáceres-Reche, P., & Alonso-García, S.
(2020). E-learning in the teaching of mathematics: An educational
experience in adult high school. Mathematics, 8(5), 840.
Munim, Z. H. (2019). Autonomous ships: a review, innovative applications and future
maritime business models. In Supply Chain Forum: An International
Journal (Vol. 20, No. 4, pp. 266-279). Taylor & Francis.
Muslu, A. (2020). The Future of Seafarers and the Seafarers of the Future from the
Perspective of Human Resources Management. In Contemporary
Global Issues in Human Resource Management (pp. 219-237). Emerald
Publishing Limited.
Narendra, A., Agarwal, M., & Shah, R. (2013). Automatic cloze-questions
generation. In Proceedings of the International Conference Recent
Advances in Natural Language Processing RANLP 2013 (pp. 511-515).
Newton, P. M., Da Silva, A., & Peters, L. G. (2020). A pragmatic master list of action
verbs for bloom's taxonomy. In Frontiers in Education (Vol. 5, p. 107).
Frontiers.
Nkhoma, C., Nkhoma, M., Thomas, S., & Le, N. Q. (2020). The role of rubrics in
learning and implementation of authentic assessment: A literature
review. In Proceedings of the 2020 InSITE Conference (Vol. 4606, pp.
237-276).
Onnasch, L. (2015). Crossing the boundaries of automation—Function allocation and
reliability. International Journal of Human-Computer Studies, 76, 12-
21.

Ow Yong, S. J. Y. (2022). Talent transformation of seafarers in the maritime industry.

Pabitha, P., Mohana, M., Suganthi, S., & Sivanandhini, B. (2014). Automatic
question generation system. In 2014 International Conference on
Recent Trends in Information Technology (pp. 1-5). IEEE.
Pan, L., Lei, W., Chua, T. S., & Kan, M. Y. (2019). Recent advances in neural
question generation. arXiv preprint arXiv:1905.08949.
Panadero, E., Tapia, J. A., & Huertas, J. A. (2012). Rubrics and self-assessment
scripts effects on self-regulation, learning and self-efficacy in
secondary education. Learning and individual differences, 22(6), 806-
813.

85
Pantouvakis, A., & Vlachos, I. (2020). Talent and leadership effects on sustainable
performance in the maritime industry. Transportation Research Part D:
Transport and Environment, 86, 102440.
Papasalouros, A., Kanaris, K., & Kotis, K. (2008). Automatic Generation Of
Multiple Choice Questions From Domain Ontologies. e-Learning, 1,
427-434.
Parasuraman, R., & Riley, V. (1997). Humans and automation: Use, misuse, disuse,
abuse. Human factors, 39(2), 230-253.
Patwardhan, S., & Pedersen, T. (2006). Using WordNet-based context vectors to
estimate the semantic relatedness of concepts. In Proceedings of the
Workshop on Making Sense of Sense: Bringing Psycholinguistics and
Computational Linguistics Together.
Pazouki, K., Forbes, N., Norman, R. A., & Woodward, M. D. (2018). Investigation
on the impact of human-automation interaction in maritime operations.
Ocean engineering, 153, 297-304.
Phillips-Wren, G., Doran, R., & Merrill, K. (2016). Creating a value proposition
with a social media strategy for talent acquisition. Journal of Decision
systems, 25(sup1), 450-462.
Pikhart, M., & Klimova, B. (2019). Utilization of linguistic aspects of Bloom’s
taxonomy in blended learning. Education Sciences, 9(3), 235.
Pino, J., Heilman, M., & Eskenazi, M. (2008). A selection strategy to improve cloze
question quality. In Proceedings of the Workshop on Intelligent
Tutoring Systems for Ill-Defined Domains. 9th International
Conference on Intelligent Tutoring Systems, Montreal, Canada (pp. 22-
32).
Plake, B. S., Huff, K., & Reshetar, R. (2010). Evidence-centered assessment design
as a foundation for achievement-level descriptor development and for
standard setting. Applied Measurement in Education, 23(4), 342-357.
Prasetyo, A. R., Nurtjahjanti, H., & Ardhiani, L. N. (2021). Impact of changes in
teaching methods during the COVID-19 pandemic: the effect of
integrative e-learning on readiness for change and interest in learning
among Indonesian university students. International Review of
Research in Open and Distributed Learning, 22(2), 87-101.
Progoulaki, M., & Theotokas, I. (2016). Managing culturally diverse maritime
human resources as a shipping company’s core competency. Maritime
Policy & Management, 43(7), 860-873.
Pujawan, I. G. N., Rediani, N. N., Antara, I. G. W. S., Putri, N. N. C. A., & Bayu,
G. W. (2022). Revised Bloom Taxonomy-Oriented Learning Activities
to Develop Scientific Literacy and Creative Thinking Skills. Jurnal
Pendidikan IPA Indonesia, 11(1), 47-60.
Pyne, R., & Koester, T. (2005). Methods and means for analysis of crew
communication in the maritime domain. Archives of Transport, 17(3-
4), 193-208.

86
Rawson, A., & Brito, M. (2023). A survey of the opportunities and challenges of
supervised machine learning in maritime risk analysis. Transport
Reviews, 43(1), 108-130.
Reddy, M. Y. (2007). Rubrics and the enhancement of student learning. Educate, 7(1),
3-17.
Reynders, G., Lantz, J., Ruder, S. M., Stanford, C. L., & Cole, R. S. (2020).
Rubrics to assess critical thinking and information processing in
undergraduate STEM courses. International Journal of STEM
Education, 7, 1-15.
Riconscente, M. M., Mislevy, R. J., & Corrigan, S. (2016). Evidence-centered
design.
Rondon, M. H. D. F., & Fontes, R. (2017). Reflections on Automation and the Need
for New Competencies in the Civil Pilot Training. Aeron Aero Open
Access J, 1(4), 139-150.
Rupp, A. A., Gushta, M., Mislevy, R. J., & Shaffer, D. W. (2010). Evidence-
centered design of epistemic games: Measurement principles for
complex learning environments. The Journal of Technology, Learning
and Assessment, 8(4).
Orhan, M., & Celik, M. (2022). An Overview through Start-up Initiatives on HRTech
in Maritime and Offshore Industries. IFSPA 2022 IFSPA 2022, 270.
Safa, M., Weeks, K., Stromberg, R., & Azam, A. A. (2018). Strategic Port Human
Resource Talent Acquisition and Training: Challenges and
Opportunities. In Advances in Human Factors, Business Management
and Leadership: Proceedings of the AHFE 2017 International
Conferences on Human Factors in Management and Leadership, and
Business Management and Society, July 17− 21, 2017, The Westin
Bonaventure Hotel, Los Angeles, California, USA 8 (pp. 205-215).
Springer International Publishing.
Scouller, K. (1998). The influence of assessment method on students' learning
approaches: Multiple choice question examination versus assignment
essay. Higher education, 35(4), 453-472.
Setyaningsih, E. R., & Listiowarni, I. (2021). Categorization of exam questions
based on bloom taxonomy using naïve bayes and laplace smoothing. In
2021 3rd East Indonesia Conference on Computer and Information
Technology (EIConCIT) (pp. 330-333). IEEE.
Shaffer, D. W. (2006). Epistemic frames for epistemic games. Computers &
education, 46(3), 223-234.
Sharunova, A., Wang, Y., Kowalski, M., & Qureshi, A. J. (2022). Applying
Bloom’s taxonomy in transdisciplinary engineering design education.
International Journal of Technology and Design Education, 32(2), 987-
999.
Shei, C. C. (2001). Followyou!: An automatic language lesson generation system.
Computer Assisted Language Learning, 14(2), 129-144.
Shute, V. J. (2011). Stealth assessment in computer-based games to support learning.
Computer games and instruction, 55(2), 503-524.
87
Singh Bhatia, A., Kirti, M., & Saha, S. K. (2013). Automatic generation of multiple
choice questions using wikipedia. In Pattern Recognition and Machine
Intelligence: 5th International Conference, PReMI 2013, Kolkata,
India, December 10-14, 2013. Proceedings 5 (pp. 733-738). Springer
Berlin Heidelberg.
Singh, H. K., Joshi, A., Malepati, R. N., Najeeb, S., Balakrishna, P.,
Pannerselvam, N. K., ... & Ganne, P. (2021). A survey of E-learning
methods in nursing and medical education during COVID-19 pandemic
in India. Nurse education today, 99, 104796.
Sobral, S. R. (2021). Bloom's Taxonomy to improve teaching-learning in introduction
to programming.
Snow, E., Haertel, G., Fulkerson, D., Feng, M., & Nichols, P. (2010). Leveraging
evidence-centered assessment design in large-scale and formative
assessment practices. In Proceedings of the 2010 Annual Meeting of the
National Council on Measurement in Education (NCME).
Snow, E., Fulkerson, D., Feng, M., Nichols, P., Mislevy, R., & Haertel, G. (2010).
Leveraging Evidence-Centered Design in Large-Scale Test
Development. Technical Report 4). Menlo Park: SRI International,
Center for Technology in Learning. Retrieved from http://ecd. sri.
com/downloads/ECD_TR4_Leveraging_ECD_FL. pdf.
Stone, D. L., Deadrick, D. L., Lukaszewski, K. M., & Johnson, R. (2015). The
influence of technology on the future of human resource management.
Human resource management review, 25(2), 216-231.
Streng, M., & Kuipers, B. (2020). Economic, social, and environmental impacts of
autonomous shipping strategies. In Maritime supply chains (pp. 135-
145). Elsevier.
Sumita, E., Sugaya, F., & Yamamoto, S. (2005). Measuring non-native speakers’
proficiency of english by using a test with automatically-generated fill-
in-the-blank questions. In Proceedings of the second workshop on
Building Educational Applications Using NLP (pp. 61-68).
Tanakitkorn, K. (2019). A review of unmanned surface vehicle development.
Maritime Technology and Research, 1(1), 2-8.
Teo, H. Q. (2021). A ranking of future competencies for maritime brokers in a digital
era. https://www.skillsea.eu/ (data retrieved 07.04.2023)
The Maritime Executive Magazine. (2018). The future of the modern seafarers?
Profession. Retrieved from https://maritime-
executive.com/editorials/the-future-of-the-modern-seafarers-
profession (Data retrieved 01.03.2023).
Tirziu, A. M., & Vrabie, C. (2015). Education 2.0: E-learning methods. Procedia-
Social and Behavioral Sciences, 186, 376-380.
Tobajas, M., Molina, C. B., Quintanilla, A., Alonso-Morales, N., & Casas, J. A.
(2019). Development and application of scoring rubrics for evaluating
students’ competencies and learning outcomes in Chemical
Engineering experimental courses. Education for Chemical Engineers,
26, 80-88.

88
Tuma, F., & Nassar, A. K. (2021). Applying Bloom's taxonomy in clinical surgery:
practical examples. Annals of Medicine and Surgery, 69, 102656.
Turan, O., Kurt, R. E., Arslan, V., Silvagni, S., Ducci, M., Liston, P., ... &
Papadakis, G. (2016). Can we learn from aviation: safety
enhancements in transport by achieving human orientated resilient
shipping environment. Transportation research procedia, 14, 1669-
1678.
Vagia, M., Transeth, A. A., & Fjerdingen, S. A. (2016). A literature review on the
levels of automation during the years. What are the different
taxonomies that have been proposed?. Applied ergonomics, 53, 190-
202.
Van der Linden, W. J., & Glas, C. A. (Eds.). (2010). Elements of adaptive testing
(Vol. 10, pp. 978-0). New York: Springer.
Vo, L. C., Lavissiere, M. C., & Lavissiere, A. (2023). Retaining talent in the
maritime sector by creating a work-family balance logic: implications
from women managers navigating work and family. International
Journal of Physical Distribution & Logistics Management, 53(1), 133-
155.
Wahlström, M., Hakulinen, J., Karvonen, H., & Lindborg, I. (2015). Human
factors challenges in unmanned ship operations–insights from other
domains. Procedia Manufacturing, 3, 1038-1045.
Waite, L. H., Zupec, J. F., Quinn, D. H., & Poon, C. Y. (2020). Revised Bloom's
taxonomy as a mentoring framework for successful promotion.
Currents in Pharmacy Teaching and Learning, 12(11), 1379-1382.
Walford-Wright, G., & Scott-Jackson, W. (2018). Talent Rising; people analytics
and technology driving talent acquisition strategy. Strategic HR
Review, 17(5), 226-233.
Williamson, D. M., Mislevy, R. J., & Bejar, I. I. (Eds.). (2006). Automated scoring
of complex tasks in computer-based testing. Psychology Press.
Wolf, K., & Stevens, E. (2007). The role of rubrics in advancing and assessing student
learning. Journal of Effective Teaching, 7(1), 3-14.
Wu, H. C., Luk, R. W. P., Wong, K. F., & Kwok, K. L. (2008). Interpreting tf-idf
term weights as making relevance decisions. ACM Transactions on
Information Systems (TOIS), 26(3), 1-37.
Yao, X., & Zhang, Y. (2010). Question generation with minimal recursion semantics.
In Proceedings of QG2010: The Third Workshop on Question
Generation (pp. 68-75).
Yao, X., Bouma, G., & Zhang, Y. (2012). Semantics-based question generation and
implementation. Dialogue & Discourse, 3(2), 11-42.
Yin, C., & Mislevy, R. J. (2021). Evidence-centered design in language testing. In
The Routledge handbook of language testing (pp. 289-305). Routledge.
Yurzhenko, А. Y. (2019). An e-course based on the LMS MOODLE to teach"
Maritime English for professional purpose".

89
Zavala, L., & Mendoza, B. (2018). On the use of semantic-based aig to automatically
generate programming exercises. In Proceedings of the 49th ACM
technical symposium on computer science education (pp. 14-19).
Zavalniuk, O., Nesterenko, V., Zavalniuk, I., & Doshchenko, H. (2021). A key
component of continuing professional development in the maritime
context.
Zhang, J., Wong, C., Giacaman, N., & Luxton-Reilly, A. (2021). Automated
classification of computing education questions using Bloom’s
taxonomy. In Proceedings of the 23rd Australasian Computing
Education Conference (pp. 58-65).
Zhang, Y., Chen, B. L., Ge, J., Hung, C. Y., & Mei, L. (2019). When is the best time
to use rubrics in flipped learning? A study on students’ learning
achievement, metacognitive awareness, and cognitive load. Interactive
Learning Environments, 27(8), 1207-1221.
Zhou, Q., Yang, N., Wei, F., Tan, C., Bao, H., & Zhou, M. (2018). Neural question
generation from text: A preliminary study. In Natural Language
Processing and Chinese Computing: 6th CCF International Conference,
NLPCC 2017, Dalian, China, November 8–12, 2017, Proceedings 6
(pp. 662-671). Springer International Publishing.
Zieky, M. J. (2014). An introduction to the use of evidence-centered design in test
development. Psicología Educativa, 20(2), 79-87.

90
APPENDICES

APPENDIX A: Hybrid talent acquisition survey & implementation protocol.

Hybrid Talent Acquisition Survey & Implementation Protocol

• Company A has a total of 12 tanker ships.

• The years of built of these ships range from 2002 to 2016.
• 4 of the ships are Oil Chemical Tankers and 8 of them are Oil
Company A Tankers.
Information • The DWTs of the ships vary between 38000 and 180000.
• Company A ships usually use the West Asia (Strait of Hormuz)
route. In this context, ships carry oil to East and Asian countries by
passing through Arabian Sea, Indian Ocean, East Africa, Cape of Good
Hope, Atlantic Ocean and Western Europe regions.
• Various documents were checked during the meeting with the
Company A.
• For example, the company's crew information, crew matrix, fleet,
investment plan for the next 10 years, revisions and company
visions, missions, shipowner expectations were analyzed. In
addition, the PSC and FSC inspections of the ships in the last 5
Evidence years were examined.
• In these reports, it has been determined that there are deficiencies in
the subjects of emergency management, human factor and safety
management system in general on Company A ships.
• Finally, to verify the analyzes and determinations, verification was
made with classification societies reports, inspections and
expectations.
Phase 1: Domain Analysis

In order to carry out a detailed analysis of the Company A, the reports,

documents, interviews and opinions received during this process will be
examined. In this regard, the basic needs of the company will be determined,
and then appropriate knowledge resource selection will be carried out.
Phase 1 Guideline
• Analyze best management practices
• Examine human factor accidents and reports
• Investigate safety management system
• Survey ISPS Code protocol
• View and research transition process
• Determine necessary rules and regulations
As a result of the interview with the Company A, it has been determined that
there are some deficiencies in line with the reports examined, especially best
Observations management practices, human factor accidents and reports, safety
management system. It has been noticed that the company has not been able
to provide adequate training to its crews on these issues. Especially within the

91
 scope of safety management system and ISPS Code protocol, there is a lack
of crew in human factor accidents and reports. The crew recruited to the
company is onboarding to the ships without adequate training and this creates
various problems for the company. In addition, as a result of the investigations,
it was stated that the company would purchase LNG and LPG vessels after
selling the Oil Tankers. At this point, it was determined that the crew, who had
knowledge on green skills, digitalization and carbon capturing, was not in the
company. Therefore, it is thought that there will be deficiencies in the
transition process of the company in the upcoming period. At this point, the
rules and regulations of company's transition process should be introduced to
the crew.

ISGOTT, Marine Engineering Compendium, Tanker Management and Self

Assessment 3, Behavioral Competency Assessment and Verification for Vessel
Operators and IMDG Code.

Options

It was seen that they have tanker ships in their fleet and that they will expand
their fleet with LNG-LGP ships in the coming period. Then, it was analyzed
that the company does not have sufficient training programs on subjects such
as safety management system and best management practices. In addition, the
Findings
repetition of human factor accidents, the lack of rules and regulations about
route and the information about the transition process in the upcoming period
were taken into account. Considering all these, ISGOTT was chosen as the
appropriate knowledge resource for the company.

92
 Phase 2: Domain Modeling

In order for the selected knowledge resource to be incorporated into the

company, detailed examinations are required in this phase. For the successful
completion of the Domain Modeling phase, the following checks must be
completed and then chapter and section selections must be evaluated from
Phase 2 Guideline
ISGOTT.

• Examine Emergency management

• Analyze PSC and FSC inspections in the last five years
• Investigate and report accidents, near misses
• Survey superintendent recommendations and strategies
As a result of the examinations, various findings were determined in the PSC
and FSC inspections of the Company A in the last five years. For example,
after the Arabian Sea crossing in one of the Oil Chemical Tanker ships with
143500 DWT, it was observed that the port state controller detected deficiencies
in the safety management system documents. It has been determined by PSC
Observations that the hot work permits on the same ship were not prepared in a timely manner
by the officers of the watch. On another ship, it was observed that the
preparation of the tanker emergency plan could not be made on time. Along
with all this, it has been reported that the newly hired crew caused accidents
and near misses after joining the ships. Finally, the findings were compared
with superintendent recommendations and strategies.

ISGOTT Part 1 (General Information), Part 2 (Tanker Information), Part 3

(Marine Terminal Information), Part 4 (Ship/Shore and Tanker/Terminal
Information).

Chapter 9 (Management of Safety and Emergencies), Chapter 10 (Enclosed

Spaces), Chapter 11 (Shipboard Systems), Chapter 12 (Shipboard Operations),
Chapter 13 (Carrying and Storing Hazardous Materials) and Chapter 14
Options (Special Ship Types).

93
 According to the observations and options, ISGOTT Part 2, Chapter 9 (Safety
Management Systems, Work planning and permit to work systems, Hot work,
Electric welding equipment, Other hazardous tasks, Emergency management,
Findings
Managing simultaneous operations, Hazards on ships with exposed transverse
frames and Repairs at a facility other than a shipyard) was chosen as the part,
chapter or section of knowledge resource.

Phase 3: Conceptual Assessment Framework

In this phase, the results of the domain analysis and domain modeling phases
should be evaluated. Also in this phase, the company's facilities, crew details
and information should be investigated for the preparation of the assessment.
Phase 3 Guideline
For this, the following three elements must be completed.

• Determine taxonomy
• Choose question type
• Employ question generation model
As a result of the interviews with the Company A, it was seen that there was a
Observations need to prepare a training program for topics such as safety management
system, work planning and permit to work systems, hot work and emergency

94
 management in ISGOTT Part 2, Chapter 9. In this context, it was discussed that
the crew could access the computer both on vacation and on board. For the
assessments to be prepared in this context, first of all, taxonomy
determination was carried out so that each crew could understand the chapter
and sections of ISGOTT from beginning to end. Then, the stage of choosing
the question type for the assessments to be created was started. Finally, a
question generation model was employed so that assessments can be created
easily, quickly and with less cost by the human resources management.

Marzano and Kendall Taxonomy, SOLO Taxonomy, Bloom’s Taxonomy,

DOK Taxonomy.

Closed questions, probing questions, funnel questions, leading questions,

clarifying questions, rhetorical questions, recall questions, contingency
questions, and multiple choice questions.

MCQ generation model, Visual question generation model, Natural question

generation model, SQuAD 1.1, COCO question generation, FairytaleQA, and
TriviaQA.

Options

95
 Taking brief, observations and options into consideration, Bloom's taxonomy
Findings from taxonomy, multiple choice question from question types and MCQ
generation model from question generation models were chosen for phase 3.

Phase 4: Assessment Delivery and Implementation

In order for the hybrid talent acquisition model to be successful, the assessments
created in the fourth phase must be presented to the crew in an appropriate
manner. At this stage, interviews with the company's HSEQ, training and
Phase 4 Guideline
human resources departments are required. In line with the information
obtained, it is necessary to apply a competency demonstration technique lastly.

• Apply competency demonstration technique

As a result of the interviews with the Company A, it was determined that the
workload of the crew was quite high. In this context, it has been understood that
traditional assessment delivery methods cannot be selected. In addition, in line
Observations with the information obtained from the HSEQ, training and human resources
departments, it was noted that the personnel on the company side also needed a
fast, effective and automation model. As a result of all these, the appropriate
competency demonstration techniques were applied.

Computer based assessment, class work and classroom testing, individual and
group assignments, theory tests and examinations, reports and essays, direct
observation, approved in-service experience, approved training ship
experience, approved simulator training and approved training program.

Options

In line with the information obtained, it has been observed that there is no
approved simulator within the Company A. In addition, considering the
Findings workload of the crew and the ship's work, it was thought that the use of
competency demonstration techniques such as reports, direct observation,
classroom testing, and individual and group assignments were not appropriate.

96
 In this context, computer based assessment was chosen because it can be
accessed from anywhere and is more convenient for automation and artificial
intelligence.

Phase 5: Recommendations

As a result of the interviews and examinations, a total of ten multiple choice questions were delivered
to the Company A in a computer-based assessment format.

97
98
APPENDIX B: Presented multiple choice questions.

Multiple Choice Question #1

Bloom Level: Remember

Bloom Keyword: List

Knowledge resource: ISGOTT Part 2, Chapter 9, Page 133-134 (742 words)

Reducted text: ISGOTT Part 2, Chapter 9, Page 134 (254 words)

Theme: Personal Protective Equipment

Question point: 10 points

Duration: 80 seconds

It is expected to retrieve, recall or recognize (e.g., recall dates of important events in U.S. history,
remember the components of a bacterial cell) relevant knowledge from long-term memory about
Personal protective equipment

This stage of learning is about memorizing basic facts, dates, events, persons, places, concepts and
patterns.

Please read the information provided carefully to answer the question correctly: PPE should be of
an approved standard and should be worn by all personnel engaged in operations. This should include
body, hands and feet, face and eyes, ears and head protection.

Depending on the information above, please choose the correct list about the topic and find the most
appropriate answer in order to pass the Personal protective equipment section successfully.

Responses

a-) Safety shoes, gloves, ECDIS, ECRS (distractor)

b-) Helmet, boiler suit, safety shoes, ECDIS (distractor)

c-) Boiler suit, spanner, telephone and shoes (distractor)

d-) Boiler suit, gloves, helmet, safety shoes and earphone

e-) ECDIS, earphone, gloves and spanner (distractor)

99
Multiple Choice Question #2

Bloom Level: Understand

Bloom Keyword: Explain

Knowledge resource: ISGOTT Part 2, Chapter 9, Page 135-136 (452 words)

Reducted text: ISGOTT Part 2, Chapter 9, Page 136 (212 words)

Theme: Permit to work

Question point: 10 points

Duration: 80 seconds

It is expected to demonstrate comprehension through one or more forms of explanation about Permit
to work (e.g., classify a mental illness, compare ritual practices in two different religions).

At this point, it is expected to explain a concept in their own words, describe a mathematical graph or
clarify a metaphor.

Please read the information provided carefully to answer the question correctly: While operators
will develop their own procedures for managing all aspects of operations and tasks, many choose to
incorporate work planning meetings and a permit to work system into their SMS to manage hazardous
tasks.

Depending on the information above, please make an explain and find the most appropriate answer in
order to pass the Permit to work section successfully.

Responses

a-) Hot work refers to any kind of work that entails the presence of ignition sources or temperatures that
are sufficiently high to cause the ignition of flammable gases, liquids, or materials. (distractor)

b-) Hot work conducted inside the specified area must be regulated by the SMS using a permit-to-work
system and certain criteria.

c-) The Chief Engineer must first determine the need and feasibility of hot work, as well as any potential
safety implications. Prior to commencing any work, it is essential for the departmental heads to evaluate
all suggested tasks within their respective departments. (distractor)

d-) While a ship is alongside a terminal, hazardous tasks should only be undertaken with the prior
agreement of the Bunker Representative. (distractor)

e-) The protocol, permission, and documentation of adherence should be maintained alongside the
entries in the Oil Record Book. (distractor)

100
Multiple Choice Question #3

Bloom Level: Apply

Bloom Keyword: Provide

Knowledge resource: ISGOTT Part 2, Chapter 9, Page 146-147 (642 words)

Reducted text: ISGOTT Part 2, Chapter 9, Page 147 (432 words)

Theme: Toolbox talks

Question point: 10 points

Duration: 80 seconds

It is expected to use information or a skill in a new situation (e.g., use Newton's second law to solve a
problem for which it is appropriate, carry out a multivariate statistical analysis using a data set not
previously encountered) about Toolbox talks.

Please read the information provided carefully to answer the question correctly: Simultaneous
Operations (SIMOPS) refer to concurrent activities occurring simultaneously within the same vicinity,
which have the potential to directly or indirectly impact the safety of other operations on the vessel or
terminal. Early identification of SIMOPS is crucial for understanding the interplay between different
operations or terminals and their respective impacts.

Depending on the information above, please provide an explanation about the topic and find the most
appropriate answer in order to pass the Toolbox talks section successfully.

Responses

a-) A toolbox talk has been carried out only Chief Engineer and Master on site. (distractor)

b-) Toolbox talks should include discussion on SIMOPS. These procedures should include the control
of her and address any possible conflicts or problems.

c-) Toolbox talk is carried out to brief the crew non-involved in operation on the hazards and safely
procedures. (distractor)

d-) Operational checks should not be carried out to confirm items identified in the pre-bunkering toolbox
talk that could have an impact on the safety of the bunkering operation have been actioned, e.g. hot
work restrictions in place. (distractor)

e-) A talk about the toolbox is necessary just for certain tasks. Prior to commencing the activity, the
person in charge (PIC) should conduct a toolbox discussion at the workplace, including all individuals
present. (distractor)

101
Multiple Choice Question #4

Bloom Level: Analyze

Bloom Keyword: Select

Knowledge resource: ISGOTT Part 2, Chapter 9, Page 147-148 (443 words)

Reducted text: ISGOTT Part 2, Chapter 9, Page 148 (321 words)

Theme: Risk assessment

Question point: 10 points

Duration: 80 seconds

It is expected to break material into its constituent parts and determine how the parts relate to one another
and/or to an overall structure (e.g., analyze the relationship between different flora and fauna in an
ecological setting) or purpose about Risk assessment.

Please read the information provided carefully to answer the question correctly: It is advisable for
all tankers and ports to do a comprehensive risk evaluation and formulate a security strategy. Tankers
should also formulate a strategy to strengthen their defences. The security strategy should include
protocols, strategies, and safeguards to effectively mitigate any identified security vulnerabilities.

Depending on the information above, please make a selection about the topic and find the most
appropriate answer in order to pass the Risk assessment section successfully.

Responses

a-) A job, apart from hot labour, that poses a danger to the ship, terminal, or workers, and requires
regulation via a risk assessment method, such as a permission to work system or a regulated procedure.
(distractor)

b-) Risk assessment is a systematic procedure that gathers data and gives numerical values to potential
hazards. This information is then used to determine priorities, evaluate different options, and aid in the
decision-making process. (distractor)

c-) An essential instrument used by companies, and a fundamental feature of all SMS for risk
management, is the waste management procedure. (distractor)

d-) In the replacement phase of the risk assessment pyramid of controls, the objective is to eliminate the
source of the hazard entirely, for example, by refraining from carrying out a planned operation.
(distractor)

e-) Risk evaluations may also facilitate the development of security strategies, plans for emergencies,
oil spill response plans, and terminals safety exclusion zones.

102
Multiple Choice Question #5

Bloom Level: Evaluate

Bloom Keyword: Summary

Knowledge resource: ISGOTT Part 2, Chapter 9, Page 149-151 (764 words)

Reducted text: ISGOTT Part 2, Chapter 9, Page 150-151 (342 words)

Theme: Hierarchy of controls

Question point: 10 points

Duration: 80 seconds

It is expected to make judgments based on criteria and standards (e.g., detect inconsistencies or fallacies
within a process or product, determine whether a scientist's conclusions follow from observed data)
about Hierarchy of controls.

Please read the information provided carefully to answer the question correctly: The Hierarchy of
control recognizes that eliminating a task altogether may provide the best protection against the risk
from that task, especially when compared to the less effective controls of additional procedures or
signage.

Depending on the information above, please make a summary about the topic and find the most
appropriate answer in order to pass the Hierarchy of controls section successfully.

Responses

a-) Prior to developing a terminal's emergency plan, it is essential to conduct a comprehensive

assessment of the terminal, such as: local emergency, terminal emergency, and big emergency.

b-) Within the hierarchy of controls, there's a prescribed sequence of priority for the selection of
preventive measures. This includes obtaining permits for hot work and enclosed space entrance, using
tools to identify dangers, doing risk assessments, and adhering to duty rosters. These measures are used
to minimise the risk of exposure to hazards. (distractor)

c-) The hierarchy of controls establishes a prescribed sequence of priorities for the selection of
preventive measures. Substitute the suggested method with a safer alternative in the elimination
components. (distractor)

d-) Within a system of controls, there exists a prescribed sequence of priority for the selection of
preventive out a suggested procedure. (distractor)

e-) Within the structure of controls, there exists a prescribed sequence of priority for the selection of
preventive measures. Within the administrative controls section, it is essential to use PPE to safeguard
the individual doing the task, such as safety glasses. (distractor)

103
Multiple Choice Question #6

Bloom Level: Create

Bloom Keyword: Explain

Knowledge resource: ISGOTT Part 2, Chapter 9, Page 152-154 (654 words)

Reducted text: ISGOTT Part 2, Chapter 9, Page 153 (253 words)

Theme: Management of change

Question point: 10 points

Duration: 80 seconds

It is expected to put elements together to form a new coherent or functional whole; reorganize elements
into a new pattern (design a new set for a theater production, write a thesis, develop an alternative
hypothesis based on criteria, invent a product, compose a piece of music, write a play) or structure about
Management of change.

Please read the information provided carefully to answer the question correctly: Management of
Change (MOC) is the process of bringing controlled and planned change, either temporary or
permanent, to operations, procedures, equipment or personnel to meet a defined goal.

Depending on the information above, please provide an explain about the topic and find the most
appropriate answer in order to pass the Management of change section successfully.

Responses

a-) The risk assessment and management procedures should exclude a MoC procedure. (distractor)

b-) The MOC method must verify that safety and environmental criteria are maintained without
compromise after the modification. (distractor)

c-) It is important to record and support the MOC process with a risk assessment, which should be
conducted after any initial risk assessments, if they exist.

d-) Exclusions from the necessity for a MOC process do not include the following changes:
Modifications to current equipment and changes in legislation and standards need alterations to the
established methods of operation in the SMS. (distractor)

e-) There is no need for MOC procedures when dealing with new cargo items or when a different kind
or dimensions of tanker visits a terminal. (distractor)

104
Multiple Choice Question #7

Bloom Level: Remember

Bloom Keyword: Indicate

Knowledge resource: ISGOTT Part 2, Chapter 9, Page 152-153 (342 words)

Reducted text: ISGOTT Part 2, Chapter 9, Page 152-153 (195 words)

Theme: Tanker emergency plan

Question point: 10 points

Duration: 80 seconds

It is expected to retrieve, recall, or recognize relevant knowledge from long-term memory about Tanker
emergency plan (e.g., recall dates of important events in U.S. history, remember the components of a
bacterial cell). This stage of learning is about memorizing basic facts, dates, events, persons, places,
concepts and patterns.

Please read the information provided carefully to answer the question correctly: Effective planning
and preparation are crucial for people to properly manage and reduce the impact of any mishaps on
board tankers. The captain and other officers should evaluate potential situations, such as a fire
occurring in the engine room, or accommodation area, an individual falling into a tank, the ship
unexpectedly drifting away from her berth, and the need for a sudden release of a tanker from its berth.

Depending on the information above, please indicate about the onboard information should be readily
available and find the most appropriate answer in order to pass the tanker emergency plan section
successfully.

Responses

a-) Specify the nature of the cargo, the quantity, and its intended distribution.

b-) Location of other engine room equipment. (distractor)

c-) Life-saving appliances and equipment plans. (distractor)

d-) Type of grey and black water and their disposition. (distractor)

e-) Type of fuel and lubrication oil; amount and disposition. (distractor)

105
Multiple Choice Question #8

Bloom Level: Understand

Bloom Keyword: Describe

Knowledge resource: ISGOTT Part 2, Chapter 9, Page 147-148 (542 words)

Reducted text: ISGOTT Part 2, Chapter 9, Page 148 (234 words)

Theme: Manifold platform

Question point: 10 points

Duration: 80 seconds

It is expected to demonstrate comprehension through one or more forms of explanation about manifold
platform (e.g., classify a mental illness, compare ritual practices in two different religions). At this
point, it is expected to explain a concept in their own words, describe a mathematical graph or clarify a
metaphor.

Please read the information provided carefully to answer the question correctly: The design of
product and chemical tankers has evolved, leading to external transverse stiffening on the tank deck.
The deck areas of these tankers are more catered and less accessible In some cases.

Depending on the information above, please describe the tanker manifold platform features and find
the most appropriate answer in order to pass the manifold platform section successfully.

Responses

a-) The manifold working platform has steel grating and, usually, a drip tray/saveall.

b-) Edge protection should not be temporarily removed to allow a hose connection. (distractor)

c-) Extra elevated platforms are not needed to access key work areas such as the manifold. (distractor)

d-) The danger of falling from a height necessitates the implementation of appropriate safety measures.
(distractor)

e-) The hazards of working on dangerous platforms, the work should not be properly planned and
effectively supervised. (distractor)

106
Multiple Choice Question #9

Bloom Level: Evaluate

Bloom Keyword: Explain

Knowledge resource: ISGOTT Part 2, Chapter 9, Page 146-147 (542 words)

Reducted text: ISGOTT Part 2, Chapter 9, Page 146-147 (345 words)

Theme: Other hazardous tasks

Question point: 10 points

Duration: 80 seconds

One is anticipated to evaluate other hazardous jobs by using criteria and standards, such as identifying
errors or fallacies in a process or product, and determining whether a scientist's findings are logically
derived from observable evidence.

Please read the information provided carefully to answer the question correctly: : A hazardous job
refers to any activity, except hot work, that poses a danger to the ship, terminal, or workers and requires
management via a risk assessment method, such as a permission to work system.

Depending on the information above, please make an explain about the features of other hazardous
tasks and find the most appropriate answer in order to pass the hazardous tasks section successfully.

Responses

a-) Testing and launching of lifeboats are not included in other hazardous tasks. (distractor)

b-) Enclosed space entry, tank inspections, diving operations and electrical work are not included in
other hazardous tasks. (distractor)

c-) It is unnecessary to create and authorise an employment licence or regulated process for every
hazardous operation. (distractor)

d-) While a ship is alongside a terminal, hazardous tasks should not only be undertaken with the prior
agreement of the Terminal Representative. (distractor)

e-) The protocol, permission, and documentation of adherence should be maintained alongside the SMS
data.

107
Multiple Choice Question #10

Bloom Level: Remember

Bloom Keyword: Outline

Knowledge resource: ISGOTT Part 2, Chapter 9, Page 146 (231 words)

Reducted text: ISGOTT Part 2, Chapter 9, Page 146 (154 words)

Theme: Electric welding equipment

Question point: 10 points

Duration: 80 seconds

The task involves the retrieval, recall, or recognition of pertinent information from memory over time
about electricity welding equipment (e.g., recalling significant dates in U.S. history, remembering the
different parts of a bacterial cell). This phase of education involves the process of committing
fundamental information, such as facts, dates, events, individuals, locations, ideas, and patterns, to
memory.

Please read the information provided carefully to answer the question correctly: Prior to each
usage, it is essential to thoroughly examine electric welding equipment to verify its optimal condition.
If deemed essential, it must be well grounded.

Depending on the information above, please carry out an outline about the welding equipment working
conditions and find the most appropriate answer in order to pass the electric welding equipment
section successfully.

Responses

a-) Gas-free spaces do not have electrical supply connections connected to them. (distractor)

b-) The current demand exceeds the capacity of the existing supply wire, leading to overloading and
subsequent heating. (distractor)

c-) The insulating properties of flexible electric lines must be in optimal condition.

d-) The cable path to the construction location is designed to provide maximum safety, only crossing
areas that have been cleared of gas. (distractor)

e-) The next step is to establish an earthing connection, where the earth return wire travels straight back
to the welding machine from the work site. The ship's construction should serve as a means to ground
the electrical current. (distractor)

108
CIRRICULUM VITAE

Name Surname : Muhittin Orhan

EDUCATION:

• B. Sc. : 2021, Istanbul Technical University, Maritime Faculty,

Marine Engineering

PROFESSIONAL EXPERINCE AND REWARDS:

• Between 2017-2021, he worked in the fields of Maritime Law, Digitalization and

Machine Learning at Istanbul Technical University Marine Engineering.
• He completed undergraduate education at Istanbul Technical University in 2021.

PUBLICATIONS, PRESENTATIONS AND PATENTS ON THE THESIS:

• Orhan, M., & Celik, M. (2022). An Overview through Start-up Initiatives on

HRTech in Maritime and Offshore Industries. IFSPA 2022, IFSPA 2022, 270.

OTHER PUBLICATIONS, PRESENTATIONS AND PATENTS:

• Orhan, M., & Bayraktaroglu, A. E. (2023). Usability Evaluation of Maritime

Websites with Different End User Groups. In Industrial Engineering in the Covid-
19 Era: Selected Papers from the Hybrid Global Joint Conference on Industrial
Engineering and Its Application Areas, GJCIE 2022, October 29-30, 2022 (pp.
224-237). Cham: Springer Nature Switzerland.
• Orhan, M., Celik, M. (2023). Application requirements of engine health
monitoring models to unmanned surface vehicles, June 25, International
Conference on Recent Engineering and Technology ICRET 2023, New Horizon
College of Engineering, Bengaluru, Karnataka.
• Orhan, M., Celik, M. (2023). Use of FMECA in engine health monitoring system
development onboard ships, June 25, International Conference on Recent

109
 Engineering and Technology ICRET 2023, New Horizon College of Engineering,
Bengaluru, Karnataka.
• Orhan, M., & Celik, M. (2023). A Fuzzy Cognitive Mapping Approach to
Conduct Deficiency Investigation under SIRE 2.0 Inspection. Transactions on
Maritime Science, 12(02).
• Orhan, M., & Celik, M. (2023). A literature review and future research agenda on
fault detection and diagnosis studies in marine machinery systems. Proceedings of
the Institution of Mechanical Engineers, Part M: Journal of Engineering for the
Maritime Environment, 14750902221149291.
• Orhan, M., Celik, M., & Kandemir, C. (2022). Estimation of human error
probabilities in marine safety services: The case of lifeboat and davits inspection.
Proceedings of the Institution of Mechanical Engineers, Part M: Journal of
Engineering for the Maritime Environment, 14750902221133884.
• Orhan, M., Celik, M. (2022). A Decision Analysis Proposal to Strength Industrial
Compliance of Maritime Education and Training Programs. International
Symposium of the Analytic Hierarchy Process 2022, ISAHP 2022.
• Orhan, M., Celik, M. (2022). Prioritization of Enabling Technologies Throughout
Digital Transformation in Ship Management Companies. International
Symposium of the Analytic Hierarchy Process 2022, ISAHP 2022.
• Orhan, M., Bayraktaroglu, A. E. (2022). Examining the Aspects Impacting the
Engagement of Volunteers on Volunteer-Match Platforms. International
Engineering and Technology Management Summit (ETMS 2022).

110