Evaluation of basic safety training using the CIPP

model
Darul Prayogo

Merchant Marine Polytechnic Semarang, Semarang, Central Java, Indonesia.

Corresponding author: D. Prayogo, [email protected]

Abstract. This study aims to investigate the impact of context, input, process, and products on
education and training services of Basic Safety Training at the Merchant Marine Polytechnic, Indonesia.
The study uses the ex post facto approach. The research location was at the Merchant Marine Polytechnic
with a sample of 86 people. The research data was collected using the questionnaire method and the
answers were analyzed using evaluative analysis techniques through descriptive-quantitative analysis.
The mean score of the post-test was higher than the mean score of the pre-test using the CIPP Model,
with 23.17%. The correlation coefficient of the pre-test and post-test was 0.895, showing a significant
correlation. The N-Gain Index was 0.799, in the high category, meaning that the implementation of the
CIPP application system model was good and effective in improving learning. Result of t-test showed that
the model was effective. CIPP also concludes that all the aspects embraced were in the good categories
for education and training services of BST at the Merchant Marine Polytechnic.
Key Words: context, evaluation, input, process, product, service.

Introduction. Transportation by ship has a positive impact on the employees who work
as the ship's crew. The Baltic and International Maritime Council (Bimco) noted that in
2020 the world needed 1593198 seafarers. However, there were only 1555281 seafarers,
so the number should be increased. It is an opportunity for those who wish to work
aboard ships, as there are more job vacancies on the seas than on land (Mamahit 2014).
One of the base educations and training programs that prospective seafarers must
follow is Basic Safety Training (BST). The program aims to provide training participants
with understanding and skills about the basics of safety on board. They must also be able
to practice the knowledge to prevent and overcome emergencies while sailing (IMO
2000).
Merchant Marine Polytechnic is an educational institution under the Ministry of
Transportation of Indonesia. The campus arranges education and training services of BST
for members of the general public who want to work as seafarers. The main objective of
the training is to improve the participant's skills, to be capable of performing a rescue in
an emergency condition. They must be able to prevent fires and extinguish them. In
addition, they are expected to understand various emergencies, occupational health and
safety, light medical care on board, and self-rescue. Participants are also required to
engage in preventing marine pollution and develop social communication aboard the ship
(IMO 2000).
At the initial field observations and interviews with education and training
managers, many deficiencies were found in the education and training services of the
BST program at the Merchant Marine Polytechnic. One of these shortcomings is the
schedule of teaching and learning activities in the classroom and field practice, which are
arranged in an unstructured or systematic manner. One schedule often collides with
another due to inconsistent lesson times. The problem is indicated due to the limited
number of lecturers and inadequate facilities. Other findings indicate that the planning,
implementation, and evaluation of education and training are not prepared based on
management principles, so the implementation is less effective. From the results of the
interview with the Head of the Seafarers' Education and Training Unit, the learning
focuses on conveying knowledge verbally, and it has not yet reached the stage of giving

AACL Bioflux, 2022, Volume 15, Issue 1.

http://www.bioflux.com.ro/aacl
557
training participants the opportunity to develop their own insights and judgment when
they face critical situations. That means that the learning methods are not sufficient in
attaining their objectives.
The CIPP (Context, Input, Process, Product) evaluation model offers a more
complex application since it covers both formative and summative evaluations. The
results of the context, input, process, and product evaluation will be useful in making
decisions (formative) and create accountable information (summative). Hence, it will add
more various strategic options applied by institutions in conducting educational programs
(Mahmudi 2011). Two CIPP evaluation results register a conventional count of 91% and a
computational count of 91.6%. Based on the analysis, the Computer-Assisted CIPP
evaluation model is able to calculate more quickly and accurately than conventional
models, although the difference is not significant. Therefore, further empirical research is
needed to discuss the controversy (Gede & Divayana 2015).
Correct safety training is beneficial to reduce the risk of work injuries that may
occur anytime and anywhere. Hence, every ship's crew is required to join safety training
(Waehrer & Miller 2009). In the findings of the effectiveness of BST on safety, safety
training is efficacious in the sailing world. However, it needs to be supported by
appropriate learning methods and models so that they can easily practice it while
working. With the support of technology, learning becomes easier and increases the
training participant literacy to understand the material (Buted et al 2014). The evaluation
of modification of BST as a response to a disaster, shows that this training has proven to
play an essential role in public awareness, and it is suggested to improve it. Communities
having experienced BST have been proven to be more alert and responsive in dealing
with disasters (Pateña et al 2016).
Context Input Process Product (CIPP) is an Information Service Evaluation Model.
It is the answer to solve problems in the evaluation of services. The CIPP evaluation
model was designed to overcome problems in assessing different services. With CIPP,
information services can be evaluated comprehensively (Muyana et al 2017).

Management of education and training. Massie (1985) revealed that management is

a process of forming cooperation in a group, which later creates a common goal. Sisk
(1969) asserted that management is the communication of various resources developed
from the planning, organizing, and controlling processes to achieve certain intentions.
Based on these ideas, management is defined as a process for solving problems with
effective methods to achieve organizational goals and objectives by employing existing
resources efficiently. Management consists of many aspects such as control,
coordination, motivation, design processes, development, and the influence of
organizational goals and resources in achieving goals (Karnani 2009). Administrative
experts such as Sergiovanni, Coombs, and Thurson stated that management is the
process of working with and through others to accomplish organizational goals efficiently
(Bafadal 2003). Based on the definitions above, management objectives always discuss
some aspects, including a) cooperation, b) conducted by two or more people, and c)
achieving common goals. Therefore, management is a process involving many people
effectively and efficiently to attain organizational purposes. Management in an
organization functions as an attempt to carry out certain activities, so its main objectives
are achieved productively and efficiently.

Basic Safety Training. This skills, education and training aim to ensure that participants
have basic competency and safety skills while sailing. Participants must practice
emergency rescue as an indicator of mastery of skills. They have the competency to
prevent fires as well as to extinguish them. The participants are also expected to
understand several types of emergencies, occupational health, and safety (K3). Apart
from the indicators above, there are other points such as medical care skills, self-rescue
during an emergency, preventing marine pollution, and maintaining social communication
on board. This BST also enables participants to comprehend the basic knowledge of
safety and to practice it to prevent and overcome problems while sailing. Accordingly,
training participants are not only proficient in theory, being also good practitioners.

AACL Bioflux, 2022, Volume 15, Issue 1.

http://www.bioflux.com.ro/aacl
558
Material and Method. This research was conducted at the Merchant Marine Polytechnic,
Makassar, Indonesia, in 2021. It is an evaluative research on research and development.
The study population is represented by training organizers, totaling 86 people. The data
were collected through questionnaires and interviews. Before the questionnaire was
submitted to the respondent, the instrument was passed through the validity and
reliability stages. Valid data in the validity test results was used for research. Two items
of data were found to be invalid during the validity test. Reliability test using Cronbach
showed a context of 0.807, input of 0.815, and product of 0.907. In addition, all CIPP
components were also reliable with a Cronbach Alpha value greater than 0.60.
To analyze the data, descriptive statistical methods were used. The methods were
then combined with the CIPP, to evaluate the implementation of the program at the
Merchant Marine Polytechnic. The next step was selecting the education and training
model through the Planning, Implementation, and Control approach. This determined the
appropriate model for the Merchant Marine Polytechnic and could be useful as a training
development strategy.
This study intends to obtain a concrete picture of the BST implementation through
the CIPP approach. The development research also reports in detail the behavior of
training participants, training activities, and exploration to determine the proper method
and development. The process also utilizes the CIPP components. After the event was
thoroughly understood, the next step was exploring the research subject to check the
overall situation in everyday life.
Stufflebeam & Zhang (2017) emphasized that policymakers must make decisions.
The context in CIPP means conducting research based on needs, problems, and
opportunities to determine the target, and to ensure the quality of graduates. Input plays
a role in planning and coordination in maximizing human resources. The process leads to
the implementation of planning, and the product serves to identify and assess its
effectiveness (Stufflebeam & Zhang 2017).
In this study, there was parametric statistical test conducted (t-test) to determine
the level of effectiveness of the model using IBM SPSS Statistics 24. The effectiveness of
the model was then tested using a t-test and the average increase from the pre-test to
the post-test results. Calculation of Gain index was carried out to determine the increase
after the implementation of CIPP management in BST. Gain test is used to determine
level of effectiveness of a treatment on expected results, which in this study was CIPP
management model in BST. The Gain index was calculated by the following formula:

Gain index = (Post-test score - Pre-test score)/(Max score - Pre-test score)

The criteria for interpreting N-Gain results are presented in Table 1 (Meltzer 2002).

Table 1
Gain Index Criteria

No Index value Criteria

1 g≤0.3 Low
2 0.3>g>0.7 Moderate
3 g0.7 High

Results and Discussion. Table 2 shows that pre-test score before implementation of
CIPP management model with correct use and procedure with management functions
starting from planning, implementation to evaluation obtained a mean of 73.7442, with a
standard deviation of 4.88259. Furthermore, result of post-test analysis after the
implementation of the CIPP management model in learning obtained a mean of 50.5698,
with a standard deviation of 3.80878. It can be said that the implementation of the CIPP
management model that has been developed has resulted in improvements and is in
accordance with the objectives of this study. Description results with numerical score are
presented in detail in Table 2. Descriptions of pre-test and post-test analysis of the CIPP
management score are presented in detail in Table 3. Paired statistics of pre-test and

AACL Bioflux, 2022, Volume 15, Issue 1.

http://www.bioflux.com.ro/aacl
559
post-test of implementation of the CIPP management model are presented in detail in
Table 4.

Table 2
Results of pre-test and post-test analyses of the CIPP management model

Descriptive Statistics
N Minimum Maximum Mean Std. Deviation
Pre Test 86 42 64 50.5698 3.80878
Post Test 86 63 93 73.7442 4.88259
Valid N
86
(listwise)

Table 3
Description of pre-test and post-test analysis of the CIPP management score

No. Score range Pre Test Post Test

1 42-49 36 0
2 50-57 45 0
3 58-65 5 2
4 66-73 0 46
5 74-81 0 33
6 82-89 0 3
7 90-97 0 2
Total 86 86
Min 42 63
Max 64 93

Table 2 shows that the maximum mean score of implementing the CIPP management
model in the pre-test was 50.56, while the mean score in the post-test was 73.74.
Accordingly, there was an increase by 23.18.
Before the implementation of the CIPP management model in BST, the highest
score obtained was at intervals 50-57 (52%), which was in the fairly good category.
Meanwhile, after the implementation of the CIPP management model in BST at Merchant
Marine Polytechnic, the highest score obtained was at intervals 66–73 (54%), which was
in the good category.

Table 4
Paired statistics of pre-test and post-test of implementation of the CIPP management
model

Paired samples statistics

Mean N Std. Deviation Std. Error Mean
Post Test 73.7442 86 4.88259 0.52650
Pair 1
Pre Test 50.5698 86 3.80878 0.41071

Table 5 explains that the CIPP management model used in BST for this study was good,
indicated by a correlation coefficient of 0.895. The results of the pre-test before
implementation and after implementation of the CIPP management model had a positive
and significant correlation. In addition, computer output Sig. was 0.000, smaller than
alpha 0.05. It demonstrates that the created CIPP management model in BST was good
and effective with pre-test and post-test results having a positive and significant
correlation. A better implementation of the CIPP in BST brings better learning outcomes.

AACL Bioflux, 2022, Volume 15, Issue 1.

http://www.bioflux.com.ro/aacl
560
 Table 5
Correlation analysis of pre-test and post-test of implementation of the CIPP management
model

Paired samples correlations

N Correlation Sig.
Pair 1 Post-test & Pre-test 86 0.895 0.000

Pre-test results showed that the mean value of CIPP management model in BST was
50.56, while the result of the post-test of implementation of the CIPP management
model in BST was 73.74, indicating that there was an increase after the implementation
of the CIPP management model. The level of effectiveness of the model applied in this
study based on the Gain Index was 0.799. Based on the Gain Index, the effectiveness of
the CIPP management model in BST implemented in Merchant Marine Polytechnic was in
the high category. Accordingly, the CIPP management model in BST developed was
effective in improving basic safety training skills at Merchant Marine Polytechnic with a
high level of effectiveness. Results of t-test on pre-test and post-test results of CIPP
model in BST are presented in Table 6.

Table 6
Results of t-test

Paired samples test

Paired Differences
95% confidence
Std. Std. error Sig. (2-
Mean interval
deviation of mean t df tailed)
Lower Upper
Post-
Pair
test 23.1744 2.25019 0.24264 22.69198 23.65686 95.508 85 0.000
1
Pre-test

Results of the t-test show that the t value was 95.508 with a significance level under
0.000 (p0.05), meaning that there was a significant difference between the average
score of the pre-test and the post-test. Therefore, it can be concluded that the CIPP
model is effective in improving management in BST at Merchant Marine Polytechnic. In
summary, the effectiveness test can be demonstrated by the following:
1. The mean score of the post-test was higher than mean score of the pre-test
(an increase of 23.17%).
2. The correlation coefficient of the pre-test and post-test was 0.895, meaning
that there was a significant correlation.
3. The result of N-Gain was 0.799, in the high category, meaning that the
implementation of the CIPP application system model was good and effective in
improving learning.
4. The result of the t-test was significant, meaning that the model was effective.

Conclusions. The BST implemented at the Merchant Marine Polytechnic using the CIPP
model is in a good category. The mean score of the post-test was higher than the mean
score of the pre-test with 23.17%. The correlation between pre-test and post-test results
was significant. The N-Gain Index was in the high category, meaning that the
implementation of the CIPP application system model was good and effective in
improving learning. The results of the t-test showed that the model was effective.
Overall, the implementation of BST at the Merchant Marine Polytechnic using the CIPP
model is good. Based on the research results, the researchers suggest to use the
application for registration, payment, and sharing of training schedules online, to
optimize the use of learning media and information technology for training
implementation in the classroom, and to emit the certificate online and deliver it timely.

AACL Bioflux, 2022, Volume 15, Issue 1.

http://www.bioflux.com.ro/aacl
561
Conflict of Interest. The author declares that there is no conflict of interest.

References

Bafadal I., 2003 [Quality management for elementary school improvement, from
centralization to decentralization]. Bumi Aksara, 102 p. [In Indonesian].
Buted D. R., Felicen M. S. S., Felicen M. S. S., Macatangay J. E. G., Andal J. F., Pangpang
K. N. R., Suayan M. C. V., Pana R. V., Leon J. D. De., 2014 Effectiveness of basic
safety training among cruise line students. Asia Pacific Journal of Multidisciplinary
Research 2(3):161-169.
Gede D., Divayana H., 2015 [Evaluation of HIV/AIDS control program using the computer
assisted CIPP model]. Konferensi Nasional Sistem & Informatika, Denpasar, Bali,
pp. 9-10. [In Indonesian].
Karnani A., 2009 The bottom of the pyramid strategy for reducing poverty: A failed
promise. DESA Working Paper No. 80, ST/ESA/2009/DWP/80, 14 p.
Mahmudi I., 2011 [CIPP: An educational program evaluation model]. Jurnal At-Ta’dib
6(1):111-125. [In Indonesian].
Mamahit B. R., 2014 [77.727 seafarers work on foreign ships]. Available at:
https://beritatrans.com/2014/05/01/dirjen-bobby-77-727-pelaut-bekerja-di-kapal-
asing/. [In Indonesian].
Massie J. L., 1985 Essentials of management. Prentice-Hall of India, New Delhi, 262 p.
Muyana S., Ahmad U., Yogyakarta D., 2017 [Context Input Process Product (CIPP):
Evaluation model]. Prosiding Seminar Bimbingan dan Konseling 1(1):342-347. [In
Indonesian].
Pateña A. D., Orence A., Mandigma L., 2016 Modified basic safety training with typhoon
awareness as a response to disaster preparedness. Asia Pacific Journal of
Education, Arts and Sciences 3(2):82-86.
Sisk H. L., 1969 Principles of management : A systems approach to the management
process. South-Western Publishing, 672 p.
Stufflebeam D. L., Zhang G., 2017 The CIPP evaluation model. How to evaluate for
improvement and accountability. Guilford Publications, 384 p.
Waehrer G. M., Miller T. R., 2009 Does safety training reduce work injury in the United
States? The Ergonomics Open Journal 2(1):26-39.
*** IMO, 2000 Model course 1.19. Proficiency in personal survival techniques.
International Maritime Organization, London, 46 p.

Received: 06 October 2021. Accepted: 08 January 2022. Published online: 28 February 2022.
Author:
Darul Prayogo, Merchant Marine Polytechnic Semarang, 9A Singosari Raya St., Wonodri, 50242 Semarang,
Central Java, Indonesia, e-mail: [email protected]
This is an open-access article distributed under the terms of the Creative Commons Attribution License, which
permits unrestricted use, distribution and reproduction in any medium, provided the original author and source
are credited.
How to cite this article:
Prayogo D., 2022 Evaluation of basic safety training using the CIPP model. AACL Bioflux 15(1):557-562.

AACL Bioflux, 2022, Volume 15, Issue 1.

http://www.bioflux.com.ro/aacl
562