International Journal of Evaluation and Research in Education (IJERE)

Vol. 12, No. 3, September 2023, pp. 1278~1289

ISSN: 2252-8822, DOI: 10.11591/ijere.v12i3.23982  1278

Exploring digital competencies domain and elements for

information technology graduates in Malaysia

Asnidatul Adilah Ismail1, Razali Hassan2

1
Information and Communication Department, Politeknik Balik Pulau, Pulau Pinang, Malaysia
2
Faculty of Technical and Vocational Education, Universiti Tun Hussein Onn Malaysia, Parit Raja, Malaysia

Article Info ABSTRACT

Article history: This paper explores the reliability of an instrument to evaluate digital
competencies domain and elements for polytechnics’ information
Received Feb 22, 2022 technology (IT) graduates in Malaysia towards future industrial revolution
Revised Jun 1, 2023 using exploratory factor analysis (EFA). This study thereby obtained 146
Accepted Jun 19, 2023 items from the previous phase (industries experts’ interview) which was later
developed according to the study’s objectives. Experts validated the items,
and after that, a pilot study was executed with 102 randomly chosen
Keywords: Polytechnics’ IT lecturers from four Malaysian Polytechnics as the
respondents for this study. Four domains had been decided which are
Digital competency Personal Effective Competencies, Functional Competencies, Essential
Industrial revolution Competencies, and Industrial Revolution 4.0 (4IR) Competencies. Elements
Information technology for each domain were created using the EFA, in which internal reliability
Polytechnics was achieved for all construct dimensions. There were 15 elements gained
through EFA for those four domains.
This is an open access article under the CC BY-SA license.

Corresponding Author:
Asnidatul Adilah Ismail
Information and Communication Department, Politeknik Balik Pulau
Pinang Nirai, 11000, Pulau Pinang, Malaysia
Email: [email protected]

1. INTRODUCTION
The new paradigm of education involves emphasizing how teachers teach and how students learn,
which are significantly emphasized in the education curriculum and teaching [1]. Advanced competences,
technology-based learning transformations, and non-cognitive abilities are all required in the current learning
process [2]. The development of information and communication technologies (ICT) has an intense effect on all
aspects of technology, especially when there is a speed in new features. Therefore, each country’s preparedness
level must be increased to meet the ever-changing demands of the world’s social, cultural, and economic
priorities [3]. These new technologies are used by people all over the world in their everyday lives for better
living conditions.
As technology evolves, more industries adopt these new technologies as the specifications of their jobs
are evolving with technology change [4], [5]. The development of technology in ICT is currently related to the
rise of the current industrial revolution, known as the Industrial Revolution 4.0 (IR4.0). Germany began
implementing the National Industry 4.0 Plan in April 2013 in line with the fact that manufacturing is their major
economic activity [4]. The Industrial Revolution 4.0 (IR4.0) is considered as a transformation from an agrarian
economy to industrial manufacturing, leading to a new economic pattern [6]. However, in 2015 IR4.0 has been
highlighted as the use of new technologies and is more relevant to adapting different work systems with a
systematic technical and practical operational level [7].

Journal homepage: http://ijere.iaescore.com

Int J Eval & Res Educ ISSN: 2252-8822  1279

This revolution undoubtedly affects the job specification and graduates’ skills in information
technology because most innovations and new technologies involve the use of digital business and ICT in their
transitions. Competence is the ability to put information into practice in order to act, exhibiting individual
competence in the process [8]. ICT has become prominent in modern societies as a tool in reforming education
systems, fostering economic growth through creating new goods and services, as well as providing access to
knowledge and expertise to promote changes in agriculture, health, and education. The use of ICT also links
communities and teachers [9]. This reform significantly affects many country's higher education system as the
higher education institution is the industry's provider [10].
The curriculum, skills of education, literacy, and education methods are evolving as ICT offers
teachers and students the tools for a more profound knowledge base [9], [11]. However, since IR4.0 is the main
topic among the industries experts, the most highlighted issue is that current workers and graduates are not
competent enough to meet the IR4.0 skills demand [5], [12]. The employees or jobseekers must keep up with
the rapid changes in workplace environment that is transitioning towards digital technology, as employers
demand wide variety of skills to fulfil current job expectations [13]. According to the Malaysia’s Ministry of
Higher Education through Graduate Tracing Survey, the percentage of polytechnic graduates who are employed
increased to 83.9% in 2018 [3] compared to 62.1% in 2015 [7]. However, beside the increase in employability
percentage, sadly not all graduates manage to secure jobs accordingly to the field of their studies. Based on
previous study, most of graduates do not get to work in their study field including IT graduates [3]. Based on
that research too, 78.5% of Information and Communication Departments graduates from the polytechnic in that
study do not work in the IT field. This situation is contradicted to the current analysis which highlights that ICT
is among the most desired course of study in which its graduates are most likely to get jobs [14].
Based on the discussion, this paper mainly exploring the type of IT graduates that should be developed
by Malaysian Polytechnics to meet the market needs of the industry and how far IT graduates from Malaysian
Polytechnics will meet the industry’s digital competencies in the future. Therefore, the purpose of this paper is
to explore digital competencies elements for each of digital competency’s domain for Malaysian Polytechnics’
IT Graduates towards Future Industrial Revolution.

2. RESEARCH METHOD
This study employs quantitative approach by using questionnaire to conduct survey as the method of
gathering data. The population for this study includes all lecturers of the information technology and
communication department (JTMK) in the Malaysian polytechnics. JTMK is chosen because in the IR4.0
transition, ICT has become one of the critical industries not only to IT industry but also to other sectors. This
is due to wide implementation of the trend in digitization, automation, and the use of ICT in various
industries [3]. Therefore, IT students need to have particular skills to ensure that they are competitive to be
hired during the era of IR4.0. The instrument is developed by researchers through the input from ICT
industry’s experts through three rounds of Modified Delphi. Next, in order to provide reliability and validity
of the instrument, the questions were thoroughly assessed, where experts in the field were consulted for this
reason. The validation of the instrument was done by experts from industries during Modified Delphi round,
supervisor, associate professors from IT field, those who have experience in IT field more than seven years,
language experts that have been certified, and also statistician experts to validate the questions before
proceeding to the next steps.
Next, the revised questionnaires were distributed to four random polytechnics to gather data. As this
study purposely aims to explore the elements for each of domains or components, Exploratory Factor
Analysis (EFA) is chosen to analyze the dimensions of the items and to measure the construct using the data
acquired. EFA is used to each construct to identify whether the items create different dimensions from prior
research. New dimensions are expected in the current study [15]–[17]. EFA is used widely in getting the
theme or component for each construct as used by previous researchers [16]. Preferred minimum sample size
for EFA to obtain valid results is 100 [8], [18]. Then, instruments were distributed to the polytechnic’s IT
lecturers from Information and Technology Department (JTMK) as: i) Politeknik Metro Kuala Lumpur
(PMKL); ii) Politeknik Metro Tasek Gelugor (PMTG); iii) Politeknik Seberang Perai (PSP); iv) Politeknik
Sultan Abdul Halim Muadzam Shah (PMS); and v) Politeknik Besut Terengganu (PBT).
Few criteria have been set for the respondents such as the respondents need to have at least five
years of experience teaching IT students in polytechnics to ensure the reliability of the study as the lecturers
need to give their overall perspective of digital competencies amongst their students based on their
experience. All of the respondents are chosen using simple random sampling method where the
questionnaires were distributed using the Google Form link as the medium of data collection. The link was
sent to Research and Innovation Officer for each polytechnic. Following that, a total of 102 questionnaires
were considered to be valid to be analyzed.

Exploring digital competencies domain and elements for information technology … (Asnidatul Adilah Ismail)
1280  ISSN: 2252-8822

The 146 items are included in the questionnaire using five-point interval Likert scale because
5-point Likert scale is less confusing and increases response rate [19], [20]. For example, the score of 1
represents “lowest competency level,” whereas 5 represents “highest competency level.” The respondents
were also asked to provide demographic details such as gender, years of service, highest academic
qualification, duration of industry attachment, and working experience in the industry.

3. RESULTS AND DISCUSSION

Exploratory Factor Analysis (EFA) was conducted for all constructs to test the dimensionality items
as this study developed the questionnaire based on previous phase. EFA reduces a big number of factors into
a manageable amount and sets the dimensions underlying measured factors to latent constructs, enabling the
development and improvement of the theory [15], [21], [22]. The EFA process includes the mean score and
standard deviation for each item measured by Kaiser-Meijer-Olkin (KMO) sampling adequacy. The values of
the Bartlett test are taken and measured based on p-value. Bartlett test is considered significant when
p-value<0.05 [22]–[24]. Total variance clarified for each construct must exceed 60% the minimum
requirement, with the factor loading exceeding 0.6 [22], [23] for all items. The items' dimension is based on
their components. Lastly, Cronbach’s alpha score is also measured to demonstrate test and scale or known as
internal reliability of the construct [23]–[25].

3.1. EFA for personal effectiveness competencies (PEC) construct

The construct for personal effectiveness competencies (PEC) was represented by 49 items in the
questionnaire for this study. However, after being analyzed, 27 elements did not fulfil the criteria in EFA.
Therefore, all 27 items are dropped from the construct. The standard deviation was computed to further
analyze the data distribution [26]. The standard deviation determines the normal distribution of the data based
on the error and variance figures to estimate the mean [25], [27], [28]. The mean and standard deviation cuts
for each item are shown in Table 1 that indicates the lowest mean value (M=2.57).

Table 1. The mean and standard deviation for every item measuring PEC
Mean Std. Deviation
PE1 Modulating their voice when speaking to suit the listener or audience 3.10 1.30
PE2 Communicate interpersonally in a familiar work situation 3.01 1.42
PE3 Confident in delivering the presentation 2.91 1.21
PE4 Expressing ideas concisely 2.81 1.15
PE5 Communicate well in English 2.81 1.40
PE6 Start communicating with confident 2.81 1.16
PE7 Overcome issues while communicating with others 2.91 1.23
PE8 Overcome issues during the presentation 2.81 1.20
PE9 Communicate with respect and politely to higher level such as head of department, 3.11 1.33
head of courses and lecturers.
PE10 Show appropriate facial expression in explanation 3.07 1.32
PE11 Respect diversity, individual differences, and perspective 3.17 1.35
PE12 Demonstrate commitment in task given 3.07 1.33
PE13 Know how to restrain themselves in a certain amount of pressure 3.07 1.42
PE14 Show professional attitude when handling task 3.20 1.37
PE15 Identify obstacles to effective teamwork 3.37 1.30
PE16 Delegate work fairly among teammate 3.10 1.32
PE17 Work independently in gaining knowledge 2.92 1.38
PE18 Independently finding new sources to finish the task 3.20 1.40
PE19 Show full interest in their work 3.18 1.35
PE20 Show effort while finishing their assignment 3.08 1.32
PE21 Have skills of making decisions 2.57 1.15

Next, the principal analysis method component (PCA) with Varimax Rotation was performed.
Table 2 shows that the value of the Bartlett test is significant (p-value<0.05). The KMO adequacy measure
value is 0.927, which is above the minimum value of 0.6, indicating that Bartlett test is significant [26], [29],
[30]. The KMO value shows that the value was greater than 0.6 and close to 1.0 and Bartlett’s test
significance value close to 0.0 indicates that the data is adequate and appropriate for next reduction
procedure [23], [24].

Int J Eval & Res Educ, Vol. 12, No. 3, September 2023: 1278-1289
Int J Eval & Res Educ ISSN: 2252-8822  1281

Table 2. The value for KMO and Bartlett’s test for PEC construct
Test value
Kaiser-Meyer-Olkin Measure of Sampling Adequacy .942
Approx. Chi-Square 2571.595
Bartlett’s Test of Sphericity df 253
Sig. .000

Total variance explained is also an extraction procedure to reduce the items to the reasonable
number before further analysis [24], [31]. This process will divide the components with eigenvalues
exceeding 1.0 and being extracted to different components as shown in Table 3 [23]. Based on the table, the
PEC construct is measured using three components where component 1 measures a construct of 31.740%,
component 2 measures a construct of 24.676%, and component 3 measures a construct of 20.318%. The
amount of budget variance for the personal effectiveness competencies construct is 76.734% which exceeds
the minimum requirement of 60%.

Table 3. Total variance explained for PEC construct

Extraction sums of squared loadings Rotation sums of squared loadings
Component
Total % of variance Cumulative % Total % of variance Cumulative %
1 14.899 64.780 64.780 7.300 31.740 31.740
2 1.496 6.504 71.284 5.676 24.676 56.417
3 1.253 5.450 76.734 4.673 20.318 76.734

Next, the result in Table 4 shows that the distribution of items for the three components for PEC
constructs. The component has been grouped to certain number of items with their respective factor loading.
In this process, only item that exceed factor loading of 0.6 will be retained [9], [30], [31]. According to that,
from 49 items, 27 items are dropped as the weighting values factor is below 0.6. The remaining items are
grouped based on the component as shown in Table 4.

Table 4. The PEC component and their respective items

Component
1 2 3
PR1 .667
PR2 .721
PR3 .749
PR4 .753
PR5 .807
PR6 .839
PR7 .788
PR8 .775
PR9 .694
IC1 .667
IC2 .790
IC3 .783
IC4 .758
IC5 .794
IC6 .695
IC7 .725
PS1 .713
PS2 .676
PS3 .688
PS4 .688
PS5 .810

Reliability analysis is a procedure used to measure the items under each construct and determined
the degree of the error [15]. The measure of reliability instruments is estimated through Cronbach’s alpha
values [23], [26]. The Cronbach’s alpha value of an instrument must exceed the minimum limit of 0.7 to be
used in the next phase [24], [29]. PEC construct has indicated that professional as the component 1 followed
by component 2 which independent and confident and third component which is passion. According to
Table 5, the Cronbach’s alpha for each component is computed and possesses a high reliability standard as
0.968, 0.953 and 0.87 for component 1, component 2 and component 3 respectively. Therefore, these three
components are recommended to be used for measuring the PEC construct in the next analysis.

Exploring digital competencies domain and elements for information technology … (Asnidatul Adilah Ismail)
1282  ISSN: 2252-8822

Table 5. The internal reliability for the PEC construct

Component N of items Cronbach’s alpha
1. Professional 9 0.968
2. Independent and confident 7 0.953
3. Passion 5 0.870
All items 21 0.972

3.2. EFA for functional competencies (FC) construct

Next, the same process need have been done to the next construct. There were 40 items represent the
construct for functional competencies (FC), and after being analyzed, 24 items did not fulfil the criteria in
EFA which represent the value of factor loading were below than 0.6 [30], [32]. Therefore, all these 24 items
are dropped from the construct. Table 6 have shown the descriptive analysis, that includes the value, means
and standard deviations of the 26 extracted factors. The lowest mean identifies basic compatibility issues
between hardware components (M=2.81) while describe the purposes of software or system testing has the
highest mean value (M=3.49). Other 24 items’ mean value score and standard deviation for each item’s
measured constructs are shown in Table 6.

Table 6. The mean and standard deviation for every item measuring FC
Mean Std. Deviation
FC1 Enter, edit and organize structured information in a database 2.92 1.26
FC2 Create database tables 3.21 1.48
FC3 Modify non-relational database tables 2.91 1.37
FC4 Understand the different hardware and software components that may be used to implement a network 2.84 1.35
FC5 Explore the basic constructs of object-oriented programs in relation to manipulation of data objects 3.26 1.27
FC6 Describe specified data protection methods 2.97 1.24
FC7 Use specified security tools to identify and prevent breaches of security 2.98 1.19
FC8 Explore different life systems and life cycles' models 2.94 1.24
FC9 Understand the importance of effective planning before coding and testing before implementation 3.43 1.26
FC10 Explore different system testing techniques 3.21 1.25
FC11 Describe relevant parts of the testing process in developing system 3.18 1.27
FC12 Describe the purposes of software or system testing 3.49 1.30
FC13 Interpret specified technical information about the test 2.93 1.27
FC14 Gather and record relevant test information and test results 2.99 1.31
FC15 Demonstrate the proper use of hardware devices 2.74 1.33
FC16 Identify basic compatibility issues between hardware components 2.81 1.28
FC17 Recognize common operational problems caused by hardware 2.88 1.26
FC18 Explain the function and purpose of software tools 3.13 1.20
FC19 Troubleshoot computer components and peripherals 2.93 1.13
FC20 Troubleshoot operating systems 3.23 1.43
FC21 Troubleshoot networks 3.13 1.34
FC22 Code using at least one basic programming language 3.40 1.31
FC23 Think like a computer with the logic of programming 3.22 1.26
FC24 Describe fundamentals of programming 3.03 1.08
FC25 Describe application lifecycle management 3.03 1.05
FC26 Describe object-oriented programming 3.23 1.33

Next, Table 7 has shown the value of the KMO for the functional competencies construct was 0.927.
The KMO value was higher than the threshold value of 0.6 as per suggested by previous researcher [20],
[22], [26]. The Bartlett’s Test of Sphericity was also significant where the value of Chi-square=2863.62 and
p-value <0.001. The KMO value for FC construct also shows that the value was greater than 0.6 and close to
1.0 and Bartlett’s test significance value close to 0.0 indicates that data for FC construct with 24 items is
adequate and appropriate for next reduction procedure [2], [33].

Table 7. The value for KMO and Bartlett’s test for FC construct
Test value
Kaiser-Meyer-Olkin Measure of Sampling Adequacy .927
Approx. Chi-Square 2863.620
Bartlett’s Test of Sphericity df 325
Sig. .000

Int J Eval & Res Educ, Vol. 12, No. 3, September 2023: 1278-1289
Int J Eval & Res Educ ISSN: 2252-8822  1283

Table 8 shows the total variance explained for FC construct. As illustrated in Table 8, FC construct
is measured using four components where components 1 measures a construct of 26.056%, component 2
measures 19.102%, component 3 measures a construct of 16.208% and lastly, component 4 measures a
construct of 15.311%. Therefore, the amount of budget variance for the FC construct is 76.676% which
exceeds the minimum requirement of 60% [15], [31].

Table 8. Total variance explained for FC construct

Extraction sums of squared loadings Rotation sums of squared loadings
Component
Total % of variance Cumulative % Total % of variance Cumulative %
1 15.070 57.960 57.960 6.774 26.056 26.056
2 1.954 7.515 65.475 4.966 19.102 45.157
3 1.539 5.919 71.394 4.214 16.208 61.365
4 1.373 5.282 76.676 3.981 15.311 76.676

The rotated component matrix results for the FC construct suggested a four-component solution as
shown in Table 9. The 26 items distribute neatly into four components that measure functional competencies.
shows the distribution of items for the four components of FC constructs. These 26 items show in Table 9
have factor loading greater than 0.6.

Table 9. FC component and their respective items

Component
1 2 3 4
DS1 .626
DS2 .729
DS3 .689
DS4 .715
DS5 .741
DS6 .824
DS7 .809
DS8 .697
DS9 .681
SD1 .699
SD2 .713
SD3 .745
SD4 .746
SD5 .688
SD6 .654
PT1 .614
PT2 .715
PT3 .655
PT4 .617
PT8 .746
PT9 .644
HS1 .682
HS2 .764
HS3 .668
HS4 .639
HS5 .640

The reliability estimates for the FC constructs are presented in Table 10. The Cronbach’s alpha
values range between 0.875 to 0.955 for four component of FC construct which represent that the construct is
strongly reliable and can be accepted. All components have Cronbach’s alpha value exceeding the minimum
value of 0.7 which indicates that they are good to be used for the next phase [15], [31].

Table 10. The internal reliability for the FC construct

Component N of items Cronbach’s alpha
1. Database, Security and System Testing 9 0.955
2. System Analysis and Design 6 0.942
3. Programming and Troubleshoot 6 0.905
4. Hardware and Software 5 0.875
All items 26 0.970

Exploring digital competencies domain and elements for information technology … (Asnidatul Adilah Ismail)
1284  ISSN: 2252-8822

3.3. EFA for essential competencies (EC) construct

The construct for essential competencies (EC) was represented by 19 items in the questionnaire for
this study as shown in Table 11. However, after being analyzed, eight elements did not fulfil the criteria in
EFA [27]. Therefore, all eight items are dropped from the construct. The standard deviation was computed to
further analyze the data distribution [15] which also determines the normal distribution of the data based on
the error and variance figures to estimate the mean [29]. The mean and standard deviation cuts for each item
are shown in Table 11 that indicates the lowest mean value (M=3.72).
Next, the principal analysis method component (PCA) with Varimax Rotation was performed.
Table 12 shows that the value of the Bartlett test is significant (p-value<0.05). The Kaiser-Meyer-Olkin’s
(KMO) adequacy measure value is 0.717, which is above the minimum value of 0.6, indicating that Bartlett
test is significant [30]–[32] and data is adequate and appropriate for next reduction procedure [28].

Table 11. The mean and standard deviation for every item measuring EC
Mean Std. Deviation
EC1 Explain the reason in choosing the solution for their decision in solving the problem 3.93 1.01
EC2 Describe the characteristics of storage devices 3.67 1.25
EC3 Explain the functions of storage devices 3.79 0.68
EC4 Explain the functions of peripheral devices 4.03 0.70
EC5 Explain the functions of core input devices 3.75 0.80
EC6 Assemble the computer 3.69 0.88
EC7 Boot the computer 4.16 0.85
EC8 Install the operating systems 3.77 1.00
EC9 Install computer software and essential hardware requirement 3.76 1.19
EC10 Utilize office effectively (Word, Excel, Power Point) 3.72 0.92
EC11 Able to make good presentation slide 4.27 0.75

Table 12. The value for KMO and bartlett’s test for EC construct
Test value
Kaiser-Meyer-Olkin Measure of Sampling Adequacy .717
Approx. Chi-Square 755.017
Bartlett’s Test of Sphericity df 55
Sig. .000

In the context of the study, the measurement of the EC construct is presented in Table 12. This
construct is assessed using three distinct components, each contributing a certain percentage to the overall
measurement. Component 1 accounts for 26.056% of the construct, component 2 represents 19.102%,
component 3 contributes 16.208%, and finally, component 4 measures 15.311% of the construct. The
construct variance for the EC construct is discussed in Table 13, where it is revealed to be 76.676%. This
percentage indicates the extent to which the actual measurement deviates from the expected value for the EC
construct. In this case, a variance of 76.676% is considered favorable, as it surpasses the minimum
requirement of 60% [34].

Table 13. Total variance explained for EC construct

Extraction sums of squared loadings Rotation sums of squared loadings
Component
Total % of variance Cumulative % Total % of variance Cumulative %
1 4.598 41.798 41.798 2.811 25.552 25.552
2 2.297 20.886 62.684 2.749 24.994 50.546
3 1.195 10.860 73.544 2.530 22.998 73.544

There were three components of EC construct measured. Table 14 shows the distribution of items
for the three components in EC constructs. Only 11 items are accepted as the factor loading exceeds the
minimum limit of 0.6 [30]. The items are grouped based on the component as shown in Table 14. Based on
Table 14, the components than being analyzed and being named according to items themed. Table 15 shows
Cronbach’s alpha values for three components in EC construct. All three components, technical computing
(0.868), initializing computer and storage (0.715) and computer-related hardware (0.856) have the
Cronbach’s alpha value exceeding the minimum value of 0.7 [35].

Int J Eval & Res Educ, Vol. 12, No. 3, September 2023: 1278-1289
Int J Eval & Res Educ ISSN: 2252-8822  1285

Table 14. The EC component and their respective items

Component
1 2 3
TC1 .641
TC2 .797
TC3 .688
TC4 .923
IC1 .627
IC2 .702
IC3 .794
IC4 .601
CR1 .780
CR2 .915
CR3 .877

Table 15. The internal reliability for the EC construct

Component N of Items Cronbach’s alpha
1. Technical computing 4 0.868
2. Initializing computer and storage 4 0.715
3. Computer-related hardware 3 0.856
All items 11 0.850

3.4. EFA for 4IR competencies (4IRC) construct

Last construct that being measured was 4IR competencies (4IRC) which represent by 36 items.
However, after being analyzed, only 19 items fit the EFA criteria which carried the factor loading that more
0.6 [23], [24], [31]. The mean value score and the standard deviation for each item and measured constructs
are shown in Table 16.

Table 16. The mean and standard deviation for every item measuring 4IRC
Mean Std. Deviation
IR1 Identify game platforms 3.58 0.84
IR2 Identify game genres 3.53 0.82
IR3 Describe a game user interface 3.43 1.11
IR4 Describe artificial intelligence (AI) 3.07 1.12
IR5 Differentiate between tool creation and game programming 3.28 1.18
IR6 Explain the basic concepts of cloud computing 3.37 1.09
IR7 Explain technical threats associated with cloud computing 3.31 0.99
IR8 Describe the impact of IoT 3.43 1.10
IR9 Describe cloud computing terminology 3.65 1.17
IR10 Describe general principle and practices of IoT 3.56 1.02
IR11 Distinguish between different types of clouds 4.20 1.26
IR12 Explain the basic concept of virtualization 3.98 1.38
IR13 Explain the basic concept of cybersecurity 3.50 1.06
IR14 Explain how to use cybersecurity 3.33 1.02
IR15 Explain the basic concept of Big Data 3.63 1.17
IR16 Describe the data science in real world 3.63 1.38
IR17 Explain the technique to share Data to work 3.21 1.25
IR18 Apply Data Science with Python 3.26 1.15
IR19 Describe cloud computing architecture 3.49 0.92

Principal analysis method component (PCA) with Varimax Rotation was performed on 36 items
measuring 4IR competencies and shows that the Bartlett test is significant (p-value <0.05). Kaiser-Meyer-Olkin’s
(KMO) value is 0.752 exceeding the minimum value of 0.6 as shown in Table 17. The value shows that the
Bartlett test is significant [26], [30], [32] and data is adequate and appropriate for next reduction procedure [15].

Table 17. The value for KMO and Bartlett’s test for 4IRC construct
Test value
Kaiser-Meyer-Olkin Measure of Sampling Adequacy. .752
Approx. Chi-Square 858.600
Bartlett’s Test of Sphericity df 171
Sig. .000

Exploring digital competencies domain and elements for information technology … (Asnidatul Adilah Ismail)
1286  ISSN: 2252-8822

In the subsequent analysis, Table 18 presents the measurement of the 4IR competencies construct.
This construct is evaluated using five components, with each component contributing a specific percentage to
the overall measurement. Component 1 represents a construct of 17.281%, component 2 measures a construct
of 12.873%, component 3 accounts for 12.592%, component 4 measures a construct of 12.036%, and finally,
component 5 contributes a construct of 11.229%. Accordingly, the construct variance for the 4IR
Competencies construct is reported to be 66.011% in Table 18. This percentage reflects the extent to which
the actual measurement deviates from the expected value for the 4IR Competencies construct. Notably, a
variance of 66.011% surpasses the minimum requirement of 60% [36], indicating a favorable outcome.
The rotated component matrix results for the 4IRC construct suggested a five-component solution,
as shown in Table 19. From 36 items, 17 items are dropped as the weighting values factor is below 0.6 [32].
The remaining items which other 19 items distribute neatly into five components as shown in Table 19.

Table 18. Total variance explained for 4IRC construct

Extraction sums of squared loadings Rotation sums of squared loadings
Component
Total % of variance Cumulative % Total % of variance Cumulative %
1 5.448 28.675 28.675 3.283 17.281 17.281
2 2.607 13.721 42.396 2.446 12.873 30.153
3 1.657 8.722 51.118 2.393 12.592 42.746
4 1.529 8.047 59.165 2.287 12.036 54.782
5 1.301 6.846 66.011 2.134 11.229 66.011

Table 19. The 4IRC component and their respective items

Component
1 2 3 4 5
GM1 .658
GM2 .651
GM3 .600
GM4 .766
AI1 .748
AI2 .829
AI3 .657
AI4 .603
IO1 .801
IO2 .729
IO3 .828
CC1 .755
CC2 .658
CC3 .705
CD1 .698
CD2 .841
CD3 .601
CD4 .829
CD5 .847

Table 20 shows Cronbach’s alpha values for five components in the 4IRC construct [37]–[39]. The
reliability value range between 0.701 and 0.55, suggesting that the items representing the constructs are
reliable and acceptable because Cronbach’s alpha value exceeding the minimum value of 0.7 [6], [40]. All
the items also reliable to be used for next phase.

Table 20. The internal reliability for the 4IRC construct

Component N of items Cronbach’s alpha
1. Cybersecurity and data science 5 0.855
2. Artificial intelligence 4 0.718
3. Cloud computing 3 0.701
4. Internet of thing 3 0.802
5. Gamification 4 0.703
All items 19 0.853

Int J Eval & Res Educ, Vol. 12, No. 3, September 2023: 1278-1289
Int J Eval & Res Educ ISSN: 2252-8822  1287

3.5. Polytechnics’ IT graduates digital competencies framework

Figure 1 shows the first phase of framework that contains constructs or known as domain, and
second-order constructs which are referred to as elements with certain number of sub-constructs [15], [41],
[42]. Every sub-construct is measured using certain number of measuring items in the questionnaire [15].
This framework consists of four construct which is personal effective competencies (PEC), functional
competencies (FC), essential competencies (EC) and 4IR competencies (4IRC). Each construct was divided
into several component and these components comes from the EFA as per explained in sub-topic 3.1 to 3.4.
As this framework is specifically developed for polytechnics’ IT students and graduates, the framework is
different from previous digital competencies framework that has been proposed by previous researcher to
accommodate research on teachers, lecturers, undergraduates, and school students [43]–[45]. This study’s
framework is more specific and detailed into digital competency of IT students. Therefore, this framework
will be validated using confirmatory factor analysis in the next phase.

Figure 1. The polytechnics’ IT graduates digital competencies framework domain and elements

4. CONCLUSION
This study contributes to the development of a digital competency’s framework, particularly for
Polytechnics’ IT students and graduates. Four domains have been set and each domain has its own respective
elements dimensions from EFA findings. Based on the EFA outcomes, the current study has figured three
components or elements for Personal Effectiveness Competencies domains, which are Professionalism,
Independent and Confident and Passion. These three-components had 0.972 Cronbach’s alpha values for the
Internal Reliability which indicates that these three components are reliable for next phase. Next domain
which is Functional Competencies domain has four elements that are Database, Security and System Testing,
System Analysis and Design, Hardware and Software and Programming and Troubleshoot. These four
components are also reliable to be used in the next phase when the Cronbach’s alpha values for the internal
reliability is 0.970. Third domain, Essential Competencies had produced three components which are
Technical Computing, Initializing Computer and Storage and Computer-related Hardware. These three
components are also reliable based on the Cronbach’s alpha value at 0.850 which is high in internal
reliability. Next, the last domain of 4IR Competencies had produced five elements such as Cybersecurity and
Data Science, Artificial Intelligence, Cloud Computing, Internet of Thing and Game Development. The
internal reliability for these five components is 0.853 Cronbach’s alpha value which indicates that they are
reliable to be used in the next phase. Therefore, based on the obtained results and framework, all of the
considered items are applicable for this study and proposed to be validated in the next phase using
Confirmatory Factor Analysis to see whether the model is fit and reliable to be used in gathering data in
polytechnics.

Exploring digital competencies domain and elements for information technology … (Asnidatul Adilah Ismail)
1288  ISSN: 2252-8822

ACKNOWLEDGEMENTS
The authors would like to thank the Ministry of Higher Education, Malaysia for the sponsorship
under the HLP program, and also thank the Faculty of Technical and Vocational Education, Universiti Tun
Hussein Onn Malaysia and Malaysian polytechnics for their support.

REFERENCES
[1] P. Nuangchalerm, “TPACK in ASEAN perspectives: Case study on Thai pre-service teacher,” International Journal of
Evaluation and Research in Education (IJERE), vol. 9, no. 4, pp. 993–999, 2020, doi: 10.11591/ijere.v9i4.20700.
[2] F. Mutohhari, S. Sutiman, M. Nurtanto, N. Kholifah, and A. Samsudin, “Difficulties in implementing 21st century skills
competence in vocational education learning,” International Journal of Evaluation and Research in Education (IJERE), vol. 10,
no. 4, pp. 1229–1236, 2021, doi: 10.11591/ijere.v10i4.22028.
[3] Malaysia Board of Technologists, “AI: The Future of Cybersecurity,” Techies, Official Bulletin, 5th ed. Malaysia Board of
Technologists, 2018. [Online]. Available: https://www.mbot.org.my/MBOT/files/b6/b61c9f4a-b897-4e8e-a91a-cc9b9ee8df41.pdf
(accessed Mar. 29, 2022).
[4] S. Lazarević and J. Lukić, “The Impact of Information and Communication Technology on Human Resources,” Sinteza 2016 -
International Scientific Conference on ICT and E-Business Related Research, 2016, pp. 369–375, doi: 10.15308/sinteza-2016-
369-375.
[5] N. Z. Othman and S. F. Hussin, “Faktor-Faktor yang Mempengaruhi Tahap Kebolehpasaran Graduan Mengikut Bidang di
Politeknik Mersing,” E-Jurnal Liga Ilmu Serantau 2019: Malaysia Indonesia (LIS 2019), 2019, vol. 1, pp. 277–289.
[6] X. Wang and L. Gao, “Development of Industry 4.0,” When 5G Meets Industry 4.0, Springer, Singapore, 2020, pp. 43–74, doi:
10.1007/978-981-15-6732-2_3.
[7] N. B. Mahadir, N. H. Baharudin, and N. N. Ibrahim, “Digital citizenship skills among undergraduate students in Malaysia: A
preliminary study,” International Journal of Evaluation and Research in Education (IJERE), vol. 10, no. 3, pp. 835–844, 2021,
doi: 10.11591/ijere.v10i3.21277.
[8] J. F. Hair, C. M. Ringle, and M. Sarstedt, “Partial Least Squares: The Better Approach to Structural Equation Modeling?” Long
Range Planning, vol. 45, no. 5–6, pp. 312–319, 2012, doi: 10.1016/j.lrp.2012.09.011.
[9] G. Falloon, “From digital literacy to digital competence: the teacher digital competency (TDC) framework,” Educational
Technology Research and Development, vol. 68, no. 5, pp. 2449–2472, 2020, doi: 10.1007/s11423-020-09767-4.
[10] H. Taherdoost, S. Sahibuddin, and N. Jalaliyoon, “Exploratory Factor Analysis: Concepts and Theory,” in Advances in Applied
and Pure Mathematics, Mathematics and Computers in Science and Engineering Series, WSEAS, 2014, pp. 375–382.
[11] I. M. Zain, “The Collaborative Instructional Design System (CIDS): Visualizing the 21st Century Learning,” Universal Journal
of Educational Research, vol. 5, no. 12, pp. 2259–2266, 2017, doi: 10.13189/ujer.2017.051216.
[12] A. M. Rahmat, A. H. M. Adnan, and N. M. Mohtar, “Hybrid Capabilities and Industry 4.0 Job Demands in Recruiting Malaysian
Graduates,” International Journal of Academic Research in Business and Social Sciences, vol. 11, no. 1, 2021, doi:
10.6007/ijarbss/v11-i1/9000.
[13] S. Pfeiffer, “Robots, industry 4.0 and humans, or why assembly work is more than routine work,” Societies, vol. 6, no. 2, 2016,
doi: 10.3390/soc6020016.
[14] World Economic Forum, “An Agenda for Leaders to Shape the Future of Education, Gender and Work,” World Economic
Forum, 2021. [Online]. Available: https://www.weforum.org/whitepapers/realizing-human-potential-in-the-fourth-industrial-
revolution/ (accessed Mar. 03, 2021).
[15] Z. Awang, SEM Made Simple: A Gentle Approach to Learning Structural Equation Modelling. MPWS Rich Publication, 2015.
[16] A. Lele, Disruptive Technologies for the Militaries and Security, vol. 132. Singapore: Springer Singapore, 2019.
[17] K. S. Taber, “The Use of Cronbach’s Alpha When Developing and Reporting Research Instruments in Science Education,”
Research in Science Education, vol. 48, no. 6, pp. 1273–1296, 2018, doi: 10.1007/s11165-016-9602-2.
[18] World Economic Forum, “The Future of Jobs Report 2020,” 2020. [Online]. Available: https://www.weforum.org/reports/the-
future-of-jobs-report-2020/digest (accessed Jan. 01, 2022).
[19] T. A. B. Yahaya, K. Idris, T. Suandi, and I. A. Ismail, “Adapting instruments and modifying statements: The confirmation
method for the inventory and model for information sharing behavior using social media,” in Management Science Letters, 2018,
vol. 8, no. 5, pp. 271–282, doi: 10.5267/j.msl.2018.4.021.
[20] A. Zainudin, Pendekatan Mudah: SEM (Structural Equation Modelling). MPWS Rich Resources Sdn Bhd (in Malay), 2018.
[21] Malaysia Ministry of Higher Education (MoHE), “Laporan Kajian Pengesanan Graduan 2015,” MoHE, 2021. [Online].
Available: https://www.mohe.gov.my/en/download/publications-journals-and-reports/lkpg/lkpg-2015 (accessed Jan. 31, 2022).
[22] A. M. Abuayyash, Z. Awang, N. A. Amirah, and O. M. A.- Nawwas, “Exploratory Factor Analysis (EFA) and Constructs
Validity for items used to measure the Impact of Human Resource Management Practices on Employees’ Job Satisfaction,” Saudi
Journal of Business and Management Studies (SJBMS), vol. 3, no. 9, pp. 1025–1036, 2018.
[23] V. Kaur, “Research Methodology,” in Knowledge-Based Dynamic Capabilities. Innovation, Technology, and Knowledge
Management. Springer, Cham, 2019, pp. 77–112, doi: 10.1007/978-3-030-21649-8_3.
[24] N. Alias, Z. Awang, and H. Muda, “Policy Implementation Performance of Primary School Leaders in Malaysia: An Exploratory
Factor Analysis,” IIUM Journal of Educational Studies, vol. 7, no. 2, pp. 22–39, 2020, doi: 10.31436/ijes.v7i2.222.
[25] N. I. Jamil, R. Ali, S. N. D. Ahmad, N. A. Yaacob, and N. Mohamed, “Factor Analysis in Identifying Domains of a Learning
Tool Instrument,” Journal of Engineering and Health Sciences, vol. 3, no. 1, pp. 17–24, 2019, [Online]. Available:
https://kuim.edu.my/journal/index.php/JEHS/article/view/584.
[26] M. I. Alkhawaja, M. Sobihah, and Z. Awang, “Exploring and Developing an Instrument for Measuring System Quality Construct
in The Context of E-Learning,” International Journal of Academic Research in Business and Social Sciences, vol. 10, no. 11,
2020, doi: 10.6007/ijarbss/v10-i11/7953.
[27] Malaysia Ministry of Higher Education (MoHE), “Higher Education Statistics 2019: Ministry of Higher Education,” MoHE,
2020. https://www.moe.gov.my/menumedia/media-cetak/penerbitan/quick-facts (accessed Jun. 03, 2022).
[28] A. Afthanorhan et al., “Gain More Insight from Common Latent Factor in Structural Equation Modeling,” Journal of Physics:
Conference Series, vol. 1793, no. 1, 2021, doi: 10.1088/1742-6596/1793/1/012030.
[29] T. V. Kondratyuk, “Fourth Industrial Revolution: What Competences Are Necessary for Employees?” Strategic Decisions and
Risk Management, no. 3, pp. 66–79, 2018, doi: 10.17747/2078-8886-2018-3-66-79.

Int J Eval & Res Educ, Vol. 12, No. 3, September 2023: 1278-1289
Int J Eval & Res Educ ISSN: 2252-8822  1289

[30] N. Anuar, A. M. Muhammad, and Z. Awang, “An Exploratory Factor Analysis of Elicited Students’ Salient Beliefs Toward
Critical Reading,” International Journal of Modern Languages And Applied Linguistics, vol. 4, no. 4, p. 101, 2020, doi:
10.24191/ijmal.v4i4.11288.
[31] Hoque, B. A. Siddiqui, Z. Awang, and S. M. A. Baharu, “Exploratory factor analysis of entrepreneurial orientation in the context
of Bangladeshi Small and Medium Enterprise (SMES),” European Journal of Management and Marketing Studies, vol. 3, no. 2,
pp. 81–94, 2018.
[32] A. S. Bahkia, Z. Awang, A. Afthanorhan, P. L. Ghazali, and H. Foziah, “Exploratory factor analysis on occupational stress in
context of Malaysian sewerage operations,” AIP Conference Proceedings, vol. 2138, 2019, doi: 10.1063/1.5121111.
[33] A. S. Shkeer and Z. Awang, “Exploring the Items for Measuring the Marketing Information System Construct: an Exploratory
Factor Analysis,” International Review of Management and Marketing, vol. 9, no. 6, pp. 87–97, 2019, doi: 10.32479/irmm.8622.
[34] W. N. A. Wan Abdul Rahman, A. K. Othman, Y. M. Yusop, Z. Awang, A. Bin Afthanorhan, and H. Zani, “Exploring and
Developing Instrument Measuring FSICU On Satisfaction Care Malay Version Among Relatives in Intensive Care Unit in Public
Hospital: Preliminary Study,” Asian Journal of Medicine and Biomedicine, vol. 6, no. 1, pp. 32–36, May 2022, doi:
10.37231/ajmb.2022.6.1.317.
[35] W. S. Alaloul, M. S. Liew, N. A. W. A. Zawawi, and I. B. Kennedy, “Industrial Revolution 4.0 in the construction industry:
Challenges and opportunities for stakeholders,” Ain Shams Engineering Journal, vol. 11, no. 1, pp. 225–230, 2020, doi:
10.1016/j.asej.2019.08.010.
[36] N. Azizan, F. Pangil, and M. L. M. Zin, “Human Capital Development in Malaysia: Issues and Challenges,” in Modeling
Economic Growth in Contemporary Malaysia (Entrepreneurship and Global Economic Growth), Emerald Publishing Limited,
Bingley, 2021, pp. 151–175, doi: 10.1108/978-1-80043-806-420211013.
[37] N. Bouranta, L. Chitiris, and J. Paravantis, “The relationship between internal and external service quality,” International Journal
of Contemporary Hospitality Management, vol. 21, no. 3, pp. 275–293, Apr. 2009, doi: 10.1108/09596110910948297.
[38] J. Flynn, S. Dance, and D. Schaefer, “Industry 4.0 and its potential impact on employment demographics in the UK,” Advances in
Transdisciplinary Engineering, vol. 6, pp. 239–244, 2017, doi: 10.3233/978-1-61499-792-4-239.
[39] A. D. Yazon, K. Ang-Manaig, C. A. C. Buama, and J. F. B. Tesoro, “Digital literacy, digital competence and research
productivity of educators,” Universal Journal of Educational Research, vol. 7, no. 8, pp. 1734–1743, 2019, doi:
10.13189/ujer.2019.070812.
[40] A. N. Azmi, Y. Kamin, M. K. Noordin, and A. N. Ahmad, “Towards industrial revolution 4.0: Employers’ expectations on fresh
engineering graduates,” International Journal of Engineering and Technology (UAE), vol. 7, no. 4, pp. 267–272, 2018, doi:
10.14419/ijet.v7i4.28.22593.
[41] J. M. Yunos, L. C. Sern, and N. H. Hamdan, “Sustainability of TVET TE programme: An exploratory sequential mixed method
design,” Advanced Science Letters, vol. 23, no. 1, pp. 220–222, 2017, doi: 10.1166/asl.2017.7138.
[42] A. R. Zakaria, M. Mohsin, G. Darusalam, and M. F. Mohamed, “Malaysia’s teacher education curriculum: Outcome and issues in
developing human capital through educational reform,” Interdisciplinary Behavior and Social Sciences - Proceedings of the 3rd
International Congress on Interdisciplinary Behavior and Social Sciences, ICIBSoS 2014, CRC Press, 2015, pp. 267–271, doi:
10.1201/b18146-49.
[43] L. M. Zawra, H. A. Mansour, A. T. Eldin, and N. W. Messiha, Utilizing the internet of things (IoT) technologies in the
implementation of industry 4.0, vol. 639. Cham: Springer International Publishing, 2018.
[44] F. Zezulka, P. Marcon, I. Vesely, and O. Sajdl, “Industry 4.0 – An Introduction in the phenomenon,” IFAC-Papers OnLine,
vol. 49, no. 25, pp. 8–12, 2016, doi: 10.1016/j.ifacol.2016.12.002.
[45] P. Levy, “Technology-Supported Design for Inquiry-Based Learning,” in Exploring Learning & Teaching in Higher Education.
New Frontiers of Educational Research. Springer, Berlin, Heidelberg, 2015, pp. 289–304, doi: 10.1007/978-3-642-55352-3_13.

BIOGRAPHIES OF AUTHORS

Asnidatul Adilah Binti Ismail PhD student at Universiti Tun Hussein Onn
Malaysia (UTHM), Master in Technical and Vocational from Universiti Tun Hussein Onn
Malaysia (UTHM), Bac. in Computer Science- Software Engineering from Universiti
Putra Malaysia (UPM). Working Experience: Lecturer in Politeknik Sultan Idris Shah,
Sabak Bernam, Selangor (2009-2014), Lecturer in Politeknik Kuching Sarawak (2015-
2017). Currently working as Lecturer in Politeknik Balik Pulau, Malaysia. She can be
contacted at: [email protected]

Razali Bin Hassan Professor at Universiti Tun Hussein Onn Malaysia

(UTHM), Batu Pahat, Johor, Malaysia. PhD holder from University of Warwick, United
Kingdom, Master Sc. (TVET) UPM, BTech. with Edu. (Electrical Technology) UTM,
Teaching Certificate from Technical Teachers Training College, (Electrical Installation;
Maintenance) Cert. in CAD (RMIT Melbourne): Ordinary members in TVET Society of
Malaysian, Members for National Union Teacher Profession, MTUN Society, Senior
members Regional Association Vocational Teacher Education (and Honorary Advisor for
Malaysian Trainer and Industries Expert Society. Working Experience: Technician in
Electrical maintenance for Toshiba Electronic, TVET teacher in Vocational School,
Lecturer in TVET, UTHM, (Electrical Technology; Graphic Engineering), Consultant,
advisor, Supervisor, panel and facilitator in TVET; Educational Technology (E-learning)
High impact consultancy experience. He can be contacted at: [email protected]

Exploring digital competencies domain and elements for information technology … (Asnidatul Adilah Ismail)