PUPIL: International Journal of Teaching, Education and Learning

ISSN 2457-0648

Jüngling et al., 2018

Volume 2 Issue 2, pp. 149-169
Date of Publication: 28th August 2018
DOI-https://dx.doi.org/10.20319/pijtel.2018.22.149169
This paper can be cited as: Jüngling, S., Telesko, R., Reber, A. (2018). Checking the Student’s Aptitude
for a Bachelor Program: Experiences with a Web-Based Tool. PUPIL: International Journal of Teaching,
Education and Learning, 2(2), 149-169.
This work is licensed under the Creative Commons Attribution-NonCommercial 4.0 International
License. To view a copy of this license, visit http://creativecommons.org/licenses/by-nc/4.0/ or send a
letter to Creative Commons, PO Box 1866, Mountain View, CA 94042, USA.

CHECKING THE STUDENT’S APTITUDE FOR A BACHELOR

PROGRAM: EXPERIENCES WITH A WEB-BASED TOOL
Stephan Jüngling
Deputy Dean Bachelor of Science in Business Information Technology, University of Applied
Sciences and Arts Northwestern Switzerland, Basel, Switzerland
[email protected]

Rainer Telesko
Institute for Information Systems, University of Applied Sciences and Arts Northwestern
Switzerland, Olten, Switzerland
[email protected]

Andreas Reber
Dean Bachelor of Science in Business Information Technology, University of Applied Sciences
and Arts Northwestern Switzerland, Olten, Switzerland
[email protected]

Abstract
In autumn 2014 the Bachelor of Science programme “Business Information Technology (BSc
BIT)” was launched. BIT is about the application of information technology in business with the
focus on building information systems. Since the start of the programme it can be observed that a
considerable number of students face difficulties in modules related to programming and
mathematics at the beginning of the study. In order to help potential applicants of the
programme to understand the kind of competencies of abstract thinking that are needed for the
BSc BIT programme ahead of the start, a web based self-assessment test was developed. The
aptitude test - built with Google Forms - currently consists of 25 predefined multiple choice

Available Online at: http://grdspublishing.org/ 149

PUPIL: International Journal of Teaching, Education and Learning
ISSN 2457-0648

questions and calculates an overall aptitude figure by summing up the single aptitude values for
the categories logical and analytical thinking, understanding algorithms, mathematics and
abstract thinking. The questions are taken from well-established test systems like ELIGO-System,
BOMAT, CASA etc. and are typically solved within less than 45 minutes. The students can check
their suitability for the study programme by comparing their overall aptitude value with a given
threshold. First test runs conducted with the tool confirm the validity of the aptitude test. The
future scope will involve more students and deal with an analysis of concrete weaknesses that
can be used as input to adapt the settings of programming and mathematics modules.
Keywords
IT-based Aptitude Test, Bachelor Program, Business Information Technology (BIT), Google
Forms

1. The Program BIT

Business Information Technology (in German: “Wirtschaftsinformatik”) is a field of
informatics focusing on how business can be supported with information technology. Therefore,
students have to understand business as well as IT to support the ongoing process of
digitalization. Students need abstract thinking skills that are involved in the analysis of business
needs. For the design of IT systems, they have to acquire the necessary innovation,
communication and modelling skills.

Figure 1: Design principles of the BIT programme

Available Online at: http://grdspublishing.org/ 150

PUPIL: International Journal of Teaching, Education and Learning
ISSN 2457-0648

In autumn 2014 the University of Applied Sciences Northwestern Switzerland (FHNW)

launched BIT as a new Bachelor of Science programme in Business Information Technology
(BSc BIT, see Figure 1) completely taught in English. The English programme is equivalent to
the successful “BSc Wirtschaftsinformatik” in German that is available since 1999. The goal of
the English BSc programme is to foster more international student exchange. The complete
programme consists of 180 ECTS points and is equally divided into five different module
groups, which are
 Foundations
 Business Administration
 Information Technology
 Business Information Technology
 and student projects
The foundations modules include mathematics, communication, ethics and law. In
business administration, students follow courses in principles of management, accounting,
marketing, economics, corporate finance, logistics and corporate strategy. Information
technology consists of requirements engineering, programming, software engineering, database
technology, internet technology, enterprise content management and information infrastructure.
Business Information Technology covers subjects like business process management, e-business,
enterprise systems, business intelligence, IT security and IT management. Through this broad
variety of topics, students learn to change perspectives from analysis to design or from business
to IT. In the module group student projects the knowledge is applied in real world projects with
companies or public organizations where they are typically working in project teams of three
students or alone during their bachelor thesis. In the end, they have all the necessary skills to lead
mixed teams, and are able to talk to business as well as IT to find new innovative business
models in the world of digitalization.
At the start of the programme, students have to pass the so-called assessment stage,
which comprises the courses Business Mathematics, Principles of Management, programming
and Introduction to BIT. Successful passing these courses is a prerequisite to continue the study
program.

Available Online at: http://grdspublishing.org/ 151

PUPIL: International Journal of Teaching, Education and Learning
ISSN 2457-0648

2. Design and Implementation of the Aptitude Test

As described above, the BIT program is about the application of information technology
in business with the focus on building information systems and managing them during the entire
life cycle. Since the start of the programme it can be observed that a considerable number of
students are facing difficulties especially with modules related to programming and mathematics
during their assessment stage. Some evidence shows that it is much more effort for those
students coming from a non-technical oriented previous education. However, since our course of
studies does not require any programming skills, the students coming from the commercial
background do not have any previous experience about their aptitude for programming, which
turns out to be one of the lectures that cause, together with mathematics the most failures during
the assessment stage. In order to provide some initial insight into the aptitude of the applicants
with respect to the required skill set four our course of studies, a web based self-assessment test
was developed within the context of a bachelor thesis (Barba, 2017).
There is no specific research about the aptitude of students for BIT-related programs.
Taking into account that programming and mathematics are two decisive topics for BIT, we can
observe that there is a lot of ongoing research about the aptitude of students resp. predicting the
success rate for programming courses (Caspersen, Larsen, & Bennedsen, 2007), (Scott &
Ghinea, 2014). Despite some initial claims (Dehnadi & Bornat, 2006) and promising initial
results, there exist up to now no really satisfactory insights about which influence factors
determine this aptitude (Bornat & Dehnadi, 2008). With regards to dedicated aptitude software,
besides a lot of questionnaires there exists a game-based tool MasterMind®, which derives the
ability to program based on actions taken in game scenarios (Lorenzen & Chang, 2006).
Up to our knowledge, no validated approach or software is available to assess the
aptitude of a student for the BIT programme and we decided to design and implement a solution
according to our needs. The next chapter provides a short overview of the stakeholder analysis as
well as the functional and non-functional requirements and the subsequent evaluation of a
suitable product.
2.1 Stakeholder Analysis
The main stakeholders for the aptitude test are not only the candidates of the Business
Information Technology programme, but also the entire faculty, including BIT deans, lecturers
as well as the management of the school. In the following table, the relevant stakeholders
together with their knowledge and goals are shown. For the elicitation of the requirements,

Available Online at: http://grdspublishing.org/ 152

PUPIL: International Journal of Teaching, Education and Learning
ISSN 2457-0648

mainly the BIT deputy dean and selected lecturers (teaching the critical subjects like
programming etc.) were involved.
Table 1: Stakeholder overview
Stakeholder Knowledge Goals
BIT deans and school Scope and goals of the  Attractive program
management BIT programme,  Sufficient number of students
statistical information
 High satisfaction degree of students
about students
BIT lecturers Module content  Reaching the module goals
 High satisfaction degree of students
 Drop out rate within acceptable limits
BIT program candidates Basic business  Program matching with the interests
knowledge (one year  Good job opportunities
practical experience)
Government (Cantons) Workforce situation  Well-trained employees
and business economy  Developing digitalization for the
Swiss economy
Future employers Digitalization need  Well-trained, innovative employees

2.2 Functional and Non-Functional Requirements of the Aptitude Test

Requirements engineering is always the first step in an IT project. This discipline deals
with the elicitation, documentation and management of customer needs, which are later on
implemented with suitable software tools. Functional requirements are concerned with the
functionality of the software. For the IT-based aptitude test, the following functional
requirements were defined:
 Public access: The system shall be accessible without prior registration.
 Integration of test questions: The system should provide test questions that cover the
necessary abstraction skills used in the assessment stage.
 Checking answers: The system should compare the given answers with the correct
solutions.
 Evaluation: Based on the given answers the system should compute an aptitude figure for
the BIT program.
 Dashboards: The system should inform the user about the results in an adequate way

Available Online at: http://grdspublishing.org/ 153

PUPIL: International Journal of Teaching, Education and Learning
ISSN 2457-0648

 Test characteristics: The test shall not build upon knowledge, which is provided during
the programme (i.e. no questions related to BIT module content in order to treat candidate
students as equally as possible.
For the project, the following non-functional requirements were defined:
 Corporate Design: The system shall use the current corporate design of the University of
Applied Sciences and Arts Northwestern Switzerland (FHNW).
 Usability: The user interface of the system shall be intuitive and enable the students to
perform the test without any assistance.
 Reliability: The system shall always create a correct evaluation of the answers.
 Performance: The system shall have response times of max 2 seconds.
 Maintainability: The system shall allow modifying or extending current content with
limited effort.
 Privacy: The system shall save the data in an anonymous way. No account or specific
application procedure for using the aptitude test shall be necessary. Furthermore all data
shall be only stored on FHNW servers (including hosting aspects)
2.3 Test Characteristics
For the success of the aptitude test, a high degree of test validity is indispensable. In our
project, the test validity is given if the questions measure the aptitude of the candidates for the
Business Information Technology programme. In order to reach that goal and to enable an
appropriate evaluation, the questions were categorized into test characteristics (Eggerth, 2006).
At first, the predominant characteristics of the relevant topics have to be selected and are
discussed in chapter 2.3.1 to 2.3.4 below. They are chosen according their importance.
2.3.1 Logical and Analytical Thinking
This characteristic relates to finding relationships and contexts and to derive unknown
knowledge from existing one. Logical thinking can be performed either in a deductive or
inductive way.
An example for inductive thinking is the following. Given that there is a logical rule
“Dogs bark”. If there is a dog named “Jimmy”, it is possible to derive the new knowledge
“Jimmy can bark” based on strong reasoning. Appropriate questions for BIT candidates may
include numerical tasks, analyzing business processes and flowcharts in programming. For this
characteristic, we used representative questions from the test systems ELIGO-System (Eligo
GmbH, 2018), Pro facts-test system and shapes.

Available Online at: http://grdspublishing.org/ 154

PUPIL: International Journal of Teaching, Education and Learning
ISSN 2457-0648

2.3.2 Abstract Thinking

Abstract thinking describes the ability to omit details and reduce the available
information to the essential. In computer science, this ability is for example necessary to create a
correct object-oriented design or an adequate data model.
Let us take the example of an “animal hierarchy”. Categorizing the terms “animal”,
“golden eagle”, “eagle” and “bird” should end up in the class hierarchy (starting with the
superclass) “animal”, “bird”, “eagle” and “golden eagle”. For this characteristic, we selected
representative questions from the test systems BOMAT (Hossiep & Hasella, 2010), (Hossiep,
Turck, & Hasella, 1999) and the core academic skills assessment (Mometrix, 2018).
2.3.3 Understanding Algorithms
Understanding algorithms is the most important precondition for writing computer
programs. Regarding the observed difficulties in programming, there is some evidence that
students have problems to develop a semantically correct algorithm with pseudo code or a
structure chart (e.g. for sorting or finding data, computing prime numbers etc.) and less problems
to transform the algorithm into a specific syntax of a programming language like Java, C# or
PHP.
Typical questions here are concerned with algorithmic constructs like conditions, loops,
variable assignments and functions (with or without parameters). For this characteristic, we
selected representative questions from the test systems pro facts- Testsystem and pro facts 360
Degree.
2.3.4 Mathematics
To include mathematics as test characteristic was an explicit request from the BIT
program management. This is mainly because students often face difficulties in both
programming and mathematics. Obviously, there are some interrelationships between these two
areas. For example, formulas in propositional logic correspond to the logical expressions in a
programming language. For this characteristic, we did not use questions from existing test
systems.
2.3.5 Subordinate Test Characteristics
Besides the predominant characteristics discussed in 2.3.1 to 2.3.4 there are some
subordinate characteristics with limited influence on the design of the test questions. Every test
question related to one of the predominant test characteristics counts for one point. If a question
is additionally related to a subordinate question, it counts for two points. For example, the test

Available Online at: http://grdspublishing.org/ 155

PUPIL: International Journal of Teaching, Education and Learning
ISSN 2457-0648

characteristic “understanding algorithms” is introduced shortly with some explanatory text or

video. Students need to quickly understand this information in order to find the correct answer.
Subordinate test characteristics such as apprehension, creativity, planning, problem solving and
velocity have been identified.
 The assessment framework of the aptitude test includes ten questions dedicated to logical
and analytical thinking, and five questions in each of the topics abstract thinking,
understanding algorithms and mathematics.
2.4 Evaluation of the Software Platform
The implementation of the aptitude test can be done in two ways:
 Alternative 1: Software development: All elicited requirements can be covered and
implemented. However, software development from scratch can be quite costly and time-
consuming.
 Alternative 2: Using an existing framework: With this approach, you are evaluating
different existing solutions and select the platform, which best fits your needs.
Mainly because of time and skill constraints, alternative 2 was chosen. A first shortlist of
possible candidates did include Google Forms, Windows SharePoint and Moodle. All of these
platforms meet our core requirements of the test namely to be able to include own questions,
define suitable response formats and to support the evaluation of results.
Before the final decision, the different platforms were evaluated in further detail based on
functional as well as non-functional requirements.
Table 2: Evaluating functional requirements
Functional requirement Moodle Docs SharePoint Google Forms
Integrating test questions ok ok ok
Checking answers ok ok ok
Correct evaluation of answers ok ok ok
Visualization of results ok ok ok
Saving results for external users ok

The functional requirements are only fully met by Google Forms. Moodle as well as
SharePoint got a lower rating for the non-functional requirements. The prototype was
implemented with Google Forms. The feature to provide individual ratings and to support further
analysis of the individual results of the test persons were the key criteria. However, it has to be

Available Online at: http://grdspublishing.org/ 156

PUPIL: International Journal of Teaching, Education and Learning
ISSN 2457-0648

mentioned that the implementation is prototypical in a sense that the feasibility of an aptitude test
for the BIT program has to be evaluated first. Important non-functional requirements related to
security and privacy has explicitly not been considered in this early project phase and may lead
to a different tool selection for a fully operational platform.
2.5 Implementation with Google Forms
After selecting the relevant categories and compiling the initial set of questions, the next
step was to design and implement the questionnaire with Google Forms that has to follow the
corporate web design guidelines of FHNW.
Below three representations of test questions are shown as they appear in the categories
logical and analytical thinking, understanding algorithms and abstract thinking. None of the
questions require previous knowledge about Business Information Technology. Most of the time,
participants have to respond to questions by selecting one solution out of a given set of answers
as shown in Figure 2.

Figure 2: Test question from the logical and analytical thinking category

Available Online at: http://grdspublishing.org/ 157

PUPIL: International Journal of Teaching, Education and Learning
ISSN 2457-0648

Figure 3: Test question from the abstract thinking category

Initial tests with around 60 participants from different backgrounds have proven the
feasibility of the test. The time needed for answering the questions was mostly between 30 and
45 minutes. Personal discussions revealed that some only answered the questions and did skip
the introduction videos. The videos have been added, because the test should not only evaluate a
final number that indicates the aptitude for the BIT course of studies, but also should provide
some insight into the field of studies and underline the importance of Business Information
Technology skill set for the business world.

3. Application of the Aptitude Test and Validation of Initial Results

In the following sections, we explain the usage of the aptitude test in the context of our
BSc BIT course of studies as well as how we analyze and evaluate the test results in order to
prove its validity as an indicator of the student’s aptitude to pass the assessment stage of the BSc
BIT program.
3.1 Purpose and Usage of the Aptitude Test
Some questions in the aptitude test come from question collections that are commonly
known as IQ test. Many people are skeptical about IQ tests, because of the potential misuse to
reduce a person’s intelligence to a single number. To circumvent such an effect for the aptitude
test, we searched for a simple way to measure the aptitude on one hand, without compromising

Available Online at: http://grdspublishing.org/ 158

PUPIL: International Journal of Teaching, Education and Learning
ISSN 2457-0648

the anonymity of the participants on the other. In order to combine both requirements,
participants are allocated to different control groups, which in turn are based on the performance
results of previous exams. We focus on the grades in programming, mathematics and the course
introduction to business information systems, which all three are responsible for the most failure
rates during the assessment stage of the BSc BIT program. The result of the aptitude test should
serve as an indicator for the aptitude for passing successfully the assessment stage. The modules
evaluated as part of the assessment stage are in turn selected as being representative for the
fundamental ability to be able to pass all courses of the BSc program. At the same time, the test
should
 Be attractive for potential students and provide some insight into the field of BIT
 Provide an opportunity to participate in a longer test under some pressure and
 Serve as an indicator towards the aptitude of abstract thinking.
All three aspects combined should serve as an indicator to the necessary skills to pass the critical
exams of the assessment stage at the end of the first semester.
3.2 Information Events
In BIT, many of our students come from an apprenticeship with a background of either IT
or business. This vocational education system is very common in Switzerland since a long time,
but is new to many of the international students, which mainly focus on universities and not on
universities of applied sciences. Some students also come from a supply school where they
graduated with a Matura or an International Baccalaureate. Most students do not have a single
wish of choosing their dedicated program and university, but take multiple options into their
shortlist of potential studies. Therefore, it is quite common, that universities provide some
information events. In BIT we provide, in addition to information events, the possibility of open
days, where interested students can join particular taster lectures in introduction to BIT,
programming and mathematics. During these open days, not only the English based programs
BSc BIT and BSc International Management but also the German speaking programmes of BSc
Wirtschaftsinformatik and BSc Betriebsökonomie are open for test lectures. The link to the
aptitude test is distributed to the participants after these events.
3.3 Student Grades of the Assessment Stage and Allocation of Control Groups
As mentioned above, students typically fail the assessment stage due to the courses
mathematics, programming and introduction to BIT. The following graph shows the distribution
of the grades from the years 2014-2016.

Available Online at: http://grdspublishing.org/ 159

PUPIL: International Journal of Teaching, Education and Learning
ISSN 2457-0648

Figure 4: Distribution of grades of the assessment stage

As mentioned above, staying anonymous in terms of the test results is one of the non-
functional requirements of the test setup. We have introduced different control groups based on
previous grades in mathematics or programming related courses. Furthermore, we distinguish
between our current FHNW internal students and the potential external applicants that are
interested to join the BSc BIT program. The participants were asked about their previous
performance ranges
 5.3 - 6 (best grade)
 4.8- 5.2 (good)
 3.8 - 4.7 (meet expectations)
 3.7 and lower (below expectations)
The participants of the test get their aptitude rating directly at the end of the test. The test
results can be evaluated by analyzing the results of the different control groups.
3.3.1 Results of the Assessment Stage
The relatively high failure rate of mainly international students was the origin of the idea
to provide the aptitude test. Table 5 shows the outcome of the assessment stage of two BIT
classes including the failure rate for Swiss and international students.

Available Online at: http://grdspublishing.org/ 160

PUPIL: International Journal of Teaching, Education and Learning
ISSN 2457-0648

Table 3: Failure rates of international students

2015 started current 2016 started current
Schweiz 22 17 Schweiz 30 17
Bulgarien 1 0 Algerien 1 0
Deutschland 1 0 Türkei 2 0
USA 1 0 Russland 1 0
Syrien 1 1 Sri Lanka 1 0
Nigeria 1 0 USA 1 0
Slowenien 1 0 Afghanistan 1 0
Estland 1 1 Pakistan 1 0
Abroad Total 7 2 Serbien 1 1
Rumänien 1 1
2015 China 2 0
Failed (CH) 23% Portugal 1 0
Failed (Abroad) 71% Italien 2 2
Ägypten 1 0
2016 Sri Lanka 1 0
Failed (CH) 43% Chile 1 0
Failed (Abroad) 79% Deutschland 1 0
Abroad Total 19 4

By comparing the failure rates of Swiss and international students, the figures showed,
that the exams were more challenging for international students. Based on some personal
discussions with students who failed, the exams were very challenging for them, since it was the
first time to have such long exams with more than 20 pages and with a duration of up to 1.5
hours. In this respect, the test is also useful for the participants to test the ability to concentrate
for a longer period.
3.3.2 Correlations between Mathematics and Programming
It is commonly known, that some correlations do exist between mathematical and
programming skills. However, figure 5 shows that the correlation is less than expected. One
possible explanation is that programming includes different skills than mathematics. Based on
the taxonomy of Bloom (Bloom, 1956) most of our mathematical skills are in the range of K1-
K4, which comprises knowledge-, understanding-, application- and analysis skills. Programming
in contrary strongly includes skills based on design and synthesis, which is up to K5 or K6
according to the revised taxonomy of Bloom (Krathwohl, 2002).

Available Online at: http://grdspublishing.org/ 161

PUPIL: International Journal of Teaching, Education and Learning
ISSN 2457-0648

Figure 5: Correlation of grades in mathematics and programming

Of course, the background of the students also has some effects on the programming
skills. Students that learned programming in an IT apprenticeship definitively have an advantage.
However, most foreign students did not have this possibility, since this type of vocational
education is not very common in other countries.
3.4 Initial Aptitude Test Results of Applicants and Current Students
As stated previously the test results are not personalized but collected anonymously by
allocating participants to the different control groups. Only the participants themselves can see
their points related to the four different categories of interest. In order to determine the validity
of the test, we collect not only data from the candidates that are interested to start the BIT
program, but also from the registered students of the programme. We use the results of our
students to get an idea of the minimum target value of the aptitude test, which would be
necessary to successfully pass the BIT programme. Our current distribution of the average points
show the following results.

Available Online at: http://grdspublishing.org/ 162

PUPIL: International Journal of Teaching, Education and Learning
ISSN 2457-0648

Figure 6: Distribution of average points of external candidates

The distribution of figure 6 is based on all the external candidates that did participate in
any of our information sessions representing the population of the potential future students.
However, it is not clear, which of these will definitely register in our BSc programs.

Figure 7: Average points of internal students

The distributions of figure 7 looks different. First, the spread is less than the current
collection of test results from external candidates. One possible interpretation is that all students
that passed the assessment stage have the basic capabilities to reach a level of 18 points or above.
On the other hand, it might be that those who really have very good skills in programming and
mathematics, finally choose a more technical course of studies, and end up in joining dedicated
BSc programmes in computer science.

Available Online at: http://grdspublishing.org/ 163

PUPIL: International Journal of Teaching, Education and Learning
ISSN 2457-0648

3.5 Control Group Based Test Results

As one of the initial control groups, the students that failed the assessment stage in 2016
were asked to participate in the aptitude test.

Figure 8: Average points of internal students

So far, the control group that failed the assessment stage had 23 points in average, and the
group that passed had 26 points, which gives some weak indication on the validity of the test.
However, the test population is currently too low to state this as a conclusion for the threshold
value.
Another parameter might be the time that students take to finish the test. Some might do
it very rapidly; others with more caution investigating more time. The following picture shows
the relation of average points and the time of participants needed to complete the test.

Figure 9: Average points and time needed to complete the test

It seems most obvious, that time is not the relevant differentiation factor. More than 40%
of the participants had less than 30 minutes; one third had less than 45 minutes. Only one
participant spent more than 1.5 hours and got less than 20 points. This could be a possible

Available Online at: http://grdspublishing.org/ 164

PUPIL: International Journal of Teaching, Education and Learning
ISSN 2457-0648

indication that it is not the right course of studies. However, the current platform does not
provide the feature of automatic time measurements, which would be an important additional
requirement for upcoming versions of the test.
In order to determine the validity of the test, our current students were asked to
participate in the test as well.

Figure 10: Average points compared to programming skills

Figure 10 shows the average points of the student population in the different control
groups based on the grades in programming while figure 11 and 12 show the average number of
points of all the students, categorized by the different performance bands based on the grades in
mathematics and introduction to BIT respectively.

Figure 11: Average points compared to mathematical skills

Available Online at: http://grdspublishing.org/ 165

PUPIL: International Journal of Teaching, Education and Learning
ISSN 2457-0648

Figure 12: Average points compared to introduction to BIT course

Figure 12 show that none of the students that failed in the BIT exam did participate in the
self-evaluation test.
Interestingly enough all three comparisons of the average number of points with the
average performance in the module exams show a positive correlation. E.g. good performance in
the self-evaluation test correlates with good grades in the appropriate modules. Bad performers
of mathematics exams got significantly less points in the test.

4. Discussion
Our initial test results provide some evidence that the aptitude test provides some
valuable predictive insights in terms of the ability to pass the assessment stage of the BIT
program. The evidence of coincidence with the number of points reached in the test and the
grades in the modules mathematics and programming is stronger than the correlation of grades
from the course introduction to Business Information Technology and the number of points in
the evaluation test. However, the number of test results are still too small and some further
influencing factors should be taken into account. One major denominator of points achieved in
the test might be the different background of the participants. Our course of studies does not
require any prerequisite IT knowledge. However, the different background certainly do have
some implications. Although figure 13 proves our experience, that our course of studies is
manageable for students with and without IT background, especially the work load of students at
the beginning is different.

Available Online at: http://grdspublishing.org/ 166

PUPIL: International Journal of Teaching, Education and Learning
ISSN 2457-0648

Figure 13: Average points from students coming from Business or IT background
Students coming from a business background need to invest substantially more time into
the programming course at the beginning, but in the end the results show, that it is also possible
to pass the assessment stage with no previous IT experience.
Worth mentioning seems the fact of the difference from the best to the least of the grades
(from 6 to 1) in all three courses. Figure 14 shows, that for the best performers, the difference in
the grades is less, than the differences for average or low performers in all three courses.

Figure 14: Correlation of grades in mathematics and programming

It underlines some qualitative statements from our students that passed successfully the
assessment stage. They stated that all students who did continuously do all the exercises in
mathematics and in programming did pass the appropriate examinations. Therefore, the grades
are highly correlated with the amount of investigation of the students into the subjects. Students
showing this attitude are consistently more successful in all modules.

Available Online at: http://grdspublishing.org/ 167

PUPIL: International Journal of Teaching, Education and Learning
ISSN 2457-0648

5. Conclusion
The BSc Business Information Technology course of studies is designed for students with
an apprenticeship and baccalaureate from the business as well as IT domain and requires a good
amount of abstract and logical thinking skills. Students that fail the assessment stage mostly
struggle with technical courses such as programming, mathematics or introduction to BIT. Our
web-based self-evaluation test is designed for prospective students to get some insight into their
personal skills in logical, analytical and abstract thinking. The overall score of the test is based
on a total number of points in four different categories. Initial tests are conducted with
prospective as well as current students and show some evidence on the validity of the self-
evaluation test. The test provides some predictive and valuable insight into the personal ability to
master the entire BIT bachelor of sciences course of studies. Although a positive correlation to
the test is given, it does not mean, that good test results in the aptitude tests automatically lead to
good grades. At most, it serves as an indicator of the necessary abstraction skills as an important
prerequisite to pass the assessment stage.
However, the validity of the tests is still limited due to the low number of test results.
Retesting with the same participants is not an option. Since the current number of questions is
very limited, it is very important the answers for the test questions are not passed along from
participant to participant.
The scope of future research includes several dimensions. For the next release of the BIT
aptitude test, we plan some improvements, especially in terms of the user interface and the
amount of test question in the database, which could be enlarged considerably. One of the major
goals of our future research aims to establish the feedback loops between the test and teaching. If
it turns out in the aptitude test that prospective students have considerable difficulties to
understand some particular concepts or topics in computer science (e.g. inheritance in object
orientation, relationships in database modelling etc.), additional exercises could be prepared to
particularly address these shortcomings earlier during the semester. Vice versa, those topics that
are usually not very well solved in the exams are good candidates for additional test questions.
Such incremental adaption of the test questions could further improve the aptitude test and help
the students to detect and address their individual shortcomings as early as possible.
Nevertheless, the current test can already give a valuable insight into the kind of skills that are
needed to thoroughly follow the course contents and to determine the required abstraction skills
that are necessary to do the exercises and pass the exams of the assessment stage.

Available Online at: http://grdspublishing.org/ 168

PUPIL: International Journal of Teaching, Education and Learning
ISSN 2457-0648

References
Barba, R. (2017). Selbsteinschätzungstest als Vorbereitung für das Wirtschaftsinformatik
Studium [Aptitude Test as preparation for the Business Information Science study
program], Bachelor Thesis. Olten.
Bloom, B. S. (Ed.). (1956). Taxonomy of educational objectives: The classification of
educational goals, by a committee of college and university examiners. New York: D.
McKay.
Bornat, R., & Dehnadi, S. (2008). Mental models, consistency and programming aptitude.
Conferences in Research and Practice in Information Technology Series, 78, S. 53-61.
Caspersen, M., Larsen, K., & Bennedsen, J. (2007). Mental models and programming aptitude.
ACM SIGCSE Bulletin, 39, S. 206-210. https://doi.org/10.1145/1269900.1268845
Dehnadi, S., & Bornat, R. (2006). The camel has two humps. Internal report, School of
Computing, Middlesex University, UK.
Eggerth, T. (2006). Konzeption und Umsetzung eines modularen Eignungstests für
Informatikstudiengänge, [Concept and Implementation of a modular aptitude test for
Computer Science Programs], Bachelor Thesis. München: Grin Publishing.
Eligo GmbH. (2018). ELIGO Psychologische Personalsoftware GmbH. Retrieved from
https://www.eligo.de/en/
Hossiep, R., & Hasella, M. (2010). Bochumer Matrizentest: BOMAT Standard. Göttingen:
Hogrefe.
Hossiep, R., Turck, D., & Hasella, M. (1999). Bochumer Matrizentest: BOMAT-advanced.
Göttingen: Hogrefe.
Krathwohl, D. (2002). A Revision of Bloom's Taxonomy: An Overview. Theory Into Practice,
41(4), 212-218. Retrieved from http://www.jstor.org/stable/1477405
https://doi.org/10.1207/s15430421tip4104_2
Lorenzen, T., & Chang, H. (2006). MasterMind: a predictor of computer programming aptitude.
ACM SIGCSE Bulletin, 38, S. 69-71. https://doi.org/10.1145/1138403.1138436
Mometrix. (2018). Core Academic Skills Assessment. Retrieved from
https://www.mometrix.com/academy/casa/
Scott, M., & Ghinea, G. (2014). On the Domain-Specificity of Mindsets: The Relationship
Between Aptitude Beliefs and Programming Practice. IEEE Transactions on Education,
57(3), S. 169-174. https://doi.org/10.1109/TE.2013.2288700

Available Online at: http://grdspublishing.org/ 169