0% found this document useful (0 votes)
27 views13 pages

Computational Thinking in Preschool Age A Case Study in Greece

Computational Thinking in Preschool Age∶ A Case Study in Greece (2023) by Zoi Kourti et al.

Uploaded by

babachuot3508
Copyright
© © All Rights Reserved
We take content rights seriously. If you suspect this is your content, claim it here.
Available Formats
Download as PDF, TXT or read online on Scribd
0% found this document useful (0 votes)
27 views13 pages

Computational Thinking in Preschool Age A Case Study in Greece

Computational Thinking in Preschool Age∶ A Case Study in Greece (2023) by Zoi Kourti et al.

Uploaded by

babachuot3508
Copyright
© © All Rights Reserved
We take content rights seriously. If you suspect this is your content, claim it here.
Available Formats
Download as PDF, TXT or read online on Scribd
You are on page 1/ 13

education

sciences
Article
Computational Thinking in Preschool Age: A Case Study
in Greece
Zoi Kourti, Christos-Apostolos Michalakopoulos, Pantelis G. Bagos *
and Efrosyni-Alkisti Paraskevopoulou-Kollia

Department of Computer Science and Biomedical Informatics, University of Thessaly, 35 100 Lamia, Greece
* Correspondence: [email protected]

Abstract: This study aims to investigate Computational Thinking (CT) in preschool age children. We
tried to assess the ability of developing CT skills in kindergarten, as well as to illustrate parents’ and
preschool teachers’ directly involved aspects on CT. More specifically, we investigated the readiness
of preschool children to engage in activities that develop CT by using Scratch Jr. Moreover, via
individual interviews, the research sheds some light on preschool children’s parents’ perceptions
on personal computers use, digital media, CT and programming. Finally, we investigated the views
and perceptions of preschool teachers through an electronic questionnaire about CT and its entry in
kindergarten. The kindergarteners’ interactive stories were evaluated with the use of a rubric which
indicates the extremely high percentage of readiness to deal with CT and programming activities in
kindergarten. The analysis of the parents’ interviews shows their distrust on readiness regarding
children’s engagement in CT. Finally, the majority of preschool teachers uses personal computers
in kindergarten to a different degree depending on their age, experience and training to develop
a variety of learning areas. Regarding CT, kindergarten teachers’ aspects appear to be swayed from
positive to neutral considering its introduction in the kindergarten classroom.

Keywords: computational thinking; preschool education; Scratch Jr.; rubric evaluation; kindergarten;
Citation: Kourti, Z.;
information and communications technology
Michalakopoulos, C.-A.; Bagos, P.G.;
Paraskevopoulou-Kollia, E.-A.
Computational Thinking in Preschool
Age: A Case Study in Greece. Educ.
Sci. 2023, 13, 157. https://doi.org/
1. Introduction
10.3390/educsci13020157 Computational Thinking (CT) has been defined as the mental skill necessary in order
to apply concepts, methods, problem solving techniques, and logic reasoning, derived from
Academic Editors: Magdalena
Ramos Navas-Parejo, Pedro Antonio
computing and computer science, with the aim of solving problems in all areas of life [1].
García-Tudela, Lucía Lomba Portela
After the statement of Wing, according to which along with reading, writing and arithmetic,
and Marco Antonio Zamora CT is a basic skill that all children should have [2], many countries already include CT in
Antuñano their formal curricula from preschool age. In this work we try to give an assessment of the
position of CT in preschool education in Greece [3].
Received: 3 January 2023 In Greece, according to kindergarten’s curriculum [4], the development of various
Revised: 31 January 2023
concepts of CT is proposed, although CT is not referred into it as a term. Specifically, it is
Accepted: 31 January 2023
proposed as the content of the ICT (Information and Communications Technology) section.
Published: 2 February 2023
For example, CT concepts are included in the introduction to programming games, where
the children attempt to “program” a classmate or a robot (e.g., BeeBot), to create patterns,
to visualize, to simulate, to model, to develop ability to judge, to make decisions and solve
Copyright: © 2023 by the authors.
problems. Additional efforts for the promotion of CT in the preschool classroom are made
Licensee MDPI, Basel, Switzerland. with developmentally appropriate tools such as programming environments based on
This article is an open access article the use of graphical interface and visual programming through the use of command tiles,
distributed under the terms and which do not necessarily require the ability of reading by the child [5].
conditions of the Creative Commons The objectives of the present research are (1) to investigate the readiness of preschool
Attribution (CC BY) license (https:// children in Greece to develop their CT, (2) to investigate views and perceptions of preschool
creativecommons.org/licenses/by/ children’s parents regarding CT and (3) to investigate the views and perceptions of
4.0/). preschool teachers about CT in the preschool class.

Educ. Sci. 2023, 13, 157. https://doi.org/10.3390/educsci13020157 https://www.mdpi.com/journal/education


Educ. Sci. 2023, 13, 157 2 of 13

1.1. Computational Thinking Definition


The continuous technological progress brings people to a constant effort to acquire
new skills and abilities, in accordance with the experiences gained through the Information
Society [6]. Reasonably, it can be assumed that over time, the necessary skills that chil-
dren must acquire as pupils/students are constantly changing—or their context is being
changed—so that they can (themselves) adapt to the developing environment [7,8].
Wing argued that in the 21st century, computational thinking will be a fundamental
skill used around the world and along with reading, writing and arithmetic, we must add
computational thinking to every child’s analytical ability [2,9]. CT is a term suggested by
Wing, based on the work of Papert who defined CT as the process of problem solving,
systems design and understanding of human behavior, based on the fundamental concepts
of computer science [10]. Since then, many more definitions have been formulated, in order
to clarify the term, to describe its properties and concepts, and to incorporate its elements in
the curricula for all levels of education [11,12]. Bers’ definition is also interesting, where CT
is defined as a type of analytical thinking that shares many similarities with mathematical
thinking (e.g., problem solving), mechanical thinking (process design and evaluation), and
scientific thinking (systematic analysis) [13]. Similarly, Barr and Stephenson define CT as
a problem-solving methodology that can be automated and applied to all subjects and is
not limited to the field of knowledge and computer science teachers [14].
Seeking to define CT and its importance, many scientists, as mentioned above, have
tried to identify CT and ‘fundamental parts’ comprising the term. The most popular among
these ‘fundamental parts’ are:
• Subtraction: it is the process of focusing on the important parts of a problem, hiding
irrelevant information, so that its key features emerge [2,15,16]. Example in the
preschool class: plastic animals such as a gazelle, a lion, a snake, an elephant and
a giraffe are placed in front of the children and they are asked what they have in
common. The removal process leads to the answer ‘they are jungle animals’.
• Generalization: it is the recognition of common elements that allows the creation of
one solution that applies to many similar problems [14]. Example in the preschool
class: while playing, children discover that if they put cylindrical bricks in the base
of their construction of a car and in the base of a robot, they roll, so they are able to
generalize that any construction equipped with a circular base will roll.
• Algorithm: it is a set of steps performed sequentially in order to solve a problem or
fulfill a goal [17]. Example in the preschool classroom: children ponder and answer
the question ‘what are the steps we follow to create a fruit salad in the classroom?’
Step one: wash the fruit, step two: peel the fruit, step three: cut the fruit, step four:
place and mix in a large fruit bowl, step five: serve in individual bowls for everyone.
• Problem Deconstruction: the process of segmenting the problem so that one can pro-
cess it into smaller, more manageable components [2,18,19]. Example in the preschool
classroom: in the classroom there is a board with the children’s names and the names
of the pedagogical corners they play at daily. The problem ‘where do I play today?’
decomposes into smaller ones, as to (a) find the card with the name, (b) notice the
name of the corner in which the child plays, (c) search between the corners to find the
card that writes it, so that they reach the solution of the problem.
• Data collection: the ability to collect data related to the problem to be solved is
considered essential [14,19]. Example in the preschool class: children worry about
‘what are the right conditions for a plant to grow?’ They plant seeds in pots, take care
of them, take photos of their growth every day.
• Data analysis: through data analysis, we reach their interpretation, drawing con-
clusions and identifying patterns [14,19]. Example in the preschool class: children
serialize photos taken during the growth of the plants, compare them, find similarities
and differences, and draw conclusions about the appropriate conditions for growing
a plant in the immediate environment.
Educ. Sci. 2023, 13, 157 3 of 13

• Data presentation: rendering data in a suitable for computer processing format,


e.g., with graphs, sentences or pictures [14,19]. Example in the preschool class: chil-
dren, after three weeks of plant growth, draw a bar chart with their growth path.
• Parallelism: the arrangement of resources for conducting parallel experiments aimed
at problem solving [19]. Example in the preschool class: children conduct parallel
experiments with plants that are in different conditions (no soil, no water, no sun, and
no air) in order to confirm the most suitable environment for their growth.
• Automation: working through machines or computers in order to make it easier to
achieve a goal [19]. Example in the preschool class: the bar chart with children’s
‘towers’ is transferred to the Microsoft Excel spreadsheet program with the help of the
preschool teacher so that children process it, print it, etc.
• Modeling: the representation of a concept or object based on important data. Example
in the preschool classroom: children creating a model of lungs of the human body
using simple materials such as balloons, straw, plastic bottle.
• Simulation: the process of performing experiments through a constructed model.
Example in the preschool classroom: children experiment with the lung model and
understand the respiration-exhalation process and the respiratory system’s function.
• Debugging: trying to find possible errors in the construction of a job that needs
correction [2]. Example in the preschool classroom: a child draws in the computer
drawing program of the personal computer (PC). S/he draws clouds and tries to fill
them with color, but instead the whole page is colored, because the cloud-shape was
not closed. S/he presses the undo button and tries to make a new plan. After several
attempts and experiments s/he concludes that a shape will be filled with color only if
it is perfectly closed around the perimeter.

1.2. Computational Thinking in Preschool Education


Programming skills play a leading role in children’s digital eloquence [20]. Through
engaging in programming, children benefit significantly as they become not only users
but also creators of technology. Furthermore, programming allows an in-depth look at
the functional world of digital media, making their use more successful. The effect of
programming in the development of mathematical skills is also important [9], since this
way children come in contact with concepts such as ‘feedback’, when, e.g., the program
they create does not correspond to the desired result or ‘variable’, or when, e.g., the course
of the hero they plan is changed by the obstacles they encounter [21].
Undoubtedly, programming skills help in the development of CT, since planning
creates problem-solving opportunities. At the same time, through programming, children
are involved in the construction efforts of models and systems; computer science itself
one could say that it derives from thoughts of engineers [22]. Finally, programming can,
by modeling the problem and its solution process, reflect and bring us into contact with
our way of thinking [23]. A glance at recent studies has demonstrated that programming
environments can be used in early childhood education and positively promote CT [24].
Programming helps develop cognitive abilities including executive functions that help
children formulate, solve and analyze problems [25], algorithmic abilities to present so-
lution logically [26], working memory and inhibition skills [27]. Scratch Jr., which is the
chosen programming tool used in the current study to examine the readiness of preschool
children to engage in CT activities, could develop students’ independent thinking, problem
solving, and CT abilities [28] and familiarize preschoolers with basic programming con-
cepts in a playful way [29]. Lastly, Scratch Jr., could foster young children’s CT concepts
(e.g., sequence, event) and practice (e.g., testing and debugging, remixing and reusing),
and enhance their enjoyment to decorate their programming projects [30].
Moreover, programming skills help in the development of CT, since it affects their
mathematical skills [31] and planning creates problem-solving opportunities. At the same
time, through programming, children are involved in the construction efforts of mod-
Educ. Sci. 2023, 13, 157 4 of 13

els and systems; computer science itself one could say that it derives from thoughts of
engineers [22].
The research on parents’ perspectives and their roles in children’s engagement in CT
is very limited [32,33]. Studies considering childrens’ usage of mobile devices show that
parents want to support their children’s learning and seek to provide a stimulating home
learning environment for them [34]. Another study considering parents perspectives on CT
lays their concerns such as their limited programming knowledge to help their children
engage on CT competencies.
What is clear through recent studies is that parents’ role in promoting their childrens’
engagement on CT is integral [32,35], therefore this study seeks to shed light into parents’
perspectives and beliefs of CT’s entrance into kindergarten.
Teachers attempts to bring CT in the classroom meet a variety of challenges, such
as subject knowledge, lack of time and support, suitable approaches to teaching topics,
students’ understanding, and ability to problem solving [36].
Teachers are also concerned about bringing CT into teaching due to limited class time
and the difficulties of addressing high level CT thinking in developmentally appropriate
ways [37].
Even after receiving professional development and support to integrate CT and com-
puter science into their classroom lessons some teachers find themselves insufficiently
prepared and sometimes do not feel confident enough teaching a new activity connected to
CT skills [38].
What some teachers believe is that CT could enhance student engagement, literacy
skills, and mathematics skills [39]. Though, according to other studies, it is believed that
CT is a difficult topic to understand which often cannot feasibly be integrated into K-12
education due to curriculum and instruction constraints [40].
Recent data suggest that CT can be promoted through a variety of software tools in the
context of preschool education. The categories are as follows [41]: The logo family program-
ming environments (e.g., Bee-Bot, Code-a-pillar), the software roamers (e.g., Ladybug leaf),
the visual programming environments (e.g., Scratch Jr.), the commercial programming
learning environments for entertainment purposes (e.g., Kodable), physical computing en-
vironments (e.g., Kibo, Arduino robot car) and lastly miscellaneous unplugged applications
and environments that promote CT without the use of a computer [42].

Indicative Presentation of Programming Toys for Children


Bee-bot from GRobotronics is the most popular robot found in preschool classes and
is suitable for children aged 3 and over. Blue-bot is its transparent version. The robot-bee
is programmed with on-board keys to move in space and helps the child to approach
spatio-temporal concepts and create a script by programming it. The advantage of the
bee-bot is that it can move on a variety of surfaces such as paper, tarpaulin, wood, carpet
and does not necessarily require a special surface, a fact which allows preschoolers to
imagine and create their own boards [43–45].
The rugged robot of GRobotronics is a robot designed for outdoor use so it can be used
on sand, soil, gravel and grass. There is a possibility to integrate a camera, so that children
can see where the robot is going to as well as storage space to use it to collect objects. The
rugged robot can be connected to a tablet and is suitable for 3 years old and up [46].
The Fisher-Price Code-a-pillar Twist game is a caterpillar robot, suitable for children
aged 3–6 years, which can be programmed with many different program combinations by
pressing or turning the caterpillar paddles, so that it moves left, right and/or emits light
and sounds. It promotes the development of the concepts of CT such as problem solving,
sequencing, and critical thinking [47].
The Fisher-Price Code ‘n’ Learn Kinderbot is a robot game, suitable for children
aged 3–6, which helps them become familiar with mathematical concepts, colors, shapes
and letters through programming opportunities. Children interact with the problem-
Educ. Sci. 2023, 13, 157 5 of 13

solving process trying to create a path for the robot and practice engineering activities
through its components [48].
The activity set for the Code & Go® Robot Mouse class of learning resources is a game
suitable for children aged 4–8 and includes four robot mice, a math pack and a board game
that are interactive learning sets and teach coding concepts such as sequencing, through
touch play without the need for an additional screen or application [49].
The Learning Resources Coding Critters Rumble & Bumble are animals that are
programmed according to an instruction book to move around the space and complete
missions while there is the possibility for each child to create her/his own program for
her/his pet. Designed for children aged 4–10 years old [50].
The KIBO is a robot designed for children aged 4–7 years old, after decades of research
by the scientific team of Marina Umaschi Bers of Tufts University, creator of the program-
ming application Scratch Jr.—which we used in our research—is aimed at both children
who are interested in ICT as well as to children who have an appeal in the arts and physical
activities. Children plan a sequence with the wooden cubes (blocks), the robot scans the
program and executes it [51].
Scratch Jr. was inspired by the popular Scratch programming language (scratch.mit.edu)
and has been redesigned to be developmentally appropriate and compatible with the cog-
nitive, personal, social and emotional development of young children. It is an introductory
programming language that allows young children (5–7 years old) to create their own
interactive stories and games. Children ‘button’ the graphic programming blocks, as they
would do with real Lego tiles, to make the characters move, dance and sing. Preschool
children can modify the characters in the color editor, add their own voices and sounds,
and then use the programming blocks to bring their characters to life [52].

2. Materials and Methods


2.1. Design
There are two main research methods, quantitative and qualitative. Assuming that
each of the two methods has significant advantages and limitations, the operation of
combining the two methods into a mixed one, can contribute to an in-depth understanding
of research problems but also help the researcher access the limitations of each method
separately [53]. In fact, a study that collects qualitative and quantitative data provides
a greater degree of reliability in the analysis and results [54]. Moreover, the mixed research
method can highlight different aspects of the same phenomenon [55].
Essentially, the present study was designed considering the elements of the triangula-
tion technique. According to Robson, there are four types of triangulation: data triangula-
tion, observer triangulation, methodological triangulation and theoretical triangulation [56].
In this research we used data triangulation, utilizing more than one methodological tool
and resources for data collection. Specifically, observation, Scratch Jr. educational software,
interview and questionnaire were used as methodological tools and data were collected
from different sample groups, such as preschool children, parents of preschool children
and preschool teachers. Following the method of triangulation, we tried to highlight the
CT of preschool children through different sources in order to ensure greater reliability in
our research data.

2.2. Sample
The sample of the study is 24 preschool children (12 preschool boys and 12 preschool
girls), 10 parents (2 male and 8 female) and 100 female preschool teachers. All individuals
were from the area of Lamia, Fthiotida, Greece. In all cases informed consent was given.

2.3. Tools
In this particular research we used data triangulation, leveraging more than
one methodological tool as well as sources for data collection. To collect information
about CT of preschool children, we used the application Scratch Jr., which is a program-
Educ. Sci. 2023, 13, 157 6 of 13

ming environment suitable for preschool children. All children participating in the study
were introduced to a Scratch Jr. software tutorial based on the suggested through the official
page by Scratch Jr. which was created by its technology development research group in
Tufts University. Considering the parents’ perception and beliefs on CT, semi-structure
interviews were conducted. Finally, the preschool teachers’ perception and opinions on
CT were gathered through an on-line questionnaire, uploaded on various kindergarten
teachers’ groups on social media. The questionnaire consisted of multiple-choice questions
and open-ended questions, designed accordingly to collect descriptive and explanatory
data on kindergarten teachers’ opinions, behaviors, characteristics, and attitudes regarding
the use of a PC or tablets in the classroom and on CT. The reason that the study combines
quantitative and qualitative data is because a mixed methods study, ensures a better under-
standing of its research problem than each species by itself [57]. The evaluation of children’s CT
after completing the activities of the Scratch Jr. tutorial was conducted using a rubric. It is
believed that rubrics have become the most consistent and popular technique for assessing
learners’ performance, as they are used for a wide range of knowledge, skills, competencies
in a variety of learning objects and activities [58]. The detailed evaluation rubric created
consists of four separate performance criteria which are analyzed and evaluated separately
from each other [59]. In the specific rubric, children were evaluated on the following
criteria: comprehension of commands (e.g., move right), comprehension of concept (e.g.,
pattern recognition), use of the tablet and original project creation, where childrens’ projects
are evaluated on their level of creativity and variety of Scratch Jr. blocks used in their
creation. All of these were evaluated by the researcher in the duration of the activity the
children were engaged with (e.g., the children were given an activity to understand pattern
recognition, the researcher takes notes on the rubric of the child’s level of understanding).
The rubric provides scores on each performance criterion and by adding them up (the
scores), a final score can be produced [60]. Completing the evaluation rubric was as follows:
during the production of projects completed in pairs with the Scratch Jr. software, the
researcher, who supervises children, completes an assessment rubric for each child. After
the evaluation on each project, the children’s overall performance is evaluated through the
results of all four rubrics completed for each child.

2.4. Procedure
Four sessions were performed in the whole preschool class and four tasks per pair,
which means that 48 tasks were completed in pairs with the method of guided collaborative
learning [61]. Sessions took place as a daily preschool activity within the program schedule.
Working in pairs was applied during free play, which allowed the researcher to focus on
the pairs during the creation of the projects in Scratch Jr. as all other children were engaged
in the classroom’s pedagogical corners.
Session 1. In the first session with Scratch Jr., all preschool children take an introductory
course on the application. The concepts of programming, command and sequence are
discussed and defined.
Session 2. In the second session children encounter the game character selection area
and learn how to change their color or size. They also learn about the background selection
process through examples.
Session 3. The third session is about speed, repeat, and the ability to adjust the
frequency of a command by selecting numbers on the tiles.
Session 4. Preschool children under the researcher’s guidance learn how to record and
add sounds to their characters. They are also introduced into alternative ways to start and
stop a program, such as by contributing another program to the script.

3. Results
3.1. Computational Thinking Level of Preschool Children through the Rubric
The analysis showed that preschool children performed well to excellent in all four
dimensions of assessment (Table 1). More specifically, they had an almost excellent perfor-
Educ. Sci. 2023, 13, 157 7 of 13

mance in creating an original project (M = 1.30, SD = 0.37) and in understanding commands


(M = 1.48, SD = 0.56). Similarly, they had almost excellent performance in understanding
concepts (M = 1.51, SD = 0.61). Moderate performance was obtained when using the tablet
(M = 1.97, SD = 0.96).

Table 1. Rubric evaluation results regarding the computational thinking level of preschool children.

Rubric Dimension M SD
Tablet use 1.97 0.96
Understanding concepts 1.51 0.61
Understanding commands 1.48 0.56
Creating an original project 1.30 0.37

Table 1 reveals the results of preschool children’s performance in tablet use. The
analysis showed that 15 of the 24 preschool children had a good to excellent performance
(values between 1 and 2), four of the 24 preschool children had a moderate to good
performance (values between 2 and 3) and five of the 24 preschool children performed
between moderate and poor (values between 3 and 4). Moreover, the analysis showed
that 20 of the 24 students had good to excellent performance (values between 1 and 2)
and four of the 24 students had moderate to good performance (values between 2 and 3).
None of the students had an average performance between moderate and poor (values
between 3 and 4). In addition, analysis showed that 22 of the 24 students had good to
excellent performance (values between 1 and 2) and two of the 24 students had moderate
to good performance (values between 2 and 3). None of the students had an average
performance between moderate and poor (values between 3 and 4). Finally, analysis
showed that 23 of the 24 students had good to excellent performance (values between 1
and 2) and one of the 24 students had moderate to good performance (values between 2
and 3). None of the students had an average performance between moderate and poor
(values between 3 and 4).
As expected, the four dimensions of the assessment are also correlated as seen in
Table 2. As we can see, all pairwise correlations are statistically significant with an r
(Pearson’s correlation coefficient) greater than 0.5. Only the pair “understanding concepts”
and “understanding commands” seems to lack a significant correlation.

Table 2. The correlation of the four dimensions of assessment. We used the Pearson correlation
coefficients. In parenthesis the p-value is given.

Understanding Understanding Creating an


Table Use
Concepts Commands Original Project
Table use 1.0000 - - -
Understanding
0.535 (0.0070) 1.0000 - -
concepts
Understanding
0.723 (0.0001) 0.322 (0.1244) 1.0000 -
commands
Creating an
0.619 (0.0012) 0.784 (<0.0001) 0.517 (0.0097) 1.0000
original project

3.2. Parents Attitude toward Computational Thinking


The results from parents’ (Table 3) interviews showed that their relationship with
digital media is very strong, with the data indicating daily long-term use of either a PC, or
a mobile phone with Android or iOS software, or a tablet.
P6 quoted: “Work time is half of entertainment time”
Most of the parents answered that they deal with the computer, the tablet and the
mobile phone more for entertainment reasons than for projects execution. Most of the
parents consider digital media to be a positively accepted human skill, a function they can
perform with their children. Fewer parents believe that long hours of digital media should
be avoided when their children are present, as there is a risk of addiction.
Educ. Sci. 2023, 13, 157 8 of 13

P1 quoted: “I let my child us the tablet on its own, but I have already checked and approved of
the applications used each time...”
Assessing the level of parental mediation, it was found that many parents choose to
be actively involved (active mediation).
P3 quoted: “...I see that he is able to grow into skills that I couldn’t when I was his age...the
more he engages with the device, the more he is able to familiarize with new apps...”
According to most parents, children have acquired several skills in handling digital
media, so children’s digital literacy can be achieved outside the classroom. Regarding the
positive changes, parents observe an increase in concentration and observation, acquain-
tance with ICT, acquisition of knowledge and also utilization of ICT as a means of gaining
new knowledge.
P2 quoted: “...even when we have agreed beforehand about the time given to use the device,
she still pushes for more, when time ends.”
In the negative consequences, parents classify the addiction and the appearance
of problematic behavior, such as unwarranted irritability. Regarding the connection of
applications and activities with cognitive object, parents replied that their children are
engaged in educational activities related to language, mathematics and the environment.
Furthermore, parents recognize that their children’s computer thinking is actually enhanced
through engaging with digital media.
Lastly, it is important to report some of parents’ beliefs on their children’s abilities as
noted through interviews.
P4 quoted: “He is rather young to be able to solve problems by using the computer.”
They seem to underestimate their children’s digital abilities although their children
have respectable skills as shown through their work with Scratch Jr.
P9 quoted: “No, I don’t think it is necessary to engage in programming activities at this age.”
Moreover, parents do not consider that preschool children are mature or capable
enough to solve problems or produce work through digital media. Plus, they do not feel
that preschool children should or can engage in programming activities.
Finally, none of the parents interviewed knew what CT is before given all related infor-
mation by the researcher or the existence of a possible connection between programming
and computational thinking.

Table 3. Parents sample.

Code Name Age Sex Education Number of Children

P1 25–29 Female High School Graduate 1


P2 35–39 Male University Degree 1
P3 30–34 Female High School Graduate 3
P4 30–34 Female University Degree 1
P5 35–39 Female University Degree 1
P6 35–39 Female University Degree 2
P7 40–44 Male MSc Degree 1
P8 35–39 Female High School Graduate 3
P9 40–44 Female High School Graduate 2
P10 35–39 Female University Degree 1

3.3. Teachers Attitudes toward Computational Thinking


The demographics of the preschool teachers that responded to the questionnaire are
given in Table 4. Results showed that only 8% (n = 8) of school units have no computers
while in 92% (n = 92) of the cases preschool teachers confirmed the existence of a computer
in their school units. In addition, it turned out that 75% (n = 75) of preschool teachers
allow the use of computers from children. Findings showed that a significant percentage of
preschool teachers use computers in the classroom for activities that are part of the learning
area of arts (n = 71, 71%), mathematics (n = 63, 63%), language (ν = 61, 61%) and ICT
Educ. Sci. 2023, 13, 157 9 of 13

(v = 50, 50%). Moreover, 71% (n = 71) of preschool teachers recognize that the cultivation
of CT of preschool children is the goal of the program in the preschool classroom, though
only 65% (n = 65) of preschool teachers recognize that the kindergarten curriculum should
include more than learning technological skills, as programming activities. Moreover, 53%
(n = 53) of preschool teachers agree or strongly agree with the fact that CT is a basic skill that
all children should have along with reading, writing and arithmetic, though 33% (n = 33)
of preschool teachers appear neutral towards this fact.
Open-ended questions showed that the majority of preschool teachers recognize the
usefulness of the computer as a tool for creative thinking and solving complex problems.
Finally, the majority of preschool teachers recognize that teaching basic programming con-
cepts develops digital eloquence, that is, the ability to design, create and link digital content.

Table 4. Kindergarten teachers’ demographic data.

n %
Male 0 0.0%
Sex
Female 100 100%
20–30 14 14.0%
31–40 34 34.0%
Age
41–50 37 37.0%
51 and over 15 15.0%
Kindergarten Teacher (4 year degree) 60 60.0%
Kindergarten Teacher (2 year degree) 7 7.0%
Education
MSc. 32 32.0%
PhD. 1 1.0%
0–5 years 18 18.0%
6–10 years 21 21.0%
11–15 years 16 16.0%
Teaching experience 16–20 years 22 22.0%
21–25 years 12 12.0%
26–30 years 7 7.0%
31 and over 4 4.0%

4. Discussion
The results indicated that preschool children performed well to excellent in all
four dimensions of assessment (concept comprehension, command comprehension, tablet
use, original project creation). More specifically, most children understood the concepts of
CT represented by software, such as the concept of programming, sequence and repetition,
which confirms the proposition that preschool children can understand basic programming
concepts [60]. Furthermore, most of the children understand the commands represented
by the blocks of the software. According to Resnick, engaging with programming envi-
ronments help children develop the process of drawing conclusions which is a skill that
they can use in the future school and their academic development [22]. It is important to
mention that preschool children had excellent performance in creating an original project.
Children in particular were asked to use their new knowledge and make something on their
own without specific instructions, thus releasing their creative thinking and relying on their
own decision-making mechanism [10,62]. Moreover, several research works have proved
that teaching the planning and development of CT contributes to preschool children’s
cognitive skills [63,64].
The evaluation revealed that preschool children had a moderate performance in terms
of the tablet use, i.e., they used it but often asked for help. This fact is backed up by a recent
survey conducted by Konstantinopoulou, where errors in Scratch Jr. are very often tablet
handling errors [65]. The particular realization does not surprise us, since visual-motor
coordination in preschool is a developing skill. In fact, dealing with ICT can support the
Educ. Sci. 2023, 13, 157 10 of 13

development of specific motor skills, such as practicing hand-eye coordination through


playful activities for the use of mouse and keyboard in educational software.
Analyzing the data that we collected from parents’ interviews, we discovered elements
about their personal involvement with technological means, their involvement or not in
the use of technological means by their children and the role of technological means in the
development of PC and cognitive skills of preschool children. Parents who participated in
the research appear to be closely connected with technology, as they use electronic digital
media on a daily basis and even consider it ‘socially normal’ [66]. Most parents are positive
about their children’s involvement in technology as long as they allow or use technology
with their child for recreational purposes, while the ones avoiding it argued that there are
risks regarding their children such as addiction. Both groups of parents tend to use digital
media with their children mainly for entertainment, ignoring other uses, such as work,
education, the search for new knowledge, etc. As regards CT, preschool children’s parents
seem to agree that the applications that their children are involved in develop CT, as they
treat “digital media as learning tools” [67]. On the contrary, they do not acknowledge any
relation regarding programming and CT, nor do they know what CT exactly includes.
Having a computer in the classroom does not necessarily imply its use by preschool
children, although 75% of teachers say they allow children to use it. It is worth mentioning
the fact that new teachers with little teaching experience (under 5 years) do not allow
the use of the computer in their classroom, a fact that could be justified by the stress of
preschool teachers during their first years at work [68]. The essential reasons for using
computers and ICT in the preschool classroom are transformation of educational practices
and integration of all curricula subjects—even physical education. From our research we
concluded that a significant percentage of preschool teachers agree that an effort has to be
made so that CT in kindergarten is developed [69].

5. Conclusions
The aim of this work was to investigate preschool children’s CT and answer questions
about their readiness to develop and use CT in programming environments, as well as to
highlight the views of parents and preschool teachers on CT. The study was conducted in
a preschool classroom with Scratch Jr. 1.2.7 software. It is worth mentioning that evaluation
rubrics demonstrated that the ability to use the tablet had an upward trend for each student
from the first to the last project. In addition to the development of CT, dealing with the
software also improved children’s ability to handle the tablet. Papadakis claimed that
children who participate in programming interventions in the classroom, even if they
have short duration, maximize their kinesthetic experiences, and still show significant
improvement in fundamental skills, such as literacy and math skills [70]. Lastly, children
who learn computational skills and computer science concepts could gain problem-solving
strategies that are considered to be a way of human thinking to facilitate their learning and
living [71].
Rightly so, “all the advanced Western countries have integrated or intend to integrate
directly into their educational systems teaching of Programming and Computer Science
already at Preschool Education level” [5]. With the current development of digital technolo-
gies and the concerns about CT literacy, how teacher education should prepare teachers to
teach CT is an important question to be studied [72].
There are few limitations of the present study that should be addressed. The first
one is the sample size of the study. The sample size (24 preschool children, 10 parents of
preschool children and 100 preschool teachers) allows us to draw some conclusions but
prevents us from generalizing the results [55]. It is reasonable that future research could
shed more light on the issue of CT in preschool children. Additionally, another limitation
is the lack of a control group of students against which the comparison would have been
performed. Alternatively, the students could have been evaluated prior to the intervention
and the relative performance may have been evaluated. The CT could also be studied
Educ. Sci. 2023, 13, 157 11 of 13

under considering other parameters; for example: (a) in relation to the place of residence of
preschool children and (b) regarding educational software of non-programming content.

Author Contributions: Conceptualization, Z.K. and E.-A.P.-K.; methodology, P.G.B. and E.-A.P.-K.;
validation, Z.K., C.-A.M., P.G.B. and E.-A.P.-K.; formal analysis, Z.K., C.-A.M., P.G.B. and E.-A.P.-K.;
investigation, Z.K., C.-A.M., P.G.B. and E.-A.P.-K.; resources, Z.K.; data curation, Z.K. and C.-A.M.;
writing—original draft preparation, Z.K.; writing—review and editing, C.-A.M., P.G.B. and E.-A.P.-K.;
supervision, P.G.B. and E.-A.P.-K. All authors have read and agreed to the published version of
the manuscript.
Funding: This research received no external funding.
Institutional Review Board Statement: Ethical review and approval of this study was given from
the appropriate departmental committee.
Informed Consent Statement: Informed consent was obtained from all subjects involved in the study.
Data Availability Statement: The data presented in this study are available on request from the
corresponding author. The data are not publicly available due to reasons of privacy.
Conflicts of Interest: The authors declare no conflict of interest.

References
1. Wang, P.S. From Computing to Computational Thinking; Chapman and Hall/CRC: Boca Raton, FL, USA, 2017.
2. Wing, J.M. Computational thinking. Commun. ACM 2006, 49, 33–35. [CrossRef]
3. Wang, D.; Wang, T.; Liu, Z. A tangible programming tool for children to cultivate computational thinking. Sci. World J. 2014, 2014,
428080. [CrossRef] [PubMed]
4. Ntoliopoulou, E.; Sousloglou, K. Kindergarten Teacher’s thoughts on the Curriculum [Apopseis nhpiagogon gia tin efarmogi ton
DEEPS gia to nhpiagogeio. Investig. Child’s World Ereun. Ton Kosmo Tou Paidiou. 2007, 7, 123–149. [CrossRef]
5. Zaranis, N.; Papadakis, S.; Kalogiannakis, M. Evaluation of educational technologies for the Promotion of Computational
Thinking in Preschool Education Aksiologisi ton Ekpaideutikon Texnologion gia tin Proothisi tis Ypologistikis Skepsis stin
Prosxoliki Ekpaideusi. Ekpaideusi kai Dia Viou Mathisi, Ereuna kai Texnologiki Anaptiksi Kainotomia kai Oikonomia.
Educ. Lifelong Learn. Res. Technol. Dev. Innov. Econ. 2019, 2, 77–86. [CrossRef]
6. Webster, F. Theories of the Information Society; Routledge: London, UK, 2014.
7. Ananiadou, K.; Claro, M. 21st Century Skills and Competences for New Millennium Learners in OECD Countries. In OECD
Education Working Papers; OECO: Paris, France, 2009. [CrossRef]
8. Silva, E. Measuring skills for 21st-century learning. Phi Delta Kappan 2009, 90, 630–634. [CrossRef]
9. Paraskevopoulou-Kollia, E.-A.; Michalakopoulos, C.-A.; Kontou, P.; Vogiatzoglou, A. The Informatics course and the performance
of the students: Views [To mathima tis Pliroforikis kai i epidosi ton mathitrion/on: Apopsis]. In Proceedings of the Greece-Europe
2020: Education, Lifelong Learning, Research and Technological Development, Innovation and Economy [Ellada-Evropi 2020:
Ekpedefsi, Dia Viou Mathisi, Ereyna ke Texnologiki Anaptiksi, Kenotomia kai Ikonomia], Piraeus, Greece, 28–30 September 2018;
pp. 374–385.
10. Papert, S. Mindstorms: Children, Computers, and Powerful Ideas; Basic Books: New York, NY, USA, 1980.
11. Barr, D.; Harrison, J.; Conery, L. Computational thinking: A digital age skill for everyone. Learn. Lead. Technol. 2011, 38, 20–23.
12. Li, Y.; Schoenfeld, A.H.; di Sessa, A.A.; Graesser, A.C.; Benson, L.C.; English, L.D.; Duschl, R.A. Computational thinking is more
about thinking than computing. J. STEM Educ. Res. 2020, 3, 1–18. [CrossRef] [PubMed]
13. Bers, M.U. Thinking About Computational Thinking. In Coding as a Playground; Routledge: New York, NY, USA, 2017; pp. 57–69.
14. Barr, V.; Stephenson, C. Bringing computational thinking to K-12: What is Involved and what is the role of the computer science
education community? Acm Inroads 2011, 2, 48–54. [CrossRef]
15. Kramer, J. Is abstraction the key to computing? Commun. ACM 2007, 50, 36–42. [CrossRef]
16. Atmatzidou, S.; Demetriadis, S. Advancing students’ computational thinking skills through educational robotics: A study on age
and gender relevant differences. Robot. Auton. Syst. 2016, 75, 661–670. [CrossRef]
17. ISTE. Available online: https://bit.ly/30fpQcB (accessed on 27 December 2022).
18. Howland, K.; Good, J.; Nicholson, K. Language-based support for computational thinking. In Proceedings of the 2009 IEEE
Symposium on Visual Languages and Human-Centric Computing (VL/HCC), Washington, DC, USA, 20–24 September 2009;
pp. 147–150.
19. NRC, N.R.C. Report of a Workshop on the Scope and Nature of Computational Thinking; National Academies Press: Washington, DC,
USA, 2010.
20. NRC, N.R.C. Funding a Revolution: Government Support for Computing Research; National Academies Press: Washington, DC, USA, 1999.
21. Arfé, B.; Vardanega, T.; Montuori, C.; Lavanga, M. Coding in primary grades boosts children’s executive functions. Front. Psychol.
2019, 10, 2713. [CrossRef] [PubMed]
Educ. Sci. 2023, 13, 157 12 of 13

22. Resnick, M.; Maloney, J.; Monroy-Hernández, A.; Rusk, N.; Eastmond, E.; Brennan, K.; Millner, A.; Rosenbaum, E.; Silver, J.;
Silverman, B. Scratch: Programming for all. Commun. ACM 2009, 52, 60–67. [CrossRef]
23. Deo, N. Graph Theory with Applications to Engineering and Computer Science; Courier Dover Publications: Mineola, NY, USA, 2017.
24. Kalogiannakis, M.; Papadakis, S. Pre-service kindergarten teachers acceptance of “ScratchJr” as a tool for learning and teaching
computational thinking and Science education. In Proceedings of the 12th Conference of the European Science Education Research
Association (ESERA), Research, Practice and Collaboration in Science Education, Dublin, Ireland, 21–25 August 2017; pp. 21–25.
25. Yang, W.; Ng, D.T.K.; Gao, H. Robot programming versus block play in early childhood education: Effects on computational
thinking, sequencing ability, and self-regulation. Br. J. Educ. Technol. 2022, 53, 1817–1841. [CrossRef]
26. Buitrago Flórez, F.; Casallas, R.; Hernández, M.; Reyes, A.; Restrepo, S.; Danies, G. Changing a generation’s way of thinking:
Teaching computational thinking through programming. Rev. Educ. Res. 2017, 87, 834–860. [CrossRef]
27. Di Lieto, M.C.; Inguaggiato, E.; Castro, E.; Cecchi, F.; Cioni, G.; Dell’Omo, M.; Laschi, C.; Pecini, C.; Santerini, G.; Sgandurra, G.
Educational Robotics intervention on Executive Functions in preschool children: A pilot study. Comput. Hum. Behav. 2017, 71,
16–23. [CrossRef]
28. Strawhacker, A.; Lee, M.; Bers, M.U. Teaching tools, teachers’ rules: Exploring the impact of teaching styles on young children’s
programming knowledge in ScratchJr. Int. J. Technol. Des. Educ. 2018, 28, 347–376. [CrossRef]
29. Papadakis, S.; Kalogiannakis, M.; Zaranis, N. Developing fundamental programming concepts and computational thinking with
ScratchJr in preschool education: A case study. Int. J. Mob. Learn. Organ. 2016, 10, 187–202. [CrossRef]
30. Tang, K.-Y.; Chou, T.-L.; Tsai, C.-C. A content analysis of computational thinking research: An international publication trends
and research typology. Asia-Pac. Educ. Res. 2020, 29, 9–19. [CrossRef]
31. Rich, P.J.; Bly, N.; Leatham, K.R. Beyond cognitive increase: Investigating the influence of computer programming on perception
and application of mathematical skills. J. Comput. Math. Sci. Teach. 2014, 33, 103–128.
32. Ehsan, H.; Rehmat, A.P.; Osman, H.; Ohland, C.; Cardella, M.E.; Yeter, I.H. Examining the role of parents in promoting
computational thinking in children: A case study on one homeschool family (Fundamental). In Proceedings of the 2019 ASEE
Annual Conference & Exposition, Tampa, FL, USA, 15–19 June 2019.
33. Yu, J.; Bai, C.; Roque, R. Considering parents in coding kit design: Understanding parents’ perspectives and roles. In Proceedings
of the 2020 CHI Conference on Human Factors in Computing Systems, Honolulu, HI, USA, 25–30 April 2020; pp. 1–14.
34. Papadakis, S.; Zaranis, N.; Kalogiannakis, M. Parental involvement and attitudes towards young Greek children’s mobile usage.
Int. J. Child-Comput. Interact. 2019, 22, 100144. [CrossRef]
35. Ohland, C.; Ehsan, H.; Cardella, M.E. Parental Influence on Children’s Computational Thinking in an Informal Setting (Funda-
mental Research). In Proceedings of the 2019 ASEE Annual Conference & Exposition, Tampa, FL, USA, 15–19 June 2019.
36. Sentance, S.; Csizmadia, A. Computing in the curriculum: Challenges and strategies from a teacher’s perspective.
Educ. Inf. Technol. 2017, 22, 469–495. [CrossRef]
37. Rich, K.M.; Yadav, A.; Schwarz, C.V. Computational thinking, mathematics, and science: Elementary teachers’ perspectives on
integration. J. Technol. Teach. Educ. 2019, 27, 165–205.
38. Duncan, C.; Bell, T.; Atlas, J. What do the teachers think? Introducing computational thinking in the primary school
curriculum. In Proceedings of the Nineteenth Australasian Computing Education Conference, Geelong, VIC, Australia,
31 January–3 February 2017; pp. 65–74.
39. Love, T.S.; Bartholomew, S.R.; Yauney, J. Examining changes in teachers’ beliefs toward integrating computational thinking to
teach literacy and math concepts in grades K-2. J. STEM Educ. Res. 2022, 5, 380–401. [CrossRef]
40. Cabrera, L. Teacher preconceptions of computational thinking: A systematic literature review. J. Technol. Teach. Educ. 2019, 27,
305–333.
41. Louka, K. Programming environments for the development of CT in preschool education: A systematic literature review.
Adv. Mob. Learn. Educ. Res. 2023, 3, 525–540. [CrossRef]
42. Fessakis, G.; Prantsoudi, S. Computer Science Teachers’ Perceptions, Beliefs and Attitudes on Computational Thinking in Greece.
Inform. Educ. 2019, 18, 227–258. [CrossRef]
43. González-González, C.S. State of the art in the teaching of computational thinking and programming in childhood education.
Educ. Knowl. Soc. 2019, 20, 1–15.
44. Grobotronics Bee-Bot. Available online: https://bit.ly/3sSF8fJ (accessed on 28 December 2022).
45. Grobotronics Blue-Bot. Available online: https://bit.ly/3dSDg2k (accessed on 29 December 2022).
46. Grobotronics Rugged Robot. Available online: https://bit.ly/3eCN13Q (accessed on 27 December 2022).
47. Mattel Code-a-Pillar Twist Game. Available online: https://bit.ly/3sRFpj0 (accessed on 28 December 2022).
48. Mattel Code ‘n’ Learn Kinderbot. Available online: https://bit.ly/3sNuoip (accessed on 29 December 2022).
49. Learning Resources Code & Go Robot Mouse. Available online: https://bit.ly/3sSmfcG (accessed on 18 December 2021).
50. Learning Resources Coding Critters Rumble & Bumble. Available online: https://bit.ly/3aG2MWo (accessed on 17 December 2021).
51. KinderLabrobotics. Available online: https://bit.ly/3xomtvF (accessed on 28 December 2022).
52. Scratchjr. Available online: https://bit.ly/32Ssz9z (accessed on 28 December 2022).
53. Creswell, J.W.; Poth, C.N. Qualitative Inquiry and Research Design: Choosing among Five Approaches, 4th ed.; Sage Publications:
Thousand Oaks, CA, USA, 2016.
Educ. Sci. 2023, 13, 157 13 of 13

54. Drazin, A. Book review: NORMAN K. DENZIN and YVONNA S. LINCOLN, The Sage Handbook of Qualitative Research.
London: Sage, 2005. 1288 pp. ISBN 07619 2757 3 (hbk)£ 85.00. Qual. Res. 2009, 9, 383–385. [CrossRef]
55. Cohen, L.; Manion, L.; Morrison, K. Research Methods in Education, 6th ed.; Routledge: Oxford, NY, USA, 2007.
56. Robson, C. Real World Research: A Resource for Social Scientists and Practitioner-Researchers, 2nd ed.; Blackwell Publishers Ltd.:
Oxford, UK, 2002.
57. Creswell, J.W.; Creswell, J.D. Research design: Qualitative, Quantitative, and Mixed Methods Approaches; Sage Publications: New York,
NY, USA, 2017.
58. Panadero, E.; Jonsson, A. The use of scoring rubrics for formative assessment purposes revisited: A review. Educ. Res. Rev. 2013,
9, 129–144. [CrossRef]
59. Petropoulou, O.; Vassilikopoulou, M.; Retalis, S. Enriched assessment rubrics: A new medium for enabling teachers to easily assess
student’s performance when participating in complex interactive learning scenarios. Oper. Res. 2011, 11, 171–186. [CrossRef]
60. Bers, M.U.; Horn, M.S. Tangible programming in early childhood: Revisiting developmental assumptions through new technolo-
gies. In High-Tech Tots: Childhood in a Digital World; Information Age Publishing: Charlotte, NC, USA, 2010; pp. 49–70.
61. Paraskevopoulou-Kollia, E.-A.; Michalakopoulos, C.-A. The Most Suitable Way of Teaching. J. Educ. Pract. 2021, 12, 4–17.
[CrossRef]
62. Shute, V.J.; Sun, C.; Asbell-Clarke, J. Demystifying computational thinking. Educ. Res. Rev. 2017, 22, 142–158. [CrossRef]
63. Kazakoff, E.; Bers, M. Programming in a robotics context in the kindergarten classroom: The impact on sequencing skills. J. Educ.
Multimed. Hypermedia 2012, 21, 371–391.
64. Kazakoff, E.R.; Sullivan, A.; Bers, M.U. The effect of a classroom-based intensive robotics and programming workshop on
sequencing ability in early childhood. Early Child. Educ. J. 2013, 41, 245–255. [CrossRef]
65. Konstantinopoulou, E.-M. Programming Mistakes of Preschool Students in Scratch Junior Software. Ta Programmatistika Lathi
Ton Mathiton Prosxolikis Ilikias Mesa Apo to Optiko Programmatistiko Periballon Scratch Junior. Master’s Thesis, University of
Patras, Patra, Greece, 2019.
66. Goffman, E. Stigma and social identity. In Deviance & Liberty; Routledge: Oxfordshire, UK, 2018; pp. 24–31.
67. Vandewater, E.A.; Rideout, V.J.; Wartella, E.A.; Huang, X.; Lee, J.H.; Shim, M.-s. Digital childhood: Electronic media and
technology use among infants, toddlers, and preschoolers. Pediatrics 2007, 119, e1006–e1015. [CrossRef]
68. Karasavvidis, I.; Kollias, V. Technology integration in the most favorable conditions: Findings from a professional development
training program. In Research on E-Learning and ICT in Education; Springer: Berlin/Heidelberg, Germany, 2014; pp. 197–224.
69. Magdalinou, K.; Papadakis, S. The use of educational scenarios using state-of-the-art IT technologies such as ubiquitous
computing, mobile computing and the internet of things as an incentive to choose a scientific career. In Proceedings of the
Interactive Mobile Communication, Technologies and Learning, Hamilton, ON, Canada, 11–12 October 2018; pp. 915–923.
70. Papadakis, S. The use of computer games in classroom environment. Int. J. Teach. Case Stud. 2018, 9, 1–25. [CrossRef]
71. Su, J.; Yang, W. A Systematic Review of Integrating Computational Thinking in Early Childhood Education. Comput. Educ. Open
2023, 4, 100122. [CrossRef]
72. Kong, S.-C.; Abelson, H. Computational Thinking Education; Springer Nature: Berlin, Germany, 2019.

Disclaimer/Publisher’s Note: The statements, opinions and data contained in all publications are solely those of the individual
author(s) and contributor(s) and not of MDPI and/or the editor(s). MDPI and/or the editor(s) disclaim responsibility for any injury to
people or property resulting from any ideas, methods, instructions or products referred to in the content.

You might also like