Session: Tips, techniques, and courseware ITiCSE 2022, July 8–13, 2022, Dublin, Ireland

A Capture The Flag (CTF) Platform and Exercises for an Intro to

Computer Security Class
Zack Kaplan Ning Zhang Stephen V. Cole
[email protected] [email protected] [email protected]
Washington University in St. Louis Washington University in St. Louis Washington University in St. Louis
St. Louis, Missouri, USA St. Louis, Missouri, USA St. Louis, Missouri, USA
ABSTRACT which cover a wide range of security topics from systems security
Cybersecurity education is becoming increasingly important as to web security and run on a freely-available customized VM image.
demand for cybersecurity professionals increases. Hands-on skills To offer students the benefits of gamification for student en-
are a critical component of cybersecurity education, and a variety gagement and learning outcomes [3][4] in our Intro to Computer
of exercise types have been developed to teach these skills. In Security course, which has used several SEED labs as a subset of our
this work, we seek to apply the benefits of gamified learning to hands-on exercises for multiple semesters, we developed and incor-
an introductory cybersecurity curriculum in the form of a set of porated Capture The Flag (CTF) exercises augmenting our existing
Capture the Flag (CTF) challenges offered as hands-on exercises for hands-on exercises in the Summer 2021 and Fall 2021 offerings of
an intro-level course. We created 20 jeopardy-style challenges of the course. We used single-player “jeopardy-style” puzzle-solving
varying difficulty based on prior research on the use of gamification CTF challenges suitable for incorporation into a typical security
in education, and we configured the open-source CTFd platform to course as assignments. The hallmarks of such challenges are that
host our challenges. Student responses to post-challenge surveys they have a tangible token of successful completion – usually an
suggest that the CTF component of the course was effective in un-guessable text string a player discovers when she solves the
improving perceived learning and student engagement. challenge – and that they are deliberately under-specified in their
solution path to varying degrees, so that a player must exercise
CCS CONCEPTS ingenuity or do outside research to solve the challenges.
In this work, we present the CTF platform and exercises de-
• Security and privacy; • Social and professional topics →
veloped for the course, and report on the implementation of the
Computer science education;
exercises as measured by a student survey taken upon the com-
KEYWORDS pletion of the CTFs. Because all the CTF exercises are designed to
reinforce concepts in the SEED labs, we hope many will be of direct
CTF, Capture The Flag, cybersecurity, gamification, student engage- interest to the cybersecurity education community.
ment, hands-on exercise
ACM Reference Format: 2 RELATED WORK
Zack Kaplan, Ning Zhang, and Stephen V. Cole. 2022. A Capture The Flag
Previous work on CTFs in cybersecurity education has shown that
(CTF) Platform and Exercises for an Intro to Computer Security Class. In
Proceedings of the 27th ACM Conference on Innovation and Technology in they produce positive learning outcomes, improved grades, and
Computer Science Education Vol 2 (ITiCSE 2022), July 8–13, 2022, Dublin, increased confidence in cybersecurity skills (see [2, 5] for exam-
Ireland. ACM, New York, NY, USA, 2 pages. https://doi.org/10.1145/3502717. ples). Our work is novel in (1) designing many new CTF challenges
3532153 specifically to complement the hands-on exercises in our Intro to
Security course, including the SEED labs used in the course, and (2)
1 INTRODUCTION in configuring the CTFd platform for a smooth, repeatable, exten-
To address the increasing demand for cybersecurity professionals sible hosting of this set of challenges by future instructors at our
with hands-on skills and the projected deficiency in qualified secu- own or other institutions.
rity professionals over the next several years [1], security educators
have developed a variety of hands-on exercises to complement tra- 3 PLATFORM AND EXERCISES
ditional learning methods. One of the most widely deployed sets of 3.1 CTF Hosting
exercises for college-level cybersecurity classes is the SEED Labs1 ,
To create and host the CTF challenges, we used the popular open-
0 This work is supported in part by the U.S. National Science Foundation under grants source CTF-hosting framework CTFd2 to publish challenges and
CNS-1916926 and CNS-2038995. track student progress, with a campus-network-connected Linux
1 https://seedsecuritylabs.org/
server hosting our CTFd instance. Students created accounts using
Permission to make digital or hard copies of part or all of this work for personal or their school email addresses, and interacted with the server via a
classroom use is granted without fee provided that copies are not made or distributed
for profit or commercial advantage and that copies bear this notice and the full citation
web browser to complete challenges and earn points. CTFd supports
on the first page. Copyrights for third-party components of this work must be honored. the import/export of challenge configuration files, allowing for easy
For all other uses, contact the owner/author(s). plug-and-play setup and backup for hosts or instructors. Features
ITiCSE 2022, July 8–13, 2022, Dublin, Ireland
© 2022 Copyright held by the owner/author(s).
such as progressive unlocking of challenges allow instructors to
ACM ISBN 978-1-4503-9200-6/22/07.
https://doi.org/10.1145/3502717.3532153 2 https://ctfd.io/

597
Session: Tips, techniques, and courseware ITiCSE 2022, July 8–13, 2022, Dublin, Ireland

Table 1: Post-CTFs survey results: average ratings of the per-

ceived impact of the CTF challenges on the categories shown.
Response options ranged from 1 (strong negative impact) to
5 (strong positive impact).

Prompt Avg. Rating Std. Dev.

Learning Experience 4.06 0.70
Understanding of Course Material 3.88 0.89
Security Skills Improvement 4.06 0.79
Security Skills Confidence 3.88 0.93
Engagement vs. Other Course Material 4.00 1.00
Course Performance 3.67 0.96
Figure 1: Server architecture and connection types for the Overall Cybersecurity Understanding 3.94 0.75
course’s CTF system. The administrator uses ssh to set up
Docker containers (including one for CTFd) on the server,
which are then accessible to users. CTFd is configured scattered throughout the instructions and hints, in order to promote
through a web interface after the containers are launched, enjoyment and decrease stress when solving.
and challenge containers have their own separate ports for
remote connections. 4 IMPLEMENTATION
The CTF platform and challenges were presented during two ses-
sions of the class, once as a pilot run during the Summer 2021
enforce dependency chains in completing exercises and to pre- session and once during the Fall 2021 session.
publish all exercises for a course at the beginning of a term, and For the Fall 2021 offering, participation was required in that
features such as an automatic scoreboard allow students to easily the equivalent of completing all 10-point challenges counted as
track their own progress throughout the semester. a graded assignment in the course, and earning additional points
counted for extra credit. A survey collecting learner feedback on
3.2 CTF Challenges the CTFs was given to students most of the way through the session,
A series of 20 CTF challenges were created for this work. The chal- and counted as the equivalent of a 10-point challenge toward their
lenges fall under the general categories of Encryption, Network- grade. The survey asked students to rate the CTFs’ impact in several
ing, Linux, Web, Steganography, and Exploitation, and comprise 3 items on a 5-point Likert scale, with 5 representing a strong positive
server-style challenges requiring Docker containers, 11 challenges impact and 1 representing a strong negative impact.
adapted to CTF style from course assignments and labs, 4 custom-
built CTF-style challenges, and 2 lightly-modified challenges based 5 SURVEY RESULTS
on previous CTF competitions. Twelve easier challenges require Results from the 𝑛=33 survey respondents are summarized in Ta-
the straightforward application of course material and SEED lab ble 1. The results suggest that students perceived increased engage-
knowledge and were worth 10 points each, while eight harder chal- ment and positive learning outcomes across a variety of metrics
lenges require creative application of course material or outside from working on the CTFs.
research and were worth 20-50 points each. Challenges include:
6 MATERIALS
• A 10-point challenge requiring students to decrypt an en-
crypted message using openSSL (a tool used in a correspond- All necessary files and instructions for setting up and managing
ing SEED lab), with a riddle giving hints about the Key and IV the CTF challenges are stored in a Github repository accessible to
encryption parameters needed to complete the decryption. course instructors and available to other instructors upon request.
• A 20-point challenge requiring students to use a brute-forcing
tool to crack the password of an encrypted zip folder. REFERENCES
[1] William Crumpler and James A Lewis. 2019. The cybersecurity workforce gap.
• A 20-point challenge requiring students to ssh into a con- Center for Strategic and International Studies (CSIS) Washington, DC, USA.
tainer and find different user credentials for the same con- [2] Juho Holmi. 2020. Advantages and challenges of using capture-the-flag games in
tainer, then log in with those new credentials and search cyber security education. Bachelor’s Thesis, University of Oulu.
[3] Chanut Poondej and Thanita Lerdpornkulrat. 2016. The development of gami-
through the many files on the system for the flag using com- fied learning activities to increase student engagement in learning. Australian
mon Linux techniques. Educational Computing 31, 2 (2016).
• A 50-point challenge requiring students to execute a buffer [4] Johnmarshall Reeve and Woogul Lee. 2014. Students’ classroom engagement
produces longitudinal changes in classroom motivation. Journal of educational
overflow exploit on a vulnerable program. psychology 106, 2 (2014), 527.
[5] Daniel Votipka, Eric Zhang, and Michelle L Mazurek. 2021. HackEd: A Pedagogical
All challenges were designed to be light-hearted, with many Analysis of Online Vulnerability Discovery Exercises. In 2021 IEEE Symposium on
jokes and puns (designed with diversity and inclusion in mind) Security and Privacy (SP). IEEE, 1268–1285.

598