Measuring Programming Experience

kaestner@cs.cmu.edu (J. Feigenspan, C. Kästner, J. Liebig, S. Apel, S. Hanenberg)
Fri, 4 Mar 2022 11:51:53 GMT

Janet Feigenspan, Christian Kästner, Jörg Liebig, Sven Apel, and Stefan Hanenberg. Measuring Programming Experience. In Proceedings of the 20th International Conference on Program Comprehension (ICPC), pages 73--82, Los Alamitos, CA: IEEE Computer Society, 2012. Most Influencial Paper Award at ICPC'22.

Programming experience is an important confounding parameter in controlled experiments regarding program comprehension. In literature, ways to measure or control programming experience vary. Often, researchers neglect it or do not specify how they controlled it. We set out to find a well-defined understanding of programming experience and a way to measure it. From published comprehension experiments, we extracted questions that assess programming experience. In a controlled experiment, we compare the answers of 128 students to these questions with their performance in solving program-comprehension tasks. We found that self estimation seems to be a reliable way to measure programming experience. Furthermore, we applied exploratory factor analysis to extract a model of programming experience. With our analysis, we initiate a path toward measuring programming experience with a valid and reliable tool, so that we can control its influence on program comprehension.