See discussions, stats, and author profiles for this publication at: https://www.researchgate.

net/publication/358716967

An Exploratory Study of Productivity Perceptions in Software Teams

Conference Paper · July 2022

DOI: 10.1145/3510003.3510081

CITATIONS READS

4 489

7 authors, including:

Anastasia Ruvimova Alexander Lill

University of Zurich University of Zurich
3 PUBLICATIONS 57 CITATIONS 10 PUBLICATIONS 37 CITATIONS

SEE PROFILE SEE PROFILE

Jan Gugler Lauren Howe

1 PUBLICATION 4 CITATIONS
University of Zurich
29 PUBLICATIONS 760 CITATIONS
SEE PROFILE
SEE PROFILE

All content following this page was uploaded by Alexander Lill on 20 February 2023.

The user has requested enhancement of the downloaded file.

 Zurich Open Repository and
Archive
University of Zurich
University Library
Strickhofstrasse 39
CH-8057 Zurich
www.zora.uzh.ch

Year: 2022

An Exploratory Study of Productivity Perceptions in Software Teams

Ruvimova, Anastasia ; Lill, Alexander ; Gugler, Jan ; Howe, Lauren ; Huang, Elaine ; Murphy, Gail ; Fritz, Thomas

Abstract: Software development is a collaborative process requiring a careful balance of focused individual effort
and team coordination. Though questions of individual productivity have been widely examined in past literature,
less is known about the interplay between developers’ perceptions of their own productivity as opposed to their
team’s. In this paper, we present an analysis of 624 daily surveys and 2899 self-reports from 25 individuals across
five software teams in North America and Europe, collected over the course of three months. We found that
developers tend to operate in fluid team constructs, which impacts team awareness and complicates gauging
team productivity. We also found that perceived individual productivity most strongly predicted perceived team
productivity, even more than the amount of team interactions, unplanned work, and time spent in meetings.
Future research should explore how fluid team structures impact individual and organizational productivity.

DOI: https://doi.org/10.1145/3510003.3510081

Posted at the Zurich Open Repository and Archive, University of Zurich

ZORA URL: https://doi.org/10.5167/uzh-216758
Conference or Workshop Item
Published Version

Originally published at:

Ruvimova, Anastasia; Lill, Alexander; Gugler, Jan; Howe, Lauren; Huang, Elaine; Murphy, Gail; Fritz, Thomas
(2022). An Exploratory Study of Productivity Perceptions in Software Teams. In: International Conference on
Software Engineering (ICSE’22), Pittsburgh, PA, USA, 22 May 2022 - 27 May 2022. ACM, 1-13.
DOI: https://doi.org/10.1145/3510003.3510081
 An Exploratory Study of Productivity Perceptions in Software
Teams
Anastasia Ruvimova Alexander Lill Jan Gugler Lauren Howe
University of Zurich University of Zurich University of Zurich University of Zurich
Zurich, Switzerland Zurich, Switzerland Zurich, Switzerland Zurich, Switzerland
[email protected] [email protected] [email protected] [email protected]

Elaine Huang Gail Murphy Thomas Fritz

University of Zurich University of British University of Zurich
Zurich, Switzerland Columbia Zurich, Switzerland
[email protected] Vancouver, Canada [email protected]
[email protected]

ABSTRACT for software development productivity [22, 27] and factors that
Software development is a collaborative process requiring a careful impact software development productivity (e.g., [29]). Recognizing
balance of focused individual effort and team coordination. Though the significant impact individuals have on the productivity of a soft-
questions of individual productivity have been widely examined ware development organization, recent research has placed more
in past literature, less is known about the interplay between devel- emphasis on the individuals involved in the development process,
opers’ perceptions of their own productivity as opposed to their and in particular, on how they view their own productivity [31, 40].
team’s. In this paper, we present an analysis of 624 daily surveys Recent research also suggests that the measurement of produc-
and 2899 self-reports from 25 individuals across five software teams tivity needs to include multiple dimensions of individual metrics,
in North America and Europe, collected over the course of three including such aspects as satisfaction, activity, and flow [29, 35].
months. We found that developers tend to operate in fluid team However, individuals do not work alone on a software develop-
constructs, which impacts team awareness and complicates gauging ment project. Software development teams, comprising multiple
team productivity. We also found that perceived individual produc- individuals, are at the heart of most complex software development
tivity most strongly predicted perceived team productivity, even projects. Individuals constantly must make trade-offs to balance
more than the amount of team interactions, unplanned work, and making progress on their individual work versus helping a team-
time spent in meetings. Future research should explore how fluid mate (e.g., [39]). Despite the necessary and central role of teams in
team structures impact individual and organizational productivity. software development, surprisingly few studies focus on team soft-
ware development productivity, and those that do, study a snapshot
CCS CONCEPTS of development. For instance, Lakhanpa et al., using a one-time
survey, found team cohesion to be the dominating factor when
· Software and its engineering → Programming teams; Soft-
investigating team performance [32]. These studies leave open
ware development methods.
many questions about the interplay between individual and team
productivity. We focus our analysis on the following:
KEYWORDS
RQ1: What is the relationship between perceived team and indi-
Team, Productivity, Software Developer, User Study vidual productivity in software teams?
RQ2: Which team factors affect individual and team productivity
1 INTRODUCTION perceptions?
Software development organizations face constant pressure to in- To investigate these questions, we undertook a three-month
crease the speed, volume, and quality of their software production. multi-modal study in which we collected daily self-reports from
As companies may not have the resources to address these goals 25 software developers within five companies in North America
by increasing team size, organizations must look to improving the and Europe. These surveys, collected on a daily and bi-hourly basis,
productivity of those involved in producing the software. A desire asked participants to report their individual and team productivity,
to improve software development productivity is not new: Boehm and to reflect on a variety of factors which affected their productiv-
wrote eloquently about the need for improved software productivity ity that day, such as team interactions, time spent in meetings and
in the late 1980s [8]. Many have since studied appropriate measures the amount of unplanned work they had done. Additionally, we
held a series of three interviews with each developer, spaced one
Permission to make digital or hard copies of part or all of this work for personal or month apart, to capture any changes in their team or work, and
classroom use is granted without fee provided that copies are not made or distributed
for profit or commercial advantage and that copies bear this notice and the full citation to ask broader questions about their perceptions of productivity
on the first page. Copyrights for third-party components of this work must be honored. and teamwork. In total, we collected 624 daily surveys, and 2899
For all other uses, contact the owner/author(s).
bi-hourly self-reports.
ICSE ’22, May 21ś29, 2022, Pittsburgh, PA, USA
© 2022 Copyright held by the owner/author(s). Applying both quantitative and qualitative analysis approaches,
ACM ISBN 978-1-4503-9221-1/22/05. we found that developers tend to operate in fluid team constructs,
https://doi.org/10.1145/3510003.3510081
1
ICSE ’22, May 21ś29, 2022, Pittsburgh, PA, USA Anastasia Ruvimova, Alexander Lill, Jan Gugler, Lauren Howe, Elaine Huang, Gail Murphy, and Thomas Fritz

which impacts team awareness and complicates measurements of

team productivity. We also found that perceived individual produc- Individual Developer Productivity
tivity most strongly predicted perceived team productivity, even Researchers have investigated how the perceived productivity of
more than the amount of team interaction, unplanned work, and individual developers relates to a number of factors. For example,
time spent in meetings. We discuss the implications of these in- job aspects that motivate developers or bring enjoyment correlate
sights and what they may suggest for future research about team with higher productivity and job satisfaction [5, 29]. Positive af-
productivity. This paper makes three contributions: fect states, indicating happier moods, have been shown to boost
• It presents a methodology for undertaking a longitudinal developer problem-solving skills and productivity [2, 25, 40, 44].
study of software development team productivity that incor- Likewise, developer’s moods were shown to influence performance
porates both individual and team observations. on certain tasks, such as debugging [28]. Interruptions, on the other
hand, are often cited as one of the most prominent factors influenc-
• It presents novel findings about developers’ perceptions of
ing productivity, increasing anxiety and error rate, and reducing
their team and the team’s productivity.
performance [3, 18, 34, 46]. Our study differs from these efforts in
• It provides a replication package with the surveys, anonymized focusing on how perceptions of individual and team productivity
daily and bi-hourly self-reported data, and analysis scripts. relate.
Section 2 relates this study and its findings to earlier efforts.
Section 3 presents the study method and data analysis approaches Team Productivity
employed. Section 4 presents the results of the study, which are Other research has looked into software development team produc-
discussed in Section 5. Section 6 reports threats to validity and tivity. Several aspects have been shown to potentially predict the
Section 7 summarizes the work. success of software teams, including organization, teamwork struc-
tures, and cohesion. Early research by Boehm [9] indicated that
productivity on a software development project is most affected
2 RELATED WORK by who develops the system and how well they are organized and
managed as a team. Bendifallah and Scacchi [6] found that variation
Traditionally, productivity is defined as the ratio of outputs over
in teamwork productivity and quality could best be explained in
inputs, where output is value produced and input is time or other
terms of recurring teamwork structures, categorized by patterns of
costs invested [47]. While this definition can work well in some
interaction. Lakhanpa et al. found team cohesion to be the domi-
contexts, like manufacturing, it is not easily applied to software
nating factor when investigating the influence of team cohesion,
development as outputs and inputs tend to be difficult to quantify.
experience, and capability on team performance [32]. A recent
We consider how our work builds on and compares to quantifying
literature review [12] revealed 37 factors to be pertinent to team
software development productivity, as well as individual developer
productivity in software development, including individual charac-
and team productivity. As our study was conducted during COVID-
teristics such as work experience [21] and skill/competence [36, 45].
19, which provides a unique work situation, we also consider how
Among interaction factors, collaboration [16, 17], team member
our study compares to other efforts to study software development
availability [21, 36] and ease of communication [54, 56] were found
during COVID-19.
to be predictive of team productivity. Ko notes that there are four
lenses through which to think about productivity in software devel-
Quantifying Software Development Productivity
opment: the individual, team, market, and the full-spectrum, and
Two primary approaches are employed in the effort to quantify
suggest that a single measure may not suffice for capturing perfor-
development work: automated process metrics and self-reported
mance [30]. Though past work has looked at individual and team
perceived productivity. Automated process metrics typically rely on
productivity separately, we aim to bridge this gap by exploring
activity metrics, such as the lines of source code (SLOC) written [22,
how individuals perceive their team’s productivity and how they
55], function points [1, 27], or change requests fulfilled [14, 43].
understand their own productivity in the team context.
These ‘objective measures’ may only be capturing a small part of
a developer’s work [52] and may not be commensurate with the
Remote Work and the Pandemic
actual value produced or progress made. Some question the very
Our study took place during the COVID-19 pandemic, with the
concept of measuring productivity [31], noting that people tend to
result that participants worked primarily from their home offices.
optimize for whatever metric is being usedÐa phenomenon known
While working from home is often claimed to improve productiv-
as Goodhart’s law [15, 24]. These metrics may underestimate work
ity [13, 20, 37], a study by Russell et al. found that working from
done for tasks that are essential, but that do not produce immediate
home is associated with greater levels of both work pressure and
tangible output, such as training for a new skill or mentoring a new
work-life conflict [49, p. 92] because work intrudes into developers’
hire. To overcome these challenges, recent efforts have considered
home lives through working unpaid overtime, thinking about work
perceived productivity and what impacts it [29, 40, 41]. In this paper,
in off hours, exhaustion, and sleeplessness [26]. It should be noted
we use self-reported productivity to explore the interplay between
that working from home in a pandemic is not representative of
individual and team productivity.
regular remote work situations in which the employee might work
from a home office or elsewhere outside the office. A recent study
explored how human and organizational factors influence software
team productivity in the pandemic, finding that the main factors
2
An Exploratory Study of Productivity Perceptions in Software Teams ICSE ’22, May 21ś29, 2022, Pittsburgh, PA, USA

5 teams from 4 companies

per participant:
Bi-hourly ? ...
? 3 interviews
Teams self-reports Work ~ 25 survey responses
day
Kick-off

Daily ~ 116 self-reports

survey

10-14 weeks

... Interview ... Interview Interview ... Follow-up

? #1 ? #2 #3 ? survey

Figure 1: Study timeline

are external interruptions, environment adaptation, and emotional 3.1 Participants

issues [7]. Ralph et al. investigated how working from home in the We recruited 25 software developers from 5 teams of 4 companies
pandemic affected software developer well-being and productivity across North America and Europe. All teams were agile and worked
[48]. Miller et al. conducted a study at a large software company to in various domains, ranging from time-tracking software to real-
understand what team culture factors have been affected by the pan- estate platforms. Team size ranged from 5 to 15 (mean=9.40), and
demic [42] and found that the ability to brainstorm with colleagues, on average, 53% of the developers of each team participated. We
difficulty communicating with colleagues, and satisfaction with recruited these participants through word-of-mouth, social media,
interactions from social activities were important factors. Rather and other personal and professional contacts. These teams were
than looking at productivity from a single perspective (individual considered to be software development teams if they identified their
level, team level, or organizational), we explore the nuanced interac- industry to be Software Development and if individual contributors
tion between individual and team productivity perceptions, and the spent at least half of their work time developing software. The
complex balance and trade-off of individual work and collaborative amount of time spent on development work was later verified with
efforts. the activity metrics given in the bi-hourly self-reports. Participants
received a smart watch as compensation for their time and effort.
Out of the 25 software developers, 21 identified as male and 4
3 STUDY DESIGN identified as female. They had a mean age of 36.22 years with a
As the activities related to software development can vary sub- standard deviation (SD) of 10.26 years. Table 2 shows an overview
stantially week to week [38], a major objective for our study was of the participants and their respective teams, the participant’s role
to consider software developers and software development teams and the amount of collected data for each participant. For simplicity,
across several weeks. The study timeline is presented in Figure 1. we categorized the 25 participating software developers into two
We targeted a study duration of 8 weeks to capture a longer period roles: individual contributor (IC) and manager. A manager was
of development (all teams participated for 10 to 14 weeks); as most considered to be anyone who plays a leadership or managerial role
teams were using an agile approach, this time span worked out in the team.
to a few sprints. Given the length of the study and the need to
gather input from the software developers, we needed to choose
study instruments that allowed collection of data frequently; we 3.2 Survey Instrument & Collected Data
used bi-hourly self-reports, daily surveys, and monthly interviews To gain a broader perspective of productivity, this multi-modal
to gather data frequently while not causing our participants to study combined bi-hourly self-reports, daily surveys, monthly in-
disengage from survey fatigue. We describe the participants in our terviews, and post-study follow-up surveys. Though the minimum
study, the survey instruments used and collected data, and report requirement for participation was 8 weeks, all five teams partici-
on the quantitative and qualitative analysis methods we applied to pated for 10 to 14 weeks (mean = 12.4, SD = 1.625). We collected
the data. 624 daily surveys and 2899 bi-hourly self-reports, with an average
In the interest of transparency and reproducibility, we have of 25 daily surveys and 116 bi-hourly self-reports per participant.
posted the survey materials, deidentified quantitative data (from Bi-Hourly self-reports were submitted on a smart fitness watch
the daily surveys and bi-hourly self-reports), and the data analysis by the participants. They were asked to report their individual and
scripts on the Open Science Framework at [50]. We have not made team productivity, stress level, and activity categories worked on
public the qualitative data from the daily surveys or interviews in the last hour. Table 1 provides an overview of the questions and
due to the risk of deanonymization, but include the corresponding answer options, and Figure 2 shows the data entry screens of the
survey and interview material for replication. smart watch application.
3
ICSE ’22, May 21ś29, 2022, Pittsburgh, PA, USA Anastasia Ruvimova, Alexander Lill, Jan Gugler, Lauren Howe, Elaine Huang, Gail Murphy, and Thomas Fritz

Table 1: Survey & Interview Questions

Question Answer Type

Bi-Hourly Self-Reports
Rate your own productivity for the past hour Very low (1) to Very high (7)*
Rate your team’s productivity for the past hour Very low (1) to Very high (7)**
Rate your own stress level for the past hour Very low (1) to Very high (7)***
What were you working on? Activity Categories****
Daily Survey
Individual Productivity
How would you rate your overall productivity today? Not Productive At All (1) to
Extremely Productive (7)
Which factors may have had an effect on your productivity today? Open-Ended
(Graph) How would you explain the pattern of your productivity ratings for today? Open-Ended (Response to Graph)
Team Productivity
How would you rate your team’s overall productivity today? Not Productive At All (1) to
Extremely Productive (7)
(Graph) How would you explain the pattern of your team productivity ratings for today? Open-Ended (Response to Graph)
Individual vs. Team Productivity
If there are difference in your own and your team’s productivity, how would you explain it? Open-Ended
Interactions
How would you rate the amount/frequency of interaction you had with your supervisor today? No Interaction At All (1) to Very High (7)
How much of the interaction with your supervisor today was related to work? Almost Entirely Not About Work (1) to
Almost Entirely About Work (5)
How much did you interact with your team today? Almost not at all (1) to Unusually Much (5)
How much of the interaction with your team today was related to work? Almost Entirely Not About Work (1) to
Almost Entirely About Work (5)
Other Factors
How much time did you spend in meetings today? Hour Buckets
How much time did you spend doing unplanned work today? Time Entry
Describe the nature of the unplanned work - what was it, how did it get assigned to you? Open-Ended
What task did you spend the most time on today? Open-Ended
Did your tasks today require more individual effort or team effort? Almost Entirely Individual Effort (1) to
Almost Entirely Team Effort (5)
Was the majority of your time today spent working remotely (out of the office)? Yes/No
How (positively) do feel about your interruptions today? Very Negative (1) to Very Positive (7)
Would you like to share anything else about your day? Open-Ended
Monthly Interviews
Could you describe your team?
Has your team context changed since you started the study?
Could you describe the nature of your work?
Has the nature of your work changed since you started the study?
Do you work more individually or actively collaborate with others?
How often do you communicate with your team and what does this communication look like?
Open-Ended
What would you say productivity means to you?
How do you go about assessing your own productivity? For your team’s?
What factors do you feel have the biggest effect on your own productivity?
What factors do you feel have the biggest effect on your team’s productivity?
What factors have the biggest effect on your stress?
Are you currently working more remotely or from the office?
* I didn’t work, No Answer; ** I don’t know, No Answer; *** Not sure, No Answer; **** Activity Categories: Break, Email/Messaging, Meeting,
Planning, Software Development Work, Browsing, Read/Write Documents, Other Individual Tasks, Other Team Tasks; Some questions were
shortened and reordered for clarity.

4
An Exploratory Study of Productivity Perceptions in Software Teams ICSE ’22, May 21ś29, 2022, Pittsburgh, PA, USA

Table 2: Study Participants (IC = Individual Contributor)

Team / Developer # Self # Daily

Participant Role Reports Surveys
Team 1 ś 13 weeks
S1 IC 117 18
S2 IC 90 13
S3 IC 129 26
S4 Manager 114 19
S5 Manager 41 28
S6 Manager 91 29
Team 2 ś 14 weeks
S7 IC 185 9
S8 IC 29 12
S9 IC 177 45
S10 Manager 122 40
S11 Manager 92 20
Figure 3: Team Productivity Graph in Daily Survey
S12 Manager 157 50
Team 3 ś 10 weeks (using a visualization generated from their bi-hourly self-reports)
S13 IC 169 27 and to reflect on any patterns or differences. Further questions
S14 IC 126 15 centered around several teamwork-related factors, such as how
S15 IC 117 31 many team interactions they had experienced that day and how
S16 IC 62 19 related these interactions were to work; time spent in meetings
S17 IC 71 11 and time spent in unplanned work; how negatively they felt about
S18 Manager 157 39 interruptions that day; how collaborative their tasks were, and
S19 Manager 70 21 whether anything else had influenced their work (open question).
Table 1 provides an overview of the questions, and Figure 3 shows
Team 4 ś 11 weeks an example of the graphs the survey presented for reflection.
S20 IC 135 10 Audio-recorded Interviews were conducted monthly (at the
S21 IC 128 33 start, middle, and end of the study). These were primarily used
S22 IC 138 18 to check in with participants and monitor for any major changes
S23 Manager 46 22 in their work or team. Additionally, we used them to question
participants about their perceptions of productivity and their team,
Team 5 ś 14 weeks
including how they assess productivity, which factors influence
S24 IC 200 50 their productivity, and how they define their team. An overview of
S25 IC 136 19 the questions asked in these interviews can be found in Table 1.
Í Follow-up surveys were employed via an online form to ask
25 2899 624
participants several additional questions, including demographics
and the criteria by which they believe their supervisor assesses
them. The full survey is included in the replication package [50].

3.3 Quantitative Analysis

We performed statistical analysis to investigate correlations be-
tween different factors (presented in Table 3), visualize ratings over
time (Figure 4), and generate a regression model of factors affecting
team productivity (Table 4). The data source included Likert-style
(a) (b) (c) ratings from both the daily surveys and bi-hourly self-reports. We
included in the analysis participants who submitted at least two
Figure 2: Screens of the smart watch application for entering bi-hourly self-reports and one end-of-day survey per day for a
(a) individual productivity, (b) team productivity, and (c) ac- minimum of two weeks (i.e., ten daily surveys and 20 bi-hourly self-
tivities done in the last hour reports in total), as we considered two weeks a minimum baseline
for balancing out possible day-to-day variation. This resulted in 24
Daily surveys were filled out via an online portal at the end of of 25 participants being included in the analysis. All values were nor-
each workday. These surveys asked participants about the individ- malized per participant using the standardization protocol: subtract-
ual and team productivity they had reported throughout the day ing mean and dividing by the standard deviation, where the mean
5
ICSE ’22, May 21ś29, 2022, Pittsburgh, PA, USA Anastasia Ruvimova, Alexander Lill, Jan Gugler, Lauren Howe, Elaine Huang, Gail Murphy, and Thomas Fritz

and deviation were calculated per participant. This normalization 4.1 The Fluid Notion of Teams
step eases comparison between participants. Quantitative analyses In the three monthly interviews, we asked participants how they
were conducted with R (version 4.0.5, https://www.R-project.org/). define their team and whether this team has changed since the
Linear regression models were created using the lme4 package start of the study. We found that the concept of team is more fluid
[4]. Tables were created in part using the apaTables package [51]. than we initially expected. In the literature, a team in software
To facilitate transparency, we have posted the survey materials, development is often treated as a well-defined and fixed concept.
deidentified quantitative data, and scripts for data analysis here For instance, in Scrumśa common agile process modelśa team is
[50]. defined as łsmallž (7±2 people) with łno sub-teamsž and people
Means, standard deviations, and correlations with confidence working łfull time in the teamž.1,2 However, in practice, the concept
intervals are shown in Table 3. of a team may be fluid, making the assessment of team productivity
Figure 4 shows the weekly average individual productivity (dot- more difficult.
ted) and team productivity (solid) for Teams 3 and 5. It can be seen All participants in our study stated that they are part of a larger
that both lines are highly individual for each team. team with up to 15 people. The participants described that these
Quantitative scores were further used to give context to qual- larger teams sometimes change due to reorganizations; over the
itative (open-text) answers from the same daily survey response. course of our study, fluctuations in the makeup of these larger teams
For example, when a participant talked about "meetings" when happened in several cases. Participants also explained that there are
responding to the factors that had an effect on their productivity, teams or sub-teams within these larger teams (6×, 43%). These
we referenced the rating they gave for individual productivity and sub-teams were typically defined in terms of the people which
whether it was lower, higher, or equal to the participant’s average participants interact with on a regular basis, whom they have the
to understand the sentiment behind the response. most knowledge about, and with whom they share tasks and goals.
7 We do have our daily meetings with my smaller team where
3.4 Qualitative Analysis we talk about what we’re working on. [...] And then every week we
Qualitative analysis focused on open-ended questions and was have a meeting that the entire team [...] and there we find out what
performed on the following data: the other half of the team is working on (S1)
Interview transcriptions: questions on team definition (in- 7 Yeah, that’s a little interesting. Because when I would talk about
forming results in 4.1) and assessing productivity (4.2 and 4.3). my team, I would always be talking about my entire team, like the
Follow-up survey: question on assessing productivity (4.3). complete team. But in the short term, like, while I’m on a particular
Daily surveys: questions on team-related activities, nature of [..] smaller team, [..] our goals would be aligned and shared. So we’d
unplanned work, and factors affecting productivity (4.4). be doing each other’s code reviews, we’d be doing verification for
For all three, we employed the reflexive thematic analysis intro- our tasks and all that. So yeah, I think if you talk about a team, in
duced by Braun and Clarke [10] from a critical-realist perspective. the sense of like, having shared goals and working together towards
The 6-step process included gaining familiarity with the data, in- them, that would be like this smaller team that I’m a part of (S1)
ductively generating codes, searching for themes/clusters of codes, Sub-teams are dynamic and can change frequently (6×, 43%),
reviewing these themes, defining and naming themes, and produc- often due to the tasks, projects, or the roles a developer takes on,
ing a final summary. One author led the process, with two other and can be quite fuzzy in terms of who is on the team or not (3×,
authors involved in the initial coding and the final theme clustering. 21%).
For both the interviews and free-form survey answers, we began 7 I moved from like one small team to a different small team. But
by coding half of the sample set and continuing until no novel that’s expected because our smaller teams are supposed to be very
codes were identified, at which point we considered convergence flexible (S1)
and saturation to be reached. In the end, we reached saturation 7 We’re split into like subgroups or squads, or whatever you want
after fully coding the interviews of 14 participants (56%) and the to call it. So our squad is actually particularly small. So I have one
daily surveys of 20 participants (80%). Below, we discuss the most other person in my squad. [..] The actual team like the full team is
frequently occurring themes. For the qualitative data, we use the × nine engineers; actually it’s 13 all together, but when I think of my
sign to indicate how many developers mentioned the theme. team, I think of it more along lines of the engineers. So that instead
of that 13 I think of the nine engineers that we have, those would be
4 RESULTS the people I talk to the most and share the common goals with (S7)
In the following, we present the key themes of our analysis: (1) At the same time there is often no single one definition of who
the fluid notion of teams, (2) the ways and difficulties of gauging is part of a team, with different people defining teams differently.
team productivity by individual developers, (3) the discrepancy Leads of (sub-)teams, for instance, have in contrast to the common
between managers’ and developers’ views on an individual’s pro- fuzzy notion a very clear picture of their team, defining them as
ductivity within a team, and (4) the interplay between team-related the people that report to them (3×, 15%).
factors and productivity perceptions. We focus primarily on the 7 People who report up to me [..] that’s who I think of as my team
team aspect of software development, though the collected data (S12)
also corroborates many earlier findings on individual productivity,
such as the positive effects of having clear goals, or the negative 1 www.scrum-institute.org/Scrum_Roles_The_Scrum_Team.php

effects of frequent context switches. 2 www.scrum.org/resources/what-is-scrum

6
An Exploratory Study of Productivity Perceptions in Software Teams ICSE ’22, May 21ś29, 2022, Pittsburgh, PA, USA

Table 3: Correlations

Variable M SD 1 2 3 4 5 6
1. Team Productivity 0 1
2. Individual Productivity 0 1 0.41**
3. Interruptions 0 1 0.28** 0.35**
4. Team Interaction 0 1 0.13** 0.10* 0.06
5. Unplanned Work 0 1 −0.09 * −0.15 ** −0.24 ** 0.11**
6. Meeting Time 0 1 0.02 0.06 0.02 0.42** 0.10*
7. Task Collaborativeness 0 1 0.05 0.07 0.08 0.53** 0.08 0.37**
M and SD are used to represent mean and standard deviation, respectively. Values in square brackets indicate the 95% confidence interval
for each correlation. The confidence interval is a plausible range of population correlations that could have caused the sample correlation
(Cumming, 2014). * indicates p < .05. ** indicates p < .01.

7 7

6 6
Productivity Rating

Productivity Rating
5 5

4 4
Individual Productivity
3 3
Team Productivity

2 2
1 2 3 4 5 6 7 8 9 10 1 2 3 4 5 6 7 8 9 10 11 12 13 14
Study Week Study Week

(a) Team 3 (7 participants, 10 weeks) (b) Team 5 (2 participants, 14 Weeks)

Figure 4: Averaged Weekly Individual and Team Productivity Ratings

Finally, developers can be part of multiple teams and/or inter- even when activity metrics are available. As a result, developers
act frequently with outside teams (7×, 35%), often more than with use several proxies to gauge team productivity.
some members of their own (bigger) team (6×, 30%).
7 I myself work on this project as project lead and partially like,
as a developer. I also work in the web based department as project Assessing team productivity is difficult, especially due to limited
lead and developer (S23) team awareness. Many participants explicitly stated that it was
7 [..] one of the UX people [..], our product marketing person and difficult for them to assess the productivity of their team, in large
technical writer [..] who I don’t technically consider on our team, part because of a lack of awareness of what their team was working
but I do work with them a decent amount on various projects (S10) on. Almost half of the coded participants (9×, 45%) mentioned at
least once that they have no awareness about their team, regardless
This fuzzy and fluid notion of teams, combined with a lack of
of them being an individual contributor or a team manager.
interaction and knowledge of the people on the bigger teams, makes
7 I just have no means or mechanism to easily determine how
the assessment of team productivity difficult. It also raises questions
much work they’ve done (S25)
of whether there can be a single, or a few, definitions of what a
7 I don’t have insight into my team’s daily productivity (S12)
team is and how to determine what makes up a good team. For the
The prevalence of remote work contributed strongly to this lack
self-reporting in our study, we asked participants to focus on the
of awareness and thus the difficulty in gauging team productivity.
łpeople that you interact with on a regular basis, and with which
From the 20 developers coded, only one participant worked mostly
you share responsibilities for outcomesž; all participants accepted
in the office, four (20%) experienced remote and office work during
this as an appropriate definition, even if that meant the definition
the study period, and the other 15 (75%) worked remotely most
of a team was fluid.
of the time. All survey responses for which participants reported
having ‘no awareness’ of their team stem from days on which the
participants worked remotely.
4.2 Gauging Team Productivity by Individuals 7 I think it’s a bit harder to see, like, the productivity when, like,
How do developers gauge the productivity of their team? Various we’re doing everything remotely (S5)
themes were identified in the final interviews regarding team pro- 7 Since I can’t see my team, I can only guess their productivity
ductivity, including that assessing team productivity is difficult (S10)
7
ICSE ’22, May 21ś29, 2022, Pittsburgh, PA, USA Anastasia Ruvimova, Alexander Lill, Jan Gugler, Lauren Howe, Elaine Huang, Gail Murphy, and Thomas Fritz

Engagement with team is often used to gauge team productivity. One team even explicitly stated at the beginning of the study that
The importance of team awareness in gauging team productivity the progress tracked in their issue tracker is not being used for
further manifests in participants often referring to their team in- assessing their team’s productivity due to the variance in task
teractions when explaining their team’s productivity in the daily complexity and size.
survey. More than half of the coded participants (11×, 55%) explic-
itly mentioned the engagement with their team in these answers, Individual productivity as a base. To gauge the team’s productiv-
talking about the meetings (11×), more general interactions with ity, participants also used the perception of their own productivity
teammates (7×), or working together (5×). as a basis and assumed that the team’s productivity was similar
7 I was having productive discussions with my co-workers, so I (3×, 15%).
knew my teammates are also being productive (S1) 7 I tend to assume my team is as productive as I am (S10)
7 Team seemed consistently productive - had some meetings with Based on their perception of their own productivity and how they
them and worked with them via slack/jira tickets/email on other thought their activities differ from a usual day, participants ad-
items (S10) justed the team’s productivity. For instance, when a participant was
The engagement theme was also very prominent for participants more distracted, had more meetings than usual, or more context
that spent time in their office. Three of the four participants that switches, they rate their team’s productivity higher than their own
spent time collocated with their teammates explicitly used the productivity.
interaction as an indicator for the team’s productivity, even if it 7 Since I can’t see my team, I assume they are being productive.
was brief and did not necessarily cover the work performed during Though I objectively think I got a fair amount done today. When I
the day. look back at the day holistically, there were moments when I felt
7 As I have seen them working and what they said, it made me more distracted, and therefore rated my productivity lower [than
the impression they were productive (S21) my team] in that moment (S10)
7 Team members walking into my room with questions and issues 7 I was having to context-switch a lot today [more than usual], but
[...] As far as I saw, they looked quite productive today (S23) I think other team members probably didn’t have to context-switch
as much (S24)
Availability of others used as indicator for productivity. One factor The close relation between individual and team productivity per-
that was mentioned several times as an indicator for the team’s ception can also be seen in Figure 4, which illustrates the averaged
productivity is the availability and responsiveness of teammates. productivity ratings for the team and individual over the course of
The more responsive teammates were, the more productive the the study. The small difference in mean team and individual produc-
team was perceived. For instance, several participants explicitly tivity for the 2899 bi-hourly self-reports (Mean𝑇 𝑒𝑎𝑚 = 4.61 ±0.78;
mentioned the responsiveness of their teammates in chat applica- Mean𝐼𝑛𝑑 =4.47 ±0.70) as well as the 624 daily surveys (Mean𝑇 𝑒𝑎𝑚 =
tions or on code review discussions as an explanation for a high or 4.98 ±0.79); Mean𝐼𝑛𝑑 =4.96 ±0.66) for the 25 participants further
low productivity rating (4×, 20%). illustrate the close relation.
7 My team was responsive when I needed them to be (S24) This close relation is expected, since the more productive each
7 Everyone I worked with was fairly responsive and we accom- individual developer is, the more productive the team can be. How-
plished a lot together (S24) ever, participants frequently delineated themselves from their team,
A few participants also related the responsiveness of teammates not referring to the team as ‘the others’ and excluding themselves. For
just to their team productivity, but also to their own productivity instance, participant S24, a member of the test team, stated:
when the lack of responsiveness might block them (3×, 15%). 7 The test team was blocked by the test environment being down,
7 [I was] able to communicate to different departments in a timely but I was not effected and in fact it probably allowed me to be more
manner so I was not blocked on my work (S5) productive on my individual tasks (S24)
Only in rare instances did participants talk about the team as a ‘we’.
Automated metrics sparsely used. All participating teams used 7 We have been able to finish some task successful in the end of
issue trackers, version control systems, and other tools that provide day (S16)
metrics on team performance. Yet only a few participants (4×, 20%) This differentiation between team productivity and individual pro-
referred to automatically collected metrics, such as issue tracker ductivity can also be seen in how solving team issues might be
activity or code review progress, for their team productivity ratings. considered detrimental to the individual productivity in some cases,
7 I didn’t see many tickets moving forward on the kanban board, even though the individual helps the team.
and didn’t get any request for peer review of code either, so it looks 7 Mostly productive until I was interrupted to help with a testing
like the team did not feel very productive (S1) issue and then I ran into the problem with a shared team resource
On the contrary, some participants referred to a ‘feeling’ of how that I had to fix (S24)
productive their team was rather than concrete metrics that they
used to gauge team productivity (2×, 10%).
7 I was not able to have a feeling about the output of the team 4.3 Gauging Individual Productivity in Teams
today (S16) How is individual productivity assessed in a team context? We were
7 In the morning I heard that many things were already done. curious to know whether developers are assessed in their teams in
In the afternoon I did not hear that much anymore so it’s more a the way they perceive they are. To explore this question, we posed
stomach feeling (S23) the following assessment questions:
8
An Exploratory Study of Productivity Perceptions in Software Teams ICSE ’22, May 21ś29, 2022, Pittsburgh, PA, USA

For individual contributors: (asked in the follow-up survey) Table 4: Factors Affecting Team Productivity
In a few words, what would you say are the criteria your manager
uses to assess your productivity? Variable Factor
For managers: (asked in the interview and the follow-up survey) Individual Productivity 0.34***
What criteria do you use to assess the productivity of your team? Feeling about Interruptions - how positive or nega- 0.10*
tive a person felt about interruptions that day
Expectations from individuals differ from the ones of managers.
Team Interaction Amount 0.07
Participating individual contributors expect managers to predomi-
Team Interaction Work-Relevance - were the team 0.00
nantly assess them by taking into account a more traditional pro-
interactions more work- or non work-related?
ductivity notion based on input and output metrics such as
Time Spent in Unplanned Work −0.06
7 how long I am spending on my tasks (S24)
Time Spent in Meetings −0.02
7 units of work completed per unit of time (S7)
Collaborativeness of Task - whether a task required −0.01
7 projects completed (S25)
more individual effort or team work
Two participants further expected the manager to actively take into
account feedback or reviews from their team members (S2, S9). * indicates p < .05; ** indicates p < .01; *** indicates p < .005
Participating managers, on the other hand, often referred to a
more holistic notion of productivity when assessing developers, one of these target the interplay between the team and the individual
which explicitly recognizes that automatic metrics such as velocity and they have often been mentioned in the past in context of in-
do not show the whole picture. dividual productivity. We also included a variable on time spent
7 Metrics such as bug fixes and flow stream help, but they are not on unplanned work, since we hypothesized that such work mostly
the end-all-be-all (S6) stems from team issues in the collaborative setting of software
7 How much toil was put in: the actual and time and effort, rather development. The answers in the daily survey on the nature of the
than just ticking off accomplishments (S11) unplanned work further confirmed our hypothesis. For instance,
Other managers adopted an even broader notion of productivity one participant reported the source of unplanned work as:
focusing on the commitments and impact: 7 A co-worker reached out to me about a problem they were facing,
7 Achieving something close to what we committed with (S4) and we both debugged it until we found a solution (S1)
7 Accomplishing the right things which will have impact (S10) We created a linear mixed regression model with the team pro-
One possible explanation for this discrepancy is an unequal inter- ductivity rating as the dependent variable, the variables mentioned
pretation of the question: perhaps łassessingž and łbeing assessedž above and listed in Table 4 as fixed effects factors, and the partici-
are not quite mirrored. The responses of individual contributors pants as random effects factor.3
suggest an interpretation of łhow does my manager know I am We saw that among these factors (see Table 4), the strongest pre-
being productive?ž, with answers mentioning indicators such as dictor of team productivity was the individual productivity reported
peer feedback and velocity metrics. In other words, participants that day (r=0.34***), with all other factors being significantly less
listed the indicators which they believe their managers use to gain strong as predictors. One possible explanation for the strong predic-
awareness of their productivity. The managers, when asked how tive capability of individual productivity is that developers identify
they assess the productivity of their team, rather answered łwhat closely with the team and assume their team’s productivity as their
does it mean for my team to be productive?ž, listing such factors as own, or vice versa: in essence, they project their own personal pro-
commitments and accomplishing the right tasks. ductivity onto the team, which we also found in the qualitative data
presented in Section 4.2. We conjecture that this is more likely to
4.4 Interplay between Team-Related Factors occur in the absence of team awareness. This correlation between
individual and team productivity is important to consider when
and Productivity Perceptions measuring team productivity for future studies; it may be wise to
Working together to create a product is a necessity when developing at the same time capture individual productivity. Though meetings
large and complex software. As a result, individuals often have to were often mentioned as a factor affecting productivity in the qual-
balance making progress on individual goals and working with the itative data (see below), we did not see a significant effect in the
team. We elaborate on these trade-offs by presenting quantitative model. This may be because meetings both drive progress forward
and qualitative data collected in the 624 daily surveys. First, we while concurrently fragmenting the workday and hindering focus.
report the statistical analysis of Likert-scale ratings collected in the
survey. Second, we present several corresponding themes we found Meetings
in the open-ended qualitative responses of these surveys. A majority of the participants (15×, 75%) stated that meetings have
a substantial impact on their individual productivity as well as
A Model for Team Productivity their team’s productivity, and that this impact can be either posi-
To investigate the interplay between team and individual produc- tive/productive (11×, 55%) or negative/unproductive (13×, 65%).
tivity, we collected data in the 624 daily surveys on team-related 7 Team meetings are generally productive for us but our meetings
factors and activities, in addition to the team and individual pro- are scheduled anywhere from 30m to 2h apart from one another.
ductivity ratings. In particular, we were interested in team-related We are meeting with 90% of the same attendees in every meeting so
activities, such as meetings, interactions, and interruptions, as well
as the focus of the task work on team or individual effort, since all 3 Note that the correlations can be found in Table 3.
9
ICSE ’22, May 21ś29, 2022, Pittsburgh, PA, USA Anastasia Ruvimova, Alexander Lill, Jan Gugler, Lauren Howe, Elaine Huang, Gail Murphy, and Thomas Fritz

there is little reason for the time gaps in between. I was still mildly 5.1 Measuring Productivity
productive because I was able to fit in non-thought intensive tasks
in between (S7) Gauging Team Productivity
7 The start of the day had a few meetings for me and my team, How to measure productivity for individual software developers
but the rest of the day was free of interruptions. The start of the day and software development teams remains a subject of inquiry
felt a little unproductive due to multiple meetings, but later I found and debate. Within the last year, some have advocated for multi-
my stride and kept working (S1) dimensional approaches to assessing individual productivity [23],
These mixed perceptions of the impact of meetings on productiv- whereas others believe "team productivity can be measured at an
ity echo the findings by Meyer et al. [40]. Meetings were generally organizational scale with simple, holistic observations [...and...]
seen as productive when there are concrete results or actionable any serious failings in individual productivity can be discovered
decisions, or when they create a sense of accomplishment for the by means of good organizational habits" [33]. Our results indicate
participants. Meetings were generally seen as unproductive if there how tricky it is to measure, assess, or gauge team productivity,
is no clear meeting agenda, key people are missing, there are too whether as a projection of individual productivity or through direct
many people, or if they are not needed. Especially when there are measurement approaches. The fluidity of teams that exist in soft-
too many meetings and/or the meetings are not necessary, the meet- ware development today would need to be incorporated into any
ings were mentioned to have a negative impact on productivity and assessment approaches. It may be that it is more important from
can reduce the time to work and the participant’s focus (13×, 65%). an organizational level to consider how software is actually being
7 Most of my team was in at least one meeting today that they produced, rather than how the organizational chart indicates the
probably didn’t really need to be in (S24) software is being produced. This kind of shift is occurring in soft-
ware development as organizations begin focusing on the streams
Supporting Others of value being produced [11].
More than half of the developers (11×, 55%) mentioned supporting
or helping other team members when they compared their indi- Effects of Team Factors
vidual with the team productivity ratings. Overall, helping others An avenue for future work would be to investigate how team char-
lead to an even distribution of being more, equally, or less pro- acteristics (e.g. size, workflow) or roles (manager, contributor) affect
ductive than the team, resulting in no clear or visible trend. Eight productivity perceptions. In this work, we chose not to perform
participants mentioned that efficient collaboration, e.g. discussing a broad analysis of many teams, favoring instead to conduct an
solutions, mentoring, helping others or generally working together in-depth longitudinal case study of a few teams to examine how
had an impact on their own as well as their team productivity. team and individual productivity relate over time. A future study
7 Even though I was interrupted to teach a teammate how to do could engage a wider sample of teams that vary systematically on
something, I know that teaching them how to do the task will be certain characteristics to investigate these questions.
valuable for the whole team in the future, so I feel pretty good about
doing it (S24)
7 Communicating and collaborating with other team members to 5.2 Enhancing Productivity
help them troubleshoot issues, give them information they needed Our results suggest possible changes to tools and processes that
(S25) may help enhance productivity in software development. These
improvements relate either to team awareness or to engagement.
Availability of Colleagues
Availability of others is not only used as an indicator for gauging Improving Team Awareness
team productivity, but as already mentioned in Section 4.2, can The results of our study suggest that developers often lack aware-
have an impact on the individual as well as the team’s productivity. ness of their team’s work and progress, corroborating previous
Multiple developers (6×, 30%) explicitly mentioned the availability findings [53]. Given that our study was conducted during the un-
of their colleagues as a productivity factor. Especially in cases when precedented remote work situation of the COVID-19 pandemic,
participants or the team is blocked due to the unresponsiveness of this lack of awareness may have been more pronounced as team
others, it can severely impact both individual and team productivity. members did not often have the chance to meet in person. Our par-
7 I had one teammate who was late to a meeting when he was ticipants used a number of digital collaboration tools, for example,
the most important person to be in the meeting, and I had another Microsoft Teams, Slack, and Zoom for communication; Screen for
teammate who was unresponsive for 30 minutes when several people para-programming; and Atlassian Jira and other digital Scrum and
were waiting on him to do something for the release deployment Kanban boards for progress tracking. Though the communication
(S24) tools help to gauge presence, and progress trackers help to provide
7 Negative: Key person not available (S16) quantitative metrics, these tools only give a surface-level measure
of productivity, and the process of combining these data can be
tedious. There is a need for improving awareness by better commu-
nication and collaboration tool design. It is, however, impressive
5 DISCUSSION that despite this lack of team awareness, coordination seems to be
Insights from this study can help inform approaches to measure effective enough for complex projects to succeed. It may be inter-
productivity as well as how to enhance productivity. esting to investigate how much team awareness software teams
10
An Exploratory Study of Productivity Perceptions in Software Teams ICSE ’22, May 21ś29, 2022, Pittsburgh, PA, USA

actually require to operate well, as such a threshold could help to One question that arises is whether measures of perceived team
optimize for minimal interruptions. productivity have value, considering that participants have limited
awareness of their team. In this work, we did not aim to capture ob-
Improving Engagement jective team productivity, but rather to understand how developers
Participants in our study recognized the need for engagement with perceive their team’s productivity even in the absence of concrete
other team members. However, they struggled with how to en- knowledge. At times, some participants did not answer the team
gage most effectively. Well-run and structured meetings have been productivity question or might have also guessed. However, the
shown to support engagement [40], but it remains challenging for thoughtful answers to the open-ended daily questions on the pro-
meetings to successfully engage people in projects despite advice ductivity patterns and differences (Table 1), which often referred to
and protocols being available on how to best structure and run such teammates’ activities and meetings, give us cause to believe that
meetings. Future research should delve into the nature of the dif- the received responses reflect the developers’ perception of team
ferent kinds of engagement required within software development productivity given the information available to them.
teams. Identifying a taxonomy of engagement types would provide
a foundation for assessing different approaches to increase engage- 7 CONCLUSION
ment. Engagement could be increased through adopting processes This work aims to bridge the gap between individual and team
such as meeting protocols, using an appropriate technological so- productivity in software development through a longitudinal user
lution, or improving the structure of communication, for example study that explores individual perceptions of productivity in a team
discussions based on issues inside the issue tracker, compared to context. Among other questions, we investigated developers’ aware-
chat channels. ness of their team, definitions and measures of productivity from
individual and managerial perspectives, and factors predicting per-
ceived team productivity. We present a methodology for undertak-
6 THREATS TO VALIDITY
ing a longitudinal study of software teams, findings about developer
perceptions of their team’s productivity, and a replication pack-
External Validity age with the study material and anonymized quantitative data. We
As our sample set featured only five teams, generalizability to other saw several themes emerge from the 624 daily survey responses
development companies and developers might be limited. These collected from 25 developers in 5 teams. The findings suggest that
companies were recruited through word-of-mouth, social media, developers operate in a fluid team construct with fuzzy, frequently
and other personal and professional contacts, and may therefore changing boundaries. Furthermore, developers often lack awareness
exhibit a sampling bias. We tried to mitigate this bias by recruiting of their team’s productivity. Finally, the most significant predictor
teams of four companies with different product focuses, in North of perceived team productivity was the reported individual produc-
America and Europe. Furthermore our sample set does not exhibit tivity, above factors such as team interaction, unplanned work and
an equal gender balance. In recruiting participants, our primary time spent in meetings. We suggest that future research should fur-
aim was to include full teams (as many members from a team as ther explore the intricacies of development in a broader spectrum
possible), and we therefore did not optimize for demographic repre- of collaboration (e.g., as an individual, between collaborators, in
sentation. Our gender distribution was 21 self-reporting males and teams, and on the organizational level) and to consider what the
4 self-reporting females (16%). This percentage of females comes fluidity and dynamism of these collaborative structures mean for
close to the 20% participation of females reported in software de- the success of software projects.
velopment in the USA [19]. It should also be noted that since the
study took place during the COVID-19 pandemic, with most (21 of ACKNOWLEDGMENTS
25) participants working remotely, the survey responses may not
reflect normalcy and should be interpreted with this in mind. The authors would like to thank all study participants and compa-
nies involved, and Joanna Triscott for support with visualizations.
Construct Validity
We performed a Thematic Analysis to analyze the participant re- REFERENCES
[1] Allan J. Albrecht. 1979. Measuring Application Development Productivity. In
sponses to the daily surveys and interviews. One potential threat Proceedings of IBM Applications Development Symposium. Monterey, 83.
could be that the open coding step for the majority of interviews [2] Teresa M. Amabile and Steven J. Kramer. 2011. The Progress Principle: Using Small
was performed by one author only. To reduce bias, three authors Wins to Ignite Joy, Engagement, and Creativity at Work. Harvard Business Press.
[3] B. Bailey, J. Konstan, and J. Carlis. 2001. The Effects of Interruptions on Task
worked together to create an initial code set and identify and discuss Performance, Annoyance, and Anxiety in the User Interface. In INTERACT.
possible themes. [4] Douglas Bates, Martin Mächler, Ben Bolker, and Steve Walker. 2015. Fitting
Linear Mixed-Effects Models Using lme4. Journal of Statistical Software 67, 1
(2015), 1ś48. https://doi.org/10.18637/jss.v067.i01
Internal Validity [5] Sarah Beecham, Nathan Baddoo, Tracy Hall, Hugh Robinson, and Helen Sharp.
The daily question about patterns in team and individual produc- 2008. Motivation in Software Engineering: A systematic literature review. Infor-
mation and Software Technology 50 (08 2008), 860ś878. https://doi.org/10.1016/j.
tivity were accompanied by a graph displaying the participant’s infsof.2007.09.004
bi-hourly self-reports for the day, to help with reflection. However, [6] Salah Bendifallah and Walt Scacchi. 1989. Work Structures and Shifts: An Em-
five participants reported issues with the graph: four saw issues in- pirical Analysis of Software Specification Teamwork. In Proceedings of the 11th
International Conference on Software Engineering (Pittsburgh, Pennsylvania, USA)
termittently and one regularly. This may have affected the accuracy (ICSE ’89). Association for Computing Machinery, New York, NY, USA, 260ś270.
with which participants recalled their productivity for the day. https://doi.org/10.1145/74587.74624
11
ICSE ’22, May 21ś29, 2022, Pittsburgh, PA, USA Anastasia Ruvimova, Alexander Lill, Jan Gugler, Lauren Howe, Elaine Huang, Gail Murphy, and Thomas Fritz

[7] Carla I. M. Bezerra, José Cezar de Souza Filho, Emanuel F. Coutinho, Alice Gama, [30] Amy Ko. 2019. Individual, Team, Organization, and Market: Four Lenses of Produc-
Ana Lívia Ferreira, Gabriel Leitão de Andrade, and Carlos Eduardo Feitosa. 2020. tivity. Apress, 49ś55. https://doi.org/10.1007/978-1-4842-4221-6_6
How Human and Organizational Factors Influence Software Teams Productivity [31] Amy Ko. 2019. Why We Should Not Measure Productivity. Apress, 21ś26. https:
in COVID-19 Pandemic: A Brazilian Survey. In Proceedings of the 34th Brazilian //doi.org/10.1007/978-1-4842-4221-6_3
Symposium on Software Engineering (Natal, Brazil) (SBES ’20). Association for [32] B. Lakhanpal. 1993. Understanding the factors influencing the performance of
Computing Machinery, New York, NY, USA, 606ś615. https://doi.org/10.1145/ software development groups: An exploratory group-level analysis. Information
3422392.3422417 and Software Technology 35, 8 (1993), 468ś473. https://doi.org/10.1016/0950-
[8] B. Boehm. 1987. Improving Software Productivity. Computer 20 (1987), 43ś57. 5849(93)90044-4
[9] Barry W. Boehm. 1987. Improving Software Productivity. Computer 20, 9 (Sept. [33] Isaac Lyman. 2020. Can developer productivity be measured?
1987), 43ś57. https://doi.org/10.1109/MC.1987.1663694 https://stackoverflow.blog/2020/12/07/measuring-developer-productivity/ (Dec.
[10] Virginia Braun and Victoria Clarke. 2006. Using Thematic Analysis in Psychology. 2020).
Qualitative research in psychology 3 (Jan 2006), 77ś101. https://doi.org/10.1191/ [34] Gloria Mark, Daniela Gudith, and Ulrich Klocke. 2008. The cost of interrupted
1478088706qp063oa work: More speed and stress. Conference on Human Factors in Computing Systems
[11] Flint Brenton. 2019. What Is Value Stream Management? A Primer For Enterprise - Proceedings, 107ś110. https://doi.org/10.1145/1357054.1357072
Leadership. https://www.forbes.com/sites/forbestechcouncil/2019/07/08/what-is- [35] Gloria Mark, Shamsi T. Iqbal, Mary Czerwinski, Paul Johns, and Akane Sano. 2016.
value-stream-management-a-primer-for-enterprise-leadership/?sh=3fbc9cc17b67 Neurotics Can’t Focus: An in Situ Study of Online Multitasking in the Workplace.
(July 2019). Association for Computing Machinery, New York, NY, USA, 1739ś1744. https:
[12] Edna Dias Canedo and Giovanni Almeida Santos. 2019. Factors Affecting //doi.org/10.1145/2858036.2858202
Software Development Productivity: An Empirical Study. In Proceedings of the [36] Katrina D. Maxwell and Pekka Forselius. 2000. Benchmarking Software-
XXXIII Brazilian Symposium on Software Engineering (Salvador, Brazil) (SBES Development Productivity. IEEE Softw. 17, 1 (Jan. 2000), 80ś88. https://doi.
2019). Association for Computing Machinery, New York, NY, USA, 307ś316. org/10.1109/52.820015
https://doi.org/10.1145/3350768.3352491 [37] Claire R. McInerney. 1999. Working in the virtual office: Providing information
[13] Wayne Cascio. 2000. Managing A Virtual Workplace. Academy of Management and knowledge to remote workers. Library & Information Science Research 21, 1
Perspectives 14 (08 2000). https://doi.org/10.5465/AME.2000.4468068 (1999), 69ś89. https://doi.org/10.1016/S0740-8188(99)80006-1
[14] Marcelo Cataldo, James D. Herbsleb, and Kathleen M. Carley. 2008. Socio- [38] André Meyer, Earl T. Barr, Christian Bird, and Thomas Zimmerman. 2019. Today
Technical Congruence: A Framework for Assessing the Impact of Technical and was a Good Day: The Daily Life of Software Developers. In Proceedings of the
Work Dependencies on Software Development Productivity. In Proceedings of the IEEE Transactions on Software Engineering (2019).
Second ACM-IEEE International Symposium on Empirical Software Engineering and [39] André Meyer, Laura Barton, Gail Murphy, Thomas Zimmermann, and Thomas
Measurement (Kaiserslautern, Germany) (ESEM ’08). Association for Computing Fritz. 2017. The Work Life of Developers: Activities, Switches and Perceived
Machinery, New York, NY, USA, 2ś11. https://doi.org/10.1145/1414004.1414008 Productivity. IEEE Transactions on Software Engineering PP (01 2017), 1ś1. https:
[15] Alec Chrystal and Paul Mizen. 2003. Goodhart’s Law: Its Origins, Meaning and //doi.org/10.1109/TSE.2017.2656886
Implications for Monetary Policy. Cent Bank Monet Theory Pract Essays Honour [40] André Meyer, Thomas Fritz, Gail Murphy, and Thomas Zimmermann. 2014.
Charles Goodhart 1 (01 2003). https://doi.org/10.4337/9781781950777.00022 Software Developers’ Perceptions of Productivity. (11 2014). https://doi.org/10.
[16] Victor Clincy. 2003. Software Development Productivity and Cycle Time Reduc- 5167/uzh-98324
tion. Journal of Computing Sciences in Colleges 19 (12 2003), 278ś287. [41] André N. Meyer, Gail C. Murphy, Thomas Fritz, and Thomas Zimmermann.
[17] Broderick Crawford, Ricardo Soto, Claudio León de la Barra, Kathleen Crawford, 2019. Developers’ Diverging Perceptions of Productivity. Apress, 137ś146. https:
and Eduardo Olguín. 2014. The Influence of Emotions on Productivity in Software //doi.org/10.1007/978-1-4842-4221-6_12
Engineering. In HCI International 2014 - Posters’ Extended Abstracts, Constantine [42] Courtney Miller, Paige Rodeghero, Margaret-Anne Storey, Denae Ford, and
Stephanidis (Ed.). Springer International Publishing, Cham, 307ś310. Thomas Zimmermann. 2021. "How Was Your Weekend?" Software Develop-
[18] Mary Czerwinski, Eric Horvitz, and Susan Wilhite. 2004. A Diary Study of ment Teams Working From Home During COVID-19. (01 2021).
Task Switching and Interruptions. In Proceedings of the SIGCHI Conference on [43] Audris Mockus, Roy T. Fielding, and James D. Herbsleb. 2002. Two Case Studies
Human Factors in Computing Systems (Vienna, Austria) (CHI ’04). Association for of Open Source Software Development: Apache and Mozilla. ACM Trans. Softw.
Computing Machinery, New York, NY, USA, 175ś182. https://doi.org/10.1145/ Eng. Methodol. 11, 3 (July 2002), 309ś346. https://doi.org/10.1145/567793.567795
985692.985715 [44] S. C. Müller and T. Fritz. 2015. Stuck and Frustrated or in Flow and Happy: Sensing
[19] datausa.io. [n.d.]. Software developers, applications and systems software. https: Developers’ Emotions and Progress. In 2015 IEEE/ACM 37th IEEE International
//datausa.io/profile/soc/software-developers-applications-systems-software Conference on Software Engineering, Vol. 1. 688ś699. https://doi.org/10.1109/
[20] Thomas H. Davenport and Keri E. Pearlson. 1998. Two Cheers for the Virtual ICSE.2015.334
Office. Sloan Management Review 39 (1998), 51ś65. [45] Edson Oliveira, Tayana Conte, Marco Cristo, and Emilia Mendes. 2016. Software
[21] Claudia De O. Melo, Daniela S. Cruzes, Fabio Kon, and Reidar Conradi. 2013. Project Managers’ Perceptions of Productivity Factors: Findings from a Quali-
Interpretative Case Studies on Agile Team Productivity and Management. Inf. tative Study. In Proceedings of the 10th ACM/IEEE International Symposium on
Softw. Technol. 55, 2 (Feb. 2013), 412ś427. https://doi.org/10.1016/j.infsof.2012.09. Empirical Software Engineering and Measurement (Ciudad Real, Spain) (ESEM ’16).
004 Association for Computing Machinery, New York, NY, USA, Article 15, 6 pages.
[22] Prem Devanbu, Sakke Karstu, Walcélio Melo, and William Thomas. 1996. Ana- https://doi.org/10.1145/2961111.2962626
lytical and Empirical Evaluation of Software Reuse Metrics. In Proceedings of the [46] Chris Parnin and Spencer Rugaber. 2011. Resumption Strategies for Interrupted
18th International Conference on Software Engineering (Berlin, Germany) (ICSE Programming Tasks. Software Quality Control 19 (03 2011), 5ś34. https://doi.
’96). IEEE Computer Society, USA, 189ś199. org/10.1109/ICPC.2009.5090030
[23] Nicole Forsgren, Margaret-Anne Storey, Chandra Maddila, Thomas Zimmermann, [47] Robert D. Pritchard. 1995. Productivity measurement and improvement: Organi-
Brian Houck, and Jenna Butler. 2021. The SPACE of Developer Productivity: zational case studies. Greenwood Publishing Group.
There’s More to It than You Think. Queue 19, 1 (Feb. 2021), 20ś48. https: [48] Paul Ralph, Sebastian Baltes, Gianisa Adisaputri, Richard Torkar, Vladimir Ko-
//doi.org/10.1145/3454122.3454124 valenko, Marcos Kalinowski, Nicole Novielli, Shin Yoo, Xavier Devroey, Xin
[24] C. A. E. Goodhart. 1984. Monetary Theory and Practice: The UK Experience. Tan, Minghui Zhou, Burak Turhan, Rashina Hoda, Hideaki Hata, Gregorio Rob-
Macmillan Press. les, Amin Milani Fard, and Rana Alkadhi. 2020. Pandemic programming: How
[25] Daniel Graziotin, Xiaofeng Wang, and Pekka Abrahamsson. 2014. Do feelings COVID-19 affects software developers and how their organizations can help.
matter? On the correlation of affects and the self-assessed productivity in software Empirical software engineering 25 (11 2020), 1ś35. https://doi.org/10.1007/s10664-
engineering. Journal of Software: Evolution and Process 27 (08 2014). https: 020-09875-y
//doi.org/10.1002/smr.1673 [49] Helen Russell, Philip O’Connell, and Frances Mcginnity. 2007. The Impact of
[26] Jeffrey Hyman, Chris Baldry, Dora Scholarios, and Dirk Bunzel. 2003. Work-Life Flexible Working Arrangements on Work-Life Conflict and Work Pressure in
Imbalance in Call Centres and Software Development. British Journal of Industrial Ireland. Gender, Work & Organization 16 (05 2007). https://doi.org/10.1111/j.1468-
Relations 41 (02 2003), 215ś239. https://doi.org/10.1111/1467-8543.00270 0432.2008.00431.x
[27] C. Jones. 1994. Software metrics: good, bad and missing. Computer 27, 9 (1994), [50] Anastasia Ruvimova, Alexander Lill, Jan Gugler, Lauren Howe, Elaine Huang,
98ś100. https://doi.org/10.1109/2.312055 Gail Murphy, and Thomas Fritz. 2022. Study Materials. In Open Science Framework.
[28] I. Khan, W. Brinkman, and R. Hierons. 2010. Do moods affect programmers’ https://osf.io/28gbx/?view_only=b65543a1d53b457aa09e708aa210fe72
debug performance? Cognition, Technology & Work 13 (2010), 245ś258. [51] David J. Stanley and Jeffrey R. Spence. 2018. Reproducible Tables in Psychology
[29] Young-Ho Kim, Eun Kyoung Choe, Bongshin Lee, and Jinwook Seo. 2019. Under- Using the apaTables Package. Advances in Methods and Practices in Psycho-
standing Personal Productivity: How Knowledge Workers Define, Evaluate, and logical Science 1, 3 (2018), 415ś431. https://doi.org/10.1177/2515245918773743
Reflect on Their Productivity (CHI ’19). Association for Computing Machinery, arXiv:https://doi.org/10.1177/2515245918773743
New York, NY, USA, 1ś12. https://doi.org/10.1145/3290605.3300845 [52] Christoph Treude, Fernando Figueira Filho, and Uirá Kulesza. 2015. Summa-
rizing and Measuring Development Activity. In Proceedings of the 2015 10th
12
 An Exploratory Study of Productivity Perceptions in Software Teams ICSE ’22, May 21ś29, 2022, Pittsburgh, PA, USA

Joint Meeting on Foundations of Software Engineering (Bergamo, Italy) (ESEC/FSE [54] S. Wagner and M. Ruhe. 2018. A Systematic Review of Productivity Factors in
2015). Association for Computing Machinery, New York, NY, USA, 625ś636. Software Development. ArXiv abs/1801.06475 (2018).
https://doi.org/10.1145/2786805.2786827 [55] C. E. Walston and C. P. Felix. 1977. A Method of Programming Measurement and
[53] Christoph Treude and Margaret-Anne Storey. 2010. Awareness 2.0: Staying Aware Estimation. IBM Syst. J. 16, 1 (March 1977), 54ś73. https://doi.org/10.1147/sj.161.
of Projects, Developers and Tasks Using Dashboards and Feeds. In Proceedings of 0054
the 32nd ACM/IEEE International Conference on Software Engineering - Volume [56] Murat Yilmaz, Rory O’Connor, and Paul Clarke. 2016. Effective Social Productivity
1 (Cape Town, South Africa) (ICSE ’10). Association for Computing Machinery, Measurements During Software Development: An Empirical Study. International
New York, NY, USA, 365ś374. https://doi.org/10.1145/1806799.1806854 Journal of Software Engineering and Knowledge Engineering 26 (04 2016). https:
//doi.org/10.1142/S0218194016500194

13

View publication stats