3988 IEEE TRANSACTIONS ON SOFTWARE ENGINEERING, VOL. 48, NO.

10, OCTOBER 2022

Exploring the Use of Chatrooms by Developers:

An Empirical Study on Slack and Gitter
Mariam El Mezouar , Daniel Alencar da Costa , Daniel M. German, and Ying Zou

Abstract—Communication is critical for the software development teams to maintain project awareness, facilitate project co-ordination
and avoid misunderstandings. The features offered in the chatrooms, such as private messaging, group conversations, and code
sharing help accommodate the communication needs of the software development teams. Therefore, chatrooms have been
increasingly adopted among the developers. Since the last study on Slack performed by (Lin et al. 2016), the audience of Slack has
more than doubled possibly leading to an evolution of the ways Slack is used; while another rich community formed around Gitter and
remains unstudied. In this paper, we perform an investigative study using qualitative and quantitative techniques to gain insights on the
use of popular modern chatrooms, specifically Slack and Gitter. Based on the survey responses from 163 developers, the interviews
with 21 developers, and the chatroom data collected from 11 Slack and 770 Gitter rooms, we are able to uncover the reasons behind
the use of Slack and Gitter, the perceived impact on the associated projects, and the quality determinants of the two chatrooms. We find
that the developers seek knowledge from the chatrooms to obtain timely feedback from experts, and in return share their expertise to
build the project community and their reputations. Furthermore, it is perceived by the Gitter developers that the chatrooms have an
impact on prioritizing the new features and the bug fixes. In Slack, the most reported impact concerns an increased project awareness,
in terms of a better tracking of the work progress. As reported on the developers’ survey, both Slack and Gitter chat services have a
visible impact on mentoring developers, and sharing the best practices. In terms of quality determinants, a non-ephemeral history and a
better history management (e.g., advanced search) could be keys for both chat services to reach their full potential.

Index Terms—Chatroom, slack, gitter, software development teams, communication

1 INTRODUCTION Modern developer chatrooms are designed to fulfill the

communication needs of the developers, including messag-
development, the communication and coor-
I N SOFTWARE
dination of software development teams are key factors
for the success of the software projects [1], [2], [3]. Specifi-
ing, file management, code sharing, and video calling. Con-
trary to other communication channels, such as, the mailing
lists or the Q&A platforms, Modern developer chatrooms
cally, informal communication among developers pro-
bring developers together in an informal setting with equal
motes project awareness [4]. However, as software
participation opportunity for everyone. Modern developer
development teams grow in size, it becomes harder to pre-
chatrooms can be either “public” or “private”. Public chat-
vent misunderstandings in the developers’ interactions.
rooms are generally used by open source projects, and
Examples of such misunderstandings are confusions
much emphasis is placed on the formation of a friendly
regarding the software requirements, uncertainty about
community who talks and shares knowledge. For example,
the right developer to contact for questions, and ambigu-
Gitter1 is an open source chatroom geared towards the open
ity about the work progress. In an attempt to address such
communities. The particularity of Gitter is the possibility of
challenges, communication channels and social media are
a Gitter room to be associated to a GitHub repository. The
often integrated with the modern development tools [5]
private chatrooms are usually used by private software
(e.g., chat, email, or microblogging services) to facilitate
projects for team communication. Slack2 is an example of a
the communication between the developers [6].
proprietary chatroom that has gained rapid uptake recently.
It was initially designed for corporate teams, but is more

Mariam El Mezouar is with the School of Computing, Queen’s University, and more adopted by the open communities as well. More
Kingston, ON K7L 3N6, Canada. E-mail: [email protected]. details on the two subject systems follow in Section 2.

Daniel Alencar da Costa is with the Department of Information Science, Uni- Research efforts have been invested to better understand
versity of Otago, Dunedin 9016, New Zealand. E-mail: danielcalencar@otago.
ac.nz.
the role of the chatrooms in software development. First, the

Daniel M. German is with the Department of Computer Science, Univer- chatrooms are considered important in supporting the infor-
sity of Victoria, Victoria, BC V8P 5C2, Canada. E-mail: [email protected]. mal communication among the developers [4]. In a large

Ying Zou is with the Department of Electrical and Computer Engineering, scale survey conducted by Storey et al. [6], private and public
Queen’s University, Kingston, ON K7L 3N6, Canada. E-mail: ying.
[email protected]. chatrooms were deemed as some of the most important
channels by nearly 15 percent of the survey participants. Sec-
Manuscript received 21 December 2020; revised 17 August 2021; accepted 21
August 2021. Date of publication 2 September 2021; date of current version 17 ond, the chatrooms (in this case Internet Relay Chat) are
October 2022.
(Corresponding Author: Mariam El Mezouar.)
Recommended for acceptance by A. Sarma. 1. https://gitter.im/
Digital Object Identifier no. 10.1109/TSE.2021.3109617 2. https://slack.com/about

0098-5589 © 2021 IEEE. Personal use is permitted, but republication/redistribution requires IEEE permission.
See ht_tps://www.ieee.org/publications/rights/index.html for more information.
Authorized licensed use limited to: The University of Hong Kong Libraries. Downloaded on March 31,2025 at 08:33:11 UTC from IEEE Xplore. Restrictions apply.
MEZOUAR ET AL.: EXPLORING THE USE OF CHATROOMS BY DEVELOPERS: AN EMPIRICAL STUDY ON SLACK AND GITTER 3989

used by developers for discussions of technical nature [7]. chatrooms by the open communities (in both Slack and Git-
Lastly, the chatrooms (specifically Slack) are used for per- ter) and the corporate teams (mostly in Slack).
sonal, team-wide, and community-wide purposes, based on Paper Organization. The rest of the paper is organized as
a survey with 104 developers by Lin et al. [8]. follows. Section 2 present the subject systems and the back-
Despite the valuable results obtained by prior research, ground. Section 3 describes the research methodology. In
modern developer chatrooms still need a more thorough Section 4, we show the results of the study. In Sections 5
investigation for the following reasons: 1) The adoption and 6, we provide a discussion and list the limitations of
of Slack since the most recent study by Lin et al. [8] in the study. Finally, we review the literature and conclude in
2016 has more than doubled, from 3 million to over 8 mil- Sections 7 and 8.
lion users. As such, the uses of Slack might have evolved
over time; 2) another rich community of developers has 2 SUBJECT SYSTEMS
been built around the usage of Gitter chatrooms and
remains almost unstudied, which led recent research— In our work, we focus on the Slack and Gitter modern chat-
such as the work from Parra et al. [9]—to collate useful rooms. In Appendix A, which can be found on the Computer
data and foster more research on Gitter; 3) the inclusion Society Digital Library at http://doi.ieeecomputersociety.
of both Slack and Gitter in the study lets us compare two org/10.1109/TSE.2021.3109617, we show a comparison
different types of chatrooms (e.g., proprietary and open between Slack and Gitter. Slack is a collaboration tool that
source), and two different types of communities (e.g., the offers features, such as persistent chatrooms (channels) orga-
open communities in both Slack and Gitter, and the cor- nized by topic, private groups, and direct messaging. Slack
porate teams mostly in Slack), and 4) beyond the uses of was launched in August 2013, and is reported to be “the fast-
the chatrooms, the impact of using the chatrooms on the est-growing business application in history” with 8 million
software projects, and the quality determinants of the daily active users, and 500K organizations that use the tool to
chatrooms are still unclear. collaborate and communicate [11]. Slack allows searching,
We designed and conducted a survey with developers indexing and integration with external applications. The
who adopt Gitter or Slack. Our survey received 163 basic Slack version is free, with the possibility to upgrade to
responses. We analyzed the survey responses, using the- advanced priced features.
matic analysis [10]. We further conducted follow-up inter- Gitter is an open source instant messaging tool for develop-
views with 21 developers to gain more insights and validate ers and users of GitHub repositories, which came out of beta
our results. Furthermore, we mined the chat data archives in 2014. Gitter enables the creation of chatrooms (called rooms)
from 770 Gitter rooms and 11 Slack rooms, and compute the for GitHub repositories, i.e., each Gitter room can be linked to
chat activity metrics, to compare the perception of the a GitHub repository page. Gitter has over 800K users, and
developers with the reality. We organize our study into the 300K rooms from 100+ countries [12]. The main feature of Git-
following three research questions: ter is a seamless integration with GitHub through GitHub fla-
RQ1. Why do developers use the chatrooms? We investigate vored markdowns in chat messages. Gitter has a unique
the reasons that motivate the developers to use the chat- activity feed showing all changes to the associated GitHub
rooms to ask and answer questions. Access to experts and a repository (e.g., commenting on a pull request or closing an
fast response time are the main reasons as to why develop- issue). Another integrated feature between Gitter and GitHub
ers ask questions in the chatrooms. In return, the developers is the user hovercards, which are based on the GitHub profiles
take the time to answer questions to further build the project and statistics (e.g., the number of followers). Similarly to
community, and to build their own personal reputation. Slack, Gitter is also integrated with other applications, such as
RQ2. What is the perceived impact of the chatroom use on the Jenkins,3 Travis CI,4 and Bitbucket.5
software development process? The most recurrent reported Overall, Slack is a proprietary chatroom, geared for the
impacts of Slack and Gitter are (respectively) managing the corporate teams but also accessible to the open communi-
communication (e.g., team updates) and guiding the project ties. An important capability of Slack is the search feature of
tasks (e.g., new issues). The developers are also able to learn the chat data. However, the feature is limited in the free
the best practices of the projects, and produce better solu- plans (i.e., only up to 10K messages). An important issue in
tions. Finally, there appears that the chatrooms improve the Slack is the discoverability of the communities (i.e., a team
productivity of the developers. can only be joined by finding its link and sending a request
RQ3. What defines the quality of the chatrooms and their to join), which limits the ‘openness’ of Slack to the public.
related chat-service?In terms of quality determinants, a ‘good’ On the other hand, Gitter is an open source platform suited
chatroom is first characterized by a knowledgeable and for the open communities. Gitter rooms can be easily found
friendly community. In terms of chat-services (e.g., the and joined. However, the content is lost quite easily, as
available features), the improvement of key features (e.g., searching the past messages is not as good as in Slack.
search management) could help the chatrooms achieve their
full potential. The majority of the surveyed developers 3 METHODOLOGY
(81.1%) report a good to excellent user experience. We study (i) the motives to use modern chatrooms, (ii) their
Contributions. A summary of of our contributions is as impact on the development process, and (iii) the
follows: 1) A better understanding of the motives and
impact of the developers’ chatrooms, 2) a comparison
3. https://jenkins.io/
between an open source (i.e., Gitter) and a proprietary (i.e., 4. https://travis-ci.org/
Slack) chat services, and 3) insights about the uses of 5. https://bitbucket.org
Authorized licensed use limited to: The University of Hong Kong Libraries. Downloaded on March 31,2025 at 08:33:11 UTC from IEEE Xplore. Restrictions apply.
3990 IEEE TRANSACTIONS ON SOFTWARE ENGINEERING, VOL. 48, NO. 10, OCTOBER 2022

characteristics of high quality chatrooms. Because our goal is In the first part (Survey part 1), we inquire about the
to perform an in-depth study of instances (i.e., Gitter and respondents’ demographics and their roles in their software
Slack) of a phenomenon in its natural context and from the projects. For example, we aim to understand if our respond-
perspective of the participants involved in the phenomenon ents use the chatrooms from the perspective of a user, main-
(i.e., the use of chatrooms) [13], we use a multiple case studies tainer, or developer. If a respondent collaborates in many
design in our research [14]. Using multiple case studies can software projects, we asked him/her to answer the survey
increase the robustness of our observations by strengthening based on the project in which he/she is most active.
or contrasting the patterns observed in the different case stud- The remaining three parts are based on the three RQs
ies (i.e., Gitter versus Slack) [14]. Our case studies consist of presented in Section 7. In part 2 of our survey, we asked our
several data collection methods, such as surveys, interviews, respondents what motivates them to both answer and ask
and mining software repositories [15] (MSR). Therefore, we questions on Slack and Gitter. We further inquire about the
use the mixed methods research design to generate our findings instances when the chatrooms are used, instead of other
(i.e., we use both qualitative and quantitative analyses) [16]. channels (e.g., mailing lists).
More specifically, our quantitative data is used to support our Prior studies [6], [18], [19], [20] report productivity con-
predominant qualitative observations, configuring our cerns related to the use of the communication and social
research design as a “QUAL + quan” mixed methods [17]. channels by developers. Consequently, we design part 3 to
First, we employ our mixed-methods design on a within-case ask the respondents whether the chatrooms have any
basis, i.e., we employ our approach to both Gitter and Slack, impact on their productivity.
individually (i.e., survey, interviews, and MSR analyses). Next, Hahn et al. [21] report that a developer is more likely to
we analyze our results on a cross-case basis by comparing our join a project when they have collaborative ties with the
observations obtained for Gitter against the observations project initiator. This finding was confirmed by Casalnuovo
obtained for Slack [14]. At this last stage, two authors (both et al. [22] who found that developers preferentially contrib-
assistant professors) re-analyze all the obtained qualitative and ute to projects in GitHub where they have prior social links.
quantitative results to identify the reinforcing and opposing pat- Accordingly, we inquire whether the use of the modern
terns across our case studies. We provide the details of each chatrooms has a similar impact (i.e., attracting developers
data collection method in the following subsections. to the project associated with the chatroom).
In the final part of the survey (Survey part 4), we focus on
3.1 Developers’ Survey the chatroom quality as perceived by the respondents. First,
In this section, we describe the process of 1) designing, 2) our respondents are requested to provide a satisfaction rat-
distributing, 3) analyzing and 4) validating our survey. We ing of the studied chatrooms. Then, we inquired about the
design the survey in four parts: a) demographics, b) motiva- features that they like most in the studied chatrooms as well
tions, c) impact, and d) quality determinants. as the features that they believe need improvement. Lastly,
Before crafting the survey for performing the research, we we asked our respondents if they were willing to be con-
developed a preliminary survey, which served as a pilot study tacted for follow-up interviews. The survey has 21 questions
to check whether our research questions were worth investi- in total, 14 of which are open-ended questions. The esti-
gating. Our pilot survey contained 5 preliminary questions mated time to complete the survey is 20 minutes. We opted
and is available online.6 The goals of our questions were to for not placing any mandatory questions as a means to
investigate the reasons behind using chatooms. We received improve our response rate. For example, studies have
85 responses to our preliminary survey, which were not shown that it is frustrating to have mandatory questions in
incorporated in our main study given the pilot nature of this surveys [23]. Mandatory questions also increase the proba-
preliminary survey. We observed that diverse reasons bility of participants dropping the questionnaire, or, even
prompted the usage of chatrooms, from handling “database worse, to provide low-quality answers in a way to skip the
issues” to getting hints about “setup and configuration.7” mandatory questions.
Therefore, we crafted a more thorough and detailed survey
to answer the research questions of this paper. We herein
refer to the more detailed survey as simply survey. 3.1.2 Survey Distribution
We obtained 163 responses to our survey, 114 and 48 We used a combination of two methods to contact the
from Slack and Gitter users, respectively. The remaining respondents: 1) we posted our invitation in the chatrooms;
response selected other (Discord) as their most used chat- and 2) we sent emails to developers. We include the invita-
services. We codified the responses to perform our analysis. tion letter sent to the potential participants in Appendix C,
We conducted a set of 21 follow-up interviews to validate available in the online supplemental material.
the results from the previous step. We provide more details Posting in the Chatrooms. In Gitter, we manually joined the
of the four steps in the sections below. public rooms that are visible on the explore page of Gitter
(i.e., 770 rooms). In Slack, we were able to join a set of 29
3.1.1 Survey Design public Slack rooms, by sending requests to join to the project
We designed our survey in four main parts8 (see owners. The only selection criterion to find respondents is
Appendix B, available in the online supplemental material). whether they were members of the chatrooms at the time.
For example, we did not consider activity levels of the
respondents or membership duration, since it is our goal to
6. https://www.surveymonkey.com/r/XK5K35K
7. https://www.surveymonkey.com/results/SM-8DNFCCPM7/ obtain feedback from all kinds of respondents. We posted
8. https://goo.gl/forms/oX4UqWUDRcykBP372 messages in the chatrooms to contact the respondents in a
Authorized licensed use limited to: The University of Hong Kong Libraries. Downloaded on March 31,2025 at 08:33:11 UTC from IEEE Xplore. Restrictions apply.
MEZOUAR ET AL.: EXPLORING THE USE OF CHATROOMS BY DEVELOPERS: AN EMPIRICAL STUDY ON SLACK AND GITTER 3991

less intrusive manner. More specifically, we identified the The first two authors collaboratively performed thematic
administrators of the chatrooms, whom we contacted analysis for every survey response, until consensus was
directly and asked for permission to distribute the survey. reached. Both authors are assistant professors. Saturation is
Some administrators ( 5) posted the survey request them- achieved after analyzing approximately 30 to 35 survey
selves, while others ( 20) gave us the permission to post responses. The first phase of the thematic analysis took place
our survey in one of the room channels (e.g., the random or in five coding sessions for Gitter and four coding sessions for
general channels). Another 11 administrators declined our Slack of about 1.5 hours each. We obtained 56 and 137 codes
request. As a result, we received 47 responses from the Git- for the survey part 2 for Slack and Gitter, respectively. An
ter respondents from different chatrooms, and another 47 additional 67 and 122 codes resulted from part 3, for Slack
from the Slack respondents from different chatrooms, bring- and Gitter, respectively. Finally, part 4 of the survey yielded
ing the total to 94 survey responses. With our survey distri- 33 and 24 codes, for Slack and Gitter, respectively. We then
bution method, we cannot assume that all the members in a generate higher level conceptual themes in the second phase
chatroom have read the survey request. Therefore, it is hard of the thematic analysis to answer our research questions
to assess the response rate. Because we sent our surveys (more details are shown in Tables 1, 2, and 3). In this paper,
separately to Gitter and Slack participants, we easily iden- we report our results on all the obtained themes.
tify the chatroom to which our participants were referring In the remaining of the paper, we refer to developers
when replying to our surveys. who participate in the survey as the respondents, and to
Email: We contacted developers through emails. Since we developers with whom we conducted the interviews as the
are not able to specifically email developers that use Slack or interviewees. We further refer to the individual respond-
Gitter, the selection criterion in the second phase is simply ents/interviewees using a code of the form S# for Slack and
developer’s participation in a GitHub repository (i.e., code G# for Gitter (# is a unique ID of each respondent, e.g., S15).
commits). To identify whether our participants were refer- The Slack respondents span from S1 to S114, and the Gitter
ring to Gitter or Slack in their answers, we explicitly asked respondents from G1 to G48.
them to identify the chatroom to which they were referring.
Our initial selection of GitHub developers with more than 3.1.4 Follow-Up Interviews
10 commits—th median number of commits across all
developers—resulted in over 4 million developers. We then Setup. 21 respondents expressed their interest in being part
chose a statistically significant random sample (confidence of a follow-up interview. The goals of the follow-up inter-
interval = 2 and confidence level = 95%), resulting in 2,400 views are: (i) to clarify the responses of the survey, (ii) to
developers. We emailed the survey request to the selected validate the themes obtained from the second phase of the
developers, receiving an additional 69 responses. At the thematic analysis, and (iii) to collect more details from the
end of both phases of deployment, we obtained a total of personal experience of developers. We have not performed
163 survey responses. The only statistical difference an additional thematic analysis at this stage as we have
between the demographics of the two deployments is an reached saturation when analyzing our survey responses.
increase in the number of female respondents (from 3.2% to Our follow-up interviews followed a semi-structured for-
13.3%, p-value = 0.016). A possible explanation for the raise mat, i.e., we started with a script of questions but we let the
of women respondents over e-mails is that women may feel respondents go off the script if he or she wishes to provide
less comfortable to participate in chatrooms due to reasons more information. We contacted the 21 interviewees
that must still be studied (e.g., chatrooms might be a male through their preferred communication channel (e.g., chat,
dominant environment). or voice call). The voice/video calls lasted in average 20
Since the survey questions are non-mandatory, we find minutes, while the chat interviews lasted around 35
that the questions were skipped by the participants 10 times minutes. We provide our interview script in Appendix G,
on average (i.e. not answered by 163 10
or 6% of the respond- available in the online supplemental material.
ents) with a median of 7. Analysis. As shown in Appendix G, available in the online
supplemental material, we prompted the interviewees with
3.1.3 Survey Analysis the themes generated during the survey analysis. As such, we
After receiving the responses to our survey, we performed used the interviews scripts to validate the generated themes,
thematic analysis [10] to analyze the collected data. Thematic and the explanations accompanying the answer as illustrative
analysis is a process involving the qualitative examination of quotes in the paper. In case an interviewee has additional
a dataset set to generate a set of themes that capture the intri- thoughts about a given question, we analyzed their answer
cacies of meaning within the data set. In the first phase of the using thematic analysis to identify new themes, not generated
thematic analysis (i.e., coding), a set of initial codes is gener- during the survey analysis step. The results shown in Section 4
ated by collapsing the data into labelling concepts. In our reflect the results obtained from both steps.
study, a survey response can be collapsed into one or more
codes, depending on the complexity and richness of the 3.2 Messages Collection From the Chatrooms
response. In the second phase of the thematic analysis, the We used the Gitter REST API to download archives of the
codes are combined into overarching themes that accurately public Gitter rooms. We then use the Gitter REST API to
depict the data. For example, the codes ‘interactive’ and ‘real- obtain the archives of the 770 joined Gitter rooms. We exclude
time’ can be categorized under the theme ‘response time’. We the 19 Gitter rooms where no conversation has started yet.
show examples of the thematic analysis of our data in In Slack, there is no central browsing page that displays the
Appendix D, available in the online supplemental material. available public rooms. Instead, each public chatroom has a
Authorized licensed use limited to: The University of Hong Kong Libraries. Downloaded on March 31,2025 at 08:33:11 UTC from IEEE Xplore. Restrictions apply.
3992 IEEE TRANSACTIONS ON SOFTWARE ENGINEERING, VOL. 48, NO. 10, OCTOBER 2022

TABLE 1
Survey Results Regarding the Motivations of Developers to use the Chatrooms

Theme % respondents agreeing Illustrating quote

to theme
Slackn¼114 Gittern¼48
Quality of the help 46% 33% “Slack allows large group discussions in real time around
technical features and bug fixes which can often lead to better
solutions.”
Response time 41% 35% “I think the immediacy of Slack is one of its big benefits: there’s
nearly always someone who can answer any given question
during the times the channels are busy. Or at least there will be
someone who can direct the questioner to a channel that would
be more likely to get an answer.”
Giving back 40% 44% “I’m using open libraries every day for free. And I want to help
the open source [communities].”
Personal gain 31% 21% “I would certainly say that as a hiring manager, I would use any
impressions formed from interactions on Slack to guide my
hiring decisions, if those developers applied to work for me”
Tasks 29% 0% “In the company I work we use [Slack] like a planning tool (well,
to discuss tasks to be done and so on)”
Internal communication 19% 0% “Our company uses [Slack] for internal communication and
collective knowledge sharing and we are expected to ask
questions.”
Requirement 18% 0% “I have to use what the creators set up”
Company’s goals 8% 2% “I am motivated to provide clear and transparent answers as
much as possible, to maximize the revenue and achieve the
company’s goals”
Custom support 1% 33% “With Gitter I can chat with [the library authors] about custom
solution, and I believe that this is the main difference”
Personal enjoyment 0% 29% “Some do it to find people to collaborate with, and others do it to
simply be amongst their peers”

unique link used to request to join the community. A thor- instance, such an automatic approach would need a high
ough internet search reveals the links of the top public Slack accuracy on identifying that a user is asking a question. Even
rooms (e.g., https://slacklist.info/). We were able to join 29 more challenging, the automatic approach would need to
public Slack rooms. To collect the room archives, we used the automatically identify that a later statement was referring to
Slack API which requires a unique token for each room. In a specific previous question, thus answering that question.
some rooms, the token is made available to all the members; These tasks are challenging because Gitter and Slack do not
while it needs to be requested from the administrators in enforce explicit references between questions and replies
others. In the end, we were able to collect the archives of 11 when discussions are being held. Thus, because the focus of
public Slack rooms. Our intention in collecting room archives our research is not on automatically identifying threads, we
is to verify/refute, when possible, the perceptions of our par- manually label 200 threads from a randomly selected chat-
ticipants obtained through our qualitative analyses. room (i.e., Samplethreads ) for our investigations. We are aware
From the chat data collected, we computed the activity that Slack offers the ability to provide nested messages,
level of the individual chatrooms. The activity level is com- which could potentially be used to automatically identify
puted as the median elapsed time between two chat mes- threads. However, we observed that relying on the nested
sages of a given chatroom. For example, if the median messages from Slack would be an incomplete solution, since
elapsed time between two messages is less than 2 seconds, we observed (through our manual analysis), that several
this signifies a high activity level. This is important to verify related messages are not nested. Moreover, Gitter does not
whether the answers from the respondents regarding the provide the feature of nested messages, which is another
activity levels of the chatrooms are in agreement with the motivator for us to perform our manual analysis.
actual chat traffic in the chatrooms. We compute the time to solution of the threads in
In addition to the activity levels, we compute the time to Samplethreads , as the time difference between the first and last
solution. The time to solution is the time difference between message in a thread. Wang et al. [29] used a similar approach
the first and last message in a thread. A thread is a set of to detect the question getting a fast response in Q&A forums.
related messages in a discussion. For example, if a user asks
a question and developers answer that question (possibly by
having a discussion), the set of messages (e.g., answers or 3.3 Demographics
other follow-up questions) related to the question compose a We present an exploratory analysis of the demographics
thread. Automatically identifying threads is challenging data that we collect from the responses of the respondents.
because it requires research on using advanced Natural Lan- Fig. 1 shows a summary of the demographics data collected
guage Processing techniques [24], [25], [26], [27], [28]. For from the Slack and Gitter survey respondents.
Authorized licensed use limited to: The University of Hong Kong Libraries. Downloaded on March 31,2025 at 08:33:11 UTC from IEEE Xplore. Restrictions apply.
MEZOUAR ET AL.: EXPLORING THE USE OF CHATROOMS BY DEVELOPERS: AN EMPIRICAL STUDY ON SLACK AND GITTER 3993

TABLE 2
Survey Results Regarding the Perceived Impact of the Chatrooms on the Development Process

Theme % respondents agreeing Illustrating quote

to theme
Slackn¼114 Gittern¼48
Issue resolution 66% 63% “It happens a lot that [participants in the chat share code snippets from
their personal projects for bug detection or algorithmic optimization],
especially with beginners trying to get help from mentors.”
Access to information 53% 42% “Having access to maintainers who provide undocumented insight is a
good way of getting the users through the door”
Brainstorm features 31% 38% “By helping people use [the project] effectively and by learning about
the problems and wishes people have, we can improve the projects
existing functionality and generate ideas for new features.”
Group awareness 62% 15% “In distributed teams (virtual, remote, etc) the chat is the closest to real-
time, real-life contact we have. it allows for typing-while-thinking,
working out problems in short iterative bursts, and developing
awareness of what each one is working on.”
Project tasks 14% 54% “The flow that I’ve seen most often is: a developer asks a question in a
channel about something they think is a bug (or about a possible
enhancement), the project team (and sometimes community) discuss it
briefly and either explain it away (not a bug, not an acceptable
enhancement) or ask for a GitHub/JIRA issue to be created.”

Gender: The majority of survey respondents (i.e., 90.7%) in second phase of the thematic analysis, as well as some of
both Slack and Gitter have identified themselves as males. the codes (shown in italic) that result from the coding phase
Development experience: A shown in Fig. 1a, almost half of of the thematic analysis. Moreover, we compare and con-
the respondents (59 out of 114 and 20 out of 48 in Slack and trast the results from Slack and Gitter. We further include
Gitter, respectively) reported an experience of 10 or more quotes from the participants to provide more insights. The
years. Slack and Gitter participants are anonymously referred to
Education: 61.4% and 62.5% of the respondents have at as S# and G# (respectively), with # as the numeric identifier
least a Bachelor degree in Slack and Gitter, respectively. A of a participant. Aside from the quantitative observations
portion of the respondents (24.6% in Slack and 18.7% in Git- described in this section, all qualitative observations are
ter) disclosed that they are self-taught. Overall, the majority based on the opinions expressed by our participants.
of developers have received a formal education in software RQ1. Why do developers use the chatrooms?
development (Fig. 1b). The first research question explores the reasons behind
Employment: For the employment status, 70.1% and 58.3% the participation of developers in the chatrooms. Table 1
of the Slack and Gitter respondents, respectively, are shows the list of themes that emerge from the analysis for
employed full-time (Fig. 1c). both Slack and Gitter. The most recurrent themes for RQ1
Project role: We asked the participants about their roles in are the quality of the help, the response time, and giving back to
the projects associated to the chatrooms. In Gitter, it was the community.
reported that over half of the respondents (i.e., 52.1%) are 1) Similarities between Slack and Gitter
users of the projects, and do not make contributions to the Quality of the Help. In both Slack and Gitter, the most recur-
code base of the projects (Fig. 1d). The second most common rent theme is the quality of the help provided by the chatrooms.
reported role is “contributor”, with a distinction between The Slack participants report that asking questions on Slack
the 29.2% active contributors (i.e., make frequent contribu- provides learning opportunities including the best practices,
tions), and the 25.0% occasional contributors (i.e., making access to experts, and inside knowledge. S18 reports that the Slack
sporadic contributions) (Fig. 1d). In Slack, the most common rooms are “where the majority of knowledgeable people in my com-
reported role is the active contributor role (49.1%), followed munities are”. The Gitter participants mention that the chat-
by the user role (38.6%). 28.1% and 10.4% of the respondents rooms are useful to provide guidance, clarification, feedback on
in Slack and Gitter, respectively, are maintainers of the proj- new ideas, and possibly better solutions. G33 stated that he usu-
ects (i.e., have project privileges, such as reviewing code ally asks questions when he knows that “there has got to be a bet-
contributions). ter way / someone who made this already”. G16 goes further and
We provide in our appendix additional analyses relating says: “I needed advice on the architecture of my commercial apps
the (i) gender; (ii) experience; and (iii) employment status of and the gitter chat saved me a lot of money which I’d have wasted on
our participants with the most recurrent themes obtained in servers if I’d gone with something different”.
each RQ (see Apprendix H). Response Time. The participants mention that the response
time in chatrooms is an important motivation for asking
questions in both Slack and Gitter. The participants describe
4 RESULTS the conversations in these chatrooms as ‘interactive’,
In this section, we answer our RQs by reporting the most ‘conversational’, and ‘instant’. We provide quantitative
common themes (shown in bold) that emerge from the insights regarding this perception later.
Authorized licensed use limited to: The University of Hong Kong Libraries. Downloaded on March 31,2025 at 08:33:11 UTC from IEEE Xplore. Restrictions apply.
3994 IEEE TRANSACTIONS ON SOFTWARE ENGINEERING, VOL. 48, NO. 10, OCTOBER 2022

TABLE 3
Survey Results Regarding the Quality Determinants of the Chatrooms

Theme % respondents Illustrating quote

agreeing to theme
Slackn¼114 Gittern¼48
Features improvement 75% 82% “Was looking forward to topics, but they’re very incomplete. While
gitter is great for in the moment support, historical topics get lost
quick. Would be great to have topics or pinned threads or something to
keep quality info easily accessible.”
“Discoverability of public Slacks is very poor – pretty much every
community Slack I’m in ends up running their own onboarding app on
Heroku (because Slack is designed for closed, paid teams, rather than
open, free communities).”
Community 52% 63% “I think a big factor is the general attitude of the channel. It’s a
welcoming place founded upon mutual respect and an absence of
elitism, and it’s a large contributor to why I am so invested in it.”
Moderation 43% 21% “[Need for] AI that can watch for uncivilized behaviour is something
that every site should be implementing even if it’s just to flag
conversations for moderator review with no action taken on the part of
the software.”

Giving Back to the Community. As far as providing the participants claim that answering questions is an oppor-
answers in the chatrooms goes, the most recurrent theme is tunity to learn by teaching, by “validating personal assumptions,
giving back to the community. Specifically, the survey partici- and having one’s personal knowledge and code be “fact checked /
pants claim that they are eager to reciprocate help. Others sanity checked by more knowledgeable people” (G42). Moreover,
believe that answering questions helps to attract new contrib- providing help is a means to build a reputation among peers.
utors to a project and grow the community. The communities 11.6% (19/163) of the participants report that they answer
in the chatrooms provide low entry barrier for the new partic- questions to build reliance and gain respect from others in
ipants, bringing a sense of equality among the members. the same community. Finally, 6.1% (10/163) of the partici-
Although most teams in the chatrooms establish community pants explain that they promote their own projects when
guidelines according to the interviewee S10, the moderation providing help to others.
is considered moderate compared to other platforms, such The participants mention further themes related to the
as Stack Overflow. G33 explains that: “there is no competition features most appreciated in the chatrooms, such as the vola-
of reputation, but rather a nicely balanced space with equal oppor- tile content. S11 clarifies that “the questions on Slack are less
tunity for everyone to be heard”. G33’s response indicates that, public, disappearing eventually from free Slack rooms, such as
unlike Q&A platforms such as Stack Overflow, which Vapor”. In Gitter, the most appreciated feature is the seam-
employ gamification mechanisms for their users to earn less integration to GitHub, such as the inline markdown and
experience points (referred at times as “reputation”), chat- the repository update panel. In terms of the topics discussed,
rooms do not track these experience points, which likely technical discussions, such as custom code review and debug-
reduces the sense of competition. G44 further mentions that ging, are common. G27 and G32 explain that debugging
there is no need to go through pre-moderation or restrictions questions are usually presented as follows: “I got unexpected
based on a score. results, what did I do wrong?”, or “Is it possible to do [task X]
Personal Gain. In addition to altruistic reasons, the survey using this library?”. According to S14, “the topics span a wide
participants report that there is some personal gain from technical spectrum ranging from the architectural level, to the
answering questions in the chatrooms. 15.3% (25/163) of nuts and bolts of specific technical tasks”. The participants also
report discussing project documentation issues, such as the
project configuration. G5 explains that “The configuration part
would be solved with good documentation, but we never found a
project with good documentation”, thus highlighting existing
issues with projects documentation (described by the partic-
ipants as voluminous, ambiguous, or incomplete).
2) Quantitative analysis of response time
Regarding the perceived response time from our partici-
pants, we investigate the chat data of the chatrooms that the
participants qualify as fast and we compute the set of chat-
room-level metrics, described in Section 3.2). For this pur-
pose, we use our previously explained time to solution
metric. In Appendix E, available in the online supplemental
material, we show the median activity levels of the chat-
rooms that were perceived as having fast responses. In the
Fig. 1. Demographics of the Slack and Gitter respondents. chatrooms from which we receive survey responses, the
Authorized licensed use limited to: The University of Hong Kong Libraries. Downloaded on March 31,2025 at 08:33:11 UTC from IEEE Xplore. Restrictions apply.
MEZOUAR ET AL.: EXPLORING THE USE OF CHATROOMS BY DEVELOPERS: AN EMPIRICAL STUDY ON SLACK AND GITTER 3995

median activity level is 77 seconds. However, the median

activity of all the chatrooms from the collected data set is
510 seconds. This indicates that our participants come
mostly from active and large chatrooms.
We further investigate whether the speed of response
has a relationship with the number of participants in a
chatroom. We find a very weak correlation between the two
factors (Spearman’s correlation = -0.17 and p-value = 8.56e-
07). We conclude that the speed of response is not associ-
ated with the number of participants in a room. The num-
ber of participants has a very strong correlation with the
number of messages exchanged in a chatroom, as expected
(Spearman’s correlation = 0.98 and p-value < 2.2e-16). Fig. 2. Summary of the uses of Slack: A comparison of the work by Lin
Additionally, we compute thread-level metrics based on et al. [8] and our study.
the manually labelled Samplethreads . We find that the
median discussion in a chatroom is 101.57 minutes, and the to a more ‘corporate’ chat-service with more emphasis on
median initial time to solution is 12.85 minutes. 61.9% of the reputation, quality, and awareness about other members.
threads get a response within an hour. In contrast, Wang 5) Insights from interviews
et al. find that 69.2% answered questions get an answer Regarding the reasons to participate in Slack and Gitter
within one hour on Stack Overflow [29]. As such, the per- chatrooms, our interviewees mention that corporate teams
ception of the survey repondents that the chatrooms are sig- are mostly required to provide assistance. Mentorship takes
nificantly faster than other platform such as Stack Overflow place over the chatrooms, in order to ease the onboarding of
is not substantially reflected by the data at hand. Further the new members. The open communities, on the other
empirical analysis based on larger samples is needed to fur- hand, display different uses of the chatrooms. First and fore-
ther confirm this observation. most, developers are drawn to the open communities chat-
3) differences between slack and gitter rooms to learn more about the project, and get custom help
The sharing of expert knowledge happens for different for specific issues. Developers are happy to reciprocate
reasons in each chatroom. For example, 17.5% of the Slack help, and consider the participation in the conversation a
respondents (20 out of 114) mention that providing guid- leisurable activity. Another aspect mentioned by the inter-
ance to the new members over Slack is required, and 7.9% (9 viewees is that chatrooms exhibit two classes of usage: 1)
out of 114) report that participation is encouraged to achieve developer support, and 2) user support. The developer sup-
the company’s goals. In this respect, S52 says the following: port is about providing guidance to developers in learning
“my senior position requires mentorship over new junior members about, and potentially contributing to, the project or tech-
in our team who require some attention. I want this task to be ful- nology associated with the chatroom. The user support hap-
filled correctly with little distraction. Thus, I am motivated to pro- pens when technical assistance is provided to the users of a
vide clear and transparent answers as much as possible, to software (e.g., installation issues). As reported by our inter-
maximize the revenue and achieve the company’s goals”. On the viewees, the Gitter open communities are almost exclu-
other hand, the Gitter respondents share knowledge for per- sively centered around user support. The Slack rooms, on
sonal enjoyment. The participants reportedly enjoy chiming the other hand, exhibit both types of usages even in the
in discussions about corner cases, or answer questions when open communities. A Slack interviewee (S21) informs us
they are bored, curious, or procrastinating on their work. that his work team has both public channels to interact with
Another example is that Slack respondents are concerned the users of the software, and private channels for internal
about building their reputations to maintain “a good communication. The Slack open communities also depend
employee image”(S41). In Gitter, the respondents are eager to on the channel feature of Slack to direct the incoming ques-
build their reputations to be recognized as experts by their tions to the right audience (e.g., #team-support, #team-
peers. Indeed, we performed a x2 test to check whether the devops). Although 5 of the interviewed developers claim
reported themes are significantly different depending on that the developer support is more common in Slack, it is
the chatroom used (e.g., Gitter or Slack). Our x2 test yields a not possible to verify as we do not have access to the data
p-value ¼ 2:345997e19 , which strongly indicates that the from the private channels.
reported themes are dependent on the chatrooms at use.
4) differences from Lin et al. [8] Slack and Gitter are used for the quality of the help
With Slack gaining more adoption and Gitter remaining received within a short response time. Sharing knowledge
unexplored, we revisit Slack and investigate Gitter for the in Slack is sometimes a required activity to discuss the
first time in our first Research Question (RQ): “Why do develop- projects’ tasks; while discussions in Gitter may occur for
ers use the chatrooms?”. Our study reveals both similarities personal enjoyment.
and differences with the uses of Slack, that were identified
by Lin et al. [8]. We show a summary of the similarities and
differences on the Venn Diagram, shown in Fig. 2. Although RQ2. What is the perceived impact of the chatroom use on the
there is an important overlap between the two studies, we software development process?
do observe an evolution of the uses of Slack, from a ‘fun’ The second research question explores the perceived
chat-service used partly for networking and social activities, impact of the chatrooms on some aspects of the
Authorized licensed use limited to: The University of Hong Kong Libraries. Downloaded on March 31,2025 at 08:33:11 UTC from IEEE Xplore. Restrictions apply.
3996 IEEE TRANSACTIONS ON SOFTWARE ENGINEERING, VOL. 48, NO. 10, OCTOBER 2022

development process (e.g., resolution of issues). Table 2 impact on their own perceived productivity. 40.5% of the
shows the list of resulting themes, for both Slack and Gitter. survey participants report a positive impact on their produc-
Some of the most common reported themes are issues resolu- tivity, as shown in Appendix F, available in the online sup-
tion, group awareness, and access to information. Group aware- plemental material. The chatroom discussions reduce the
ness includes knowledge about members of the project, the ‘endless trials and errors’ to determine the best way to do a
project areas they are working on, their current progress, task. G32, a maintainer of a GitHub project, explains that
and their plans [4]. ‘the chatrooms offload some of the support to the community.
1) Similarities between Slack and Gitter They are especially good at helping newbies. That saves the core
Help With Issue Resolution. According to S15, S16, G7, and team cycles to invest into the project.’ However, a non-negligi-
G14, the actual resolution time of an issue is likely shortened ble portion of the survey participants (15.9%) believe in a
because of the presence of many ‘brains and eyes’ to help negative impact on the productivity. S26 believes that the
out, and the possibility to tag the concerned developers. S10 productivity is slowed down because of the FoMo (Fear of
adds that a possible reason is that the discussions in Slack Missing out). G38 confirms that there is always something
are much faster than in GitHub, where response sometimes going on in the chat, which is an incentive to read and par-
takes several days. S31 goes even further and states that ticipate. Other survey participants (37.5%) have a more
sometimes issues that would have been reported on GitHub nuanced opinion about the impact of the chatrooms on the
are not reported at all, as they are fixed through an productivity (i.e., mixed or no impact). The participants find
exchange in the chatroom. However, S13 argues that it does that although productivity might be reduced, the benefits of
not apply to the resolution time of the more complex issues. using the chatrooms even out the damage caused. S29
Others explain that the impact is not on the resolution time explains that the chatrooms help speed up cases where
itself, but rather on the visibility of the issues because ‘louder progress is stalled, however, it is sometimes misused to ask
voices tend to win’. S17 explains that people pressing him questions when an email is more appropriate. Related to
on Slack for an issue fix are probably going to be prioritized, that, the participants complain that the inability or difficulty
simply because of exposure. G24 reveals that several issues to search for past discussions lowers their productivity.
have been found thanks to Gitter users asking questions, Overall, it seems that the use of the chatrooms requires self-
specifying that these issues would have gone undiscovered discipline to avoid getting side-tracked, as explained by
otherwise. Specifically to Gitter, the panel showing the S14. Additionally, S10 explains that it is possible to make
GitHub repository updates in the associated Gitter room use of the features provided in order to manage (i.e.,filter or
(e.g., a new comment on an issue) is important. G17 claims mute) the chat notifications.
witnessing a GitHub issue getting more activity because 2) Differences between Slack and Gitter
one person responded to it on GitHub, and others noticed it We find that the most reported impacts are the group
from the Gitter channel. awareness and guiding the project tasks (bug fixes and new fea-
Access to Information. The participants claim that one of tures) in Slack and Gitter, respectively. Our x2 test yield a
the important impacts is the access to information, such as the p-value ¼ 6:725e8 , which strongly indicates that the differ-
best coding practices of a project and the peripheral knowledge. ence in the reported themes is dependent on the chatroom
For example, it is the opinion of S19 that “the chatroom dis- at use. Group awareness is critical for the software develop-
cussions allow for learning the best coding practices in the Vapor ment teams, especially when working remotely. In Slack,
project, and for prompting developers to think of better coding the teams are able to collaborate remotely, make decisions faster
approaches”. Eventually, developers are able to produce in smaller teams, and share updates. It is also believed by S25
higher quality products, according to S12. Furthermore, that the Slack rooms help minimize the unnecessary talks
developers are able to obtain up-to-date resources about a and meetings for the collocated teams, and hence allow
project/technology from the chatrooms discussions. G23 developers to dedicate more time to the quality of the prod-
mentions the Angular project as an example of a fast grow- uct. However, S41 explains that the negative side is that
ing technology, where “the tutorials from even a few months decisions are accumulated in the chatroom itself, without
ago may be obsolete”. In addition to the up-to-date resources, being organized or centrally recorded. On the other hand,
G32 argues that “the chatrooms’ discussions contain quite a bit the discussions in the Gitter chatrooms help the maintainers
of ‘peripheral’ knowledge, such as discovering other technologies, learn about the problems and wishes people have in an
patterns, and news”. informal setting, so they can improve the existing function-
Help in Brainstorming Features. The design of the projects ality of the projects and generate ideas for new features
benefits from the feedback in the chatrooms. Specifically, S14 (G11). This confirms the use of Gitter for user support
explains that it helps shape new features, identify shortcom- within the open communities. In addition to user support,
ings of the design, and test pre-release versions. Similarly to the project maintainers utilize the chatroom to quickly ‘gut
Slack, the Gitter participants report that the use of the chat- check’ an idea before putting together a full pull request on
rooms has helped in brainstorming new ideas and resolving GitHub. It is reported that the discussions on Gitter are
issues, with a reduced effort. In this regard, we find that helpful in raising issues (that are possibly critical) in the proj-
Slack and Gitter are similar to the mailing lists, which are ect associated to the chatroom. The issues are further dis-
also used to discuss the activities of the projects [4]. cussed, and possibly solved through code reviews. G14
Productivity. The productivity of developers is a much explains that discussions on Gitter often lead to new pull
debated topic in the era of socially-enabled software devel- requests and issues being opened and closed. However,
opment, and the developers’ chatrooms are no exception. G21 states that important issues related to the project are
The responses from the participants reflect conflicting discussed solely on GitHub.
Authorized licensed use limited to: The University of Hong Kong Libraries. Downloaded on March 31,2025 at 08:33:11 UTC from IEEE Xplore. Restrictions apply.
MEZOUAR ET AL.: EXPLORING THE USE OF CHATROOMS BY DEVELOPERS: AN EMPIRICAL STUDY ON SLACK AND GITTER 3997

3) Insights from interviews yields a p-value ¼ 0:01383, which strongly indicates that the
When it comes to the impact on the associated project, importance given moderation is dependent on the chatroom
managing the internal comunication of the team comes on at use.
top for the corporate teams. Developers report that ‘pinging’ 3) Insights from interviews
a co-worker in the chatroom is less invasive than stopping by Our interviewees did not provide much further insights
their office. Furthermore, the exchange in the chatroom compared to what we have captured from our survey. Both
allows for passive knowledge sharing among the co-work- Gitter and Slack interviewees further stressed the impor-
ers, an important aspect of maintaining project awareness. In tance of a more structured search with the proper managa-
an interview with S10, a member in a dozen Slack teams both ment of historical data.
public and private, he reports that “company Slack usage tends
to be much more structured and proactively administered, by which The quality of the community, in terms of friendliness,
I mean the use of different user roles and different channel access”. activity and expertise are key determinants of a
The projects associated to open communities benefit from chatroom’s quality in both Slack and Gitter. However,
the chatrooms by attracting new contributors, who consis- there is more request for moderation in Slack, compared
tently participate in the chatrooms. to Gitter. Several features (e.g., bots and history manage-
ment) could be improved in Slack and included in Gitter.
Slack and Gitter reportedly help developers to support
the issue resolution process, to have access to information,
and to brainstorm features. However, it is perceived that
Slack has more impact on improving the group awareness; 5 DISCUSSION
while Gitter has an impact on guiding the project tasks In this section, we provide further insights regarding the
(e.g., new features). demographics of our participants, discuss possible implica-
tions from our findings, and outline key differences
between the two chat-services, (Slack and Gitter).
RQ3. What defines the quality of the chatrooms and their Demographics Analysis: while we present here the key
related chat-service? takeaways from our demographics, a more detailed discus-
To better inform the design decisions of the chatrooms, sion can be found in Appendix H, available in the online
we investigate the elements that characterize the quality of supplemental material. In terms of motivation to participate in
the chatrooms in RQ3. Table 3 shows the list of the themes chatrooms, most experienced participants are mainly moti-
that emerge from the survey analysis. The survey partici- vated by giving back to the community, whereas most inexpe-
pants report the following ratings of Slack and Gitter rienced participants are motivated by receiving fast
respectively: Excellent (33.9% - 41.3%), Good (47.2% - 41.3), responses and quality help. Regarding the impact of chatrooms
Average (12.3% - 17.4%), Below average (3.8% - 0.0%), and in the software project, we observe that less experienced par-
Poor (2.8% - 0.0%). The most common themes in RQ3 are the ticipants perceive that access to information and feature brain-
community, and the features. storming are the major impacts that chatrooms have in the
1) Similarities between Slack and Gitter projects, whereas more experienced participants stress that
Community. The most reported quality determinant in the issue resolution and tasks are the aspects that are mostly
chatrooms can be summarized as follows: “a friendly, inclu- impacted. Lastly, regarding the perceived quality of chatrooms,
sive community of folks with deep technical knowledge” we note that very experienced participants (i.e., 10+ years of
(G19). S11 argues that in terms of expertise, “it is better to experience) praise the community aspect of chatrooms,
have a good mix of experienced users and novices”. 55.8% (91/ which is also related to the previous motivation of “giving
163) of the participants stress on the fact that a welcoming back to the community”. As for less experienced partici-
and safe atmosphere is key to encourage contribution and pants (0-4 years of experience), the focus is on the improve-
exchange. In addition, the occasional participation of the ment of features. We suspect that less experienced
project leaders or maintainers in the discussions sets apart participants have a mindset set more towards productivity
the good chatrooms from others. (so they can climb the ladders more quickly).
Features. In terms of features, it appears that the Slack Possible implications: We discuss the following aspects as
participants are concerned about improving the current fea- possible implications of the findings presented in Section 4.
tures; while the Gitter participants request new features, Project promotion and adoption. The responses from the
such as bots and history management. In both chatrooms, it is survery respondents indicate that Slack and Gitter are both
a common complaint that knowledge is lost. In Slack, the appreciated for the quality of help received, within a short
conversations in the chatrooms under the free plans are vol- response time. For the project owners, the previous finding
atile, and the search features provided are quite basic. In could be an incentive to further support the online commu-
Gitter, the search option is almost useless according to G23. nities through the chat-services, in order to promote the
Consequently, a non-ephemeral history and a better history project and boost its adoption. Another finding that sup-
management (e.g., advanced search) could be key features ports this claim is the reported role of the chat-services in
for chatrooms to reach their full potential. community building. A healthy and thriving developer com-
2) Differences between Slack and Gitter munity around a project could support its adoption.
Moderation. Based on the results shown in Table 3, Time to market. The time factor (i.e., interactive and real-
moderation of the chat (e.g., staying on topic) is twice more time conversations) was a commonly reported theme as a
important in Slack than it is in Gitter. Indeed, our x2 test motivation to use the chat-services. As such, we conjecture
Authorized licensed use limited to: The University of Hong Kong Libraries. Downloaded on March 31,2025 at 08:33:11 UTC from IEEE Xplore. Restrictions apply.
3998 IEEE TRANSACTIONS ON SOFTWARE ENGINEERING, VOL. 48, NO. 10, OCTOBER 2022

that a possible implication of using the chat-services, such some knowledge of other teams members and of the ongo-
as Slack and Gitter, could possibly enable a shortened time ing tasks. This is enabled on Slack through access control
to market. For instance, in the case of a closed-source proj- and multiple channel-based discussions, each aimed at spe-
ect, resolving pending issues or ambiguities among a team cific matters (e.g., design, deployment, testing, etc). On the
members in a timely manner can speed up the issue resolu- other hand, the openness of Gitter enables to hear from a
tion process, or the clarification of a set of requirements. larger pool of participants (i.e., reducing the feedback bar-
Allocation of tasks. In both Slack and Gitter, the respond- rier), which in turn enables to better guide the project tasks
ents report being able to build a reputation, and establish (e.g., prioritizing features or issues fixes). These findings
their expertise in certain areas. The implication of such find- can guide the choice of which chat-services is best suited for
ing is that the project owners could use this knowledge for a given project.
more targeted allocation of tasks in the future, or for more Perception of the users. Other differences observed
efficient recruitment of future contributors. This implication between Slack and Gitter could be explained by how the
applies to both the open-source and close-source projects. users of the chat-services perceive the tools (i.e., a company
Quality of the end product. Slack and Gitter allow for access communication tool versus a leisurely tool). Indeed, Gitter
to information possibly not available anywhere else, to users are more likely to report that Gitter has no impact on
brainstorm features, and to speed up the resolution process. their productivity (22.9%), compared to 6.9% of Slack users.
For instance, access to information not available in other Gitter users join on their own volition, with no requirement
project documentation may improve the productivity of the to ask or answer questions; while a number of Slack
new developers, or speed up the onboarding of new devel- respondents (9%) report that Slack participation is required.
opers. Additionally, some of the survey respondents Indeed, personal enjoyment was reported as one of the rea-
reported that the discussions in the chatrooms enable to sons to answer questions on Gitter (18.9%), but was never
optimize the solution to implement a feature or to resolve mentionned by the Slack respondents. This could possibly
an issue. As such, a possible implication is an improved end explain why Slack users have more mixed-feelings about
product, thanks to the access to a pool of developers with their productivity when using Slack, as shown in Table A5,
intricate knowledge of the technology used to build a soft- available in the online supplemental material. Similarly,
ware product. moderation (e.g., staying on topic) is twice as mentioned on
Retention of developers. Another commonality between the Slack as it is mentioned on Gitter, as a quality determinant
two chat-services is their contribution to community build- of chatrooms.
ing. The use of the chat-services to build a sense of belong-
ing, particularly for geographically distributed teams, could
possibly encourage the retention of developers, and encour- 6 THREATS TO VALIDITY
age future contributions. It is also important to highlight This section discusses the threats to validity of our study.
that the observed motivations to use chatrooms (see Table 1) Threats to conclusion validity concern the relation between
is what distinguishes modern developer chatrooms from the treatment and the outcome. The conclusion validity
other communication channels, such as GitHub pull- threat mainly comes from the inherent bias that comes from
requests or issue discussions. Chatrooms provide a more dealing with human subjects. For instance, in terms of dem-
immediate and informal access to resources. ographics, we observed that half of our participants report
Possible reasons behind the observed differences: both Slack an experience of > 10 years in software development,
and Gitter are chat-services that overlap in a number of the 90.7% are male, and 100% are involved in the more active
functionalities offered (e.g., group/private chat, code tag- chatrooms, as measured by the number of participants
ging, user tagging, and integrations with code hosting serv- (median = 709) and the number of exchanged messages
ices). However, we still observe a number of key differences within a year (median = 17,836). Therefore, our findings
in the ways the two chat-services are used and perceived by may be biased towards more experienced male developers
their users. We discuss below the possible intertwined rea- participating in the active chatrooms. To offset this bias, we
sons behind the observed dissimilarities. asked the participants in the interviews about their experi-
Focus of the discussions. In Gitter, the heart of the discus- ences with beginner developers, and with the use of the
sions appears to be the project itself. On the other hand, the smaller chatrooms in terms of the number of participants
Slack discussions are also focused on the people involved, and the number of messages exchanged. Additionally,
and their management (i.e., who is doing what when). There almost 70% of the participants are Slack users. It was not
is a number of the findings from Section 4 that support this our intention to recruit more Slack participants. However,
claim. For example, discussions about internal communica- the total number of Slack users is over 8 million; while Gitter
tion and project tasks are mentionned in total by 37.9% of has just over 800K users. Therefore, our population of par-
the participants in Slack, and 0 times in Gitter. Moreover, ticipants reflects the popularity of each chat-service. To off-
the highest reported impact in Slack is on group awareness set bias during thematic analysis (i.e., a manual process
(e.g., team updates and progress awareness); while Gitter subject to human error), we performed the analysis collabo-
reportedly impacts aspects of project improvement, such as ratively, and carried discussions until consensus was
opening new issues and pull requests. Maintaining group reached.
awareness through open-access chatrooms in Gitter may be Another potential bias in our work lies in the fact that the
quite challenging due to the nature of the chat services, that survey distribution in the case of Slack is skewed towards
easily allows posts from anyone with no apparent structure public chatrooms, since we have not had access to private
within the discussion. Indeed, project awareness requires chatrooms. However, regardless of the limited access to
Authorized licensed use limited to: The University of Hong Kong Libraries. Downloaded on March 31,2025 at 08:33:11 UTC from IEEE Xplore. Restrictions apply.
MEZOUAR ET AL.: EXPLORING THE USE OF CHATROOMS BY DEVELOPERS: AN EMPIRICAL STUDY ON SLACK AND GITTER 3999

private chatrooms (in the case of Slack), we were still able to First, as we also learn from our participants, messages get
study the motivations of participants that use Slack in a cor- lost quite fast in chatrooms, which reduces the visibility of
porate setting. We envision that studying more private chat- our survey requests. Additionally, in a number of chat-
rooms could enrich our insights regarding the usage of rooms, the admins deemed our request as off-topic or as not
Slack in corporate settings, since private organizations will in line with the regulations of the chatroom. As such, our
likely use private chatrooms in Slack. requests were never posted or simply deleted from the chat-
Lastly, another potential bias in our conclusions is due to rooms. Nevertheless, we still believe that our sample of par-
the nature of the projects (e.g., size, domain, complexity) of ticipants can provide valuable insights for the software
our participants. It may be that their perception regarding engineering community.
the motivation to use chatrooms may be tightly related to
the complexity of their project for example. However, our
goal in this work is to study the overall perceptions of our 7 RELATED WORK
participants instead of investigating the impact that the In this section, we discuss the related work and explain how
nature of the projects may have on the perceived usefulness they relate to our three research questions.
of chatrooms. Motivation of Chatrooms Uses. Developer chatrooms have
Threats to internal validity concern our selection of subjects attracted the attention of researchers for different rea-
and analysis methods. To better understand the use of the sons [32]. This is in part due to the important role of chat-
chatrooms by developers, we opted to use a survey to reach rooms in managing knowledge, as reported by Waska et al.
the participants. The survey inclusion criterion in the first [33]. Specifically, chatrooms support three types of knowl-
deployment phase is the membership in the chatrooms. edge transfer: the knowledge embedded (i) in developers’
This suggests a bias towards developers that favor the use heads [18], (ii) in artifacts [34], and (iii) in communities [8].
of the chatrooms, over the general population that might Storey et al. [18] argue that the modern chatrooms mimic
have differing views on the chatrooms. To mitigate this the face-to-face communication and allows the transfer of
bias, we targeted a more general population of developers tacit knowledge. Shihab et al. [34] find that the project arti-
in the second phase of the survey (active users of GitHub). facts (e.g., test cases and source code) are constantly dis-
We observed an agreement between the results of the two cussed in chatrooms. These discussions complement IDEs
deployment phases. and Version Control Systems (VCS). To support the commu-
Finally, there are some risks involved in our chosen nities, Lin et al. [8] report that software developers congre-
methodology for the coding process. We adopted the same gate in chatrooms to discuss what the community has
methodology used by Treude et al. [30] in which coders learned over time. Lin et al. [8] further reckon that commu-
would sit together and discuss the codes until saturation nication, information discovery, and the feeling of belong-
was reached. Although our methodology saved us time, the ing to a community are some of the most reported
risk of this methodology is the generation of bias in case one motivations behind using Slack, for example.
of the coders has a more vocal personality than others. The Impact of Chatrooms on Software Development. As the
However, we do not think that this is our case, since we usage of the modern chatrooms is increasing at a fast pace,
made sure that every coder had a reserved time to express researchers have been investigating the impact of the chat-
his/her ideas in the coding sessions. Another limitation of rooms on the software projects. Gutwin et al. [4] studied the
not performing independent coding, is that we do not pro- group awareness in open source projects, and reported that
vide a measurement of Inter Coder Reliability (ICR) [31]. The chatrooms (in their case IRC) are beneficial for ad-hoc com-
main purpose of computing an ICR is to make sure that munication and the overhearing of informal and technical
coders have a common understanding of the coding scheme discussions. An analysis by Hendel et al. [7] revealed that six
that is produced. On the other hand, we believe that our ses- distributed development teams use the chatrooms to achieve
sions can also reduce misunderstandings effectively synchronous communication with occasional asynchronous
because every deviation in the interpretation of a given exchanges. Chatterjee et al. [35] observed that developer
code was discussed instantaneously. Given that our coders chats in Slack provide more information on API mentions
discussed all responses together (as opposed to working than Q&A (e.g., Stack Overflow) posts. Storey et al. [18]
with samples), we believe that our methodology is sound. reported that the chatrooms support the one-on-one discus-
Threats to external validity concern the possibility to gener- sions and the knowledge exchange, among developers. Sto-
alize our results. Our study focuses on two widely-used rey et al. [18] also explained that private chat still falls short
developer-oriented chat-services (i.e., Slack and Gitter). in terms of supporting the sharing of contextual information.
Although our findings are specific to Slack (a proprietary Although previous studies have highlighted some
service geared towards corporations) and Gitter (an open advantages of using chatrooms in software development,
source service aimed at the open source communities), the impact that the chatrooms have on the projects being
many observations could be applicable to other chat-serv- developed is still unclear. The results of the survey adminis-
ices that offer similar services, such as Microsoft teams, tered in the scope of this study reveal the following impacts:
Cisco Jabber, or Chatter.
Although we strive for higher response rates (e.g., by
Our study confirms the results by Gutwin et al. [4]
providing incentives to 30% of our participants), we still that chat-services support group awareness, particu-
face challenges. For example, we posted our survey solicita- larly in Slack.
tion in 770 Gitter rooms. However, we received only 47
Both Slack and Gitter are found to enable access to
responses. There are two possible reasons for this problem. information unavailable elsewhere (e.g., peripheral
Authorized licensed use limited to: The University of Hong Kong Libraries. Downloaded on March 31,2025 at 08:33:11 UTC from IEEE Xplore. Restrictions apply.
4000 IEEE TRANSACTIONS ON SOFTWARE ENGINEERING, VOL. 48, NO. 10, OCTOBER 2022

knowledge of a project), which ultimately enables namely Slack and Gitter. We find that the chatrooms are
the learning and use of the best practices. used by the developers to exchange timely and expert

In Gitter, the feedback loop from a project’s users is knowledge, motivated by intrinsic reasons (e.g., support the
reduced, which possibly allows an evolution of the project community), and extrinsic reasons (e.g., improve the
project that is more in line with the expectation of reputations). We further attempt to identify the impact of
the users (e.g., in terms of which bugs to fix or fea- the use of the chatrooms on the software projects. Our find-
tures to implement). ings show that chatrooms reportedly impact the communi-
The Quality of Chatrooms. The quality of the communica- cation management, the development directions, and the
tion channels in software development is determined by the resolution time of issues. The developers’ productivity can
features that are offered by these channels (e.g., code be positively impacted by the use of chatrooms. We find
highlighting) and their communities (e.g., friendly develop- that quality determinants of chatrooms are mainly the activ-
ers). For instance, Internet Relay Chat (IRC) is a communica- ity levels and expertise of the community members. In our
tion channel that is commonly used by open source future studies, we will investigate further aspects in the
developers. However, Chowdhury and Hindle [36] identify chatrooms, such as (1) the quantitative impact on the proj-
a non-negligible amount of off-topic discussions in IRC dis- ects (as opposed to the perceived impact reported in this
cussions. Off-topic discussions can be a detriment to the effi- study); (2) the nature of the topics discussed; (3) the partici-
ciency of any chatroom service. For example, a considerable pation patterns of the developers and non-developers; (4)
amount of off-topic discussions can hinder knowledge orga- whether different levels of experience with chatrooms influ-
nization and may keep developers distracted. With this ence the perceptions of our participants; and (5) how gender
issue in mind, Chowdhury and Hindle [36] proposed classi- can influence the participation dynamics on chatrooms.
fiers to filter out off-topic discussions in programming IRC Regarding the participation patterns of developers, we will
channels. The classifiers are trained using Stack Overflow study whether existing Natural Language Processing techni-
data as positive examples of on-topic discussions, and You- ques are suitable for automatically identifying and classify-
Tube video comments as the opposite. Although Stack ing discussion threads [39]. Lastly, we plan to extend our
Overflow is praised for the quality of the questions and future studies to include other modern developer chat-
answers, it is also criticized for its “unwelcoming rooms, such as GitHub Discussions.
community” and the high entry barrier for the begin-
ners [37], [38]. For example, Vasilescu et al. [37] run a com- REFERENCES
parison of the representativeness and activity of genders
[1] F. P. Brooks Jr, The Mythical Man-Month: Essays on Software Engi-
between Stack Overflow and the mailing lists, and reveal neering, Anniversary Edition, 2/E. Noida, Uttar Pradesh, India:
that women disengage sooner despite similar activity levels Pearson Education India, 1995.
to men. Mamykina et al. [38] claim that the high entry bar- [2] B. Curtis, H. Krasner, and N. Iscoe, “A field study of the software
rier for beginners is established by a need for “reputation design process for large systems,” Commun. ACM, vol. 31, no. 11,
pp. 1268–1287, 1988.
points”. For example, a user cannot vote for the quality of [3] J. D. Herbsleb and A. Mockus, “An empirical study of speed and
an answer if his/her reputation is not at a certain level. communication in globally distributed software development,”
Although the quality determinants of mailing lists and IEEE Trans. Softw. Eng., vol. 29, no. 6, pp. 481–494, Jun. 2003.
[4] C. Gutwin, R. Penner, and K. Schneider, “Group awareness in dis-
Q&A systems, such as Stack Overflow, have been well tributed software development,” in Proc. ACM Conf. Comput. Sup-
investigated, the quality determinants of modern chat- ported Cooperative Work, 2004, pp. 72–81.
rooms, such as Slack and Gitter, are still unclear. Investigat- [5] M. Chui, J. Manyika, and J. Bughin, “The social economy: Unlocking
ing the quality determinants of modern chatrooms is value and productivity through social technologies,” McKinsey
Global Inst. 2012.
important because their usage has increased exponentially. [6] M. Storey, A. Zagalsky, F. F. Filho, L. Singer, and D. M. German,
In this regard, our study reveals the following: “How social and communication channels shape and challenge a
participatory culture in software development,” IEEE Trans. Softw.

Although it may be a conscious design decision to Eng., vol. 43, pp. 185–204, Feb. 2017.
keep the chatrooms history volatile (or at the very [7] M. Handel and J. D. Herbsleb, “What is chat doing in the work-
place?,” in Proc. ACM Conf. Comput. Supported Cooperative Work,
least hard to search), the survey respondents report 2002, pp. 1–10.
a need for mechanisms to manage the chat history, [8] B. Lin, A. Zagalsky, M.-A. Storey, and A. Serebrenik, “Why devel-
by providing better search features and history man- opers are slacking off: Understanding how software teams use
agement. For example, a project design decisions slack,” in Proc. 19th ACM Conf. Comput. Supported Cooperative Work
Soc. Comput. Companion, 2016, pp. 333–336.
may be discussed in the chat, and with time the [9] E. Parra, A. Ellis, and S. Haiduc, “Gittercom: A dataset of open
rationale behind such decisions may be lost and source developer communications in gitter,” in Proc. 17th Int.
hard to recover. Conf. Mining Softw. Repositories, 2020, pp. 563–567.
[10] V. Braun and V. Clarke, “Using thematic analysis in psychology,”

Although good moderation is a reported quality Qualitative Res. Psychol., vol. 3, no. 2, pp. 77–101, 2006.
determinant in both Slack and Gitter, the most [11] “Slack - about us,” Accessed: Sep. 30, 2018. [Online]. Available:
reported theme is the quality of the community, in https://slack.com/about
terms of both expertise and friendliness. [12] “Gitter - explore,” Accessed: Sep. 30, 2018. [Online]. Available:
https://gitter.im/explore/
[13] M. D. Gall, W. R. Borg, and J. P. Gall, Educational Research: An
Introduction. London, U.K.: Longman Publishing, 1996.
8 CONCLUSION [14] R. K. Yin, Case Study Research and Applications: Design and Methods.
Thousand Oaks, CA, USA: Sage Publications, 2017.
We design a survey to assess the motivations of the devel- [15] A. E. Hassan, “The road ahead for mining software repositories,”
opers when using two widely used developers’ services, in Proc. Frontiers Softw. Maintenance, 2008, pp. 48–57.
Authorized licensed use limited to: The University of Hong Kong Libraries. Downloaded on March 31,2025 at 08:33:11 UTC from IEEE Xplore. Restrictions apply.
MEZOUAR ET AL.: EXPLORING THE USE OF CHATROOMS BY DEVELOPERS: AN EMPIRICAL STUDY ON SLACK AND GITTER 4001

[16] J. W. Creswell, Educational Research: Planning, Conducting, and [36] S. A. Chowdhury and A. Hindle, “Mining stackoverflow to filter
Evaluating Quantitative. Upper Saddle River, NJ, USA: Prentice out off-topic IRC discussion,” in Proc. 12th Working Conf. Mining
Hall, 2002. Softw. Repositories, 2015, pp. 422–425.
[17] NV. Ivankova, JW. Creswell, and VL. Plano Clark , “Foundations and [37] B. Vasilescu, A. Capiluppi, and A. Serebrenik, “Gender, represen-
approaches to mixed methods research,” First Steps Res., Pretoria: tation and online participation: A quantitative study of
Van Schaik, pp. 253–282, 2007. stackoverflow,” in Proc. Int. Conf. Soc. Inform., 2012, pp. 332–338.
[18] M.-A. Storey, L. Singer, B. Cleary, F. Figueira Filho, and A. Zagal- [38] L. Mamykina, B. Manoim, M. Mittal, G. Hripcsak, and B. Hartmann,
sky, “The (r) evolution of social media in software engineering,” “Design lessons from the fastest q&a site in the west,” in Proc. SIG-
in Proc. Future Softw. Eng., 2014, pp. 100–116. CHI Conf. Human Factors Comput. Syst., 2011, pp. 2857–2866.
[19] L. Singer, F. Figueira Filho, B. Cleary, C. Treude, M.-A. Storey, and [39] O. Ehsan, S. Hassan, M. El Mezouar, and Y. Zou, “An empirical
K. Schneider, “Mutual assessment in the social programmer ecosys- study of developer discussions in the gitter platform,” ACM Trans.
tem: An empirical investigation of developer profile aggregators,” Softw. Eng. Methodol., 2021, pp. 1–39.
in Proc. Conf. Comput. Supported Cooperative Work, 2013, pp. 103–116.
[20] L. Singer, F. Figueira Filho, and M.-A. Storey, “Software engineer- Mariam El Mezouar received the PhD degree in
ing at the speed of light: How developers stay current using computing from Queen’s University, in 2019. She
twitter,” in Proc. 36th Int. Conf. Softw. Eng., 2014, pp. 211–221. is currently an assistant professor with the School
[21] J. Hahn, J. Y. Moon, and C. Zhang, “Emergence of new project teams of Computing, Queen’s University, Canada. Her
from open source software developer networks: Impact of prior col- research interests include mining software reposi-
laboration ties,” Inf. Syst. Res., vol. 19, no. 3, pp. 369–391, 2008. tories, collaborative software development, and
[22] C. Casalnuovo, B. Vasilescu, P. Devanbu, and V. Filkov, the social aspects of software engineering.
“Developer onboarding in github: The role of prior social links
and language experience,” in Proc. 10th Joint Meeting Foundations
Softw. Eng., 2015, pp. 817–828.
[23] S. Stieger, U.-D. Reips, and M. Voracek, “Forced-response in
online surveys: Bias from reactance and an increase in sex-specific
dropout,” J. Amer. Soc. Inf. Sci. Technol., vol. 58, no. 11, pp. 1653– Daniel Alencar da Costa received the PhD degree
1660, 2007. in computer science from the Federal University of
[24] R. Collobert, J. Weston, L. Bottou, M. Karlen, K. Kavukcuoglu, and Rio Grande do Norte (UFRN), in 2017, followed by
P. Kuksa, “Natural language processing (almost) from scratch,” J. a postdoctoral fellowship at Queen’s University,
Mach. Learn. Res., vol. 12, pp. 2493–2537, 2011. Canada, from 2017 to late 2018. He is an assistant
[25] C. D. Manning, M. Surdeanu, J. Bauer, J. R. Finkel, S. Bethard, and professor with the University of Otago, New Zea-
D. McClosky, “The stanford corenlp natural language processing land. His research goal is to advance the body of
toolkit,” in Proc. 52nd Annu. Meeting Assoc. Comput. Linguistics: knowledge of software engineering methodologies
Syst. Demonstrations, 2014, pp. 55–60. through empirical studies using statistical and
[26] P. Chatterjee, K. Damevski, N. A. Kraft, and L. Pollock, “Software- machine learning approaches as well as consulting
related slack chats with disentangled conversations,” in Proc. 17th and documenting the experience of software engi-
Int. Conf. Mining Softw. Repositories, 2020, pp. 588–592. neering practitioners.
[27] P. Chatterjee, K. Damevski, and L. Pollock, “Automatic extraction
of opinion-based q&a from online developer chats,” in Proc. IEEE/
ACM 43rd Int. Conf. Softw. Eng., 2021, pp. 1260–1272. Daniel M. German is currently a professor with the
[28] O. Ehsan, S. Hassan, M. E. Mezouar, and Y. Zou, “An empirical Department of Computer Science, University of
study of developer discussions in the gitter platform,” ACM Trans. Victoria. His research interests include in the areas
Softw. Eng. Methodol., vol. 30, no. 1, pp. 1–39, 2020. of mining software repositories, open source soft-
[29] S. Wang, T.-H. Chen, and A. E. Hassan, “Understanding the fac- ware ecosystems and the impact of intellectual
tors for fast answers in technical Q&A websites,” Empir. Softw. property in software engineering.
Engi., vol. 23, no. 3, pp. 1552–1593, 2018.
[30] C. Treude, F. Figueira Filho, and U. Kulesza, “Summarizing and
measuring development activity,” in Proc. 10th Joint Meeting Foun-
dations Softw. Eng., 2015, pp. 625–636.
[31] C. O’Connor and H. Joffe, “Intercoder reliability in qualitative
research: debates and practical guidelines,” Int. J. Qualitative Meth- Ying Zou is the canada research chair in software
ods, vol. 19, 2020, Art. no. 1609406919899220. evolution. She is a professor with the Department of
[32] R. Alkadhi, J. O. Johanssen, E. Guzman, and B. Bruegge, “React: Electrical and Computer Engineering, and cross-
An approach for capturing rationale in chat messages,” in Proc. appointed to the School of Computing, Queen’s Uni-
ACM/IEEE Int. Symp. Empir. Softw. Eng. Meas., 2017, pp. 175–180. versity, in Canada. She is a visiting scientist with the
[33] M. M. Wasko and S. Faraj, “”It is what one does”: Why people IBM Centers for Advanced Studies, IBM Canada.
participate and help others in electronic communities of practice,” Her research interests include software engineer-
J. Strategic Inf. Syst., vol. 9, no. 2–3, pp. 155–173, 2000. ing, software reengineering, software reverse engi-
[34] E. Shihab, Z. M. Jiang, and A. E. Hassan, “Studying the use of neering, software maintenance, and service-
developer IRC meetings in open source projects,” in Proc. IEEE oriented architecture. For more information please
Int. Conf. Softw. Maintenance, 2009, pp. 147–156. visit: http://post.queensu.ca/zouy
[35] P. Chatterjee, K. Damevski, L. Pollock, V. Augustine, and N. A.
Kraft, “Exploratory study of slack q&a chats as a mining source
for software engineering tools,” in Proc. IEEE/ACM 16th Int. Conf. " For more information on this or any other computing topic,
Mining Softw. Repositories, 2019, pp. 490–501. please visit our Digital Library at www.computer.org/csdl.

Authorized licensed use limited to: The University of Hong Kong Libraries. Downloaded on March 31,2025 at 08:33:11 UTC from IEEE Xplore. Restrictions apply.