AI Impact 1
AI Impact 1
Wendy Ceccucci
[email protected]
Business Analytics and Information Systems Department
Quinnipiac University
Hamden, CT 06518 USA
Alan Peslak
[email protected]
Department of Information Sciences and Technology
Penn State University
Dunmore, PA 18512 USA
Kiku Jones
[email protected]
Business Analytics and Information Systems Department
Quinnipiac University
Hamden, CT 06518 USA
Abstract
GitHub Copilot, developed by GitHub, is a new tool aiding developers with a range of tasks, including
the generation of code snippets, documentation assistance, and the formulation of implementation
strategies. Comparable AI development tools, such as Tabnine and AWS Code Whisperer, also serve as
developmental aids but are utilized to varying and much lesser extents. Our research examines the
adoption of GitHub Copilot and similar AI development tools among professional developers and other
users using the Stack Overflow Annual Survey. The study reveals notable age-related disparities in tool
usage, with younger individuals showing a markedly higher propensity to employ these technologies
compared to older users. Additional significant insights include variations in usage based on the type of
developer, professional status, and the influence of user attitudes towards AI on the adoption of GitHub
Copilot among developers.
Keywords: AI Developer Tools, Artificial Intelligence, GitHub Copilot, Technology Adoption, AI Trust
developer tools among IT developers. The study these aspects, the study seeks to contribute to
aims to provide an understanding of how these the responsible and effective integration of AI in
tools are reshaping the development landscape, software development practices.
the attitudes of developers towards them, and the
trust issues that may arise. 2. LITERATURE REVIEW
programming has been extensively studied, learning courses into their curricula, which equips
highlighting it as a crucial design requirement for new graduates with the skills and familiarity
these tools. Trust is considered a key prerequisite needed to leverage these tools effectively.
for ensuring the safety of the resulting software
products (Hasselbring & Reussner, 2006; Lipner, Moreover, the rapid pace of technology means
2004). that younger individuals often have a 'digital
native' advantage. They tend to be more adaptive
According to a study by Wang, et al. (2024), to changes in the tech landscape, including the
developers’ trust is rooted in the AI tool’s use of sophisticated AI platforms that require a
perceived ability, integrity, and benevolence, and steep learning curve. A report by GitHub, The
is situational, varying according to the context of State of the Octoverse (Daigle & GitHub Staff,
usage. Existing AI code generation tools lack the 2024), highlights that newer programmers are
affordances for developers to efficiently and more likely to utilize AI coding assistants,
effectively evaluate the trustworthiness of AI- attributing this to their up-to-date training and
powered code generation tools. inherent flexibility in adopting new workflows.
Several other demographics and factors can Additionally, the software development industry
influence a developer's trust in AI tools, and itself fosters a culture of innovation and
understanding these can help tailor AI solutions continuous learning, which resonates well with
to better meet user needs. These factors include the mindset of younger developers. They are
developer’s geographical location, gender, and often driven by the potential to streamline coding
socioeconomic status. processes, enhance productivity, and solve
complex problems efficiently with the help of AI
Developers from different regions may have tools. In practical terms, younger developers
varying levels of trust in AI tools based on cultural utilize AI tools for a range of tasks including
attitudes towards technology and data privacy. writing and reviewing code, automating repetitive
Although their research did not focus on tasks, and even in conceptual phases of
developers specifically, Grassini and Ree (2023) development like brainstorming and prototyping.
found that respondents from the USA The convenience and efficiency offered by AI tools
demonstrated higher levels of hopefulness for AI align with the fast-paced, results-oriented
technology compared to those from the UK. Their environment preferred by this demographic. The
finding suggests that cultural context does play a influence of AI on younger developers is not just
role in shaping individuals' perceptions of AI transforming their individual workflows but is also
technology. shaping the future dynamics of team
collaborations and project management in
In the same study, they found that male software development. As these young
respondents showed higher hopes for AI systems developers progress in their careers, their
than female respondents. This finding is preference for AI-enhanced workflows is likely to
consistent with previous research, which reported catalyze broader adoption of these technologies
that males perceive AI as more useful (Arujo, et across the industry, potentially leading to more
al., 2020) and generally more favorable than innovative solutions in the overall approach to
females (Lozano, et al., 2021). Likewise, a study software development.
by Armutat, et al. (2024) found that men tended
to view AI applications more positively, rate their 3. METHODOLOGY
own AI competencies higher, and have more trust
in the technology compared to women. This study explores the usage of AI tools by
software developers by leveraging data from the
Age and AI Coding Tools 2023 Stack Overflow survey. The Stack Overflow
The integration of AI tools into the workflow of Developer Survey is widely acknowledged as the
software developers has shown a notable age- most comprehensive survey targeting coders
related trend, with younger developers worldwide. Annually, it covers a wide range of
demonstrating a higher propensity to adopt such subjects, from developers’ technology
technologies. This trend can be attributed to preferences to their career aspirations. According
several factors. First, younger developers are to Stack Overflow’s website:
generally more exposed to the latest “For 13 years, we've delivered industry-
technological innovations during their education leading insights regarding the developer
and early careers, making them more receptive community. This is the voice of the
to AI-driven solutions. Educational institutions developer. Analysts, IT leaders,
are increasingly incorporating AI and machine reporters, and other developers turn to
Surprisingly, those learning to code are lower Showing the benefits of learning to code utilizing
than these groups at 23%. the AI developer tools and the time saving in
completing projects will likely be a positive for
both the organization and the developer. If there
are developers on site who are already proficient
in using the AI developer tools, it may be
beneficial to the organization to have these
experienced developers work with those who are
just beginning to discover the AI developer tools.
In addition, those developer types who have the
highest usage rate of AI developer tools may be
the best people to talk to the other developers
who are not utilizing the tools to discuss their
experiences to provide credibility to the
recommendation to use the tools.
6. REFERENCES
Table 9: Developer’s Level and use of GitHub Araujo, T., Helberger, N., Kruikemeier, S. &
Copilot Vreese, C. (2020). In AI we trust?
Perceptions about automated decision-
When we examine the developer type and use of making by artificial intelligence. AI &
GitHub Copilot, we find that Blockchain, Society, 35, 611-623.
Developer Advocates, and Front-end developers https://doi.org/10.1007/s00146-019-
have the highest usage rate. They all exceed 00931-w
30% usage (Appendix 1). Database, enterprise,
and embedded developers all are lowest end at Armutat, S., Wattenber, M., & Mauritz, N. (2024).
less than 15%. Artificial Intelligence – Gender-Specific
Differences in Perception, Understanding,
5. CONCLUSIONS and Training Interest. Proceedings of the 7th
International Conference on Gender
GitHub Copilot was found to be used the most of Research, 7(1), 36-43.
all AI powered developer tools regardless of a https://doi.org/10.34190/icgr.7.1.2163
person’s profession. When looking strictly at Asaduzzaman, M., Mashiyat, A. S., Roy, C., &
software developers, the same result was found Schneider, K. (2013). Answering questions
with GitHub Copilot being used by 25% of the about unanswered questions of stack
respondents and the next tool coming in at only overflow. Proceedings of the 10th Working
5%. An additional finding to help tool developers Conference on Mining Software Repositories,
is that a majority of developers have a favorable 97-100.
view of AI and somewhat trust it. The favorable https://doi.org/10.1109/MSR.2013.662401
view of AI does seem to impact a developer’s 5
choice in using the AI developer tool. However,
trust in AI appears to have limited impact. This is Barua, A., Thomas, S., & Hassan, A. (2014). What
an area where future researchers will want to look are developers talking about? An analysis of
further to determine what other factors are topics and trends in stack
possibly moderating this relationship. overflow. Empirical Software
Engineering, 19, 619-654.
Age has a clear impact on the use of AI developer https://doi.org/10.1007/s10664-012-9231-
tools. As the age group increased, the use of AI y
developer tools decreased. This finding, which is
Cheng, R., Wang, R., Zimmermann, T., & Ford,
supported in the literature, may be helpful to
D. (2024). “It would work for me too”: How
organizations as they review their workforce and
Online Communities Shape Software
determine potential training or motivating
Developers’ Trust in AI-Powered Code
opportunities to engage different age groups to
Generation Tools. ACM Transactions on
participate in using the AI developer tools.
Interactive Intelligent Systems. 14(2), 1–39.
Another finding which can be helpful to
https://doi.org/10.1145/3651990
organizations is that those learning to code do not
use the AI developer tools as much as CodeSignal (2024). Trends Report Developers &
professional developers or even hobbyists. AI Coding Assistant Trends. CodeSignal.
Retrieved July 26, 2024, from Lipner, S. (2004). The trustworthy computing
https://codesignal.com/report-developers- security development lifecycle. Proceedings
and-ai-coding-assistant-trends/ of the 20th Annual Computer Security
Applications Conference, 2–13.
Daigle, K. & GitHub Staff (2024). The State of the
https://doi.org/10.1109/CSAC.2004.41
Octoverse. GitHub. Retrieved on July 26,
2024, from https://GitHub.blog/news- Lozano, I., Molina, J. & Gijón, C. (2021).
insights/research/the-state-of-open-source- Perception of artificial intelligence in
and-ai/ Spain. Telematics and Informatics,
63. https://doi.org/10.1016/j.tele.2021.101
Desai, A. & Deo, A. (2022). Introducing Amazon
672
CodeWhisperer, the ML-powered coding
companion. AWS. Retrieved July 26, 2024, Palacios-Gonzalez, E., Fernandez-Fernandez, H.,
from Diaz, V., Garcia-Bustelo, B., & Lovelle, J.
https://aws.amazon.com/blogs/machine- (2008). A review of Intelligent Software
learning/introducing-amazon-codewhisperer- Development Tools. Proceedings of the 2008
the-ml-powered-coding-companion/ International Conference on Artificial
Intelligence, 585-590.
Dakhel, A., Majdinasab, V., Nikanjam, A., Khomh,
F., Desmarais, M., & Jiang, Z. (2023). GitHub Shani, I., & GitHub Staff (2023). Survey reveals
Copilot AI pair programmer: Asset or AI’s impact on the developer experience.
Liability?, Journal of Systems and Software, GitHub. Retrieved July 26, 2024, from
203. https://GitHub.blog/2023-06-13-survey-
https://doi.org/10.1016/j.jss.2023.111734 reveals-ais-impact-on-the-developer-
experience/
Ernst, N., & Bavota, G. (2022). AI-Driven
Development Is Here: Should You Worry?
Snyk (2023). 2023 Snyk AI-Generated Code
IEEE Software, 39(2) 106-110.
Security Report. Snyk. Retrieved on July 26,
https://doi.org/10.48550/arXiv.2204.07560
2024, from https://snyk.io/reports/ai-code-
GitHub (2024). GitHub Developer Survey. security/
GitHub. Retrieved July 26, 2024, from
Stack Overflow Retrieved July 26, from
https://GitHub.blog/news-
https://survey.stackoverflow.co/2020#over
insights/research/survey-reveals-ais-impact-
view
on-the-developer-experience/#developers-
want-more-opportunities-to-upskill-and- Treude, C., & Robillard, M. P. (2016). Augmenting
drive-impact API documentation with insights from stack
overflow. Proceedings of the 38th
Grassini, S. & Ree, A. (2023). Hope or Doom AI-
International Conference on Software
ttitude? Examining the Impact of Gender,
Engineering, 392-403.
Age, and Cultural Differences on the
https://doi.org/10.1145/2884781.2884800
Envisioned Future Impact of Artificial
Intelligence on Humankind. Proceedings of Wang, R., Cheng, R., Ford, D., & Zimmerman, T.
the European Conference on Cognitive (2024). Investigating and Designing for
Ergonomics 2023, 1-7. Trust in AI-powered Code Generation Tools.
https://doi.org/10.1145/3605655.3605669 Proceedings of the 2024 ACM Conference on
Fairness, Accountability, and Transparency,
Hasselbring, W. & Reussner, R. (2006). Toward
1475-1493.
trustworthy software systems. Computer,
https://doi.org/10.1145/3630106.3658984
39(4), 91–92.
https://doi.org/10.1109/MC.2006.142 Zhang, B., Liang, P., Zhou, X., Ahmad, A., &
Waseem, M. (2023). Demystifying Practices,
Jaworski, M., & Piotrkowski, D. (2023). Study of
Challenges and Expected Features of Using
software developers' experience using the
GitHub Copilot. International Journal of
GitHub Copilot Tool in the software
Software Engineering and Knowledge
development process. ArXiv,
Engineering, 33(1) 1653-1672.
abs/2301.04991.
https://doi.org/10.1142/S021819402341004
https://doi.org/10.48550/arXiv.2301.04991
8
Kedar, S. (2024). Tabnine vs. GitHub Copilot.
Tabnine. Retrieved on July 26, 2024, from
https://www.tabnine.com/blog/tabnine-
versus-GitHub-copilot/
Appendix 1