This article has been accepted for publication in IEEE Engineering Management Review.

This is the author's version which has not been fully edited and
content may change prior to final publication. Citation information: DOI 10.1109/EMR.2024.3487007

A Comprehensive Machine Learning Framework for

Evaluating Agility of a Software Development
Organization
Suman De, Product Manager and Scrum Trainer, SAP Labs India, and Research Scholar, CCDS, IIT Kharagpur

Abstract—Software Project Management has undergone focus on individuals and interactions in creating a functional
significant development over the years due to the progress made software version, prioritizing collaboration rather than
in various techniques. Scrum is widely regarded as one of the most extensive documentation, and incorporating customer input. To
often employed methodologies, encompassing a range of stages mitigate the risks to a system, it is advisable to integrate the
from refinement to retrospective. It empowers individual teams to
techniques with scrum events to enhance the development of
take ownership of their work and deliver on their commitments
within a specified timeframe. This presents numerous issues for
the systems and software products discussed in this article.
upper-level management regarding product delivery and Although processes have evolved, the emphasis on preparing to
engineering. This study examines a novel method of estimating fulfill a feature request has traditionally relied on the Person
work based on complexity instead of estimating effort based on Days (PDs) metric. [3] Velocity is a useful tool in agile
hours. It presents a new approach to combining this framework development for resource allocation and user story
and using it to measure and predict organizational efficiency using customization, shifting focus from time-based estimates to
K-Nearest Neighbor and Decision Tree algorithms. This article backlog prioritization. It helps teams assess planning precision,
will examine scenarios that elucidate how velocity-driven planning performance, and goals. Velocity measures individual team
can assist a software organization in effectively delivering business
performance over time, demonstrating adaptability and
value to its customers. It will utilize a synthetic dataset to
demonstrate how current datasets can assist a Scrum Master or
response to historical occurrences. It's a crucial tool for
Project Lead in forecasting and organizing future sprints and understanding development and assessing progress over time,
dividing backlogs into manageable tasks using tools like Jira. This rather than comparing teams.
results in higher accuracy, as seen for Organization 2 with 87% Agile teams may use their flexibility to make informed
and 95% for Organization 3, respectively via experiments, and decisions, improve backlog planning and optimizing resource
showcases the maturity growth over time. utilization. In unclear scenarios, machine learning techniques
like Decision Trees or k-Nearest Neighbors (k-NN) can
Index Terms— Project planning, Scrum, Sprint velocity, Machine enhance the precision of forecasting of success or failure for
Learning, Productivity in Software development, Forecasting
expected tasks in a sprint. By using historical data inside these
models, we assess the precision of characteristics relevant to
I. INTRODUCTION
sprint outcomes, so aiding Scrum Masters in making more

P
LANNING a software development project in any informed decisions for future iterations. These models use the
organization introduces the prospects of the following proximity of data points, with k-NN using Euclidean distance
sequence of steps adopting a model for software to classify outcomes based on nearest neighbors, and Decision
development like Scrum, Kanban, ScrumBan, and so on. The Trees segmenting data to evaluate feature importance. This
synergy of manufacturing methods has given rise to Agile and research explores a story-point based approach for software
has seen a massive adoption of processes like Scrum, Kanban, development that prioritizes complexity of a task rather than
ScrumBan, eXtreme Programming, etc., and is used widely by time estimations in evaluating the velocity of a team. This
most software development organizations. [1] It follows well- provides more accurate prediction of team capacity
defined steps and creates a channel for funneling features into requirements and sprint completion probabilities, while
stories to simple backlog items. The most significant source of improving decision-making in continuous development setup.
evolution is the time taken to deliver a feature and the Therefore, the algorithms used in this context uses machine
importance of stakeholder feedback. It directly contributes to learning techniques, such as k-NN, for outcome prediction,
the tension triangle of quality that is impacted by cost, time, and emphasizing team maturity, adaptability, and data-led decision-
scope. In a typical scenario, a product is expected to be feature- making for backlog optimization.
heavy, and at the same time, the time taken to complete the Research primarily focused on effort-based forecasting
scope with the limited cost is also considered. [2] methods that were incapable of dynamically adapting and
Scrum has emerged as the predominant strategy for large-scale effectively managing complexity before implementing the
development organizations as agile methodologies have narrative point-based approach using KNN and decision trees.
progressed. Scrum events facilitate the development team's The disparity was caused by the absence of a robust, data-driven

Authorized licensed use limited to: b-on: Universidade de Coimbra. Downloaded on December 04,2024 at 15:32:13 UTC from IEEE Xplore. Restrictions apply.
© 2024 IEEE. Personal use is permitted, but republication/redistribution requires IEEE permission.See https://www.ieee.org/publications/rights/index.html for more information.
 This article has been accepted for publication in IEEE Engineering Management Review. This is the author's version which has not been fully edited and
content may change prior to final publication. Citation information: DOI 10.1109/EMR.2024.3487007

framework that could utilize historical data to anticipate the magnitude of potential losses, initiating risk assessment and
success or failure of future projects, thereby improving the mitigation is imperative as early as possible. Multiple risk
precision and uniformity of predictions. The hypothesis here indicators can be found in both academic literature and real-
states that KNN and decision trees can be used in a story point- world applications. Due to the vast number of reported hazards,
based estimate strategy to significantly improve the accuracy sifting through the lists for dangers and narrowing down
and dependability of sprint planning and project success relevant issues becomes challenging. This study introduces a
forecasts in Scrum projects compared to traditional effort-based novel ML technique that generates risk alerts by analyzing
estimation approaches. This paper will highlight related works software project characteristics and other relevant criteria. The
followed by a short explanation of story point-based study also examines the effectiveness of risk labeling following
development and then the proposed methodology of combining categorization. [7]
this framework with ML algorithms, and Results achieved
through the experiments. The paper also covers an industry III. STORY POINT-BASED DEVELOPMENT
perspective section and concludes with remarks on the The development process does not follow a "develop first,
proposed framework. validate later" approach but instead emphasizes continuous
feedback from the client through minor iterations. Each
II. LITERATURE REVIEW incremental iteration prioritizes value creation for the end-user
This section examines pertinent literature on the application of by offering features that make total sense to a consumer. These
Machine Learning in the context of Software Project concise and tangible elements represent critical client needs and
Management and Agile methodologies. Machine Learning are commonly expressed as narratives. As an illustration,
techniques can be incorporated at different phases of Software suppose Google began developing its search engine for the
Development to enhance project activities. Forecast enhances initial time. An Epic could involve the development of a Search
project management by leveraging Machine Learning (ML) to Engine. [8] Additionally, it can have many story points, such as
streamline and optimize processes. To achieve this objective, "As a user, I should have the capability to input unrestricted text
they have implemented multiple predictive models to automate and receive pertinent data based on the text." Setting aside the
and improve a segment of the decision-making process. Their algorithmic complexity, the objective is to develop a search
current models can automatically categorize tasks, assign control that internally queries a database containing vast data
resources and roles to them, and predict their duration. The and retrieves relevant things based on matching patterns. To
predictions are made in real-time via the app and sent to more enhance the intricacy, an additional narrative to this grand tale
than two hundred diverse companies and thousands of daily would be, "As a user, I should possess the capability to receive
visitors. Tree-based models and logistic regressions are often pertinent recommendations as I commence inputting the text."
effective when used with data tables. When dealing with While both tales may hold significant value, it is crucial to
Unstructured Data, such as text, it might be challenging to prioritize the first narrative point to release the initial version of
extract all the signals. Over the years, Natural Language the product. This story point does not address intelligent search
Processing (NLP) has developed various intelligent methods to algorithms but focuses on providing meaningful matches only
extract valuable information from unstructured text for various after the entire text has been searched. This provides a
purposes. [4] To effectively respond to changes in the software foundation for a software development organization by
industry, it is necessary to implement enhancement efforts at providing clarity on which story to select and prioritize based
every stage of the software development process. This study not only on difficulty but also on the needs of the end-user.
utilized machine learning (ML) to enhance the teams' Developing stories to interpret customer needs facilitates
contributions in the retrospective meeting, a component of the comprehension for development teams, enabling them to offer
improvement-focused agile product development process incremental updates to the customer and receive ongoing
known as LAPIS (Logo Agile Process Improvement System). feedback. Each story is assigned a value that is not dependent
LAPIS, developed by LOGO, draws heavily from lean on any individual team. A scrum team has the authority to
manufacturing theory. [5] determine the complexity of a particular story based on factors
Estimating software effort is crucial in the software such as resource availability, skill level, and, most importantly,
development life cycle, which project managers and software the impact it has on the customer or the value it provides for the
engineers carry out. During the early stages, the only available business. The stories are categorized using T-shirt sizes, with
options are software system functionalities. The cost estimate is each size representing a specific value. This value is typically
a specific objective linked to the planning phase of software measured in hours or Person Days, depending on the skill level
project management. Methods such as Artificial Neural of the scrum team. During the refinement phase, each team
Network (ANN), Support Vector Machines (SVM), K-star, and collectively determines the tasks, stories, or Backlog Items
Linear Regression are evaluated on a publicly available dataset (BLIs) they have and selects sprintable topics for each sprint.
using real-world software. These results showed that a machine The team's benchmark for success is determined by the
learning technique can be trusted to predict the future effort of cumulative value of stories they can accomplish and that are
a software system. [6] Software risk management is a crucial approved by the Product Owner. Throughout each sprint, the
component of software development management. Due to the team gains a comprehensive awareness of the number of stories

Authorized licensed use limited to: b-on: Universidade de Coimbra. Downloaded on December 04,2024 at 15:32:13 UTC from IEEE Xplore. Restrictions apply.
© 2024 IEEE. Personal use is permitted, but republication/redistribution requires IEEE permission.See https://www.ieee.org/publications/rights/index.html for more information.
 This article has been accepted for publication in IEEE Engineering Management Review. This is the author's version which has not been fully edited and
content may change prior to final publication. Citation information: DOI 10.1109/EMR.2024.3487007

or business value it can produce. This is determined by • Each internal node tests an attribute.
summing up the value points delivered by the team and dividing • Each branch corresponds to an attribute value.
them by the number of sprints in which they were computed. • Each leaf node assigns a classification.
For instance, if a team has effectively completed features or The data files were converted to .csv files for further processing.
stories with a value of twenty-five in five consecutive sprints, The first step was to convert these datasets (training and test)
the team's velocity is five. Hence, the aim is to plan for the into numerical mappings of the attributes. The datasets were
current sprint with the same target velocity to make accurate then appended to have a generalized form of converted datasets.
estimations of tasks. Each data frame without a numerical value was then mapped to
Although this perspective is considered best for estimating, a numerical value, and the older fields (or columns) were
real-life situations are much more complex. Breaking down dropped. The obtained dataset was then converted to a CSV for
large tasks into smaller, manageable pieces and maybe working further processing.
on them across multiple iterations can provide challenges. The The generated CSV file is then processed by taking the dataset
management is potentially interested in the feasibility of and splitting the dataset into the training and test datasets as
completing a specific project within a designated timeframe and merged in the previous execution file through the function
seeks the participation of multiple teams to collaborate on splitTrainingTestData. It furthers the steps of creating the
various aspects. Understanding the velocity of a newly decision tree using the function createDecisionTree while using
constituted team with limited knowledge of necessary skill sets the function of calcEntropy and calcInformationGain to
might provide a challenge. Predicting the velocity of a scrum calculate the Entropy and Information Gain, respectively. The
team is a complex process that requires careful attention. This received output is then saved in a JSON file and also printed as
process is important because it enables better decision-making part of the program using the function savePrintOutput.
for the Product Management team when selecting stories and Subsequently, the test data set, which was prepared and kept
creating a roadmap for the product line. using the split initially, is used to predict the accuracy of the test
data using the testModel and predictAccuracy functions,
IV. PROPOSED METHODOLOGY respectively. For the prediction and decision tree, a training
Forecasting the accuracy of a projected series of tests can be dataset of 1728 instances was taken. The accuracy prediction
valuable, but it necessitates a substantial amount of data to was done on a test data set of fifteen instances and gave an
achieve precision. In this study, we have utilized supervised accuracy of 82.35 percent.
learning approaches in Machine Learning to forecast [9] the B. K-Nearest Neighbor Classification
potential velocity of a team or the outcome of a single sprint.
This technique involves the development of software that aims
This research suggests employing Decision Trees and K-nearest
to forecast the success or failure of a sprint for a given set of
neighbor Algorithms to enhance comprehension of the velocity
test data. The KNN classification rule is used to determine the
of a scrum team. The abundance of data available from previous
majority category label of its k-nearest training samples. To
projects or from the same team working on different projects
avoid disputes, it is common to choose k as an odd number. The
forms the foundation for employing Machine Learning
k = 1 rule is the nearest-neighbor categorization rule. The
algorithms to forecast and make informed judgments regarding
training data used for this particular use case contained thirteen
future initiatives. The datasets often contain numerous columns,
characteristics, with the final column representing a binary class
including Business Priority, Roadmap Priority, Number of
label (0/1). The objective was to forecast the category labels for
Stories, Number of Resources, Size, Estimation, and Resultant
4 test situations and save them in an output file. This algorithm
of existing sprints. In this instance, these characteristics have
uses the Euclidean distance metric as its distance measure. This
been employed to construct a Decision Tree, as elucidated in
function accepts the filename as input and utilizes a reader from
the following explanation. The KNN approach utilizes a distinct
the CSV module to read the file's content. The dataset that is
dataset containing thirteen features, which can also be
received is thereafter subjected to processing to eliminate the
employed for prediction. [11-16] There are various algorithms
headers. The remaining rows are changed from string to float
that can be used and have been referenced before this study, and
for all columns except for the last column, which contains class
this research focuses on KNN and Decision Trees. [17-22] The
labels. This conversion happens via the
algorithm has been investigated using Euclidean Distance as the
'convertStringColumnToFloat' function. The column
distance metric with input parameters No of Backlog Items,
representing the class label, called "target," was subsequently
Task, Team Strength, Planned Points, Completed Points,
transformed from a string data type to integers using the
Result, and class. The output is accuracy and potential
'convertStringColumnToInteger' function. As stated in the
prediction of a new project to be a success or failure based on
assignment, the user is allowed to choose the value of K as a
the existing dataset.
user-defined variable with three specified constraints. If the
A. Decision Trees user enters a value less than five or pushes the 'Enter' key
A Decision Tree is generated based on the Training and Test without inputting any value, the value of K is set to 9 for the
datasets provided to decide on velocity and comparison for the first two situations. If the user inputs any value that does not fall
test data based on features given in the dataset. A decision tree into the two circumstances, the user will receive a prediction
is used to represent learned target functions where: based on the user-specified value of K.

Authorized licensed use limited to: b-on: Universidade de Coimbra. Downloaded on December 04,2024 at 15:32:13 UTC from IEEE Xplore. Restrictions apply.
© 2024 IEEE. Personal use is permitted, but republication/redistribution requires IEEE permission.See https://www.ieee.org/publications/rights/index.html for more information.
 This article has been accepted for publication in IEEE Engineering Management Review. This is the author's version which has not been fully edited and
content may change prior to final publication. Citation information: DOI 10.1109/EMR.2024.3487007

Fig 1. The decision tree for the dataset of Organization 2 is further interpreted in the Results & Discussion section.

After the initial processing, the test data undergoes the same level of agility.
processing as described in the previous scenario of loading a
CSV file. Each row in the testdataset is treated as a separate test Algorithm 1 Predict class with respect to neighbors
instance and is predicted using the 'predictClassification' def predictClassification(train, testInstance, neighborsParam):
function. This function utilizes the 'getNeighbors' function to neighbors = getNeighbors(train, testInstance,
examine neighboring data points. The 'getNeighbors' function neighborsParam)
digs down into each training data point based on the Euclidean outputValues = [row[-1] for row in neighbors]
distances produced for each item using the prediction = max(set(outputValues),
'calculateEuclideanDistance' function. Every neighbor is key=outputValues.count)
assessed in a similar manner, and the prediction output for each return prediction
test instance is saved in the variable "label". The prediction
output is determined by the most often observed class values Explanation: The function predictClassification takes training
among the neighbors, using the max function. The program data (train), a test instance (testInstance), and a parameter for
appends each output to the previous output in a variable called the number of neighbors (neighborsParam). It first calls the
'output'. This variable is then saved in a file and printed as part getNeighbors function to get the nearest neighbors to the test
of the program using the 'savePrintOutput' function. The code instance. It then obtains the class labels of the nearby examples
snippet that involves predicting and then finding the neighbors and predicts the class that is most often represented among
using Euclidean Distance as a measure may be found in them. If the closest neighbors are classified as [0, 1, 1], the
Algorithm 1, Algorithm 2, and Algorithm 3. expected class for the forecast will be 1 or Success, since it is
This article examines two simulated datasets and analyzes the more predominant.
correctness of the results. The datasets are synthetically
generated with reference to real data using the technique Algorithm 2 Calculating Euclidean Distance
mentioned in TableGAN as indicated in the paper Synthetic def calculateEuclideanDistance(row1, row2):
Data Generation for Asset Administration Shells in Industry 4.0 distance = 0.0
Scenarios. [23] It also explores the information conveyed by for i in range(len(row1)-1):
each dataset for the larger organization, which serves as the distance += (row1[i] - row2[i])**2
foundation for strategic recommendations. This facilitates the return sqrt(distance)
establishment of a standard by which businesses can assess their

Authorized licensed use limited to: b-on: Universidade de Coimbra. Downloaded on December 04,2024 at 15:32:13 UTC from IEEE Xplore. Restrictions apply.
© 2024 IEEE. Personal use is permitted, but republication/redistribution requires IEEE permission.See https://www.ieee.org/publications/rights/index.html for more information.
 This article has been accepted for publication in IEEE Engineering Management Review. This is the author's version which has not been fully edited and
content may change prior to final publication. Citation information: DOI 10.1109/EMR.2024.3487007

the organization. Organization 1 should have a goal of improving

Explanation: The calculateEuclideanDistance function accuracy, which would be the measuring parameter for its agility,
mentioned above determines the Euclidean distance between which would, in turn, impact its organizational efficiency.
the two data points (row1 and row2). The distance is determined
by aggregating the squared differences of each characteristic in TABLE I. Result for the first dataset of Organization 1.
the two rows and then extracting the square root of the sum. The Precision Recall F1-score Support
outcome or output obtained from the analysis is then taken into Failure 0.57 0.27 0.36 15
consideration to quantify the distance between two data points Success 0.31 0.62 0.42 8
and vice versa. For example, the Euclidean distance between Accuracy 0.39 23
the locations (1, 2) and (4, 6) is computed as √[(4-1)² + (6-2)²] Macro Average 0.44 0.45 0.39 23
= 5. Weighted Average 0.48 0.39 0.38 23

Algorithm 3 Identify the most significant/related neighbors The next dataset, which is of Organization 2 as shown in Table II,
def getNeighbors(train, testInstance, neighborsParam): presents a better result in terms of accuracy. This dataset was tested
distances = list() with both Random Forest Classification (accuracy: 0.83) and basic
for trainInstance in train: Decision Tree (accuracy: 0.87). This tells us that the organization
dist = calculateEuclideanDistance(testInstance, in question is better equipped to use scrum methodologies and has
trainInstance) adopted the practices throughout its organizational projects.
distances.append((trainInstance, dist))
distances.sort(key=lambda tup: tup[1]) TABLE II. Result for the second dataset of Organization 2.
neighbors = list() Precision Recall F1-score Support
for i in range(neighborsParam): Failure 0.89 0.94 0.91 17
neighbors.append(distances[i][0]) Success 0.80 0.67 0.73 6
return neighbors Accuracy 0.87 23
Macro Average 0.84 0.80 0.82 23
Explanation: The getNeighbors function identifies the closest Weighted Average 0.87 0.87 0.87 23
neighbors of the test instance by calculating the Euclidean
distance between the test instance and every instance in the
The above comparison was performed on dummy datasets, as
training dataset. It uses these distances and arranges them in
getting organizational data is confidential and private. However,
ascending order, and then produces the specified number of
every organization keeps a record of the same parameters
nearest neighbor instances. For instance, if the training set has
considered for this research and forms the basis of comparison
distances [1.5, 0.5, 2.0] from the test instance and the number
between its maturity as an agile organization over a specific period.
of neighbors is specified as 2, the function will provide the two
The dearth of comparison mechanisms makes this novel research
closest neighbors with distances 0.5 and 1.5.
on how organizations rate themselves in terms of agility and adapt
different mechanisms to improve organizational efficiency, which
also has a direct impact on the Objectives & Key Results (OKRs)
V. RESULTS AND DISCUSSIONS of the organization. The synthetic data automatically preprocesses
The data set considered here presents an approach to how the the data and generates value for all scenarios, but for a real dataset,
accuracy value of the model helps decide on proceeding with a it may use measures of central tendency in case of missing values,
potential project. In this scenario, we compare the behavior of two which can be addressed as per data availability.
organizations and how the mechanism presented in the paper helps
differentiate between an agile and a non-agile organization. The TABLE III. Result for the first dataset of Organization 3.
real datasets from which this is generated are multinational Precision Recall F1-score Support
organizations (MNCs) that have a global presence in terms of Failure 0.97 0.94 0.96 35
demography. In the first case, the results show that the projects Success 0.93 0.96 0.94 26
could be of different types, as the accuracy of 0.39 is very low. It Accuracy 0.95 61
offers a deep dive into the consideration that the complexity of Macro Average 0.95 0.95 0.95 61
projects in the consideration are inherently complex and do not Weighted Average 0.95 0.95 0.95 61
capture the data effectively (see Table I). It is often tough to model
the problem and will require further usage over time to improve Furthermore, another synthetic dataset was used to understand the
accuracy. This clearly indicates that the organization should results of a potential Organization 3 using Random Forrest
improve its estimation practices, which are volatile and should classification. The results were more promising, as can be seen in
work towards bringing standardization to scrum processes. Table III. This was further strengthened and captured using a
Velocity calculation differs from team to team, and to achieve a confusion matrix and ROC curve. The AUC value was 0.63, which
proper agile setup, the organization should employ methods to indicated a certain level of predicting power, although it had a
improve the accuracy and take it as a KPI to define the agility of greater number of projects considered than before.

Authorized licensed use limited to: b-on: Universidade de Coimbra. Downloaded on December 04,2024 at 15:32:13 UTC from IEEE Xplore. Restrictions apply.
© 2024 IEEE. Personal use is permitted, but republication/redistribution requires IEEE permission.See https://www.ieee.org/publications/rights/index.html for more information.
 This article has been accepted for publication in IEEE Engineering Management Review. This is the author's version which has not been fully edited and
content may change prior to final publication. Citation information: DOI 10.1109/EMR.2024.3487007

predicting outcomes, effectively handles complexity, and

adapts over time, thereby enhancing sprint planning and
resource allocation. However, carrying out its implementation
requires a substantial quantity of historical data and a
considerable degree of proficiency in machine learning.
Conventional approaches are more straightforward to put into
action, can be used in a wide range of situations, and are made
simpler; yet, they are vulnerable to personal opinions, lack
adaptability, and face difficulties in handling intricate tasks.
The suggested method's data-driven approach has the potential
to significantly improve the accuracy of estimates and the
success rates of projects. Nevertheless, the process of setting up
and the interdependence of data pose significant difficulties.
Conventional techniques have a proven track record of being
precise; nevertheless, their reliance on personal judgment might
lead to uneven estimations.
Fig 2. Random Forest Confusion matrix for dataset 3

In conclusion to the analysis as shown in Table IV,

Organization 1 exhibits low performance in terms of precision,
recall, and F1-scores, particularly in failure identification, with
an overall accuracy of 0.39, indicating a need for improvement.
Organization 2 shows better performance, achieving higher
precision (0.89) and recall (0.94) for failure predictions, with an
overall accuracy of 0.87. Organization 3 outperforms both,
delivering near-perfect results across all metrics, with a 0.95
accuracy, making it the most effective in predictions.
Organization 1, however, requires significant performance
optimization. The suggested approach for estimating Scrum
projects, which utilizes algorithms like K-nearest neighbors
(KNN), decision trees, and even Random Forest Classification,
has many advantages over traditional effort-based estimate
approaches. Using past data improves the accuracy of Fig 3. ROC Curve for the Dataset 3

TABLE IV. Comparative analysis of the results for all 3 organizations.

Metric Organization 1 Organization 2 Organization 3
Failure Precision 0.57 0.89 0.97
Failure Recall 0.27 0.94 0.94
Failure F1-Score 0.36 0.91 0.96
Failure Support 15 17 35
Success Precision 0.31 0.8 0.93
Success Recall 0.62 0.67 0.96
Success F1-Score 0.42 0.73 0.94
Success Support 8 6 26
Accuracy 0.39 0.87 0.95
Macro Average Precision 0.44 0.84 0.95
Macro Average Recall 0.45 0.8 0.95
Macro Average F1-Score 0.39 0.82 0.95
Weighted Average Precision 0.48 0.87 0.95
Weighted Average Recall 0.39 0.87 0.95
Weighted Average F1-Score 0.38 0.87 0.95
Total Support 23 23 61

Authorized licensed use limited to: b-on: Universidade de Coimbra. Downloaded on December 04,2024 at 15:32:13 UTC from IEEE Xplore. Restrictions apply.
© 2024 IEEE. Personal use is permitted, but republication/redistribution requires IEEE permission.See https://www.ieee.org/publications/rights/index.html for more information.
 This article has been accepted for publication in IEEE Engineering Management Review. This is the author's version which has not been fully edited and
content may change prior to final publication. Citation information: DOI 10.1109/EMR.2024.3487007

over a certain time frame. This innovative research examines

VI. BUSINESS RELEVANCE AND ADVANTAGES how businesses assess their own agility and employ various
Scrum is presently one of the most widely utilized project strategies to enhance organizational efficiency, which directly
affects the Objectives & Key Results (OKRs) of the company.
management models. It is utilized by numerous firms, including
However, the lack of comparison mechanisms hinders this
SAP, Google, Netflix, Sage, Siemens, and Deloitte. Scrum and
research.
Agile are equally effective in smaller organizations. These
strategies are flexible and innovative approaches to project
ACKNOWLEDGMENT
management. Scrum emphasizes the practice of adjusting during
the project rather than engaging in excessive planning prior to its This research work is successful thanks to the support of our
commencement. This makes it ideal for dynamic organizations parent bodies of SAP Labs India, IIT Kharagpur, and SIBM
with constantly evolving tasks. Scrum is founded on shorter bursts Nagpur. The authors would also like to thank the organizers of
of effort known as sprints, which produce value at an early and Symbicon 2022, where this topic was first presented and later
frequent rate. This mitigates risk and enhances stakeholder adapted for a full-fledged paper. The author would like to thank
relationships, especially in large multinational organizations. the co-author, Mrs. Ivy Baroi, of the paper presented at the
The present study improves the understanding of adoption by conference for the efforts in a base paper presentation at the
offering insights into the calculation of velocity and the prediction conference and for all the support and effort in performing the
of sprint success or failure based on specific feature values. It literature survey for this research. We would also like to thank
significantly enhances project management scenarios by Dr. Pabitra Mitra of the Centre for Computational & Data
improving the estimation of projects that adhere to scrum Sciences for his insights on Machine Learning that were used
approaches. Changing the goal class to assess how estimations may in the proposed methodology.
be enhanced with a team of specific velocity also allows for more
effective refinement sessions that are based on facts and enhance REFERENCES
operational efficiency. The value of velocity prediction is in its [1] S. De, "Minimal Satisfiable Product: A Novel Concept in Agile Project
Management for better adherence to Customer Requirements," 2021
ability to forecast future roadmap or long-term distribution of Epics International Conference on Smart Generation Computing,
to teams, using complexity as a basis rather than relying on hourly Communication and Networking (SMART GENCON), Pune, India,
data. While most firms implement scrum, it is frequently noted that 2021, pp. 1-6, doi: 10.1109/SMARTGENCON51891.2021.9645781.
[2] S. De, "A Novel Perspective to Threat Modelling using Design Thinking
it is not executed well. Machine learning algorithms are effective and Agile Principles," 2020 Sixth International Conference on Parallel,
in mitigating human bias and optimizing the utilization of scrum in Distributed and Grid Computing (PDGC), Waknaghat, India, 2020, pp.
software development projects, and they help relate to the maturity 31-35, doi: 10.1109/PDGC50313.2020.9315844.
of an organization. [3] SETH, A. (2020, May 12). Agile estimation using T-shirt sizing. PM Tips,
1. Available: https://pmtips.xyz/2020/05/12/t-shirt-estimation/
[4] Triana, M., & Henckel, T. (2021, April 27). ML in project management:
VI. CONCLUSION Classification & regression problems with text. Forecast, 1.
[5] Tekbulut, T., Canbaz, N., & Kaya, T. Ö. (2020). Machine learning
Scrum is a widely adopted software development application in LAPIS agile software development process. In 2020
methodology for enterprises of all sizes. It involves a process of Turkish National Software Engineering Symposium (UYMS) (pp. 1-6).
modest, iterative creation of features accompanied by a robust IEEE.
[6] M. Hammad and A. Alqaddoumi, "Features-Level Software Effort
feedback mechanism. This paper has discussed two
Estimation Using Machine Learning Algorithms," 2018 International
components of scrum: complexity-based story point estimation Conference on Innovation and Intelligence for Informatics, Computing,
and the utilization of machine learning techniques for and Technologies (3ICT), Sakhier, Bahrain, 2018, pp. 1-3, doi:
predicting the velocity of a Scrum team. The text discussed 10.1109/3ICT.2018.8855752.
[7] Joseph, H. R. (2015). Poster: Software development risk management:
recent research and projects related to the application of
Using machine learning for generating risk prompts. In 2015 IEEE/ACM
Machine Learning at different stages of software development. 37th IEEE International Conference on Software Engineering (pp. 833-
It then delved into the intricacies of scrum and how stories may 834). IEEE.
be scheduled based on both time and complexity, considering [8] Sońta-Drączkowska, E., & Mrożewski, M. (2020). Exploring the role of
project management in product development of new technology-based
the experience of the scrum team. This study also investigated
firms. Project Management Journal, 51(3), 294–311.
the application of Machine Learning in forecasting the outcome https://doi.org/10.1177/8756972819851939
of a sprint based on the available characteristics. This study also [9] Natarajan, A. (2022). Reference Class Forecasting and Machine Learning
discussed the utilization of Decision Trees and K-nearest for Improved Offshore Oil and Gas Megaproject Planning: Methods and
Application. Project Management Journal, 53(5), 456–484.
neighbor categorization, as well as its application in the context
https://doi.org/10.1177/87569728211045889
of commercial relevance. This article discusses the application [10] OpenAI. (2024). ChatGPT [Large language model]. https://chatgpt.com
of Machine Learning in many aspects of the scrum framework, [11] Wijayasekera, S. C., Hussain, S. A., Paudel, A., Paudel, B., Steen, J.,
revolutionizing the software development process by making it Sadiq, R., & Hewage, K. (2022). Data Analytics and Artificial
Intelligence in the Complex Environment of Megaprojects: Implications
data-driven rather than relying on traditional methods.
for Practitioners and Project Organizing Theory. Project Management
The comparison in this research paper was conducted using Journal, 53(5), 485–500. https://doi.org/10.1177/87569728221114002
simulated datasets due to the sensitive and secret nature of [12] Boonstra, A., & Reezigt, C. (2023). A Complexity Framework for Project
obtaining organizational data. However, each company Management Strategies. Project Management Journal, 0(0).
https://doi.org/10.1177/87569728221142229
maintains a record of the same metrics used in this research,
[13] Yang, Q., Bi, Y., Wang, Q., & Yao, T. (2021). Batch-based agile program
which serves as the foundation for comparing its level of agility management approach for coordinating IT multi-project concurrent

Authorized licensed use limited to: b-on: Universidade de Coimbra. Downloaded on December 04,2024 at 15:32:13 UTC from IEEE Xplore. Restrictions apply.
© 2024 IEEE. Personal use is permitted, but republication/redistribution requires IEEE permission.See https://www.ieee.org/publications/rights/index.html for more information.
 This article has been accepted for publication in IEEE Engineering Management Review. This is the author's version which has not been fully edited and
content may change prior to final publication. Citation information: DOI 10.1109/EMR.2024.3487007

development. Concurrent Engineering, 29(4), 343-355. Communications Technology and Society (ICTAS), Durban, South
https://doi.org/10.1177/1063293X211015236 Africa, 2019, pp. 1-6, doi: 10.1109/ICTAS.2019.8703639.
[14] S. De and V. Vijayakumaran, "An Efficient Algorithm in Project [23] S. De and P. Mitra, "A Novel Technique of Synthetic Data Generation for
Management for Resource Scheduling and Conflict Management using Asset Administration Shells in Industry 4.0 Scenarios," in IEEE
Graph Coloring Technique," 2020 IEEE International Conference for Transactions on Artificial Intelligence, doi: 10.1109/TAI.2024.3409516.
Innovation in Technology (INOCON), Bangluru, India, 2020, pp. 1-6,
doi: 10.1109/INOCON50539.2020.9298193.
[15] N. P. Augustine, B. Balaji and S. De, "Novel Practices in Software
Documentation for Continuous Delivery and Integration Using Feature
Toggles," 2023 3rd International Conference on Smart Generation AUTHOR PROFILE
Computing, Communication and Networking (SMART GENCON),
Bangalore, India, 2023, pp. 1-6, doi:
10.1109/SMARTGENCON60755.2023.10441952. Suman De is a Product Manager and
[16] A. El Yamami, S. Ahriz, K. Mansouri, M. Qbadou and E. H. Illoussamen, Scrum Trainer at SAP working for the
"Rethinking IT project financial risk prediction using reference class
forecasting technique," 2018 4th International Conference on
Global Adoption & xperience CoE unit.
Optimization and Applications (ICOA), Mohammedia, Morocco, 2018, He is also pursuing a Ph.D. at IIT
pp. 1-6, doi: 10.1109/ICOA.2018.8370586. Kharagpur in the Centre for
[17] Lourens, M., Raman, R., Vanitha, P., Singh, R., Manoharan, G., & Tiwari, Computational and Data Sciences, as a
M. (2022). Agile Technology and Artificial Intelligent Systems in
Business Development. In Proceedings of 5th International Conference
working professional. In addition, he
on Contemporary Computing and Informatics, IC3I 2022 (pp. 1602- holds the position of Visiting Professor
1607). DOI: 10.1109/IC3I56241.2022.10073410 at the Symbiosis Institute of Business Management, where his
[18] Costa, A., Ramos, F., Perkusich, M., Neto, A. D. S., Silva, L., Cunha, F., areas of expertise include Business Analytics, Cloud
Rique, T., Almeida, H., & Perkusich, A. (2022). A Genetic Algorithm-
Based Approach to Support Forming Multiple Scrum Project Teams.
Computing, and Marketing. He has obtained his Master of
IEEE Access, 10, 68981-68994. Technology degree in Software Engineering from BITS Pilani
https://doi.org/10.1109/ACCESS.2022.3186347 and has authored over 40 research articles, which have been
[19] Mishra, S. K., & Singh, V. K. (2019). Location coder-decoder android published in prestigious journals such as IEEE, Elsevier, and
application using init agent specific ontology for agile software
development. International Journal of Innovative Technology and
Springer. Additionally, he has contributed to four book chapters
Exploring Engineering, 9(1), 3056-3059. on cutting-edge industrial subjects. Suman is an accomplished
https://doi.org/10.35940/ijitee.A9134.119119. Scrum Trainer and mentor for Research initiatives. He has
[20] R. Balabhadrapathruni and S. De, "A Study on Analysing the impact of gained recognition as a Thought-Leader for his valuable
Feature Selection on Predictive Machine Learning Algorithms," 2020
Sixth International Conference on Parallel, Distributed and Grid
contributions to external conferences. He has served as a
Computing (PDGC), Waknaghat, India, 2020, pp. 10-15, doi: Keynote speaker at six research conferences and is affiliated
10.1109/PDGC50313.2020.9315801. with other IEEE and Springer Conferences as a member of the
[21] S. Nootyaskool and P. Ounsrimuang, "Optimization algorithm using Technical Program Committees. Additionally, he has chaired
scrum process," 2016 International Conference on Machine Learning and
Cybernetics (ICMLC), Jeju, Korea (South), 2016, pp. 245-250, doi:
sessions at International Conferences and was recently
10.1109/ICMLC.2016.7860908. nominated as the Youth Engineering Icon of India by the
[22] L. Butgereit, "Using Machine Learning to Prioritize Automated Testing Institution of Engineering and Technology (IET).
in an Agile Environment," 2019 Conference on Information

Authorized licensed use limited to: b-on: Universidade de Coimbra. Downloaded on December 04,2024 at 15:32:13 UTC from IEEE Xplore. Restrictions apply.
© 2024 IEEE. Personal use is permitted, but republication/redistribution requires IEEE permission.See https://www.ieee.org/publications/rights/index.html for more information.