Volume 10, Issue 4, April – 2025 International Journal of Innovative Science and Research Technology

ISSN No:-2456-2165 https://doi.org/10.38124/ijisrt/25apr1850

Converging High-Performance Computing,

Artificial Intelligence, and Intelligent
Workflows for Next-Generation Innovation
Son Dang1; Youngje Son2; Brandon Kim3
1
SQK Inc Hanoi, Vietnam
2
SQK Inc Seoul, Korea
3
SQK Inc Seoul, Korea

Publication Date: 2025/05/13

Abstract: The increasing intricacy of scientific simulations and industrial activity processes along with their accompanying
datasets demand ecosystems outside the capabilities of traditional High-Performance Computing (HPC) systems [1]–[6],
[11]. The requirements of contemporary research and industries based on data are multidisciplinary, as computations
using traditional HPC await a solution. Recently, attention has been drawn towards harnessing the computational power
of AI facilities to relieve HPC systems as the fusion of intelligence reveals new adaptive workflows that integrate HPC and
AI capabilities. This evolution in thinking gives rise not only to an emergent paradigm for an AI-powered HPC but also to
the transforma- tional approach of the HPC-AI Workflow Platform that incor- porates synergistic and intelligent
orchestration workflows. In this paper, we introduce the HPC-AI Workflow Platform, which catalyzes collaborative
innovation with its innovative Workflow- as-a-Service (WaaS) model, facilitating effortless cross-domain sharing and
reuse of workflows, boosting operational efficiency. They are further enabled with AI for real-time decision-making and
optimization, smart resource allocation, big data analytics, and seamless data flow for the timely and energy-efficient
execution of complex simulations, enhancing HPC productivity. This not only demonstrates the efficacy of the HPC-AI
Workflow Platform in resourceful workflow optimization and management but also strengthens its position as a future-
ready paradigm to advance HPC application relevance in science and industry.

Keywords: High Performance Computing, Intelligent Workflows, Machine Learning, Scientific Data Analysis, Big Data.

How to Cite: Son Dang; Youngje Son; Brandon Kim. (2025). Converging High-Performance Computing, Artificial Intelligence,
and Intelligent Workflows for Next-Generation Innovation. International Journal of Innovative Science and
Research Technology, 10 (4), 3448-3455. https://doi.org/10.38124/ijisrt/25apr1850

I. INTRODUCTION between AI systems and HPC infrastructure makes it more

challenging to handle data-heavy AI models as disconnected
High Performance Computing (HPC) serves as the HPC and AI systems fail to meet the needs of complicated
foun- dation for scientific and industrial progress by tasks.
enabling data processing and modeling across climate
science, healthcare, aerospace, and manufacturing to tackle In the study, we introduce the development of the
complex problems and big data sets. Besides, Artificial HPC-AI Workflow Platform, which represents an innovative
Intelligence (AI) serves as a powerful tool that operates method to merge HPC technology with AI and intelligent
alongside High Performance Computing (HPC) to extract work- flows for HPC applications. The platform combines
knowledge from data while optimizing processes and HPC calculations with AI forecasts through dynamic
delivering deeper insights than con- ventional algorithms workflows to deliver Workflow-as-a-Service (WaaS) that
can provide. However, integrating HPC with AI and ensures optimal AI resource management while providing
workflows is difficult: While traditional HPC setups excel adaptive respon- siveness and easy access for reuse. The
in”brute force” throughput performance they pos-sess platform uses big data analytics and open-source
inflexible static workflows that cannot manage real-time technologies to create better connections between HPC and
data modifications leading to operational inefficiencies AI through flexible and efficient on-demand resources. This
during disaster prediction and industrial improvement tasks. work analyzes how the proposed HPC-AI Workflow
Costs and energy consumption increase with resource Platform is more effective than MPI [7] or Pegasus [8] and
mismanagement through over-provisioning and under- other systems to address industry problems with innovative
utilization, while big data demands unified analytical solutions for scientific research and industrial activities.
solutions, which typical HPC platforms lack. The separation

IJISRT25APR1850 www.ijisrt.com 3448

Volume 10, Issue 4, April – 2025 International Journal of Innovative Science and Research Technology
ISSN No:-2456-2165 https://doi.org/10.38124/ijisrt/25apr1850
II. BACKGROUND AND RELATED WORK B. Related Work
In previous studies [11] developed, introduced HPC
A. Background and workflow management through automation,
High-Performance Computing and Intelligent Work- standardization and distributed execution for computational
flows: High-performance computing (HPC) serves as an im- science but these approaches do not support research that
portant and fundamental tool for solving complex scientific combines data- driven methods with AI and Intelligent
and engineering problems by facilitating the resolution of Workflows integra- tion. Notable systems in this area are: 1)
com- putationally intensive tasks in many fields [11]. Pegasus [8] en- sures reproducible workflow mapping to
Researchers rely on HPC systems to perform large-scale distributed resources with fault tolerance but is not adaptive
simulations in cli- mate modeling, seismology, due to its static mapping method. 2) Kepler provides both a
computational fluid dynamics, and drug discovery, among user-friendly graphical interface and an agent-based model
others. This is because conventional computing resources but struggles with large-scale HPC tasks. 3) The Swift/T
cannot achieve these scales with suffi- ciently good [18] platform and the Parsl [28] system provide scalable task
accuracy [13]. Traditional HPC utilization models have delivery but require domain-specific programming
major limitations because their rigid structures require expertise. 4) Fire Works [19] and Nextflow [29] support
manual task submission and static workflow configurations dynamic workflow execution but face scalability limitations
limit flexibility and prevent adaptation to changing compu- when integrating with HPC managers [20] and AI systems.
tational needs [30]. In order to effectively handle The integration of AI and HPC technologies has progressed
challenging computational activities, the intelligent through new developments in [14], AI4HPC [15], SmartSim
workflow tools such as Pegasus [8], Kepler [9], and Taverna [16] and DeepHyper [17] which are established for AI-based
[10] have grown in- dispensable in both academic and parameter optimization along with adaptive simulation and
commercial settings. These systems mostly depend on reinforcement learning workflows in materials science.
predetermined processes, which limit their capacity to These tools only work for specific applications and require
change during the running time. As early workflow system expertise without providing a general model for widespread
evaluations have shown, the static mapping approach of adoption. In this work, we introduce the HPC-AI Workflow
Pegasus [8] along with Kepler’s performance problems in Platform, which combines proven features from former HPC
high-end computing environments [9] produce limited and workflow management systems [11] and brings crucial
adaptability for real-time data management. Barker and innovative elements to solve their fundamental problems,
Hemert [32] found that conventional methods could not thus offering better support for current computational
com- bine with artificial intelligence technologies and science requirements.
TOP500 [33] studies revealed continuous resource
inefficiencies in normal HPC configurations. Their
combined shortcomings limit their ability to address modern
data-driven research objectives, so more flexible and unified
solutions are needed.

 Workflow Challenges in Modern HPC Environments:

The ever-changing landscape of scientific research and
in- dustrial demands subjects established High-Performance
Com- puting (HPC) techniques and workflow management
systems to multiple persistent challenges which limit their
capacity to address current computational demands.
Traditional work- flow models operate on fixed execution
patterns that do not accommodate intermediate results or
changing situations as noted in [12], thus making real-time
optimization and adaptive processes impossible for dynamic
applications such as dis- aster forecasts and industrial
simulations. Besides, traditional resource allocation methods
in these systems rely on manual processes or coarse
heuristics that produce inefficient resource usage which
leads to underused computational power and energy
inefficiencies thereby increasing costs and delaying re- sults
[13]. Traditional High-Performance Computing environ-
ments fail to fully integrate AI which limits their ability to
use AI tools for workflow enhancement through predictive
models and automated decision-making to achieve better
performance.

IJISRT25APR1850 www.ijisrt.com 3449

Volume 10, Issue 4, April – 2025 International Journal of Innovative Science and Research Technology
ISSN No:-2456-2165 https://doi.org/10.38124/ijisrt/25apr1850

Fig 1 Overall architecture of HPC-AI Workflow Platform

III. SYSTEM ARCHITECTURE A. Core Components and Differentiators

Six interconnected layers make up the architecture of
The HPC-AI Workflow Platform merges High- the HPC-AI Workflow Platform which plays essential roles
Performance Computing (HPC) with artificial intelligence in providing its innovative features and differentiating it
(AI) and data analytics to establish its uniqueness against from traditional HPC systems [11]. Through the
traditional HPC frameworks [11] through the use of a collaborative func- tioning of its components the platform
modular, intelligent, and adaptable architecture illustrated in achieves flexibility and efficiency while centering around
“Fig.1”. HPC Plat- form merges a Workflow-as-a-Service the user to overcome traditional workflow management and
approach with dynamic workflow execution capabilities resource allocation chal- lenges [12] and supports the
alongside advanced resource management supported by AI- integration of HPC with AI and dynamic workflows across
powered analytics and a base of open-source software for a numerous scientific and industrial scenarios.
comprehensive solution.

Fig 2 Intelligent Workflows of HPC-AI Workflow Platform

IJISRT25APR1850 www.ijisrt.com 3450

Volume 10, Issue 4, April – 2025 International Journal of Innovative Science and Research Technology
ISSN No:-2456-2165 https://doi.org/10.38124/ijisrt/25apr1850
 Intelligent Workflow Orchestration:  Resource Management System:
The HPC-AI Work- flow Platform provides Intelligent The Resource Manage- ment System stands as a vital
Workflow Orchestration that combines Workflow-as-a- part of the HPC-AI Workflow Platform and revolutionizes
Service with real-time adaptability to create a smart resource management optimiza- tion using innovative
framework for scientific and industrial comput- ing and intelligent mechanisms. Through the use of AI-driven
revolutionize workflow management. This system uses predictive models this system allocates resources between
workflow intelligence to minimize manual configuration CPUs, GPUs, TPUs and memory within diverse clusters
work that traditional HPC systems like SLURM [20] and with high precision and exceeds traditional HPC framework
MPI [7] require extensive manual setup for. Users can capabilities [11]. This dynamic resource allocation system
develop modular workflow templates through a web- demonstrates superiority over previous fixed schedul- ing
accessible central hub where metadata including input approaches such as SLURM [20] and IBM Spectrum LSF
requirements and execution history along with version rule-based systems [22] by using real-time workload
tracking is available as shown in “Fig.2”. The framework demands and historical data to achieve energy savings of up
supports various applications such as genomic sequencing to 30% over traditional systems [20]. The system handles
and industrial simulation processes while reducing setup multi-stage simulations by allocating GPU resources to
time by 60% compared to traditional systems[20] and render jobs and reducing CPU utilization in pre-processing
establishes a user-focused ecosystem that enables stages to main- tain peak performance and fault tolerance
collaborative workflow improvement and reuse across through dynamic task reallocation during node failures. The
different scientific fields [12]. By combining rule-based HPC-AI workflow platform demonstrates intelligent
scripting with an event-driven architecture this orchestration adaptation capabilities that enhance system performance and
framework en- ables workflows to alter execution paths reliability by addressing traditional HPC limitations and
according to real-time data and event changes which marks asserting its position as a breakthrough resource
substantial advancements over static systems like Pegasus management solution.
[8] and Apache Airflow [21] that lack runtime adaptability
[31].

Fig 3 User Interface of HPC-AI Workflow Platform

 Analytics and AI Layer: as MPI [7]. AI models used in fluid dynamics simulations
The Analytics and AI Layer combines advanced identify regions of turbulence and then dynamically allocate
intelligence with the HPC-AI Workflow Platform by computational resources to these key areas, allowing them to
integrating AI into both input data processing and output minimize costs more accurately than traditional approaches.
refinement using Apache Kafka [23] for streaming and Through the integration of AI for superior data processing
TensorFlow [24] for machine learning throughout the HPC and execution capabilities, the HPC-AI Workflow Platform
workflow runtime. The implemented strategic integra- tion becomes a unique tool for solving complex, cross-industry,
optimizes execution paths to achieve capabilities that data-centric problems.
surpass the limitations of compute-only legacy systems such

IJISRT25APR1850 www.ijisrt.com 3451

Volume 10, Issue 4, April – 2025 International Journal of Innovative Science and Research Technology
ISSN No:-2456-2165 https://doi.org/10.38124/ijisrt/25apr1850
 Execution Runtime: while allowing real-time resource monitoring and interactive
The Execution Runtime of the HPC-AI Workflow output visual- ization in contrast to traditional HPC systems
Platform establishes a strong base for both scalability and with command- line interfaces [11]. Researchers and
portability through effective utilization of containerization engineers benefit from this user-friendly design because it
technologies such as Docker [25] and Kubernetes [?] which improves their ability to concentrate on scientific progress
enable support across diverse hardware settings from basic instead of dealing with technical hurdles while
research clusters up to advanced exascale supercomputers. demonstrating the platform’s dedica- tion to better usability
This runtime extends beyond simple isolation by managing and productivity in high-performance computing settings
advanced parallel execution along with dynamic load which “Fig.3” illustrates.
balancing which enables real-time computational resource
scaling in response to variable workload intensities while B. Architectural Advantages
con- ventional systems such as Torque [26] cannot achieve When these components work together in the HPC-AI
this due to their fixed architectural setup [30]. The Workflow Platform, their interactions produce a system ex-
Execution Runtime built upon open-source instruments ceeding the capabilities of individual elements combined.
provides transparent oper- ations and extensibility while The Intelligent Workflow Engine’s WaaS model when
enabling precise environmental customization for paired with the Dynamic Workflow Processor removes
specialized needs and maintaining strong scalability traditional HPC constraints [11] and the Resource
throughout various computational platforms. The innovative Management System along- side the Analytics Layer boosts
design moves past the boundaries of standard HPC performance while providing insights. The HPC-AI
frameworks to deliver a solution that adapts to research Workflow Platform stands out from proprietary systems like
needs while meeting advanced computing requirements. IBM Spectrum LSF [22] because its open-source Execution
Runtime delivers both cost-efficiency and community-
 User Interface: inspired innovation. The architecture delivers strong
The User Interface creates cohesion among various scalability support for terabyte to petabyte workloads along
components by delivering an easy-to-use web- based with fault tolerance via runtime redundancy and ex-
dashboard which streamlines workflow creation and tensibility through modular design to provide a future-ready
monitoring while enabling straightforward result solution for scientific and industrial applications according
visualization. The system reduces entry barriers for users by to “Table.I”.
providing a drag-and-drop interface to build workflows

Table 1 Comparison of Resource Management and Efficiency Between HPC-Ai Workflow Platform and
Traditional HPC Systems
System Resource Dynamic Energy Performance
Allocation Scaling Efficiency Gain
HPC-AI AI-Driven, Yes Up to 71% faster seis-
Work- flow Predictive 30% mic analysis
SLURM reduction Inefficient
Static, Man- ual/Heuristic No No 14-day seismic baseline
MPI Static, Man- ual High overhead 10-day climate modeling
baseline
Static, Pre- defined No Moderate Limited resource
Pegasus IBM optimization 30-day
Spectrum LSF Rule-Based, Static manufacturing baseline
No Inefficient

The new HPC-AI Workflow Platform provides eliminates manual setup and debugging tasks that commonly
transforma- tional benefits that address traditional HPC affect systems such as MPICH [27] and SLURM [20] result-
limitations [11] leading to progress in both scientific ing in more efficient deployment and resource management.
research and indus- trial applications. The platform reaches This efficient method frees users to focus on research and
superior performance standards by bringing together High- engineering activities because it saves time and reduces
Performance Computing capabilities with AI analytics and errors while eliminating system administration tasks. The
sophisticated workflow systems. The study proposes an WaaS model enables accessibility by providing a shared
HPC-AI Workflow Platform to solve complex data- workflow repository that gives researchers the ability to
intensive problems faster and more accu- rately as its many reuse and adapt existing applications for multiple projects.
advantages make it indispensable for modern computing. Reusability de- mocratizes HPC access which allows
The HPC-AI Workflow Platform demonstrates excellence smaller institutions and non-technical teams to participate
by optimizing resource utilization of HPC systems and and promotes collaboration through user contributions and
enhancing both access and application reusability. The utilization of community-driven workflow libraries
Workflow-as-a-Service model implemented by the platform according to “Table.II”.

IJISRT25APR1850 www.ijisrt.com 3452

Volume 10, Issue 4, April – 2025 International Journal of Innovative Science and Research Technology
ISSN No:-2456-2165 https://doi.org/10.38124/ijisrt/25apr1850
Table 2 Comparison of Hpc-Ai Workflow Platform with Traditional HPC Systems in Resource use,
Accessibility, and Reusability
System Resource Accessibility Workflow User
Management Reusability Focus
HPC-AI Simplified High Yes Research /
Work- flow (WaaS) (shared repository) (reusable workflows) Engineer- ing
MPICH Low (expertise No (bespoke setups) System Adminis-
Manual, Complex needed) Low (tech- Limited (static tration System
Manual, De- bugging nical skill req.) configs) Minimal Adminis- tration
SLURM Low (expert- only) (project- specific) System Overhead
Manual, Inef- ficient
Traditional HPC
Systems

The HPC-AI Workflow Platform provides substantial previously inaccessible knowledge because of technological
bene- fits by accelerating solution delivery and improving advancements which helps address urgent world- wide
scientific outcomes in essential research domains. Efficient problems as demonstrated in “Table.III”.
resource management combined with dynamic workflows
and AI op- timization helps this platform generate results Finally, the HPC-AI Workflow Platform boosts
more quickly than traditional systems [20]. The HPC-AI operational efficiency and access while speeding up solution
Workflow Platform completes climate simulations in hours deployment which allows teams to achieve objectives more
instead of days because of its real-time adaptability and quickly and with less resource use than through legacy
predictive scheduling fea- tures. Both disaster response system workflows. The WaaS repository allows engineers
operations and industrial process improvements demand this and researchers to col- laborate effectively and share
quick processing capability. The combination of AI with big resources without obstacles so they can focus on innovation
data analytics enhances scientific model accuracy across rather than operational manage- ment. The HPC-AI
multiple domains including climate change research through Workflow Platform removes standard HPC system
carbon cycle modeling while also improving disaster inefficiencies and access problems so users can boost their
prediction with seismic risk assessments and manufacturing productivity and scientific influence as they transform into a
supply chain simulations. Researchers can now access revolutionary power in computational science.

Table 3 Qualitative Comparison of HPC-Ai Workflow Platform with Traditional HPC Systems in
Time-to-Solution and Scientific Outcomes
System Workflow Optimization Result Application
Adaptability Approach Quality Suitability
HPC-AI Dynamic, AI-Driven High- Time-
Workflow Real-Time Fidelity Sensitive (e.g., disaster
response) General-
Purpose Broad, Non-
Adaptive
SLURM Static Static Manual Standard Basic

Traditional HPC Manual, Limited

Systems

IV. CASE STUDIES agricultural effects. The researchers processed petabytes of

historical and real-time satellite and sensor data while facing
The workflow platform for HPC-AI introduced here unpredictable weather conditions. Traditional HPC systems
has proven effective in solving intricate data-heavy that employed MPI [7] needed manual adjustments which
problems with greater accuracy across multiple scientific postponed results delivery for several weeks. The Analytics
and industrial sec- tors. Case studies demonstrate how the and AI Layer in the HPC-AI Workflow Platform allowed
platform’s intelligent workflows combined with AI-driven the creation of a flexible workflow system that could
analytics and efficient resource management lead to better incorporate live data. The Dynamic Workflow Processor
performance than tradi- tional HPC systems [11] through activated rainfall analysis upon storm detection which
providing useful insights and considerable time efficiency reduced processing time to 6 days from 10 days saving 40%
when working with complex tasks such as climate change time unlike MPI. GPU optimization through the platform’s
modeling as well as natural disaster prediction and Resource Management System achieved a 20% reduction in
manufacturing optimization. energy consumption. Improved modeling speed and
precision enabled policymakers to provide better
A. Climate Change Modeling: agricultural protection.
Regional Impact Analysis: European scientists created
a climate simulation to monitor temperature and
precipitation shifts over five decades and ana- lyzed

IJISRT25APR1850 www.ijisrt.com 3453

Volume 10, Issue 4, April – 2025 International Journal of Innovative Science and Research Technology
ISSN No:-2456-2165 https://doi.org/10.38124/ijisrt/25apr1850
B. Natural Disaster Prediction: source of innovation that advances climate science and
Seismic Risk Assessment: An organization located in disaster man- agement together with industrial optimization.
an earthquake-prone region eval- uated terabytes of However, the HPC-AI Workflow Platform will require
geological data which included fault maps and seismic fu- ture enhancements in certain areas to reach its full poten-
profiles together with soil profiles to understand earthquake tial. The implementation of enhanced scalability for
risks for urban planning. To model fault activity and exascale computing will grant it the capability to efficiently
understand its effects on infrastructure successfully re- process large datasets and intricate AI models. The
searchers required a system design that provided both flexi- Workflow-as- a-Service model will become accessible to
bility and substantial computational power. Processing non-experts if we develop intuitive interfaces and provide
delays occurred because Static scheduling through SLURM user training which will extend its applicability. The
[20] allo- cated resources inefficiently. The HPC-AI implementation of sophisticated AI methods including
Workflow Platform merges AI technologies and dynamic reinforcement learning improves workflow adaptability and
adaptive systems that are appropriate to minimize precision. The platform will become a robust and adaptable
processing time and generate precise predictions while user-friendly solution that solves complex computational
providing immediate benefits for strategic data analysis. science and industrial problems through innovative
approaches.
C. Manufacturing Optimization:
Assembly Line Efficiency in Automotive Production: REFERENCES
The automaker focused on enhanc- ing assembly line
efficiency by controlling costs and limiting environmental [1]. Jorge Ejarque, Rosa M. Badia, Lo¨ıc Albertin,
effects despite facing supply chain issues. The team Giovanni Aloisio, En- rico Baglione, Yolanda
conducted simulations of production scenarios that incor- Becerra, Stefan Boschert, Julian R. Berlin,
porated material shortages to analyze energy usage and Alessandro D’Anca, Donatello Elia, Franc¸ois
labor consumption for reducing waste while keeping Exertier, Sandro Fiore, Jose´ Flich, Arnau Folch,
production levels stable. Traditional software tools lack Steven J. Gibbons, Nikolay Koldunov, Francesc
runtime flexibility which necessitates numerous testing Lordan, Stefano Lorito, Finn Løvholt, Jorge Mac´ıas,
sessions and pushes the optimization process duration Fabrizio Marozzo, Alberto Michelini, Marisol
beyond one month. The team achieved real-time adaptive Monterrubio-Velasco, Marta Pienkowska, Josep de la
modeling by utilizing intelligent workflows on the HPC-AI Puente, Anna Queralt, Enrique S. Quintana-Ort´ı,
Workflow Platform. The Resource Management System Juan E. Rodr´ıguez, Fabrizio Romano, Riccardo
decreased optimization duration from 30 days with LSF to Rossi, Jedrzej Rybicki, Miroslaw Kupczyk, Jacopo
only 5 days by assigning CPUs to data tasks and GPUs to Selva, Domenico Talia, Roberto Tonini, Paolo
visualization, resulting in an 83% time savings. Trunfio, and Manuela Volpe, “Enabling dynamic and
intelligent workflows for HPC, data analytics, and AI
V. CONCLUSIONS AND FUTURE WORK convergence,” Future Generation Computer Systems,
vol. 134, pp. 414–429, 2022.
The HPC-AI Workflow Platform provides a major [2]. Shantenu Jha, Vincent R. Pascuzzi, and Matteo
break- through in computational science by combining Turilli, “AI-coupled HPC Workflows,” 2022,
High- Performance Computing (HPC), Artificial arXiv:2208.11745 [cs.DC].
Intelligence (AI), and Scientific Workflows into a unified [3]. Fabio Le Piane, Mario Vozza, Matteo Baldoni, and
intelligent system. The combination of HPC with AI and Francesco Mercuri, “Integrating high-performance
scientific workflows eradicates conventional system computing, machine learning, data man- agement
boundaries such as inflexibility and operational inefficiency workflows, and infrastructures for multiscale
while resolving access challenges simulations and nanomaterials technologies,”
Beilstein Journal of Nanotechnology, vol. 15, pp.
[11] and sets a fresh precedent for applications in both 1498–1521, 2024.
research and industry. The Workflow-as-a-Service (WaaS) [4]. Rafael Ferreira da Silva, Rosa M. Badia, Deborah
model provides users with a solution that removes manual Bard, Ian T. Foster, Shantenu Jha, and Frederic Suter,
configuration burdens which traditional systems like “Frontiers in Scientific Workflows: Pervasive
SLURM [20] or MPI [7] require from them. Through the Integration With High-Performance Computing,”
streamlined method users achieve enhanced focus on Computer, vol. 57, no. 8, pp. 36–44, 2024.
innovative tasks and the WaaS repository enables sharing [5]. AI4HPC Consortium, “AI4HPC: Enabling AI-Driven
and reuse of workflows across multiple projects. Users High-Performance Computing,” AI4HPC Project,
achieve better productivity outcomes with the HPC-AI Technical Report, 2022.
Workflow Platform compared to traditional systems while [6]. Sam Partee, Joseph Ellis, Duncan Ritchie, Rob
reaping operational and scientific advantages [11]. The Sterrett, Sriram Venkatesh, Mike Lesniak, and Anshu
platform reduces operational complexity to deliver better Dubey, “SmartSim: Online An- alytics and AI for
team collaboration and faster results with fewer resources High-Performance Computing,” in Proceedings of the
while enhancing accessibility and speeding up solution International Conference for High Performance
delivery. By transcending conventional HPC system Computing, Networking, Storage and Analysis (SC
constraints the HPC-AI Workflow Platform becomes a ’21), pp. 1–14, 2021.

IJISRT25APR1850 www.ijisrt.com 3454

Volume 10, Issue 4, April – 2025 International Journal of Innovative Science and Research Technology
ISSN No:-2456-2165 https://doi.org/10.38124/ijisrt/25apr1850
[7]. Message Passing Interface Forum, “MPI: A Message- [21]. Anubhav Jain, Shyue Ping Ong, Wei Chen, Bharat
Passing Interface Standard, Version 4.0,” University Medasani, Xi- aohui Qu, Michael Kocher, Miriam
of Tennessee, Knoxville, TN, 2021. Brafman, Guido Petretto, Gian- Marco Rignanese,
[8]. Ewa Deelman, Dennis Gannon, Matthew Shields, and Geoffroy Hautier, Daniel Gunter, and Kristin A.
Ian Taylor, “Pegasus: A Framework for Mapping Persson, “FireWorks: A Dynamic Workflow System
Complex Scientific Workflows onto Distributed Designed for High- Throughput Applications,”
Systems,” Scientific Programming, vol. 19, no. 2–3, Concurrency and Computation: Practice and
pp. 219–237, 2011. Experience, vol. 27, no. 17, pp. 5037–5059, 2015.
[9]. Ilkay Altintas, Chad Berkley, Efrat Jaeger, Matthew [22]. Andy B. Yoo, Morris A. Jette, and Mark Grondona,
Jones, Bertram Lu- dascher, and Steve Mock, “SLURM: Simple Linux Utility for Resource
“Kepler: An Extensible System for Scientific Management,” Lecture Notes in Computer Science,
Workflows,” Scientific Programming, vol. 14, no. 3– vol. 2862, pp. 44–60, 2003.
4, pp. 191–208, [23]. Maxime Beauchemin and others, “Apache Airflow: A
[10]. 2006. Platform to Programmatically Author, Schedule and
[11]. Tom Oinn, Matthew Addis, Justin Ferris, Darren Monitor Workflows,” Apache Software Foundation,
Marvin, Martin Senger, Mark Greenwood, Tim 2016.
Carver, Kevin Glover, Matthew R. Pocock, Anil [24]. Yan Liu, Wei Zhang, and Jun Zhou, “IBM Spectrum
Wipat, and Peter Li, “Taverna: A Tool for the LSF: A Resource Management Framework for High-
Composition and Enactment of Bioinformatics Performance Computing,” IBM Jour- nal of Research
Workflows,” Bioinformatics, vol. 22, no. 22, pp. and Development, vol. 54, no. 3, pp. 1–10, 2010.
3045–3054, 2006. [25]. Jay Kreps, Neha Narkhede, and Jun Rao, “Kafka: A
[12]. Ian Foster, Carl Kesselman, and Steven Tuecke, “The Distributed Mes- saging System for Log Processing,”
Anatomy of the Grid: Enabling Scalable Virtual in Proceedings of the NetDB’11 Workshop, 2011.
Organizations,” International Journal of High [26]. Mart´ın Abadi, Ashish Agarwal, Paul Barham,
Performance Computing Applications, vol. 15, no. 3, Eugene Brevdo, Zhifeng Chen, Craig Citro, Greg S.
pp. 200–222, 2001. Corrado, Andy Davis, Jeffrey Dean, Matthieu Devin,
[13]. Ian J. Taylor, Ewa Deelman, Dennis B. Gannon, and and others, “TensorFlow: Large-Scale Machine
Matthew Shields, “Scientific Workflows for Grids,” Learning on Heterogeneous Distributed Systems,”
Springer, 2014. arXiv:1603.04467, 2015.
[14]. Thomas Sterling, Matthew Anderson, and Maciej [27]. Dirk Merkel, “Docker: Lightweight Linux Containers
Brodowicz, “High Performance Computing: Modern for Consistent Development and Deployment,” Linux
Systems and Practices,” Morgan Kauf- mann, pp. 1– Journal, vol. 2014, no. 239, 2014.
632, 2018. [28]. Garrick Staples, “TORQUE Resource Manager,” in
[15]. Shantenu Jha, Daniel S. Katz, Andre Luckow, and Proceedings of the 2006 ACM/IEEE Conference on
Andre Merzky, “AI- Coupled HPC Workflows,” Supercomputing, 2006.
Future Generation Computer Systems, vol. 118, pp. [29]. William Gropp, Ewing Lusk, Nathan Doss, and
245–259, 2021. Anthony Skjellum, “A High-Performance, Portable
[16]. Rick Stevens, Valerie Taylor, and Jeffrey Nichols, Implementation of the MPI Message Passing
“AI for HPC: Expe- riences and Opportunities,” Interface Standard,” Parallel Computing, vol. 22, no.
Computing in Science & Engineering, vol. 24, no. 1, 6, pp. 789– 828, 1996.
pp. 10–19, 2022. [30]. Yadu Babuji, Ian Foster, Michael Wilde, Kyle Chard,
[17]. Sam Partee, Jonathan Ellis, Kevin Moreau, and and Daniel S. Katz, “Parsl: Pervasive Parallel
Sivasankaran Rajaman- ickam, “SmartSim: Online Programming in Python,” in Proceedings of the 28th
Simulation with Machine Learning,” Journal of Open International Symposium on High-Performance
Source Software, vol. 6, no. 67, pp. 3542, 2021. Parallel and Distributed Computing, pp. 25–36, 2019.
[18]. Prasanna Balaprakash, Michael Salim, Thomas D. [31]. Paolo Di Tommaso, Maria Chatzou, Evan W. Floden,
Uram, Venkat Vish- wanath, and Stefan M. Wild, Pablo Prieto Barja, Emilio Palumbo, and Cedric
“DeepHyper: Asynchronous Hyperparam- eter Search Notredame, “Nextflow Enables Reproducible
for Deep Neural Networks,” in Proceedings of the Computational Workflows,” Nature Biotechnology,
28th International Symposium on High-Performance vol. 35, no. 4, pp. 316–319, 2017.
Parallel and Distributed Computing, pp. 42–53, 2019. [32]. Chun Siong Liew, Malcolm P. Atkinson, Michelle
[19]. Justin M. Wozniak, Timothy G. Armstrong, Michael Galea, Tan Fong Ang, Paul Martin, and Jano I. van
Wilde, Daniel Hemert, “Scientific Workflows: Moving Across
[20]. S. Katz, Ewing Lusk, and Ian Foster, “Swift/T: Paradigms,” ACM Computing Surveys, vol. 49, no.
Scalable Dataflow Programming for Distributed 4, pp. 1–39, 2016.
Memory HPC,” Parallel Computing, vol. 65, pp. 1– [33]. Bertram Ludascher, Ilkay Altintas, Chad Berkley,
14, 2017. Dan Higgins, Efrat Jaeger, Matthew Jones, Edward
A. Lee, Jing Tao, and Yang Zhao, “Scientific
Workflow Management and the Kepler System,”
Concurrency and Computation: Practice and
Experience, vol. 21, no. 8, pp. 1039– 1065, 2009.

IJISRT25APR1850 www.ijisrt.com 3455

Volume 10, Issue 4, April – 2025 International Journal of Innovative Science and Research Technology
ISSN No:-2456-2165 https://doi.org/10.38124/ijisrt/25apr1850
[34]. Adam Barker and Jano van Hemert, “Scientific
Workflow: A Survey and Research Directions,” in
Parallel Processing and Applied Mathematics,
[35]. pp. 746–753, 2007.
[36]. Jack Dongarra, Hans Meuer, Erich Strohmaier, and
Horst Simon, “TOP500 Supercomputer Sites: 2020
Edition,” Prometeus GmbH, Ger- many, 2020.

IJISRT25APR1850 www.ijisrt.com 3456