JOURNAL OF LATEX CLASS FILES, VOL. 14, NO.

8, AUGUST 2015 1

Prompt Sapper: LLM-Empowered Software

Engineering Infrastructure for AI-Native Services
Zhenchang Xing, Member, IEEE, Qing Huang, Member, IEEE, Yu Cheng, Liming Zhu, Senior
Member, IEEE, Qinghua Lu, Member, IEEE, and Xiwei Xu, Member, IEEE,

Abstract—Foundation models, such as GPT-4, DALL-E have brought unprecedented AI “operating system” effect and new forms of
human-AI interaction, sparking a wave of innovation in AI-native services, where natural language prompts serve as executable “code”
directly (prompt as executable code), eliminating the need for programming language as an intermediary and opening up the door to
personal AI. Prompt Sapper has emerged in response, committed to support the development of AI-native services by AI chain
engineering. It creates a large language model (LLM) empowered software engineering infrastructure for authoring AI chains through
human-AI collaborative intelligence, unleashing the AI innovation potential of every individual, and forging a future where everyone can
be a master of AI innovation. This article will introduce the R&D motivation behind Prompt Sapper, along with its corresponding AI
chain engineering methodology and technical practices.

Index Terms—Foundation Models, AI Chain, Software Engineering, AI Native IDE.

1 I NTRODUCTION
”The greatest danger in times of turbulence is not the traditional programming and AI training, and allows (non-
turbulence - it is to act with yesterday’s logic.” technical) individuals to create, customize and compose AI
— Peter Drucker services by describing “what to do” (e.g., task workflow and
Just as 40 years ago, personal computers (PCs) has data characteristics), and designing human-AI interaction
brought about the transformation of information age; 15 through natural language, rather than focusing on “how to
years ago, the proliferation of smartphones further drove do” as in Software 1.0/2.0, i.e., data structures, algorithms,
the development of mobile computing and applications, API calls, data labeling, and model training.
making our daily lives and work more convenient and effi- The bottleneck of personal AI (analogous to PC 40 years
cient. Now, the rapid development of artificial intelligence ago) is rapidly opening up thanks to foundation models. If
(AI) technology, represented by foundation models [1] such we regard the foundation models as AI “operating system”,
as the well-known GPT-4 [2] and DALL-E [3] , is bringing the generative capabilities invoked by prompt-as-code are
about the long-awaited AI platformization effect and a new the “API” to the AI “operating system”, but they are not
trend of technological revolution. AI-native services running on the AI “operating system”.
In the Project AI 2.0 [4], Kai-Fu Lee believes that the We are in the stage of transforming the capabilities of
current popular chatbots and text-to-image generation are foundation models into practical AI services, which presents
just the tip of the iceberg in this revolution. As illustrated in two opportunities: 1) allowing everyone (not just AI or
Figure 1, foundation models unlock Software 3.0. The most software engineers) to create personalized AI services, and
essential characteristic of Software 3.0 lies in “prompt as 2) enabling people to share and hire AI services. Andrej
code”, that is, we can interact with AI and instruct AI with Karpathy envisioned that this natural language program-
our intent in natural language “code” (i.e., prompts) directly ming paradigm could potentially expand the number of
to create AI-native services on top of foundation models. In “programmers” to 1.5 billion [7] .
Software 1.0, the code is written in a programming language Despite the rapid emergence of AI services based on
by developers. In Software 2.0 [5], the network behavior foundation models (see https://gpt3demo.com/), their de-
is learned from data, but the neural network code is still velopment still requires traditional software development
written in a programming language by developers. methods, offsetting the AI democratization effect of founda-
Foundational models would go beyond making soft- tion models, and thus many great ideas remain stagnant at
ware 1.0/2.0 development more productive, for example by the early ideation stage and fail to progress further. While
Github Copilot (i.e., prompt to code). They are changing ChatGPT demonstrates impressive capabilities, including
not only who can create software but also what types of generating and testing code, chat is not inherently an
software can be created [6]. Software 3.0 frees people from effective software development mechanism. Furthermore,
prompt-based AI services relies on prompt engineering
magic which lacks transparency and is hard to control, test,
• Z. Xing is with CSIRO’s Data61, Australia. reuse and maintain, akin to the state of ad-hoc coding before
E-mail: [email protected]
• L. Zhu, Q. Lu and X. Xu are with CSIRO’s Data61, Australia. the advent of software engineering 55 years ago.
• Q. Huang and Y. Cheng are with Jiangxi Normal University, China. Q. In response to the AI 2.0 and Software 3.0 revolution,
Huang is the corresponding author. what we need is not just flashy prompts and chatbot, but
Manuscript received XXX XX, XXXX; revised XXX XX, XXXX. a transformative software engineering infrastructure that
JOURNAL OF LATEX CLASS FILES, VOL. 14, NO. 8, AUGUST 2015 2

Fig. 1. Foundation Models Unlock Software 3.0 (The most essential characteristic of Software 3.0 is “prompt as code”)

allows everyone to participate and truly benefit from AI. knowledge (e.g., sentiment ratings), analyze requirements,
decompose tasks, and generate and evolve AI chains. AI
chains consist of AI-native workers that directly invoke
2 AI C HAIN E NGINEERING emergent capabilities of foundation models by natural lan-
Our vision is to reshape software landscape through gen- guage prompts (i.e., prompt as code), without the need for
erative AI. To realize this vision, we propose AI chain programming language as an intermediary. AI chain work-
engineering1 . We regard foundation models as AI “operat- ers uphold well-established software engineering values,
ing systems” whose generative capabilities can be invoked such as modularity, extensiblity, reusability.
by prompt-as-code, without requiring a programming lan- As shown in Figure 3, we are committed to develop a
guage as an intermediary. AI chains are a novel form of systematic AI chain methodology (Goal 1: Promptmanship),
software product that assembles prompt calls to founda- the corresponding AI-native development and deployment
tion models, together with calls to traditional AI models, environment (Goal 2: AI chain IDE), and the AI services
external data or APIs (if needed), according to specific marketplace and ecosystem (Goal 3).
workflows, thereby delivering AI-native services. AI chain First, prompt-as-code does not imply that writing
functions as both a medium for human-AI collaboration, prompts is the only task AI-native service development
and as the tangible outcome of such collaboration. They entails. We aim to summarize the best practices of prompt
intuitively map to task workflow and are “coded” in nat- engineering and place them within the broad context of
ural language prompts, directly learnable and moldable by software engineering, complementing important software
ordinary people (AI chain builders). engineering methods that have been overlooked (such as
As illustrated in Figure 2, the AI chain builders do not software processes, system design, testing). This creates
need to switch to “programmer mindset” as in Software an AI chain methodology that upholds well-established
1.0/2.0 but stay within their task contexts for building AI- software engineering principles and values to improve the
native services. In Software 1.0/2.0, the user requires a good transparency, controllablity, testability and maintainability
knowledge of the solution space, for example, different of AI services built on top of foundation models.
types of models and APIs and how to use them in source
Second, chatbot, regardless of the capabilities of the
code. AI assistance (e.g., Github Copilot) could lower this
underlying LLM, is a only “command-line console” to the
technical barrier, but the user still needs to know some tech-
AI “operating system”. We aim to develop an AI-native
nical concepts and tools (e.g., NER, HuggingFace, NLTK)
integrated development environment (IDE) that supports
in order to write the effective prompts for code generation.
the whole process of AI chain development, from ideas
In Software 3.0, the LLM assists the user in acquiring task
to services. Through the tangible implementation of the AI
chain methodology, AI chain IDE will support individuals to
1. As an emergent paradigm, we focus on the development of AI-
native services through AI chain engineering, and leave the mainte- develop high-quality, foundation model-based AI services.
nance and other DevOps activities as future work. Leveraging natural language interface, LLM-empowered
JOURNAL OF LATEX CLASS FILES, VOL. 14, NO. 8, AUGUST 2015 3

Fig. 2. Building AI-Native Services through AI Chain Engineering, with LLM Assistance and Prompt-as-Code

interactive guidance and automatic AI chain generation, this 3 P ROMPTMANSHIP

IDE lowers the technical barrier for AI chain development
and improve the development efficiency. Over the decades of its development, software engineering
has accumulated a wealth of effective methods and prac-
tices, many of which can be applied to AI chain engineering.
Third, we aim to develop an AI services marketplace However, with the transformative shift in human-AI interac-
to promote the development of the AI services ecosystem. tion brought about by foundation models, it is necessary to
We envision this AI services marketplace would be different examine and adjust traditional software engineering meth-
from code repositories like Github and data/model hubs ods and practices to accommodate the emerging human-
like HuggingFace, as the AI-native services are primarily centered natural language programming paradigm.
prompt driven and offer practical compositional AI capabil- On one hand, LLMs encode vast amounts of knowledge
ities due to the AI platformization brought by foundation and possess powerful conversational abilities. We can lever-
models. We propose to investigate responsible AI chain age them to assist AI chain engineers in acquiring domain
engineering methods and technologies to enhance the trans- knowledge, understanding problems, and gaining inspira-
parency, accountability, and security of AI services. tion for problem-solving. At the same time, to mitigate the
inherent challenges of errors and illusions in LLMs, we need
to have a deep understanding of their capabilities (known
We envision that AI chain builders adhere to prompt- as “mechanical sympathy”) and employ effective prompt
manship, develop AI chains in the AI-native IDE, and design patterns. On the other hand, LLMs not only change
share them in the marketplace. These AI-native services can who can develop AI services but also profoundly alter the
be deployed in various platforms to serve the end users types of AI services that can be developed. This requires
through the chatbot interface or other GUIs. Our vision a shift from code-centric development tools of the past to
is to establish an open, collaborative, secure, and sustain- human-centric tools that enable (non-technical) individuals
able AI chain ecosystem that provides support for digital to focus on problem-solving and engage in intuitive analy-
transformation and upgrade across various industries. We sis, design, construction, and evaluation of AI chains.
want to empower individuals and businesses to unleash Guided by these design considerations, we propose
their creativity and intelligence in the new era of AI 2.0 and an AI chain methodology (promptmanship) as shown in
Software 3.0, allowing them to benefit from it and achieve Figure 4. We define the functional units of AI chains as
the vision of symbiosis between humans and AI. “workers”. Promptmanship encompass not only traditional
JOURNAL OF LATEX CLASS FILES, VOL. 14, NO. 8, AUGUST 2015 4

Fig. 3. Software Engineering for AI Chain: Vision and Goals

Fig. 4. Promptmanship: AI Chain Process, Concepts, and Activities

software concepts such as requirements, object composition stereotypes for input transformation, task processing and
and collaboration, and object roles but also specific AI chain output validation, and 12 well-adopted prompting best
concepts. For instance, we differentiate three types of work- practices (e.g., prompt caveats, aspects and decorators).
ers (corresponding to three software paradigms: Software We view AI chain engineering as an iterative rapid
1.0/2.0/3.02 ), four levels of worker-AI interaction modes prototyping process with many-step collaboration between
based on increasing reasoning capabilities, nine worker human and AI. This process consisting of four iterative
stages: Explore, Design, Build, and Deploy. Each stage
2. We believe Software 3.0 will not cover 100% the problem space involves concurrent activities (inspired by the Rational
where Software 1.0 and Software 2.0 workers can perform more ef- Unified Process [8]), including task modeling, system de-
fectively and economically. For example, Software 1.0 can process
deterministic logic precisely, and Software 2.0 will be efficient and sign (requirements analysis, task decomposition, AI/non-
economic for the text classification task. AI separation of concerns, workflow rehearsals), AI chain
JOURNAL OF LATEX CLASS FILES, VOL. 14, NO. 8, AUGUST 2015 5

implementation, and testing. However, each stage has a testing. This is, it is an LLM-empowered software engineer-
different emphasis (as depicted by the corresponding bar ing framework for AI chain, i.e., AI4SE4AI. The Sapper IDE
heights). Different activities generate or refine different AI can be seen as an “incubator for AI services” as it leverages
chain concepts (shown within the blue intervals below the the power of foundation models to create more AI services
concepts). Throughout all activities, we propose a “magic through the deep collaboration between human and AI. It
enhances magic” activity that leverages the knowledge and not only fulfills the users’ needs for AI-native services but
conversational abilities of large language models to assist also inspires them to explore more possibilities and help
AI chain engineers in acquiring task knowledge, gathering them create even better AI services.
and analyzing requirements, and gaining an understanding
of model capabilities (i.e., mechanical sympathy).
At the end of an iteration, an AI chain will be constructed 5 AI S ERVICES M ARKETPLACE
suitable for production. It is fundamentally modular, as
To demonstrate the applicability of our promptmanship and
opposed to epic prompts that ask the LLM to do the whole
Sapper IDE, we have built a set of AI service showcases
thing all at once (so called LLM maximalism by Matthew
based on foundation models, including writing assistants,
Honnibal). The whole AI chain, as well as its individual
poem to paint generation, mental health assistant, inter-
workers, have been tested and optimized, and errors can
view training, programming assistant, quiz maker. We are
be attributed to different workers and fixed. When require-
actively building an AI service marketplace and will allow
ments change or get extended, new workers can be added
users to directly share their AI chain projects from the Sap-
or existing ones can be updated or replaced. Workers can
per IDE to the marketplace. We anticipate that the commu-
also be reused across different services.
nity will contribute a multitude of excellent personalized or
vertical market AI services, fostering a rich and diverse AI
4 S APPER IDE services ecosystem centered around foundational models.
Open-source AI chain projects will provide ample learning
Our Sapper IDE differs from existing code-centric develop- materials and inspiration to onboard beginners.
ment tools, because AI chains will be developed by many This AI services marketplace would be different from
non-technical individuals. In the era of Software 3.0, these code repositories like Github and data/model hubs like
individuals possess vast demands and domain knowledge, HuggingFace, as AI-native services are primarily prompt
but they lack proficiency in computer science, software de- driven and offer the compositional AI capabilities due to
velopment, and AI background knowledge. Consequently, the AI platformization brought by foundation models. Of
they face challenges in effectively expressing their require- course, this will bring new software engineering challenges,
ments and knowledge to AI and leveraging foundation especially in responsible AI and AI services supply chain
models to create the desired AI services. management, which are our active research agenda to ex-
Therefore, our foremost design principle is “human- pand our promptmanship and sapper IDE to address these
centric”, which manifests in three aspects: First, we reify the challenges, for example, to embed our AI risk assessment
promptmanship (AI chain methodology) into the AI chain framework [9] and Responsible AI pattern catalogue in the
IDE, enabling anyone to effectively apply best AI chain promptmanship, and to develop AI Bills of Materials to
practices and methods. Second, we leverage the knowledge secure AI services supply chain [10].
and conversational capabilities of LLMs to develop AI co-
pilots (acting as virtual product manager, system designer
and tester) that provide the whole-process AI chain develop- 6 R ELATED W ORK
ment support for non-technical individuals. Third, we offer
a no-code AI chain analysis, design, construction, testing, The concept of AI chain has been widely applied in various
and deployment process, making it easy for anyone to scenarios recently. We believe it is a fundamental strategy
transform their ideas into AI services. for determining and expanding the capability and aware-
Based on these design principles, we have developed ness boundaries of AI. We categorize these works into three
the Sapper IDE3 , an AI-native platform that fully supports major categories: task-specific AI chains [11], [12], [13], [14],
the AI chain process, concepts and activities proposed in [15], [16], [17], task-agnostic agent frameworks [18], [19],
our promptmanship. The Sapper IDE adopts a human-AI [20], [21], [22], [23], [24], [25], [26], AI chain programming
teamwork strategy, where humans, assisted by AI, mainly support [27], [28], [29]. Prompt Sapper draws inspiration
handle the two ends (expressing requirements, acceptance from these projects and tools, but it has significant differ-
tests), while AI operates in the middle (generating and ences from them in three aspects:
testing AI chains). As such, different from the current IDE Human-AI collaborative intelligence. Prompt Sapper em-
designed around code editor, the emphasis of Sapper IDE is phasizes the collaborative interaction between AI and hu-
on the LLM-empowered exploration and design view that man users, with human focusing on requirements and ac-
iteratively scaffold requirement analysis and system design ceptance while AI constructing AI chains and performing
and on iterative AI chain testing and debugging, rather than tasks. It marries human intelligence with artificial intelli-
on the visual programming as in [6]. gence through AI chains, effectively addressing complex
The Sapper IDE is built on a suite of LLM-empowered problems and achieving shared goals. This human-AI col-
AI co-pilots for AI chain analysis, design, construction and laborative intelligence fosters enhanced overall efficiency,
reduced error rates, and empowers human users to fully
3. Visit https://promptsapper.tech to try our Sapper IDE. harness the potential of AI. As illustrated in Figure 5, this
JOURNAL OF LATEX CLASS FILES, VOL. 14, NO. 8, AUGUST 2015 6

Fig. 5. Spectrum of Human-AI Interaction

distinctive approach sets Prompt Sapper apart from human- and Software 3.0. This AI4SE4AI framework not only sub-
driven conversational bots (e.g., ChatGPT) that demand hu- stantially enhances the development efficiency and project
man expertise and thought process and AI-dominated agent quality of AI services but also supports flexible service
frameworks (e.g., AutoGPT) that rely on the LLM’s divide- reuse and assembly, as well as continuous improvement and
and-conquer and self-correction capabilities, highlighting its optimization of AI services to meet ever-changing demands.
innovative and unique value proposition. Low-code LLM [30] and LLM pragmatism [31]share the
Requirements for programming skills. Compared to other similar task decomposition idea as Prompt Sapper, but they
projects [27], [28], [29], Prompt Sapper significantly low- do not provide a holistic AI4SE4AI framework.
ers the barrier to entry for creating complex AI services Sapper IDE stands apart from existing Cloud IDEs or
tailored to user needs. It introduces a suite of LLM-based coding tools like Repit [32], Codespace [33], Jupyter Note-
virtual product manager, architect, and prompt engineer book [34]. While these Cloud IDEs primarily address the
to assist users in acquiring domain knowledge, analyzing complexity of setting up and configuring software devel-
task requirements, and constructing AI chains. Addition- opment environments, Sapper IDE tackles the complexity
ally, Prompt Sapper provides an intuitive and user-friendly of the AI-native service development process. Recently, sev-
interface, enabling users to effortlessly interact with AI eral low/no-code AI tools have been introduced, such as
and prototype AI functionalities without the need for ad- Microsoft AI Builder [35], Zapier [36], and superbio.ai [37].
vanced computing or programming skills. This approach These tools enable drag-and-drop visual programming, al-
broadens the spectrum of people able to benefit from the lowing users to utilize or customize pre-built models within
advancements in AI and underscoring the distinct position their workflows. However, the workflows they support are
of Prompt Sapper in the AI landscape. often challenging to define or customize. Moreover, they
The AI4SE4AI framework. Prompt Sapper values the close primarily focus on the coding stage of the software de-
integration of software engineering and AI, striving to create velopment process (the bottom-right corner in Figure 4).
a systematic AI4SE4AI framework. Within this framework, They do not adequately support novices in effectively ex-
Prompt Sapper leverages AI technology to significantly im- pressing their needs and knowledge to AI during the early
prove the efficiency of software engineering processes, such stages of the software development process, nor leveraging
as requirements analysis, AI chain design, construction, and foundation models to build custom workflows and per-
testing. At the same time, Prompt Sapper adheres to and sonalized AI services. Programming assistants like GitHub
expands upon the best practices of software engineering Co-pilot [38], Replit Ghostwriter [39], and similar products
to adapt to the new software landscape driven by AI 2.0 aim to lower coding barrier and improve coding efficiency
JOURNAL OF LATEX CLASS FILES, VOL. 14, NO. 8, AUGUST 2015 7

within Software 1.0/2.0. They represent useful but incre- [11] S. Arora, A. Narayan, M. F. Chen, L. J. Orr, N. Guha, K. Bhatia,
mental improvements to existing IDEs (i.e., prompt to code), I. Chami, F. Sala, and C. Ré, “Ask me anything: A simple strategy
for prompting language models,” arXiv preprint arXiv:2210.02441,
whereas Sapper IDE supports a disruptive transformation in 2022.
software development (i.e., prompt as code). [12] T. Schick, J. Dwivedi-Yu, Z. Jiang, F. Petroni, P. Lewis, G. Izacard,
AI chain engineering carries a certain notion of inten- Q. You, C. Nalmpantis, E. Grave, and S. Riedel, “Peer: A collabo-
tional programming [40]. Both advocate task decomposition rative language model,” arXiv preprint arXiv:2208.11663, 2022.
[13] K. Yang, N. Peng, Y. Tian, and D. Klein, “Re3: Generating longer
and intention expression. The difference lies in that inten- stories with recursive reprompting and revision,” arXiv preprint
tional programming aims to separate precise intent from arXiv:2210.06774, 2022.
source code so that software becomes easier to create and [14] A. Creswell, M. Shanahan, and I. Higgins, “Selection-inference:
maintain in the Software 1.0/2.0 paradigms, while AI chain Exploiting large language models for interpretable logical reason-
ing,” arXiv preprint arXiv:2205.09712, 2022.
engineering aims to turn fuzzy informal intent into AI- [15] S. M. Kazemi, N. Kim, D. Bhatia, X. Xu, and D. Ramachandran,
native services in the era of Software 3.0. With the support “Lambada: Backward chaining for automated reasoning in natural
of foundation models, Software 3.0 is realizing the long- language,” arXiv preprint arXiv:2212.13894, 2022.
pursued vision of end-user programming: empowering or- [16] I. Singh, V. Blukis, A. Mousavian, A. Goyal, D. Xu, J. Tremblay,
D. Fox, J. Thomason, and A. Garg, “Progprompt: Generating
dinary individuals to harness the full power of computers situated robot task plans using large language models,” arXiv
and AI, rather than relying solely on applications developed preprint arXiv:2209.11302, 2022.
by professional programmers. [17] Q. Huang, J. Zhu, Z. Li, Z. Xing, C. Wang, and X. Xu, “Pcr-chain:
Partial code reuse assisted by hierarchical chaining of prompts on
frozen copilot,” in Proceedings of the ACM/IEEE 45th International
7 C ONCLUSION Conference on Software Engineering: Companion Proceedings, 2023.
[18] Y. Shen, K. Song, X. Tan, D. Li, W. Lu, and Y. Zhuang, “Hugging-
We are witnessing the technological revolution in AI and gpt: Solving ai tasks with chatgpt and its friends in huggingface,”
software engineering. Prompt Sapper advocates the best arXiv preprint arXiv:2303.17580, 2023.
practices and methodologies of AI chain engineering, driv- [19] R. Abdelghani, Y.-H. Wang, X. Yuan, T. Wang, H. Sauzéon,
and P.-Y. Oudeyer, “Gpt-3-driven pedagogical agents for train-
ing the development and proliferation of AI-native services. ing children’s curious question-asking skills,” arXiv preprint
We will adopt an “outbound and inbound” approach to arXiv:2211.14228, 2022.
bridge the last mile between AI-native services and end [20] C. Wu, S. Yin, W. Qi, X. Wang, Z. Tang, and N. Duan, “Visual
users, bringing AI chain engineering methodologies, tools, chatgpt: Talking, drawing and editing with visual foundation
models,” arXiv preprint arXiv:2303.04671, 2023.
and practices to broader audience, while promoting the de- [21] Y. Liang, C. Wu, T. Song, W. Wu, Y. Xia, Y. Liu, Y. Ou, S. Lu,
velopment of the AI service marketplace and ecosystem. We L. Ji, S. Mao et al., “Taskmatrix. ai: Completing tasks by con-
envision that Prompt Sapper on top of foundation models necting foundation models with millions of apis,” arXiv preprint
will fulfil the vision of personal AI, and propel society and arXiv:2303.16434, 2023.
[22] M. Fezari and A. A.-D. Ali-Al-Dahoud, “From gpt to autogpt: a
the economy towards a more intelligent future. brief attention in nlp processing using dl.”
[23] G. Li, H. A. A. K. Hammoud, H. Itani, D. Khizbullin, and
B. Ghanem, “Camel: Communicative agents for” mind” explo-
ACKNOWLEDGMENTS ration of large scale language model society,” arXiv preprint
The authors would like to thank... arXiv:2303.17760, 2023.
[24] Y. Nakajima, “Babyagi,” https://github.com/yoheinakajima/
babyagi, 2023, accessed: 21/05/2023.
R EFERENCES [25] hyperwriteai. (2023) Hyperwrite ai. https://www.hyperwriteai.
com/. Accessed: 21/05/2023.
[1] R. Bommasani, D. A. Hudson, E. Adeli, R. Altman, S. Arora, S. von [26] H. Face. (2023) Hugging face transformers agents documen-
Arx, M. S. Bernstein, J. Bohg, A. Bosselut, E. Brunskill et al., “On tation. https://huggingface.co/docs/transformers/transformers
the opportunities and risks of foundation models,” arXiv preprint agents. Accessed: 21/05/2023.
arXiv:2108.07258, 2021. [27] H. Chase. (2023) Langchain documentation. https://python.
[2] OpenAI, “Gpt-4 technical report,” arXiv preprint arXiv:2303.08774, langchain.com/. Accessed: 21/05/2023.
2023. [28] Dust. (2023) Dust documentation. https://dust.tt/. Accessed:
[3] A. Ramesh, M. Pavlov, G. Goh, S. Gray, C. Voss, A. Radford, 21/05/2023.
M. Chen, and I. Sutskever, “Zero-shot text-to-image generation,” [29] Primer. (2023) Primer documentation. https://primer.ought.org/.
in International Conference on Machine Learning. PMLR, 2021, pp. Accessed: 21/05/2023.
8821–8831. [30] Y. Cai, S. Mao, W. Wu, Z. Wang, Y. Liang, T. Ge, C. Wu, W. You,
[4] K.-F. Lee. (2023) Ai 2.0 blog. https://www.chuangxin.com/blog/ T. Song, Y. Xia, J. Tien, and N. Duan, “Low-code llm: Visual
ai-2-0. Accessed: 21/05/2023. programming over llms,” 2023.
[5] A. Karpathy. (2017) Software 2.0. https://karpathy.medium.com/
[31] M. Honniabl. (2023) Against llm maximalism. https://explosion.
software-2-0-a64152b37c35. Accessed: 21/05/2023.
ai/blog/against-llm-maximalism. Accessed: 21/05/2023.
[6] T. Wu, E. Jiang, A. Donsbach, J. Gray, A. Molina, M. Terry, and C. J.
[32] replit. (2023) Replit. https://replit.com/. Accessed: 21/05/2023.
Cai, “Promptchainer: Chaining large language model prompts
through visual programming,” in CHI Conference on Human Factors [33] Y. Nakajima, “Codespaces,” https://github.com/features/
in Computing Systems Extended Abstracts, 2022, pp. 1–10. codespaces, 2023, accessed: 21/05/2023.
[7] A. Karpathy. (2023) Programming with english: A new [34] replit. (2023) Jupyter notebook. https://jupyter.org/. Accessed:
paradigm with expanded possibilities. https://mem.ai/p/ 21/05/2023.
J7iliZ9gtgeWRa9oJ4rC. Accessed: 21/05/2023. [35] microsoft. (2023) Microsoft ai builder. https://powerautomate.
[8] P. Kruchten, The rational unified process: an introduction. Addison- microsoft.com/zh-cn/ai-builder/. Accessed: 21/05/2023.
Wesley Professional, 2004. [36] zapier. (2023) Zapier. https://zapier.com/. Accessed: 21/05/2023.
[9] B. Xia, Q. Lu, H. Perera, L. Zhu, Z. Xing, Y. Liu, and J. Whittle, [37] superbio. (2023) superbio.ai. https://www.superbio.ai/. Ac-
“Towards concrete and connected ai risk assessment (c2 aira): A cessed: 21/05/2023.
systematic mapping study,” 2023. [38] github. (2023) Github copilot. https://github.com/features/
[10] B. Xia, T. Bi, Z. Xing, Q. Lu, and L. Zhu, “An empirical study on copilot. Accessed: 21/05/2023.
software bill of materials: Where we stand and the road ahead,” [39] replit. (2023) replit ghostwriter. https://replit.com/site/
2023. ghostwriter. Accessed: 21/05/2023.
JOURNAL OF LATEX CLASS FILES, VOL. 14, NO. 8, AUGUST 2015 8

[40] K. Czarnecki and U. W. Eisenecker, Generative Programming: Meth-

ods, Tools, and Applications. USA: ACM Press/Addison-Wesley
Publishing Co., 2000.