PUBLIC

openSAP
Generative AI at SAP

Unit 1

00:00:07 Welcome to this OpenSAP course on Generative AI with SAP. My name is Sean Kask, Chief
AI Strategy Officer at SAP
00:00:14 Artificial Intelligence, and I will be leading you through the course. As Philip shared in the
teaser, this course consists of five units.
00:00:23 There is a course assignment and a discussion forum where we encourage you to participate.

00:00:28 This is a nutshell course which is open to learners of all levels. So the first three units we will
introduce you to the basics
00:00:36 of artificial intelligence and how we went from AI to generative AI, as well as what we need to
do to adapt generative AI to business processes.
00:00:45 The last two units, we will look specifically at how we introduce generative AI into the SAP
portfolio.
00:00:51 So how you can extend SAP systems with generative AI and how we embed that natively into
our products.
00:00:58 Let's get started with Unit 1. In Unit 1, you will learn the definitions of intelligence and from that
what is
00:01:05 artificial intelligence. We'll learn about how humanity has tried to implement artificial
intelligence
00:01:11 over the past five or six decades, and we'll learn about the relationship between AI, machine
learning, deep learning,
00:01:20 leading us up to generative AI. So first, I would like to just take a moment to think,
00:01:29 what do we mean when we talk about intelligence? So if you say that person is very intelligent
or that looks like an
00:01:35 intelligent solution, what does it actually mean? And I like the simple definition here from Max
Tegmark.
00:01:42 He defined intelligence as the ability to accomplish complex goals. Jeff Hawkins,
neuroscientist, defines intelligence as the ability
00:01:53 to see patterns and predict outcomes based on previous experiences. So this incorporates
some aspect of learning and prediction.
00:02:03 One of the founders of AI research from Stanford, he defines intelligence as the quality that
enables an entity to function appropriately
00:02:12 and with foresight in its environment. This includes things that humans can do, like generating
language, creating music,
00:02:19 for example, playing challenging games, all these kinds of things. Quite simply, then, if we go
from intelligence to what is
00:02:30 artificial intelligence, artificial intelligence is simply intelligence exhibited by non-biological
systems, so by machines.
00:02:39 What do we mean by this? In humans and in animals, intelligence is taking place in our brains.
00:02:47 So, these are neurons on a substrate of biological tissue, okay, where inside of your brain.
00:02:54 In computers, we simply have digital neural networks that are exhibiting some kind of
intelligence on a substrate of,
00:03:00 you know, at the moment, silicon chips. And when we talk about artificial intelligence
00:03:07 in computers, for the most part, we're talking about something called narrow AI.
00:03:11 So, narrow AI is the ability to accomplish a narrow set of goals, otherwise called weak AI.
00:03:17 So, for example, we will create a model that is very good at doing one thing typically.
00:03:23 So, a model that is good at classifying cat pictures. That model typically cannot excel at
something like playing chess
00:03:31 or different kinds of goals and different kinds of intelligence.
00:03:38 There's also a term called artificial general intelligence, and that is basically human level
intelligence displayed by machines.
00:03:46 That's the ability to understand and learn any intellectual task that a human can, otherwise
called strong AI.
00:03:54 I bring this up because with generative AI, there's a lot of discussion, is that getting towards
artificial general intelligence?
00:04:02 It's not quite yet artificial and general intelligence, although as we'll see later in the course,
generative AI does have the ability
00:04:09 to generalize and access all different kinds of tasks. So, humanity's first attempt at scaling
artificial intelligence
00:04:19 was called symbolic AI. With symbolic AI, we set a human in front of a computer and we codify
knowledge.
00:04:27 So these can be rules, deterministic programming for anyone who's a programmer, these so-
called expert systems.
00:04:35 And we're basically, if you're solving a problem, you have the if-then-else type rules, which
works pretty well.
00:04:44 And that's actually one of the reasons why a company like SAP was so successful. So back in
the 80s and 90s, we'd put an expert in front of a computer
00:04:52 and say, how do you match invoices, for example, how do you close your books, how do you
run a warehouse, and encode this into the system.
00:05:00 And that worked pretty well. So that got us pretty far, and that will always serve us as
humanity.
00:05:07 And we still see this all around us today. For example, in semantic web, ontologies, automated
planning,
00:05:13 and scheduling systems. But, let's have a little thought experiment
00:05:19 here. How many rules would you need, so in a deterministic programming setting,
00:05:24 to describe something as simple as identifying a picture of a cat versus a dog in a computer?
00:05:31 So for human we might say, well, you know, a cat has whiskers perhaps, they're a little bit
longer than a dog, although that's not
00:05:37 always true. Maybe a cat's face is a little bit more squished, a little fluffier.
00:05:42 This is very difficult to explain to a computer, which is looking at pixels, zeros and ones, and
RBG settings inside of a piece of data.
00:05:51 And quickly that becomes nearly impossible, that's an impossible task, frankly, to describe
using rules and deterministic programs.
00:05:59 And the British philosopher Michael Plany put it very well, and he basically said, we know more
than we can tell.
00:06:07 So anyone who can ride a bike or kick a soccer ball, for example, you know how to do this.
00:06:14 If you were to try to describe very explicitly, because computers need explicit information,
Thank you very
00:06:20 much. How to do that, how do you kick a ball, what muscle in your leg is twitching,

2 / 28
00:06:26 at which force and which angle, and how do you plant your foot, and all these kind of things.
00:06:30 It quickly just becomes impossible. And this is on the nature of explicit knowledge, where you
can explicitly write
00:06:39 things down and describe it, versus tacit knowledge, which is more just knowing how to do it.
00:06:44 And so very quickly you see that we run into limits describing these complex rules and the
relationships between the inputs and outputs
00:06:52 in a computer system. And that's where artificial intelligence, modern approaches
00:06:59 to artificial intelligence with machine learning come along. So this is probabilistic, and rather
than explicitly programming these rules
00:07:07 into the system, we are showing examples to the computer. So we're showing it examples of
the input data and examples of the output
00:07:15 data and allowing it to learn by itself the relationships between these. The most popular and
the leading paradigm right now is called supervised learning.
00:07:27 And going back to the cat example, we'll simply take pictures of cats and pictures of dogs with
a label, and we'll show that to the computer.
00:07:36 And we'll apply some kind of machine learning algorithm. This is the learning algorithm in the
background.
00:07:41 There are many of these, different approaches have different pros and cons, there's not one.
00:07:46 And we're going to allow it to learn these relationships and train a model, simply by showing it
examples.
00:07:54 And then once we have this trained model, we can put this into a production system and we
can show it new examples that it hasn't seen before.
00:08:02 This is called inference, and it's making some kind of prediction on the outcome.
00:08:09 And if you think about it, the importance for software engineering and a company like SAP is
this allows us to extend the reach of computer systems.
00:08:19 Because, again, deterministic programming, symbolic AI, expert systems, that will always be
there.
00:08:26 That gets us pretty far. We still do that.
00:08:29 But artificial intelligence, especially machine learning, allows us to work with unstructured data,
like pictures, sound, text,
00:08:38 which is most of the data, not just data and tables. And it allows us to pick up these very
complex relationships that we have that
00:08:46 may be just simply too complicated to capture in rules. So there are roughly three approaches
to how we do artificial intelligence
00:08:58 and machine learning with statistics. The one that we just showed was called supervised
learning.
00:09:04 So we're simply training the model with labeled examples. So we're telling it, here's the input
data you get, dear machine,
00:09:11 and here's the output we want, and we've somehow labeled that. And the output is typically
some kind of regression,
00:09:17 so we have a continuous number that comes out of it, and most of it is frankly classification.
00:09:21 Today, I'm going to be talking about ticket classification, so does it fall into category 1, 2, 3
when we've shown it something?
00:09:27 An example could be ticket classification in a company. So you get service tickets that come
in, and then you classify the ticket.
00:09:33 Is this a ticket for HR, or for travel and expense, or an IT ticket, or a product ticket, or
whatever?
00:09:40 And that's something we can scale. Related is unsupervised learning.
00:09:46 So in unsupervised learning, the data set is not labeled. We're simply showing all of this data
to the computer,

3 / 28
00:09:53 and the computer is finding structures and patterns out of that data, and humans then have to
interpret that.
00:10:00 So for example, you know, something like fraud detection using clustering.
00:10:04 So we could cluster all of the transactions or data in a system based on a whole bunch of
different dimensions,
00:10:11 different attributes of the data. And we get clusters, and you might have one that sticks out,
that doesn't fall
00:10:17 into those clusters, that is statistically outside of the range.
00:10:20 And maybe we flag that and we say, huh, let's investigate that, could that be fraud?
00:10:27 And the last major paradigm in statistical approaches to AI is something called reinforcement
learning.
00:10:33 Not to be confused with retraining a supervised learning system. And in this we have an agent
that is in some environment
00:10:44 and it's taking an action in the environment. So we're not starting with label training data or
anything like that.
00:10:49 So it's basically observing where it is. It takes some kind of action.
00:10:53 And then based on that action, it's either rewarded or punished and it trains itself.
00:10:58 So it's learning what we call a policy. So the example of this is, you know, computers that can
play chess,
00:11:05 for example, or video games. So, you put them in.
00:11:08 AlphaGo is a big example, which is a very complex board game. You put them into the system.

00:11:16 They try all sorts of different things. In the beginning, they can't do anything.
00:11:19 And just over time, they kind of learn how to navigate the environment. Bringing this back
towards generative AI.
00:11:28 So, generative AI, and related to the foundation models, utilizes a certain kind of algorithm
called neural networks.
00:11:38 This is a attention-based transformer model. The labels, though, this is a derivative of
supervised
00:11:46 learning, but the labels are not labeled by people. The labels are somehow inherent in the
structure of the data.
00:11:55 So, for example, a large language model will predict the next word, so the label is the next
word in a sentence.
00:12:02 And that's something that we'll explore in more detail in the next unit. So, just before we go on,
I want to make a note on probabilistic thinking
00:12:13 and uncertainty and being comfortable with these kind of systems because most managers,
most people, especially enterprises,
00:12:20 grew up with these rules-based expert systems, deterministic programming. And now we come
to artificial intelligence,
00:12:26 which is actually probabilistic in nature. So, it's not saying this is a picture of a cat because it's
coming out of rules
00:12:33 in a table. It's saying this is a picture of a cat with a probability of,
00:12:37 or confidence score of, you know, 0.95 or something, you know. And, you know, some people
are just uncomfortable with that.
00:12:46 So I've left a picture up there on the screen for a few seconds for people to look at.
00:12:50 Don't rewind, please, okay, without rewinding. What did you see in that picture, and how
certain are you with what you saw?
00:13:02 Take a minute. Is this a concert that you saw, or is this a cotton harvester?
00:13:09 This picture has been around for a while on the internet. I've been doing this for years with big
groups of people.

4 / 28
00:13:17 Typically, 80% of the people say this is a cotton harvester, or sorry, this is a concert, excuse
me, when in fact it is a cotton harvester.
00:13:24 So, as amazing as human intelligence is, people are not 100% accurate, and all knowledge
has some level of uncertainty and probability in it.
00:13:35 People need to be comfortable with that in enterprise settings. In a computer, when we teach it
via machine learning,
00:13:42 it's similar. But it can scale and it can execute tasks, not always with 100% accuracy,
00:13:50 but typically much faster and even more accurately than people can do. And of course we
always stress in these probabilistic type systems,
00:13:58 we need people working together with artificial intelligence rather than just letting the system
do everything by itself.
00:14:05 Good, so to summarize these approaches and what we've discussed today. So intelligence is
just the ability to achieve complex goals.
00:14:14 And then artificial intelligence is simply intelligence exhibited by machines. This includes many
different approaches.
00:14:21 Artificial intelligence is a very, very big field. It's not just machine learning or deep learning.
00:14:26 This slide includes approaches like symbolic AI that we discussed all the way through to
cutting-edge stuff like hardware and neuromorphic computing
00:14:35 that is still evolving. Machine learning right now is probably the dominant paradigm
00:14:41 in artificial intelligence, which is leading to many of these advances.
00:14:45 So, simply, computers learn from examples without being explicitly programmed. There's many
different kinds of algorithms you can do this.
00:14:52 So, if we're using a regression, for example, or a logistic model, all the way down to deep
learning, these are still artificial intelligence.
00:15:02 And within that, the dominant paradigm, though, is deep learning. So deep learning is simply,
you know, one class of algorithms that we
00:15:12 use for statistical modeling and these are typically artificial neural networks.
00:15:17 And there's all different kinds of neural networks as well, like convolutional neural networks,
RNNs, and the newer class,
00:15:24 which has led to a lot of these advancements, transformers. Bringing this back to generative
AI, generative AI is,
00:15:32 again, a subfield of this deep learning, the machine learning, based on something called
foundation models and using a very specific kind
00:15:40 of neural network called an attention-based transformer model. And as we'll see in the next
unit, they exhibit all of these
00:15:48 interesting tasks like emergent capabilities that come out of the content. So we'll continue with
that in the next section and close this
00:15:57 and thank you very much.

5 / 28
Unit 2

00:00:06 Welcome to unit two of this open SAP course on generative AI with SAP. In this unit, we're
going to get an introduction into generative AI.
00:00:16 So you will learn what is behind all the news about AI and its impact on the economy.
00:00:21 We will define foundation models, large language models, and generative AI. We will
understand why generative AI is a new approach
00:00:31 to artificial intelligence, and we'll finish with some examples of generative AI.
00:00:38 So it's difficult to open a newspaper or look on the internet, open YouTube right now, without
hearing something
00:00:45 about generative AI. It is all over the news.
00:00:49 We see that the US Congress, for example, is holding hearings on generative AI. We see that
entire industries like publishing, video games,
00:00:56 are all being transformed with generative AI. We see economists discussing the impact of this
on society and the economy,
00:01:05 and we see companies like SAP making announcements around generative AI and AI in our
products.
00:01:14 What is going on? So I want people to understand that this hype that we hear, all the news
00:01:19 around generative AI, is real. So this will impact what we do and how we live, like past
technologies.
00:01:26 If we look at mobile telephones, if we look at the internet, if we look at personal computers,
these all profoundly impact society
00:01:33 and how we work. Generative AI will be no different.
00:01:37 There was a study that came out recently from McKinsey, among others, that said that, look,
artificial intelligence,
00:01:45 so what we learned previously, machine learning, was already set to add between 11 to 17
trillion dollars of value to the economy
00:01:54 by 2040. With Generative AI, they expect an additional $2.5 to $4.5
00:02:00 trillion added to global GDP. To put this in perspective, the gross domestic product, or GDP,
00:02:07 of Germany is about $4 trillion right now. And surprisingly, many companies have already
started their journey
00:02:15 with generative AI. So with more traditional artificial intelligence,
00:02:20 I would say between a fourth to a third to a half of companies had some artificial intelligence in
production.
00:02:27 And that's been around for a while. Generative AI, already we see around a third of companies

00:02:32 are using generative AI regularly in at least one business function. They all expect to increase
their investment in AI overall and most
00:02:43 companies have seen some increase in revenue or some positive impact on the company as a
result of adopting generative AI.
00:02:52 Let's look at definitions. What is generative AI?
00:02:55 We're going to start with foundation models. So foundation models are neural networks trained
on huge volumes of data
00:03:04 using a self-supervised learning objective that can be applied to many different tasks.
00:03:11 Let's look at an example of one kind of foundation model to understand what these are.
00:03:16 This is large language models. So large language models are simply foundation models
00:03:21 trained on text, including computer code. What do we mean by a self-supervised learning
objective?

6 / 28
00:03:29 Basically, we take a large amount of data. I mean, this can be billions, even trillions of words
that we've gotten
00:03:37 from certain sources. Many of these big companies have scraped this from the internet,
00:03:41 and the label in the data is the next word. So, people do this.
00:03:46 So, if we give an example like London is in blank, most people will think, oh, maybe England,
maybe United Kingdom, maybe Europe, okay, because remember,
00:04:00 knowledge is probabilistic, and the machine is learning this. If we give the machine now more
context, so we're giving more context
00:04:07 into the training, and I say two hours south of Toronto, London is in, and now people who know
a little bit of geography
00:04:16 won't think, okay, England, this is in Ontario or in Canada. And again, trained on billions, in
some cases even trillions of words.
00:04:29 Generative AI is an application of foundation models that can create new output, and this can
be text, images, sound, video, etc.,
00:04:39 based on simple user input called prompts. And not all foundation models are generative and
not all generative AI
00:04:46 is based on foundation models, but the majority of it is. Chat GPT, that most people are
familiar with,
00:04:55 is simply an application built on top of actually several large language models and foundation
models.
00:05:03 This is a model that has been optimized for dialog, so conversing back and forth. I will mention
that where these are headed very quickly, and probably the future
00:05:13 of foundation models in generative AI, is multimodal models. These are models that can work
with several forms of data,
00:05:21 for example converting text into images or looking at images and videos and describing that in
text.
00:05:31 So what are the capabilities of foundation models? So, as we described, we take huge
amounts of data, we train that, and we have some
00:05:40 big foundation model. And what can we do with that now?
00:05:43 What are the characteristics of foundation models? One is that they're huge, okay?
00:05:49 So, these are measured typically in billions or even trillions of what are called parameters.
00:05:55 So parameters are just the numbers behind the model that it's learned and trained on billions
or trillions of what are called tokens.
00:06:07 For all respects, a token can be simply a word, for example. It's a way that we've made this
data machine readable.
00:06:16 Important to understand is that foundation models are stateless. I think computer programmers
will get this right away,
00:06:23 but it means that they respond only to data that is being sent to the model, but they remain in
their original state, so they're not remembering or learning
00:06:33 from the interactions of what has been sent. If you've sent something to it, it sends you a
response.
00:06:38 It remembers nothing from what you've sent to it. And as we'll look in the next slide, the
performance and the capabilities
00:06:47 seem to, in general, scale as these models increase in size. Why is this a new approach to AI?

00:06:56 In Unit 1, we learned a little bit about supervised learning and narrow AI. We train a model to
do one thing very, very well specifically, and that's all it
00:07:08 can do. This takes typically a lot of labels.
00:07:12 We need labeled training data and high quality of data. With generative AI, we move from the
supervised learning

7 / 28
00:07:20 paradigm to a self-supervised learning paradigm. The label is somehow inherently in the data,
so that we've defined this
00:07:27 learning objective in a way that scales and kind of labels itself out of all this data we're feeding
it.
00:07:35 We've moved from predefined to emerging capabilities. So, for example, we train this model to
identify cat pictures,
00:07:43 that's all it can do, it's predefined, that's all we want it to do, to, as we'll see in a second,
models that simply have capabilities
00:07:52 coming out of it that no one expected. So they still discover capabilities that these models can
can can can do.
00:08:01 From single purpose to multi-scenario AI, so again one model trained to do one thing, to one
model that can do a whole bunch
00:08:08 of things and handle all different kinds of scenarios. And most models in the past, to be fair,
everything from Netflix
00:08:17 to Uber, were classifying artificial intelligence, so it's making some class coming out of it based
on supervised learning,
00:08:26 to AI that is generative and it can actually create new data out of the model.
00:08:32 What this means is that users can easily interact with the model without requiring a lot of
technical knowledge.
00:08:40 Today I'm going to talk about how to send data to a model like ChatGPT without necessarily
being a data scientist and without having to understand how
00:08:52 the training lifecycle works. We get these capabilities that come out of, for example, large
language models
00:09:00 which are the predominant paradigm today. We can do classic natural language processing
tasks like classification,
00:09:08 like extracting entities, extracting keywords out of data. We can summarize text.
00:09:14 We can search large volumes of data and provide answers to questions. We can create all
sorts of content like music, videos.
00:09:25 I even saw a rap song that was created by artificial intelligence recently, and it can even
generate computer code.
00:09:34 So, how did we get here? As I mentioned in Unit 1, there are many different kinds
00:09:40 of algorithmic approaches to AI. All sorts of different architectures in a neural network.
00:09:47 This is how we build the neural network. There was a paper published back in 2017, which
introduced
00:09:54 the attention-based transformer architecture. At the time, it seemed interesting, and companies

00:10:01 started to experiment with this. So, one of the first models produced out of that was GPT,
00:10:06 and it was very small, so it had around 100 million of these so-called parameters, and it had
some interesting properties.
00:10:15 And over the years, they started scaling this. So they used more and more training data.
00:10:20 The models got bigger and bigger and bigger. So GPT-2 was introduced in 2019 with 1.6
billion parameters,
00:10:27 again, very interesting. GPT-3, which is the model behind ChatGPT that everyone is so
familiar with,
00:10:35 was actually released in 2020, and that had 175 billion parameters. And suddenly we had
these emergent capabilities
00:10:44 coming out of the model. So it could help explain math problems, for example,
00:10:49 that it couldn't do in the past. It could do sentiment classification with just a little bit of input
00:10:56 and guidance. And that seems to be the trend.
00:11:00 So these models are getting bigger and bigger. There are families of models.

8 / 28
00:11:05 So all sorts of different companies now are creating foundation models. There are open source
versions available.
00:11:12 They tend to get bigger and bigger as well. GPT-4, which was released in the middle of this
year, which is now mostly running
00:11:19 under ChatGPT, is rumored to be at over one trillion parameters.
00:11:25 And as I said, it seems that the bigger the models get, more compute, more data, more
parameters, they tend to be
00:11:32 more accurate, you can adapt them to more things, and they simply just perform better.
00:11:38 Today, I'm going to talk about what's the trend, but there's also a trend towards smaller
models, benchmarking, so we compare the performance of these
00:11:47 on certain tasks, narrow tasks, as well as some of the bigger ones. What are some examples?

00:11:55 So, as I mentioned, generative AI processes prompts. A prompt is simply instructions or cues
or data that you're sending to the model,
00:12:07 and that's guiding it to produce the desired output. A simple example from a text-to-image
multimodal model.
00:12:17 I've here given the model a prompt, so I've simply typed in natural language and I've done
nothing else.
00:12:23 Hasso Plattner, the founder of SAP, giving a high five to a robot. And the model has produced
several images for me based on that prompt.
00:12:35 So here we see someone who looks like Hasso giving a high five to a robot.
00:12:39 In the other picture, it's kind of gotten it wrong, so I have to ease the robot, giving it a high five.

00:12:45 But I think you get the idea. Large language models are also good at creating text.
00:12:52 So for example, we can ask it to write an email introducing our logistics company called
Treasure Island Transports to a retail prospect.
00:13:01 So I want to sell to this person, and because we're Treasure Island Transports, I wanted to
write it in the style of a pirate.
00:13:09 So how a pirate would speak, and keep it under a certain number of words, and voila, there we
go.
00:13:16 We have an email. I'll deploy you there.
00:13:18 We'd be Treasure Island Transports, Masters of the Seven Seas, etc. introducing our company
to this prospect.
00:13:27 And they're also good at summarizing text and information and actually getting insights out of
large volumes of data.
00:13:35 So, for example, I've taken a thousand-word document here on SAP artificial intelligence, and
I've asked it to summarize something
00:13:42 out of that document. So I want to understand the key benefits that were listed there, it there,
00:13:47 and I want it to do it very concisely. So I'm asking the model, I'm instructing it to do this
00:13:51 in less than 100 words and to use bullet points for the answer. This is the prompt that I've sent
it, along with all of the text
00:13:58 from the document. And voila, there we go.
00:14:03 So it's describing the benefits of SAP Business AI. So, you know, we embed it, we can use it to
extend systems, etc.
00:14:17 They're also good at kind of classic NLP tasks. So we can use this to classify and extract text.

00:14:23 So, for example, let's say I want to do some sentiment analysis and understand the feedback
on the course, or I'm in a company and we
00:14:32 get service tickets and comments and I want to understand the sentiment in these, I'm asking it
to classify the sentence as negative, neutral,

9 / 28
00:14:42 or positive sentiment, and I want to extract the object of what they're talking about from that
short sentence.
00:14:50 And what it returns is, this is positive in sentiment because it said, I love your course.
00:14:57 And the object being referred to in the sentence is your course on OpenSAP, which is
otherwise known as Named Entity Recognition.
00:15:08 And, importantly for all the developers here, this seems to work very, very well with creating
code, either working code or code that needs
00:15:18 developers to make it ready for production, based on simple text descriptions.
00:15:23 So you can describe what you want a system to do, and what language you want that to write
it in, and it will produce code that can be used.
00:15:32 Good. So these straightforward yet impressive examples
00:15:37 are simply prompting generic generative I models. In the next unit, we'll learn how to adapt
these models to business context
00:15:47 and also a little bit about the limitations of generative AI models and how we overcome them.
00:15:53 So thank you very much.

10 / 28
Unit 3

00:00:06 Thank you for joining Unit 3 of this OpenSAP course, which covers how we adapt generative
AI to business context.
00:00:14 In the last unit, we saw some impressive examples of what generative AI is capable of.
00:00:19 In this unit, we will learn about some of the limitations of generative AI models, and we will
learn about methods
00:00:25 to address these limitations by grounding generative AI in context and adapting it to business
context.
00:00:31 And we will end with best practices when adapting generative AI. So generative AI's ability to
answer broad general knowledge questions is very
00:00:41 remarkable. We've seen, for example, it passed the bar exam in the United States,
00:00:46 which you need to become a lawyer and often score better than trained lawyers.
00:00:51 We've seen it able to pass the SAT exam in the United States, which is used for school
entrance exams, and it's even passed the medical licensing
00:01:00 exam, which is incredibly impressive for our language model.
00:01:04 And because of that, people may have the impression that it can tackle any kind of problem
out of the box.
00:01:11 However, the goal of this unit is to help users be aware of these limitations, keep their
expectations in check, and understand how to make Generative Eye
00:01:20 Enterprise ready so that it can handle business problems. Let's look at a few examples of
some of these limitations.
00:01:27 The first is called hallucination. So hallucination is where large language models can generate
plausible sounding yet
00:01:37 false answers. And related to that, sometimes if you ask it a question
00:01:42 in different languages, it might provide different facts back to you even.
00:01:47 For example, there's a research paper that I recommend everyone read about the challenges
and applications of large language models,
00:01:53 where the researchers show that they ask it basically to create some citations for papers, and
the first citation that it gives is correct.
00:02:05 It's a real citation. So that is a fact, it is a true answer.
00:02:08 The next citation doesn't exist at all. So the model has just completely invented an academic
citation that sounds real,
00:02:16 but if you look up, if you try to find it, the article doesn't exist. And the third example is the
correct article,
00:02:23 but it's somehow invented or hallucinated the authors of the paper. This makes it difficult to
trust the output of large-language models.
00:02:32 Relatedly, some lawyers in the United States in court use chat GPT and large-language
models to generate their case,
00:02:45 as well as referencing certain cases that they submitted to court. When the judge looked into
these, none of them actually existed.
00:02:53 The model had just hallucinated and invented these facts. They were fine and they learned a
valuable lesson.
00:03:00 Another challenge and limitation is that the knowledge of a model is basically frozen in time
from when the model was trained.
00:03:10 So it can be difficult to get up to date and specific knowledge into the model. For example, I
asked it who won the NBA preseason game last night between the L.A.
00:03:21 Lakers and the Golden State Warriors. So the model can't know this because the model was
trained in 2021

11 / 28
00:03:29 and of course at the time of recording we're in 2023. The first answer here is correct, so the
model actually tells me as a user,
00:03:38 I can't know that. I'm limited in my training data.
00:03:41 The second example goes ahead and hallucinates a fake score. So if you ask the same
question to the model many times, it keeps giving
00:03:49 you different scores which sound plausible but are not real. Today we're looking at a business
problem.
00:03:58 We can ask what libraries and SDKs are available with SCP AI Core or with some of our
products.
00:04:04 And this one, again, correctly tells me that its training ended in 2021 in September, so it can't
know that.
00:04:13 But then it goes ahead and gives me a plausible sounding answer that is actually wrong.
00:04:17 And the ground truth is that we do offer SDKs, API clients, SDKs, etc. in AI Core, which was
actually launched in October 12, 2021, a month
00:04:29 after the model was even trained. So it couldn't have known that, yet it tried to give me
00:04:35 a plausible sounding result. Also, users may be tempted to assume that LLMs work like
computers
00:04:43 or like calculators and can solve complex optimization problems, or forecasting, or things like
this, many of them are not so good at math,
00:04:54 so they have inconsistent math abilities and a limited notion of time. So, for example, I asked it
a question here that I think most humans would get
00:05:05 right. I said Jane was elected class president as the mother in 1973 when she was 12
00:05:11 years old. Her daughter Jill was elected class president in 2012 when she was 13 years
00:05:17 old. Who was older when she was elected class president?
00:05:21 Jane or Jill? And the correct answer, of course, is that when elected,
00:05:25 Jill was 13 and Jane was 12, therefore Jill was older. When we ask three of the biggest leading
large language models this question,
00:05:34 they all get it wrong. And even the reasoning was strange.
00:05:39 So it says Jane was 12 years old, well in 2012 Jill was 13 years old, therefore Jane was older.

00:05:45 So it just doesn't get it right, and I think astute experts following this do realize that there are
some methods to improve the performance of large-angles
00:05:57 models when dealing with math, which can be very, very impressive. Here I've asked it to
solve a quadratic equation, and I've done a little bit
00:06:05 of prompt engineering here, so I've asked it to show its work and to list the answer, and pretty
amazingly,
00:06:12 it does get the answer correct. Since we don't trust implicitly the output of a large language
model without checking
00:06:19 it, without verifying it, I do check online using a different tool and it did, in fact, find the correct
answer
00:06:26 to the quadratic equation. Good, so now that we are aware of some of the limitations of large
language
00:06:34 models and generative AI, let's look at some of the methods to ground and adapt this in
business context.
00:06:40 So grounding is defined as providing additional information and context that may not be
available as part of the foundation model's initially
00:06:48 training data. So grounding can help reduce hallucination, it can help provide
00:06:53 up-to-date results, and improve the performance of the model on specific tasks.
00:07:00 So remember to create a generative AI foundation model, we've essentially taken this huge
amount of data, probably a lot of it is scraped

12 / 28
00:07:08 from the internet, we have this huge model with billions, even trillions of parameters inside of it,
and we have some limitations.
00:07:15 And let's say we want to now apply that to a specific use case or business context.
00:07:20 What can we do? How do we ground and adapt these?
00:07:23 Today's one of the methods. So there are two big buckets for how we do this.
00:07:27 The first is providing task-specific instructions. So this is passing information in the prompt to
the model
00:07:36 to improve its performance. Because as we remember from the last unit, large language
models and generative AI
00:07:42 models are stateless. Today, I will present the first bucket, prompt engineering.
00:07:54 We will provide more information to describe the task. These include approaches like zero-
shot and few-shot learning.
00:08:03 We will cover each of these in more detail in the rest of this unit. The second approach is
called retrieval augmented generation, or RAG.
00:08:14 So here we allow the large-diameter model to access external information and we extend the
domain knowledge by injecting information
00:08:25 via various techniques such as embeddings, knowledge graph, search. And importantly, this
can actually provide references back to where it got
00:08:34 the information. And so we'll provide a source on why it's giving the output
00:08:39 that it's given. And the third one, which is very, very exciting, a little
00:08:44 more experimental, these are called orchestration tools. So, these are basically providing tools
to a large language model so that it
00:08:52 can perform various tasks, it can access external information, it can do things like calculations,
and we'll look into that in more detail.
00:09:01 The other big bucket is actually going back and retraining and fine-tuning the model itself.
00:09:08 So remember, we have this huge model full of many, many parameters inside of it, and we're
going to actually
00:09:14 retrain the model. So we're going to adjust those weights in the model.
00:09:19 Fine-tuning is the way that we do this, so it's a very good approach for improving the
performance on domain-specific tasks or through various instruction
00:09:31 tuning methods. Another one which we won't cover here is called reinforcement
00:09:37 learning with human feedback. This is how companies like OpenAI have fine-tuned chat GPT
to respond in
00:09:47 a certain way. So basically, all of the responses that it gives, people are looking at those
00:09:52 and then they're telling the model, yes, this was a good response or no, this wasn't.
00:09:56 And then it's taking all of that feedback and going back to adjust the model to change how it
performs over time.
00:10:05 And these methods are useful, but they're not really enough to make generative AI ready to
put into an enterprise context and production ready.
00:10:13 There are additional processes and governance around the model, of course, that are very
important.
00:10:18 One is AI ethics. Basically, don't put use cases into production that may cross
00:10:25 certain ethical boundaries or may be potentially harmful to humans in the first place, including
human in the loop in the design
00:10:33 process. So allowing humans to work together with AI.
00:10:37 So, for example, if the output of a large language model is not always consistent, or it may
have hallucinations,
00:10:44 it's important to have a human in the process checking this. There are technical ways we can
also do that, such as output validation

13 / 28
00:10:54 and cross-checks. If, for example, if the model is recommending
00:10:58 a certain purchase order number and we can actually check that against the master data, is
that in the right format,
00:11:04 does that even exist? Also, various testing and red teaming, where basically humans
00:11:11 try to break the model before it goes into production. And of course, having continuous
feedback and monitoring,
00:11:19 because over time sometimes the model behavior can change a little bit. We want to check
that, you know, in a month or in two months
00:11:26 or six months from now that it's still consistently giving the right kind of output.
00:11:33 Let's look at the first of these in detail, which is prompt engineering and where you should
always start when trying to build a generative AI use case.
00:11:42 So prompting, as we learned in the last module, is giving the model detailed information to
reliably produce the desired output.
00:11:52 And the easiest way to do that is what is simply called zero-shot learning. So we provide a
very simple prompt to the model and we get a result
00:12:01 out of it. Let's say, for example, we have a use case where we want a large language model
00:12:06 to create a job description for us. It's a job posting we're going to put on the internet.
00:12:12 People will read that. They will apply for the job.
00:12:15 A simple way would simply be write a job description for a support engineer, and you can try
this if you have access to large language models.
00:12:24 It does a pretty good job at that out of the box, but probably not perfect, not exactly what you
need.
00:12:31 Now what we can do is add more information. We can try a prompt engineering technique
called instruction following.
00:12:37 So we're writing the command tense and we're telling the model what it should do.
00:12:41 So for example, write a job description, keep it less than 300 words, include the following skills,
which we would list.
00:12:49 We give that as input. It then takes it and it will produce a job description that is a little bit
better.
00:12:55 And we could even take that a little bit further, and we could mix this, we could have
instructions with in-context learning,
00:13:02 including examples. In an SAP system, for example, we would have this user input,
00:13:07 this user prompt, where they're asking to create a job description.
00:13:11 And in the back end, we can inject more information into the prompt, like job title, for example.

00:13:18 We can give instructions that, for example, it should ensure diversity and fairness, use
subtitles for required skills,
00:13:25 make bullet points, etc. Today, I'm going to be talking about how to create a good job
description
00:13:35 for your company. We can provide those simply into the prompt.
00:13:40 Keep in mind, there's no free lunch. We're adding more and more information to get more
reliable, better outputs.
00:13:52 But the context window is getting bigger. This means we're sending more and more data, more
and more tokens
00:13:57 to the system. This typically increases the cost, because you're paying the more data
00:14:02 you're sending to these models typically, it's consuming more GPUs in the background, and it
can increase things like response
00:14:10 time. But this is how we get the system to behave better.
00:14:15 Another example of prompt engineering is with SAP Business Documents. So, for example,
we can use optical character recognition,

14 / 28
00:14:24 OCR, to extract text out of a scan, for example, or a PDF. We pass that on to a large language
model, and here we have the text
00:14:36 preserved with the 2D structure. We inject more information into the prompt,
00:14:42 so we want to extract certain fields, for example. We maybe give examples of those fields.
00:14:49 And what we get out of the large language model, because I've asked it using a command
here to extract the fields as adjacent,
00:14:57 is those fields extracted in a machine-readable way from the document that we can now post
into an ERP system or other business
00:15:06 application. And of course, you know, it doesn't always get it right.
00:15:12 We learned that machine learning and generative AI is always probabilistic, so we have some
kind of output validation.
00:15:17 Maybe the ID number is in the wrong format or it's done something that it shouldn't expect and
we can actually verify that in the back-end system.
00:15:27 The next adaptation approach that we'll look at is Retrieval Augmented Generation or RAG,
and we'll use the example of embeddings.
00:15:36 So first, what are embeddings when you see this? So embeddings are simply numerical
representations of data,
00:15:45 of information that retain the semantic and contextual meaning.
00:15:49 What does that mean? Let's take an example.
00:15:51 So let's say we have product documentation and we want to make that product documentation
accessible by large language models and generative AI so that users
00:16:01 can search it and they can interact with it and get insights and information out of that.
00:16:05 So we take the product documentation and we first encode it as embeddings using some kind
of machine learning algorithm, and then we store this
00:16:15 in a vector database. What is a vector?
00:16:18 So we have a sentence like, configure business processes by opening da-da-da, and we turn
that into a vector.
00:16:25 It's simply a big row of numbers, okay, that is understood by a machine but not really by
people.
00:16:33 And this vector, these numbers, capture the meaning in the text. So, for example, the word
apple has a different vector,
00:16:41 there's different numbers in the row. In the sentence, Apple makes phones, where it's the
subject of the sentence,
00:16:47 versus, I have an Apple iPhone 15. So those two vectors are probably similar, but slightly
different, versus
00:16:54 a completely different vector, Apple is an ingredient in pies. So this will be completely different.

00:17:01 And the beauty of this is, once we represent all of this business data as embeddings, it can be
easily searched and retrieved
00:17:07 using techniques like vector similarity scoring. So we can simply mathematically compare what
is the user asking for versus
00:17:15 what information are we looking to pull out of the system. So how would we apply that?
00:17:21 So we've already said that we have this business documents that should be searchable by the
users.
00:17:28 They should be able to ask questions to it, and what we've done is we've embedded this
knowledge and then we can retrieve
00:17:35 the relevant items given a question using large language models and other foundation models,
and we can then generate an answer
00:17:43 from the best results, again, using large language models. So, for example, a user might ask,
how can I adapt my business processes

15 / 28
00:17:51 for VAT calculation? That sentence is first turned into an embedding model.
00:17:57 We're then using these techniques to retrieve the relevant results out of the backend, which
are also vectorized as an embedding.
00:18:07 We take all this information and we inject it into the prompt, and give that back to the large-
diameter model, which is then going to generate an
00:18:14 answer for us. So, you know, you need to configure your business process and add a step
00:18:19 by whatever that is. And the beauty of this is that it grounds the prompt
00:18:27 with relevant information, so it's telling the prompt, create the answer, not from what you know,
what you've been
00:18:32 trained on, but create the answer from this text here. And it can even provide references back
to the source material that were used
00:18:40 to generate this. Today, I'm going to talk a little bit about how to get the answer
00:18:42 to the documentation and see where did I get this answer from, so that I know I can trust it.
00:18:49 The other way to provide these task-specific instructions is an emerging technique called
agents or orchestration tools.
00:19:01 So, these basically give large language models access to tools, like the ability to access an
API or various libraries that provide a reasoning structure
00:19:14 for the model. And we allow these so-called agents to work together and find an answer.
00:19:21 For example, the user can input some kind of task, asking it to do something. And the agent
will basically build a plan.
00:19:33 So it's going to retrieve, for example in this case, API specifications based on what the user's
input.
00:19:42 It will then generate a plan for how it's going to extract the information out of these.
00:19:47 And then it goes ahead and calls the API, collects the information, and presents the
information back to the user.
00:19:54 So again, I mean, this is pretty advanced, so you're basically giving the model a description of
an API.
00:20:03 It knows which one to use from that. It can write its own JSON file.
00:20:07 It can call the information and pass the information back to the users.
00:20:12 And the last one we'll cover is fine tuning. So, as I said, this is adjusting the existing foundation

00:20:20 model's parameters, the weights, to perform better at a specific task by retraining it on a new
dataset.
00:20:27 So we have our big pre-trained foundation model as a starting point, and now we're going to
retrain that using curated domain on our task-specific data,
00:20:39 typically labeled with examples. Thank you.
00:20:42 Today, I'm going to be talking about how to fine-tune a model. What we find is that can
improve the performance on certain tasks.
00:20:54 Most people, when they look at large-range models, tend immediately to think, I need to fine-
tune a model.
00:21:02 I think there are considerations about whether that's actually the first thing you should do or
not.
00:21:05 I think that's the last thing. The opportunity of fine-tuning is, again, it can improve the accuracy

00:21:12 and performance on certain kinds of tasks. So, for example, extracting information from
business
00:21:17 documents, we can fine-tune that on labeled examples of information we've extracted from
business documents,
00:21:24 and it will improve it at that specific task. And even these smaller and medium-sized models,

16 / 28
00:21:31 even 5 to like 50, 60, 70 billion parameter models can re-truexceed the performance of the
more costly, massive models
00:21:40 using this technique. However, this may reduce but not eliminate problems like hallucination.
00:21:48 The model may not necessarily forget the existing biases and information that it already had.
00:21:54 It does not reliably learn new factual information. So it's not going to learn new facts by doing
this.
00:21:59 For facts, you should turn to techniques like rags or orchestration tools. You need a lot of
curated training data to fine-tune.
00:22:09 It can be quite expensive and costly to retrain them, and currently it's difficult to generalize the
volume of required training data
00:22:17 that you need to achieve meaningful results, meaning this is a lot of experimentation.
00:22:23 Just as a point here, consider and test task-specific instructions, so just prompt engineering,
00:22:32 before jumping ahead and investing in fine-tuning models. So to wrap up, some
recommendations on adapting foundation models.
00:22:43 Do not trust generic AI models to answer factual questions, especially in the context of a
business, especially in a business context.
00:22:53 Instead, be aware of those limitations and apply these grounding and adaptation techniques to
ensure that the output is relevant and reliable.
00:23:04 Do not start immediately with fine-tuning, as we just discussed. That's a very special use case.

00:23:09 Instead, start with task-specific instructions like prompt engineering and retrieval augmented
generation.
00:23:16 Do not use the biggest model by default. So, these huge models like GPT-4, for example,
00:23:22 currently are very, very impressive. They work very, very well.
00:23:25 They can also be costly, and they may not actually be the best one to use for that use case.
00:23:32 So instead, test and adapt different generative AI models and optimize based on price, price
and performance.
00:23:40 And do not productize a technically functional generative AI use case without these
established processes, like design with human in the loop,
00:23:48 like having AI ethics in place. So ensure that you have proper governance and design for
generative AI.
00:23:55 So this concludes the first part of the course, which introduces the fundamentals of generative
AI.
00:24:01 In the next units, we will switch the focus on how SAP helps our customers to build and
leverage generative AI in their business processes.
00:24:10 Thank you.

17 / 28
Unit 4

00:00:07 Hi everyone, welcome to Unit 4, Extending SAP Applications with Genitive AI.
00:00:14 My name is Hadi Hares and I'm a Lead Product Manager at SAP AI. Let us look into what we
have for objective in Unit 4.
00:00:27 We want that you understand the Business AI approach, Genitive AI and its capabilities,
examples of Genitive AI Hub,
00:00:37 and the BTP Reference Architecture. So let's jump directly into it, what we have in the
Business AI approach.
00:00:47 It's full-packed, obviously, and we also cover this in much more detail in unit 5.
00:00:53 Bear with me, we will look into the AI Foundation piece, which here is in the middle, to
understand what we have in offering
00:01:02 for our embedded AI capabilities, but also for true. So what is BTP AI Foundation?
00:01:13 It's basically a starting point for SAP developers to build applications and extending them,
leveraging our vast AI offerings.
00:01:24 It includes everything you need from embedding business-ready AI into your application as
extensions, like ready-to-use AI services.
00:01:34 You could do AI workload management and newly introduced degenerative AI management
probabilities.
00:01:43 And to ensure it with your business data connectivity here, which is the middle part.
00:01:50 But what is important is obviously the access to foundation models, which is the lowest part.
00:01:56 For that we have SAP Build, which is an outlook into the future, and the Partner Build.
00:02:02 We will briefly touch on that. So what we have in offer for you as an AI developer or developer
in BTP,
00:02:12 it's first the same foundation we use for ourselves to embed AI into our own products.
00:02:19 Quite important. It's designed to have security and governance in mind,
00:02:28 and it's ground up at enterprise-grade levels that you would need and expect also from SAP
offerings.
00:02:38 And in a world of disruptive technologies like foundation models, you would obviously need
access to all different foundation models being hosted,
00:02:48 being remote, accessible, or later stage, as mentioned, sub-built or fine-tuned.
00:02:55 Besides the various possibilities you see here on the slide, what will be important for you to be
able to leverage grounding capabilities
00:03:05 via the vector engine here, and the data management obviously piece, to ensure that your
enterprise data give a proper context to your LLM
00:03:16 so that hallucinations are mitigated. And for this, I proudly can announce the next piece,
00:03:24 which is the Ingenitiv AI Hub. So what we have an offering for you for Ingenitiv AI Hub is three
things.
00:03:33 We have tool sets, which gives you purpose-built tools, tools and services for your
development phase.
00:03:42 We will briefly touch on the deeper level here in the next slide. But first, let's go over that.
00:03:49 So then we have in the middle part the access. And access is here that you can accelerate
outcomes with instant access to top rated
00:03:58 foundation models for multiple providers. So you don't need to spend time there and think
which is good foundation
00:04:05 models. We do that on our side.
00:04:08 And then, obviously, last piece is trust and control, where you can supercharge your mission-
critical processes

18 / 28
00:04:16 while preserving security and privacy, which is quite important in the business AI world, right?

00:04:22 And all this comes together to unlocking your business with Genitive AI. So let's have a deeper
look at what is in offer there.
00:04:33 So bear with me, the slide is full packed, but it gives you a good overview over what we will
provide today and also in the future.
00:04:42 So the Genitive AI Hub provides developer instant access to partner build foundation models,
as you see here in the middle in the gray box.
00:04:50 And we have there available for you instant or already available with Microsoft Azure AI
models plus Falcon 4DB.
00:05:01 And in the future, we will add more and more models over time.
00:05:05 Planned, we have Aleph Alpha plus Meta Llama 2. With this access, developers can submit a
prompt to multiple
00:05:16 LLMs to power mission-critical processes with complete control and transparency with built-in
features like prompt history,
00:05:26 as you see here in the prompt management section, and also deliver the enterprise-grade
security
00:05:31 and data privacy, as you see here in the trust and control layer.
00:05:36 You will be also able to leverage grounding capabilities, which is the lower part from business
data and context on SAP HANA Cloud,
00:05:44 to base grounding on enterprise data for proper context to your use. And for that, we're
leveraging the newly announced HANA Cloud Vector Store
00:05:58 capabilities. So let's jump then to the next slide.
00:06:04 We have here a deep dive now to the toolset section. And what we have here in place is three
things that you can interact with Genitive
00:06:11 AI Hub. One is the Playground, that you will see in a short demo
00:06:15 I have brought with me, which is basically a way to explore for you with different models, with
different capabilities,
00:06:23 to our generative AI Hub. Yeah, basically as the name suggests, it's Playground, so you can
use that
00:06:29 to experiment, do tests, and so on. Then we have the client way, which is a normal way to
productively use
00:06:37 Genitive AI, hub in your application against the API that we provide. And there you can have
basically all kinds of clients there, right?
00:06:47 And what we also provide here are more support like with libraries and SDKs so that you can
do it also in runtimes like Python for yourself.
00:06:57 So, what the tool set provides you then is basically the experimentation phase for your prompt
engineering phase and later stage, obviously,
00:07:08 then enhancing this in your application with generative AI. Now, jumping to the access part, we
have access to different things.
00:07:19 One really important one is the partner build, LLMs or foundation models. As mentioned, we
have Azure OpenAI, Falcon 4DB and later
00:07:29 the other upcoming models. And what does this give you?
00:07:34 It's basically first a connection to easily and seamlessly to add any supported LLM into your
workflow and to your process.
00:07:46 Then acceleration for you, because it's thanks to SAP managed legal and commercial
framework, you don't need to spend time on that
00:07:56 and you just can directly instant have access on it and use it. And then at a later stage, you
also might be thinking of switching

19 / 28
00:08:05 between models to find and upgrade the best suited technology for your need and for your use
case.
00:08:12 So that's also a way then to ensure that Genitive AI Hub is also works for you for the next time,
not only short time.
00:08:25 Let's then jump into the trust and control layer, which is the foundation piece, let's say, for
doing this and to do it in a safe way.
00:08:36 One is the security data privacy way here, which is ensuring your applications are native and
embedded with SAP security, so that you can rest assured that all SAP
00:08:50 standards are there. Then we have the control thing, where we have model responses
00:08:59 with grounding, where you can control your LLM in a way that you bring context to it, and then
the response that you get out is
00:09:08 really grounded and no hallucinations there. And with this, you can then extend your solution
00:09:16 with advanced capabilities. Also looking now an outlook into the future here is supports
00:09:23 for agents, plugins, content moderation so that you can really think more about your BTP
application and less about the things
00:09:32 that, the integration piece that you might need, right. And one last but not least point here is
the multi-tenancy functionality,
00:09:41 so that when you build a BTP application that is multi-tenant aware, we support you here also.

00:09:49 So now let's jump into a short demo that I brought with me. So what we see here is the prompt
management section
00:09:55 in Genitive AI Hub, and we have couple of prompts saved from yesterday, and now we want to
work on where we left off.
00:10:02 We select the collection grounding because that's where we left off yesterday and now going
into what we saved yesterday and see the message and the response
00:10:12 and the versions and now going into the open in editor part. And you see here, we have the
message.
00:10:18 You can adjust this. Let me write this quickly.
00:10:26 And now I'm selecting still Falcon 40b and then I'm giving the grounding capabilities here with
the graph documentation for HANA.
00:10:37 And now I'm running this against Falcon 40B and after a quick time we get the response back
from the Falcon 40B LLM and that's the response.
00:10:49 But I'm not super happy with it because I'm a SAP HANA developer and I want to have a
working example also for that.
00:10:59 And what I'm doing now is wanting to change first the model where I know it works better from
my experience.
00:11:06 And for this, I will now go to the section, model section, and choose a different model.
00:11:10 From past experience, I know that GPT-4 works really great, provided examples, and now I'm
choosing and running it again.
00:11:19 And after a quick time, we see then the response for this. And you see here, it's a much better
response for my work.
00:11:29 I get directly an example in GraphScript and I can now go to my HANA IDE and work on it and
check if that works for me.
00:11:41 And with that, you see the capabilities in the playground which we ship. So now that we saw a
quick demo of the playground,
00:11:52 let's see how this works in the real application and in production. For this, I brought with me the
BTP reference architecture for Genitive AI.
00:12:02 And this presents you a multi-tenant application crafted for a potential SAP partner or
customer tailored for BTP.

20 / 28
00:12:11 So now imagine if you look into this picture here, we have a scenario of a comprehensive
SaaS solution to enhance customer support, let's say,
00:12:21 utilizing advanced email insights and automation. The system analyzes incoming mails using
large language models and a length
00:12:32 chain, as you see in the middle here, to offer core insights such as categorization, sentiment
analysis, or urgent assessment.
00:12:43 So one of the innovative features here is involving using anonymization here because you
know with emails we have personalized data so we
00:12:53 want to anonymize that and the email embedding as you saw here in the HANA cloud vector
similarity search persistency where you have similar historic emails
00:13:05 and are aiding you in understanding how similar requests were handed previously and then
foster this as input for your customer service.
00:13:18 Furthermore, you can picture how a potential response for customer could look like.
00:13:23 You could generate a response with the influencing factor of the grounding from the HANA
cloud and providing a much better experience
00:13:34 for your customers here in the customer support example. Plus, with the flexibility to connect
with SAP
00:13:44 systems like S4HANA, you add even enterprise dimension to it, allowing seamless integration
of process and data to it.
00:13:56 So, when we look into this picture, we see a lot of things are possible from Playground, as you
see also here in the picture,
00:14:05 plus to productive using Genitive AI with our Genitive AI Hub, and giving you as a developer a
lot of possibilities to drive this really
00:14:16 interesting times in AI. With that, I would conclude, and thanks a lot for listening.

21 / 28
Unit 5

00:00:06 Welcome to Unit 5. We're going to look in the next roughly 20 minutes at business use cases
00:00:13 where generative AI is applied in an SAP context. My name is Jana Richter and I'm Vise
President for Artificial Intelligence,
00:00:21 Product Success at SAP. Let's move into the unit itself.
00:00:27 What are we going to do? Well, I will give you a brief overview of the SAP Business AI portfolio

00:00:33 and the main goals where we apply artificial intelligence. Then we will dive into a couple of use
cases, just a selection for this chapter.
00:00:43 And, of course, towards the end, I would like to bring some key takeaways across.
00:00:49 Let's get started. Well, if we look into the SAP Business AI product portfolio,
00:00:55 when applying AI to SAP business solutions and business processes, we typically have three
main areas where AI is applied.
00:01:04 First of all, we drive business value by automating tasks. So think of all the tedious repetitive
tasks
00:01:12 that your teams are actually drowned in and typically the amount of work is ever increasing.
00:01:18 Well, we aim to use here AI and machine learning algorithms to complete repetitive tasks in a
higher automated manner
00:01:27 so that your employees, your colleagues can really focus on the complex business problems in
working on innovation and and with your customers,
00:01:36 and having a higher level of automation in place. Well, when people work with systems and
have the time to dive into deeper
00:01:44 complex problems, of course you would like to interact with the system in a very natural human
way, even for things that you might not do on
00:01:52 a daily basis. And here AI really gives us the possibilities,
00:01:57 specifically with the whole leapfrogs around generative AI, to facilitate a more human
interaction with machines.
00:02:06 So it's not that you as a human have to think, well, where do I find what in SAP solutions, but
that you can interact in natural
00:02:14 language and the SAP processes and systems react accordingly and guide you through the
process.
00:02:21 And lastly, of course, whenever you take decisions and work on complex problems, the
amount of data that is a basis for your decision is ever
00:02:30 increasing. Just imagine supply chains and all the factors that actually influence
00:02:37 your supply chain, that influence the demand of your customer base.
00:02:41 If you want to do daily demand planning, there are so many factors that you have to take into
account that AI can really help to give you
00:02:49 insights into this database, to do more efficient planning activities, to get proactive, alerting
recommendations on what to do next and what to focus on.
00:03:01 So those are the three main areas where AI provides business value. And you have seen this
picture in the previous chapter already.
00:03:10 Well, SAP applies AI really to business solutions and thus, what is super important is that
whatever we do is actually relevant in the business
00:03:19 context and is relevant to the business process. Secondly, of course, reliability is key.
00:03:26 So we have to make sure that deeply embedded into the SAP solutions, the right data is taken
into account, the right integration is happening,
00:03:35 and that of course that you as users can see where AI is applied and where you get
recommendations, which algorithms give you some logic and some hints.

22 / 28
00:03:46 And thirdly, of course, responsibility is also key, the whole ethical aspects, the whole data
compliance to use data in a GDPR compliant
00:03:55 format and to really avoid any manifestation of bias by implementing AI. And for that we have a
super strong ethical framework and processes
00:04:04 in place where we apply AI as any other technology into the SAP portfolio. Going a level
deeper, you have heard about Juul already.
00:04:14 Juul is the new co-pilot that really helps you to understand your business and that grows
across the entire business portfolio.
00:04:22 And I will show you in a few minutes a quick demo how this looks like. And this really spans all
the SAP solutions, cloud,
00:04:30 ERP, human capital management, span management, where we embed dual, but also other
AI capabilities along the lines of automation,
00:04:39 natural user experience, and giving additional insights, optimizations and predictions.
00:04:45 Across the entire portfolio, those capabilities are included. But of course, on the technology
side, we do have reusable components,
00:04:53 centralized architectures in place that you've seen again in unit four of this course, a bit more
in depth and that we refer to as SAP Business
00:05:03 Technology Platform and the AI Foundation. And we do have a variety of AI partnerships with
leading generative AI
00:05:11 model providers to really leverage the latest and greatest large language models and GenAI
algorithms in the different business contexts.
00:05:21 Next to that, of course, we do have deep learning models, HANA database specific algorithms
and an evolution towards really SAP
00:05:30 optimized foundation models as part of that overall AI foundation stack in the overall planning.
00:05:39 What I like to emphasize is that of course the topic AI goes along with SAP since a very long
time with the whole buzzword
00:05:47 and the whole keyword of intelligent enterprise. We really embed this since a couple of years
00:05:52 already across the entire portfolio and that's why I want to spend a minute in showcasing a few
of the already existing use cases.
00:06:01 We can see in the place of cloud ERP that solutions like SAP Cash Application, where we
match incoming payments with invoices, help customers
00:06:12 all around the globe to really automate their business to a higher level. And we see individual
customers who can reach up to 1.5 million savings
00:06:23 every year by hiring this higher level of automation. In the human capital management space,
within success factors,
00:06:30 we do have learning recommendations in place, so where learners get recommendations,
what really helps them to elevate
00:06:37 their skill profile. And here we see 4.2 million learners across all the enterprises who every
month
00:06:44 work with those learning recommendations. I want to skip the middle example because I will
come back to that
00:06:51 in a second. In the customer relationship management space, I like to refer to a use case
00:06:56 that probably you don't see as a user, but where AI is of course super relevant. Looking at
customer data cloud and intrusion detection,
00:07:04 where we have AI algorithms behind the scenes that make sure, well, who's accessing the
system and is this a valid attempt?
00:07:11 Or do we need two factor authentication or even block in this authentication? And here we see
at round about 2 billion of events that are scored every month
00:07:21 alone in the customer data cloud. And where you can imagine that those models need to be
adapted frequently

23 / 28
00:07:27 and have to have super quick response times in milliseconds. In business technology platform
we see more than 10,000 interfaces
00:07:35 and scenarios provided in an intelligent manner through the BTP integration advisor.
00:07:42 Let me come back to the middle part. One use case that really excites me a lot is all around
document automation
00:07:49 and one that many of you might be familiar with is in the spend management and business
network place where where with Concur invoice management,
00:07:57 we can have up to 440 million euros annual savings if we look at the overall user base that
now doesn't have to extract all
00:08:07 the relevant information from invoices manually anymore, but by scanning the documents and
having an AI algorithm that extracts header data and line items
00:08:18 automatically from your invoices and your users just need to check whether all the entries are
valid and then proceed and progress.
00:08:26 And actually, if you think about that use case, this is super interesting because there's a lot of
document-based processes
00:08:33 across the entire SAP portfolio. And I've just shown one example here.
00:08:39 Well, document information extraction, for those of you who are already familiar with those use
cases and these capabilities, exists since a while
00:08:48 already in its base form where we automate core business documents like invoices, like
payment advices, like sales orders, really for those
00:09:00 selected document types and where you can realize use cases yourself with BTP and where
we have embedded this across the entire portfolio.
00:09:08 But now what you see on the right side, newly we supercharge our business document
processing with generative AI in a premium flavor.
00:09:18 And what this allows us by using large language models behind the scenes is that we can
support a large variety of business document types out of the box
00:09:28 with most languages supported end-to-end. end.
00:09:30 So that gives you up to 40 languages for virtually any kind of document. And to give you an
idea what this means, which kind of use cases we are realizing
00:09:41 now with this technology, let me show you one example in transportation management.
00:09:47 Imagine, well, if trucks are arriving at your destination with goods from your suppliers and you
actually need them as part
00:09:56 of your production process. We see in many companies, well, they are still
00:10:00 tedious and cumbersome steps to take delivery notes with those truck deliveries, take this
information and bring it into the system before this
00:10:08 can be processed. And now with document information extraction and generative AI included,

00:10:14 we can automatically extract entities from those delivery notes. And of course, it's not just a
pure extraction,
00:10:22 but it's really that the SAP transportation management application is adjusted to have that
automatic scanning,
00:10:30 extraction of entities, and then subsequent processing steps automated to a higher degree.
00:10:36 Which means that you get automatic checks with greater accuracy and minimized errors,
because you don't have humans who are under stress with a large list
00:10:44 of tutorialists that are waiting to be processed, but you have a process that is running in the
background and can take care of processing
00:10:51 and extracting all the information that is needed. And that can of course significantly reduce
the hours
00:10:57 spent on manual checks and waiting times and ultimately improve your operational efficiency.

24 / 28
00:11:04 You can try out Document Information Extraction Premium as part of BTP free trial already
today.
00:11:13 I want to show you some more examples in a quite different space. So where can we also
apply generative AI and the large language model capabilities.
00:11:23 I mentioned beforehand it's about interacting with the systems in a human manner, in natural
language.
00:11:31 And a pretty amazing feature is the Just Ask capability as part of SAP Analytics Cloud.
00:11:37 So imagine you're looking at your dashboards and you would like to understand out of that
data the relevant information.
00:11:44 Well, if you can use natural language now, you can faster get insights because you can just
type a question in natural language and get then
00:11:52 the values derived from the business semantics. And of course we need here trusted large
language models that help us to get
00:12:00 the relevant information, know the business context in a secure manner.
00:12:05 To give you an idea how this looks like, let me show you a quick demo in this context.
00:12:12 We can get started and look into SAP Analytics Cloud and the Just Ask capabilities live here.
00:12:20 So you can enter in a natural way which product has the lowest cost and the lowest sales cost
and really ask this in a very
00:12:30 natural manner. The large language model behind the scenes will understand what you're
looking
00:12:34 for and give you the results. And what you can see is that automatically to this chart and to this
data
00:12:41 we apply the right filters, the right sorting that we're interested in the bottom result when
looking at the sales cost of our products.
00:12:51 This is just a quick example to show you how you can now interact in a natural language and
very quickly get the insights that you are interested in.
00:13:01 Let me continue along those lines and in a use case that is even broader, illustrating how you
can interact in natural language with your SAP systems
00:13:11 and even across the entire SAP portfolio scope. Juul is our digital assistant, our co-pilot,
00:13:20 that guides you through the day and that really works as a co-pilot that truly understands your
business and your business context in the SAP
00:13:28 universe. You want to benefit from the relevant insights as part of the SAP systems
00:13:34 and quickly want to get answers on demand about the things that are relevant now. And this of
course helps you to achieve better outcomes when you
00:13:44 create content and code because you can quickly and instantly get help with whatever you
want to do.
00:13:50 Still this means that of course you as a user stay in control, take the decision making, have
control over the data privacy
00:13:57 and get assistance. Let me show you a demo on how this is going to evolve and how this is
planned
00:14:05 to look like. For this, let me log into a system and go into SAP Start as my daily start
00:14:13 page and then interact with Jewel. And you can see here I again look at information where I'm
interested
00:14:20 in a chart similar to what we've seen in SAP Analytics Cloud but can get this embedded into
our copilot tool.
00:14:28 In addition, you can ask natural language questions to the knowledge base and get information
like where is my relevant cities and then even
00:14:37 do something freestyle here when saying, well, if I want to open up a pop-up store, which
name could it have?

25 / 28
00:14:44 The real SAP connection is coming now when you say, well, if I want to open this pop-up
store, actually I need some help in hiring a store manager.
00:14:53 So, in opening a position and creating a relevant job posting. And also here, Juul helps me and
guides me through this process in a very
00:15:02 natural language manner and in a very intuitive manner, helping me to take an existing
employee, copy from that profile,
00:15:10 of course make a couple of changes, like for example, if you want this to be in a different
location, we can pick and adapt that we
00:15:19 want to have that new position in Geneva instead of Boston. And then move ahead in really
creating the job posting within the system.
00:15:30 Moreover, if you really want to create a job posting out there that applicants can look at, you
need a good description.
00:15:37 And also here, generative AI and large language models can help you to generate some
proposals for texts that you then as a manager, of course,
00:15:46 can adapt. But you do have a start point already that is connected to your SAP
SuccessFactors
00:15:51 systems and that is really a good start point to adapt. In this demo here, let's not do the
adaptation,
00:15:59 but let's rather proceed and create the position and the posting and trigger the according
workflows.
00:16:07 So typically, of course, you do have a approval workflow behind the scenes that we're going to
trigger with this process.
00:16:14 Having done so, the interesting piece is that you can also then navigate right away into the
system of record like SAP SuccessFactors and look into
00:16:24 the workflow in its current status and all the relevant details that have been filled by Jewel
automatically.
00:16:31 So you have right away the hook from JOOL into the system of record. If you're interested in
what actually is happening behind the scenes,
00:16:44 let me dive a level deeper in telling you, well, how does all this connect? You've seen in the
previous chapter already the generative AI hub
00:16:53 and actually partner models like from Microsoft, IBM, Aleph Alpha that are provided through
the generative AI hub
00:17:02 with the highest level of trust. Of course, those environments and the same logic
00:17:08 is applied to Jewel and the large language models there. And we do have a deep integration
with SAP Start as the central entry point
00:17:16 and the relevant business applications that you're regularly working with. Additionally, of
course, in order to make that really relevant in
00:17:24 the business context and give the most suitable results, this needs to be grounded on your
own business data,
00:17:32 on the own systems connected with HANA Cloud, Knowledge Graph, Data Sphere, and
connected to SAP Foundation models that are planned to be provided
00:17:41 in the future and customer data and third-party plugins. I've just shown you some of the
examples where we can apply generative AI
00:17:53 to business processes. There's of course a lot more that we can do and that we can't cover in
20
00:17:58 minutes of this chapter from summarizing text. So if you have a large interaction, for example,

00:18:05 around customer communication, summarizing the relevant content and extracting action
items is something that is needed, for example,
00:18:14 in an SAP CX pod for you. We've seen that we can have writing assistants like the one that
just helped

26 / 28
00:18:20 us to create job descriptions. But you can think that even further in various contexts,
00:18:25 either in success factors itself, when you want to create interview questions or goals, but also
across the entire portfolio,
00:18:33 you might want to create out of metadata and bullets, really full speaking texts in a variety of
languages, where large language models can help you
00:18:42 to do so. We've seen the answer, or the example of question
00:18:46 answering with Juul and the information retrieval capabilities. We've seen this in JustAsk as
part of SAP Analytics Cloud,
00:18:55 but there's a whole chapter of opportunities as well around generating coding, for example as
part of the BTP portfolio for automatic code and data
00:19:05 model generation. Thus, I would just like to show you that we've just seen a very small subset
00:19:13 of the planned use cases across the entire portfolio. SAP is introducing artificial intelligence
and generative AI, and you can see here
00:19:23 that this is actually a mixture. Of course, AI is a broad field, generative AI really opens up a
whole set
00:19:30 of new possibilities, but we do see more classical approaches like forecasting algorithms,
simulations, deep learning models remaining relevant,
00:19:39 of course, as well. Four scenarios that just have been provided this year already.
00:19:45 With that, I would like to wrap up the last chapter. I hope that you took away that generative AI
will impact your work
00:19:54 and your industry. So it's great that you already prepare yourself and your organization.
00:20:00 I want to encourage you to look at all the use cases that SAP is providing that are already
available, that are part of the roadmap, and make use of them.
00:20:10 Of course, also leverage business technology platform for building custom generative AI
extensions to SAP and business
00:20:18 applications. So, get started today, and to proceed further, good luck
00:20:25 with the exams and the tests after the session. Thanks a million, and bye.

27 / 28
© 2023 SAP SE or an SAP affiliate company. All rights reserved.
See Legal Notice on www.sap.com/legal-notice for use terms,
disclaimers, disclosures, or restrictions related to SAP Materials
for general audiences.