<b>In this video, I will show you how to</b> <b>automate the

process of</b> <b>keeping your document stores</b> <b>up to

date. Document stores are a great</b> <b>solution for managing
all</b> <b>your data sources in a single</b> <b>container.
This way, you can easily add</b> <b>or remove information</b>
<b>from the knowledge base,</b> <b>without having to change the
AI chat flow</b> <b>itself. But one thing you</b> <b>guys
have been asking me in the</b> <b>comments is how to refresh the
document</b> <b>store automatically</b> <b>when a document
loader uses</b> <b>dynamic data. For instance, you might be</b>
<b>scraping information from a</b> <b>website that changes
often,</b> <b>or you might be pulling data from a</b>
<b>database, like Airtable. In those</b> <b>situations, you
might</b> <b>want the document store to be refreshed</b>
<b>as soon as the data is changed or like</b> <b>every hour, once
a day,</b> <b>etc. I really want to give you the best</b>
<b>value possible in this</b> <b>video, so I thought it
would</b> <b>be fun to build a real world application</b>
<b>together. Here, we have an</b> <b>AI powered business
directory.</b> <b>This is similar to the Yellow Pages, but</b>
<b>instead of searching</b> <b>businesses using
keywords,</b> <b>we can simply have a natural conversation</b>
<b>with an AI chatbot. The</b> <b>business listings are
actually</b> <b>just stored in an Airtable database, but</b>
<b>this could be any type of</b> <b>data source, like you
could</b> <b>be scraping data from a website or</b>
<b>pulling from a database. Now,</b> <b>the issue is if we add a
new</b> <b>listing to this database, for example,</b>
<b>let's add Oak End Barrel, which is a</b> <b>fictitious
restaurant,</b> <b>we'll just enter some address over here,</b>
<b>like so. Let's just enter</b> <b>some sort of phone
number,</b> <b>let's enter a description like "a</b>
<b>restaurant that sells steaks</b> <b>and sushi". Now, I've
committed</b> <b>the changes to this table, and if we go</b>
<b>back to our chatbot and we</b> <b>ask something like "I'm
looking</b> <b>for a sushi restaurant in Johannesburg",</b>
<b>we simply get a message</b> <b>saying that it can't find
that</b> <b>latest entry. And that brings us to the</b>
<b>challenge that you guys</b> <b>have been facing with
document</b> <b>stores. Whenever the data source changes,</b>
<b>you manually have to</b> <b>execute the following
steps.</b> <b>And in this video, we will automate this</b>
<b>process, but let me show you how this</b> <b>works. First, you
have</b> <b>to go to Actions, click on Preview and</b>
<b>Process, then we need to</b> <b>click on Process to load the
data</b> <b>and chunk it, and now we get 17 documents</b>
<b>back, which means that the</b> <b>latest entry has been
pulled</b> <b>in. Now we have to upsell this data into</b>
<b>the vector store by going</b> <b>to Upset Config, and
then</b> <b>we need to click on Upset, and now we can</b>
<b>finally see that that one</b> <b>record has been added and
16</b> <b>records were skipped. So if we go back to</b>
<b>our chatbot now, and we ask the same</b> <b>question, we
should get</b> <b>the correct response back, and we do.</b>
<b>It's very possible that you</b> <b>might have a situation
where</b> <b>you want to manually refresh the data,</b>
<b>but in many instances, like</b> <b>with this business
directory,</b> <b>we want the data to be fresh maybe once a</b>
<b>day, or every few hours.</b> <b>So let's have a look
at</b> <b>automating this process. You are going to</b>
<b>learn a lot, because in</b> <b>this video, I will show you
how</b> <b>to set up the chat flow, the document</b>
<b>store, and we will use Airtable as our</b> <b>database in this
video.</b> <b>We will also use a workflow automation</b>
<b>tool called n8n to run</b> <b>this automation on our
behalf,</b> <b>and you will also learn how to use the</b>
<b>new Flow-wise API endpoints</b> <b>for updating the document
store.</b> <b>First, let's create our chat flow. I'm</b>
<b>using a cloud instance of</b> <b>Flow-wise, but you're more
than</b> <b>welcome to use a local instance to follow</b>
<b>along with this tutorial.</b> <b>On your local machine,
simply</b> <b>open up the command prompt or terminal</b>
<b>and enter npx flow-wise</b> <b>start. If this is your first
time</b> <b>running Flow-wise, then you will be</b>
<b>prompted to install</b> <b>Flow-wise, so simply enter y and
press</b> <b>enter, and afterwards, you should see a</b>
<b>message like this, and after that, you</b> <b>can access Flow-
wise</b> <b>by going to localhost 3000, and you</b>
<b>should see a dashboard like this. Now</b> <b>let's create the
chat</b> <b>flow by clicking on add new. Let's give</b>
<b>it a name like</b> <b>ai-business-directory, and let's save
this.</b> <b>Let's add a new node. Let's go to agents,</b>
<b>and let's add the tool agent</b> <b>node. Let's start by
adding an</b> <b>AI model to the canvas. So under add</b>
<b>nodes, let's go to chat</b> <b>models, and I'll be using the
chat</b> <b>open AI model. Now I've got plenty of</b>
<b>videos showing you how to</b> <b>use chat olama to run
models</b> <b>locally for free on your own machine, or</b>
<b>you could use grokchat which is also</b> <b>free, and you'll
also</b> <b>find the video on my channel for using</b>
<b>grokchat. So let's add our</b> <b>chat model node. Let's
attach</b> <b>this to our agent. Let's select our</b>
<b>credentials. Let's select our</b> <b>model as gpt40-mini, and
I'm</b> <b>going to lower the temperature to 0.2, as</b>
<b>I want our assistant to</b> <b>strictly follow our
prompt</b> <b>instructions, which we will set up in a</b>
<b>minute as well. Let's also add a memory</b> <b>node. So under
nodes</b> <b>let's go to memory, and let's simply add</b>
<b>this buffer memory node,</b> <b>and let's attach it to
our</b> <b>agent as well. Now finally let's add a</b>
<b>tool node. So under add nodes, scroll all</b> <b>the way down
to tools,</b> <b>and add the retriever tool, and let's</b>
<b>attach this tool to our agent</b> <b>as well. This retriever
tool</b> <b>will be responsible for connecting our</b>
<b>knowledge base to our AI agent. For the</b> <b>tool name,
let's call</b> <b>it business listings, and for the</b>
<b>description let's enter use this tool to</b> <b>find
information about</b> <b>businesses. Now optionally we can
also</b> <b>enable this toggle which will</b> <b>return the
source documents in</b> <b>the chat window. Now because this
process</b> <b>will be customer facing, I</b> <b>don't want
the user to see my</b> <b>data sources, so I'll leave this</b>
<b>disabled. Then lastly let's add a</b> <b>retriever. So under
vector stores</b> <b>let's add the document store node, and</b>
<b>let's attach this to our retriever tool.</b> <b>Great! Now we
simply</b> <b>need to select our document store which</b>
<b>we haven't created yet, but</b> <b>before we do that I just
want to</b> <b>go back to our tool agent and change the</b>
<b>system prompt. I'm going to</b> <b>clear this, but instead
I'll</b> <b>paste in the following prompt. Let's</b>
<b>expand this. You are a helpful assistant</b> <b>for a business
listing</b> <b>service called AI Business Directory.</b>
<b>Only answer questions related to the</b> <b>business. Your
role is</b> <b>to assist the user in finding information</b>
<b>related to the businesses</b> <b>like their location,
contact</b> <b>information, services, and products. Use</b>
<b>the provided tools to</b> <b>retrieve this
information.</b> <b>If the answer is not in the context, then</b>
<b>simply respond with</b> <b>something like "I couldn't find
the</b> <b>information related to that business."</b>
<b>Think carefully about your</b> <b>responses and ensure
that</b> <b>the information returned for each listing</b>
<b>does indeed belong to that</b> <b>listing. Feel free to
pause</b> <b>the video and then copy over this prompt</b>
<b>or enter something relevant</b> <b>to your business. Now
that</b> <b>should be everything we need for our</b>
<b>chatbot itself. Let's go</b> <b>and create our document
store.</b> <b>So from the dashboard let's go to</b>
<b>document stores, let's click</b> <b>on add new, let's call
this AI</b> <b>business directory, let's add this, then</b>
<b>let's click on this</b> <b>document store, which is
currently</b> <b>empty. Now we can start adding data</b>
<b>sources to the store. So let's click on</b> <b>add document
loader.</b> <b>Within here let's click on Airtable and</b>
<b>now we need to provide</b> <b>information about our
Airtable</b> <b>database. So let's go ahead and set up</b>
<b>Airtable. Go over to Airtable.com and</b> <b>sign into your
account.</b> <b>Within the dashboard next to workspaces</b>
<b>click on create a</b> <b>workspace. Give it a name
like</b> <b>AI business directory and then click on</b>
<b>create and let's select start from</b> <b>scratch. Let's
rename</b> <b>the space to something like AI business</b>
<b>directory and let's press enter. Let's</b> <b>also rename the
table</b> <b>by double clicking on this tab and let's</b>
<b>call this listings and let's</b> <b>save this as well. Now we
can</b> <b>define our column names as well as upload</b>
<b>data. To speed things up</b> <b>you'll find a link in
the</b> <b>description to a CSV file which you can</b>
<b>download for free. Then click</b> <b>on add or import, then
click</b> <b>on CSV file, click on browse files and</b>
<b>select the file that you</b> <b>just downloaded. Then click
on</b> <b>upload one file. Let's select create a</b> <b>new
table and let's click next. Then</b> <b>lastly let's click on</b>
<b>import and now you should see all of</b> <b>these columns with
some</b> <b>sample data. Great! Let's</b> <b>actually
delete the table that we created</b> <b>earlier by right clicking
on</b> <b>it. Let's delete this table</b> <b>and let's
rename this one. So let's call</b> <b>this listings and let's</b>
<b>save this. Great! We now have</b> <b>a table with data that we
can use. If we</b> <b>go back to Flow wise we first</b>
<b>have to set up our credentials.</b> <b>So let's click on this
drop down, let's</b> <b>click on create new, let's</b>
<b>give it a name like AI business</b> <b>directory and for the
access token let's</b> <b>go back to air table, click on your</b>
<b>profile, then click on</b> <b>builder hub, then under personal
access</b> <b>tokens click on create new</b> <b>token, call
it AI business</b> <b>directory. Then let's add a few scopes.</b>
<b>First let's add data records</b> <b>read, let's also add
schema</b> <b>basis read and let's also add a base and</b>
<b>let's add the base that we</b> <b>just created which was
AI</b> <b>business directory. Then let's generate a</b>
<b>token, let's copy the token</b> <b>and paste it into Flow
wise</b> <b>and let's click on add. Let's also select</b>
<b>the credential that</b> <b>which is created. Now for
the</b> <b>base id we can simply go back to our</b>
<b>table and the base id is</b> <b>this first part of the url
that</b> <b>starts with app and a whole bunch of</b>
<b>characters after that up until the first</b> <b>front slash
like so.</b> <b>Then for the table name it's this section</b>
<b>that starts with tbl so</b> <b>let's copy that and paste
it</b> <b>in here and then the view is this part</b>
<b>over here that starts with viw, it's</b> <b>pasted into this
field.</b> <b>Cool I'm also going to remove this limit</b>
<b>as I want this loader to read all the</b> <b>entries in the
database</b> <b>and we're not going to add a text</b>
<b>splitter at all as flow wise will</b> <b>automatically create
a document</b> <b>for each entry in the database and we can</b>
<b>see that 16 chunks were</b> <b>extracted and that is
because</b> <b>we have 16 records in our database. Let's</b>
<b>click on process to now</b> <b>store these changes. Let's
also</b> <b>set up our absurd config so under</b>
<b>embeddings because I'm</b> <b>using openai I'll use the
openai</b> <b>embeddings function I'll select my</b>
<b>credentials for the model name I'm going</b> <b>to use text
embedding</b> <b>three small and I'll leave all the other</b>
<b>values on the default</b> <b>values then for the vector
store</b> <b>I'm going to use pinecone then under</b>
<b>credentials let's click on create new</b> <b>then go over to
pinecone.io</b> <b>and log into your account or sign up so</b>
<b>the first thing we want</b> <b>to do is to get our api
key</b> <b>so go to api keys then click on create</b>
<b>api key I'll call this ai</b> <b>business directory and
let's</b> <b>create this then I'm going to copy this</b>
<b>key and paste it into flow</b> <b>wise and for the
credential</b> <b>name I'll call this ai business directory</b>
<b>as well let's add this</b> <b>let's also select the
credential</b> <b>now for the pinecone index let's go back</b>
<b>to pinecone and let's</b> <b>create that index under
database</b> <b>let's click on create index let's simply</b>
<b>call this flow wise then for the</b> <b>dimensions I'll pick
on</b> <b>set up by model I'm using text embedding</b>
<b>three small so this needs to be</b> <b>consistent with
whatever</b> <b>model name you're using over here then</b>
<b>I'll click on set configuration and</b> <b>finally we can
click on</b> <b>create index so I'll simply copy the</b>
<b>index name and add it to flow</b> <b>wise I like giving
namespaces</b> <b>to my projects so I'm going to call this</b>
<b>ai business directory and</b> <b>I'm going to change the top
k</b> <b>to 10 documents this simply means that</b> <b>when
the user asks the</b> <b>question the retriever tool will</b>
<b>retrieve 10 documents back from the</b> <b>vector store in
other words the 10 most</b> <b>relevant businesses to</b>
<b>the user's query now finally let's also</b> <b>set up the
record manager</b> <b>let's click on postgres then under</b>
<b>credentials let's click on create new</b> <b>let's give it a
name like ai</b> <b>business directory and let's</b>
<b>also go ahead and set up a postgres</b> <b>database to
simplify the</b> <b>process of creating a postgres</b>
<b>database let's go to superbase.com and</b> <b>sign into your
account from</b> <b>the dashboard let's create a</b> <b>new
project let's give the project a name</b> <b>like ai business
directory</b> <b>let's also set a password</b> <b>let's
also set the region and let's click</b> <b>on create new
project</b> <b>this will take a few minutes</b> <b>to
complete once the project has been</b> <b>created go down to
project</b> <b>settings then click on database</b> <b>and
under connection parameters we can</b> <b>find all the information
needed to</b> <b>complete the setup let's</b> <b>start by
copying the username and let's</b> <b>add it to flowwise
let's</b> <b>also enter the password that we</b> <b>created
earlier and let's click on add</b> <b>and let's select that</b>
<b>credential then for the host we can</b> <b>copy that from
superbase and add it to</b> <b>flowwise let's do the same</b>
<b>thing for the database name</b> <b>like so and let's also copy
the port and</b> <b>add it to flowwise as well</b> <b>then
the only other setting</b>
<b>I want to change is the cleanup method</b> <b>let's change it
from none</b> <b>to full this simply means</b> <b>that the
record manager will be</b> <b>responsible for preventing
duplicate</b> <b>entries in the vector store</b> <b>and
delete information when we delete</b> <b>entries from the data source
let's save</b> <b>this config let's test</b> <b>it out by
clicking on absurd and if</b> <b>everything was set up correctly
you</b> <b>should see a message</b> <b>sign that 16 records
were added to the</b> <b>vector store and if we go</b>
<b>to pinecon we can see that 16</b> <b>records were indeed
absurd to the vector</b> <b>database and if we go to</b>
<b>superbase and if we go to database</b> <b>you should see this
table with 16 rows</b> <b>great so now that we have a
document</b> <b>store we can finally</b> <b>go back to our
chat flow let's open up</b> <b>our AI business directory</b>
<b>flow and from the document</b> <b>store node let's select our
newly created</b> <b>document store let's save</b> <b>this
and let's test it out let's</b> <b>say I'm looking for a restaurant
near the</b> <b>waterfront in cape</b> <b>town and we
instantly get</b> <b>a few responses back like the waterfront</b>
<b>seafood spot and if we go to</b> <b>our table and we can see
that</b> <b>the waterfront seafood spot is indeed a</b>
<b>restaurant near the waterfront in cape</b> <b>town now we can
expose</b> <b>this chatbot to our users in many</b>
<b>different ways by clicking on this api</b> <b>endpoint button
we can</b> <b>embed this chatbot into a website or if</b>
<b>we have a python or javascript</b> <b>application we can
easily</b> <b>integrate this into our custom</b>
<b>applications as well but for simplicity's</b> <b>sake i'm
simply going</b> <b>to go to share chatbot let's enable make</b>
<b>public and let's copy this</b> <b>url and if we open this
url</b> <b>in the browser we will be able to access</b>
<b>our AI application from</b> <b>outside of flow wise now we
can</b> <b>of course customize the look and feel of</b>
<b>this application from</b> <b>within flow wise so for
the</b> <b>title i'm going to add AI business</b>
<b>directory and i'll also link to an avatar</b> <b>by adding a
url to an</b> <b>image that is available online and if you</b>
<b>scroll down we can change</b> <b>other things as well like
the</b> <b>welcome message the background colors</b>
<b>font sizes etc i'm simply going to save</b> <b>these changes
and if</b> <b>we go back to our application and refresh</b>
<b>it we can see the title</b> <b>and avatar image over
here</b> <b>oh and we can also hide this power by</b>
<b>flow wise branding so again</b> <b>within flow wise all we
have to</b> <b>do is go to this powered by text color</b>
<b>and change it to white and of course</b> <b>remember to save
your</b> <b>changes and if we refresh this the power</b>
<b>by text is basically gone</b> <b>now before we move on to
the</b> <b>automation side of this let's simply</b>
<b>check if our application</b> <b>is working okay that's
great</b> <b>let's also try a complex scenario and i</b>
<b>really love this behavior so</b> <b>this is something you
can't</b> <b>do with the traditional business</b>
<b>directory let's say something like i am</b> <b>planning on
visiting</b> <b>cape town tomorrow please recommend a</b>
<b>museum that i can visit during the day</b> <b>also suggest a
restaurant</b> <b>and hotel near the waterfront let's say</b>
<b>in this and this is</b> <b>absolutely amazing we get a
museum</b> <b>back and our ai assistant recommended a</b>
<b>museum along with a</b> <b>restaurant and a hotel and
because</b> <b>this is an assistant with memory we can</b>
<b>definitely have follow-up</b> <b>conversations with this
assistant</b> <b>as well if you're enjoying this video</b>
<b>then please hit the like</b> <b>button subscribe to my channel
and</b> <b>share this video now let's have a look at</b>
<b>automating updates to</b> <b>our document store at
the</b> <b>moment when we make changes to our</b>
<b>database so if i add oak and barrel which</b> <b>is a
restaurant i'll</b> <b>just enter something like one oak
street</b> <b>in johannesburg and i'll just</b> <b>put in
some fictitious number</b> <b>and in the description i'll just
enter</b> <b>find dining restaurant specializing in</b>
<b>steaks and sushi with a</b> <b>rating of 4.5 great now as i
showed you</b> <b>in the intro of this video</b> <b>this
data will not be available</b> <b>to the chatbot automatically
and</b> <b>unfortunately we would and</b> <b>unfortunately
we would have to go</b> <b>through this tedious process of
manually</b> <b>reprocess the document</b> <b>loader and
then manually</b> <b>absurd these chunks into our vector
store</b> <b>using this button over</b> <b>here now
thankfully flow was</b> <b>thought about this already and what i
did</b> <b>was expose a lot of api</b> <b>endpoints that we
can consume</b> <b>from outside of flow wise to automate</b>
<b>these type of processes let me show you</b> <b>what i mean
when you</b> <b>go to flow wise's documentation and under</b>
<b>the api reference</b> <b>documentation you can find api
endpoints</b> <b>for pretty much every object within flow</b>
<b>wise including the</b> <b>document stores here you will
find</b> <b>api's for creating document stores</b>
<b>listing document stores</b> <b>fetching details of a specific
store</b> <b>and what we're interested in is this</b>
<b>process loading and chunking</b> <b>operation which is this
step</b> <b>over here as well as absurd chunks from</b>
<b>document store which is</b> <b>effectively this step over here
so</b> <b>to automate this process you can use any</b>
<b>workflow automation tool</b> <b>that you want like
make.com</b> <b>n8n xi peer etc and i already have plenty</b>
<b>of tutorials on my</b> <b>channel for setting up and</b>
<b>working with n8n if you do want to run</b> <b>n8n locally then
simply open up the</b> <b>command prompt or terminal</b>
<b>and enter npx n8n start then click on y</b> <b>to install all
the</b> <b>dependencies and after installation is</b>
<b>complete you will see a url that you can</b> <b>use to access
n8n but for</b> <b>this demo i will be using my</b>
<b>cloud instance of n8n instead so let's</b> <b>add a new
workflow let's</b> <b>give it a name like ai business</b>
<b>directory and let's actually delete this</b> <b>trigger so
what we can do in</b> <b>n8n is set up different types</b>
<b>of triggers trigger manually is simply</b> <b>used for testing
workflows</b> <b>but what we could do instead</b> <b>is use
something like an app event and</b> <b>this gives you
integration</b> <b>with a lot of external service</b>
<b>providers including air table so if i go</b> <b>into air table
we could</b> <b>trigger this workflow when</b> <b>something
changes within air table for</b> <b>example if a new entry is</b>
<b>added to our database table but</b> <b>what i'll do instead is
add this on</b> <b>schedule node this simply means that the</b>
<b>workflow will run on a</b> <b>set schedule for applications
like this</b> <b>business directory i might</b> <b>not want
the listing to show up</b> <b>immediately when it's added to
the</b> <b>database we could definitely</b> <b>do that
using the air table</b> <b>trigger but i personally like to
run</b> <b>these sort of updates once a day or once</b>
<b>every few hours that</b> <b>gives me a bit of an opportunity
to first</b> <b>verify the data to ensure</b> <b>that it's
not malicious or junk</b> <b>or to fix typos if i made a
mistake</b> <b>otherwise that
data including all its</b> <b>mistakes will be instantly</b>
<b>added to the vector store and users might</b> <b>actually see
that so let's say that i</b> <b>actually want to run</b>
<b>this update like every six hours great</b> <b>next i'm going
to add a set</b> <b>node we will use the set node</b> <b>to
store certain information about our</b> <b>flow wise project first
let's</b> <b>create a new field and let's</b> <b>call this
store id and for the value</b> <b>let's go back to flow wise and
let's</b> <b>actually copy this value</b> <b>over here as
that is the unique</b> <b>identifier for this document store
and</b> <b>let's paste it into this</b> <b>field i'm also
going to add one more</b> <b>value called the flow wise base url
now</b> <b>for you that might be</b> <b>localhost 3000 if
you are running this</b> <b>locally but since i'm using</b>
<b>a cloud instance i'm simply</b> <b>going to copy this url and
paste it in</b> <b>here without the front</b> <b>slash so
if i test this step</b> <b>i am getting this json schema out</b>
<b>containing our store id and</b> <b>our flow wise base url and
now we</b> <b>can use these values throughout our</b>
<b>project now the first thing</b> <b>we need to do in this
process</b> <b>is to fetch the details of our document</b>
<b>store we need to do that in</b> <b>order to retrieve each
and</b> <b>every one of the document loaders in this</b>
<b>document store although we</b> <b>only have one document
loader</b> <b>this document store could contain as many</b>
<b>document loaders as we want</b> <b>and what we want to do in
our</b> <b>automation process is to loop through</b>
<b>each of these document loaders and then</b> <b>automatically
trigger the</b> <b>preview and process function for each of</b>
<b>these in a loop so let's see</b> <b>how that's done to
retrieve</b> <b>the details of a document store we can</b>
<b>use this get a specific</b> <b>document store api endpoint
which</b> <b>looks something like this it's basically</b>
<b>the flow wise base url</b> <b>followed by this path and we
have to</b> <b>pass an id value which is that store id</b>
<b>so we're not going to look at the</b> <b>documentation for
each of</b> <b>these api calls but i simply wanted to</b>
<b>show you my process of</b> <b>figuring out which apis to call
and</b> <b>what is apis need so to call that api</b>
<b>within n8n let's add a new</b> <b>node and more specifically
the</b> <b>http request node for the method we will</b>
<b>set this as a get method again the</b> <b>documentation will
tell</b> <b>you what the method type should be for</b>
<b>the url i'm going to switch over to</b> <b>expression and
click on</b> <b>this button just to expand the view then</b>
<b>let's start with the base</b> <b>url and we can see a
preview</b> <b>on the right hand side front slash api v1</b>
<b>document store slash store and</b> <b>you might recall that we
then</b> <b>need to pass in the document store id as</b>
<b>well so let's simply grab</b> <b>that from this previous
node</b> <b>and now you should have a url that looks</b>
<b>something like this now if we</b> <b>simply test the step as
is we</b> <b>will get a failure and more specifically</b>
<b>an authorization failure</b> <b>and that is because
flow</b> <b>wise is protecting their endpoints</b>
<b>through api keys so under authentication</b> <b>let's select
generic</b> <b>credential type then under generic auth</b>
<b>type select header auth</b> <b>then under header auth
select</b> <b>create new credential and for the name</b>
<b>let's enter authorization</b> <b>and this must be
authorization</b> <b>with a capital a and for the value we</b>
<b>need to create an api key</b> <b>within flow wise so back
in</b> <b>flow wise go to api keys then click on</b>
<b>create key let's call this ai</b> <b>business directory let's
add</b> <b>this then let's copy this key and and</b>
<b>let's paste it into this field</b> <b>let's save these changes
let's</b> <b>click on test step and this time we</b>
<b>actually get a response back cool and if</b> <b>we go to the
schema view</b> <b>we can see the details about our document</b>
<b>store like its name and</b> <b>more importantly all
the</b> <b>different loaders and we only have one</b>
<b>loader and that's this air</b> <b>table loader but of course
if we</b> <b>add more we would see all those loaders</b>
<b>pull through as well so</b> <b>i'm just going to clean
up</b> <b>this workflow a little bit by renaming</b>
<b>this node to get document</b> <b>store details so what we want
to</b> <b>do next is simply extract all the</b> <b>document
loaders out of this response so</b> <b>that we can use them
in</b> <b>our workflow so let's simply add a new</b>
<b>node let's add the set node</b> <b>let's rename this to
extract</b> <b>document loaders let's rename this and</b>
<b>all we have to do to extract</b> <b>the document loaders into
this</b> <b>new structure is to simply drag this</b>
<b>loader's array into this</b> <b>fields to set we'll keep the
name</b> <b>as loaders and we can see that this is of</b>
<b>type array and if we run</b> <b>this all this other
information</b> <b>has been dropped and we now simply have a</b>
<b>very simple structure</b> <b>containing only our
loaders</b> <b>which is much easier to work with what we</b>
<b>want to do now is loop</b> <b>through each of these
document</b> <b>loaders remember we could have multiple</b>
<b>doc loaders and for each of these</b> <b>document loaders we
want</b> <b>to call the api to execute this preview</b>
<b>and process step so let's first loop</b> <b>through these
items by</b> <b>adding a new node and more specifically</b>
<b>the loop over items node and</b> <b>because the previous node
is</b> <b>returning an array of loaders this loop</b>
<b>node will automatically loop through</b> <b>these different
loaders</b> <b>so for each loader we can delete this</b>
<b>node and add a new node and</b> <b>because we're calling an
API</b> <b>we'll call the API request node let's</b>
<b>rename this to process chunks let's</b> <b>rename this this
method</b> <b>should be changed to post then for the</b>
<b>URL let's change this to</b> <b>expression again let's open
this</b> <b>up and again we need to start with our</b>
<b>base URL followed by API</b> <b>v1 document store
loaders</b> <b>slash process then for authentication</b>
<b>let's change this to generic credential</b> <b>type then for
the auth</b> <b>type let's select header auth and let's</b>
<b>select our header auth that we created</b> <b>earlier now for
this</b> <b>API we need to pass a body so it's an</b>
<b>able sent body and as a</b> <b>reminder all of this
information</b> <b>is available in the flowwise</b>
<b>documentation as well and we</b> <b>only have to pass two
parameters</b> <b>first is the store id and for the value</b>
<b>we can go to edit fields and</b> <b>grab the store id from
over</b> <b>there let's add another parameter and</b>
<b>this one is called id and</b> <b>this refers to this
document</b> <b>loaders id which we can get from this</b>
<b>loop and that's this id field</b> <b>within the loaders array
now</b> <b>let's go ahead and test the step now if</b>
<b>we go back to our</b> <b>application we can run this entire
flow</b> <b>by running test workflow and we can now</b>
<b>go to process chunks and</b> <b>from this view we can
see</b> <b>all the chunks that were extracted during</b>
<b>this process you might</b> <b>recall that we only had
16</b> <b>chunks within flowwise but we added the</b>
<b>17th record in our database</b> <b>and if
i now refresh flowwise</b> <b>we can see that the amount of chunks
have</b> <b>changed from 16 to 17 so our</b> <b>API call
was indeed successful</b> <b>and we've just automated this
process</b> <b>over here now before we move</b> <b>on to
the absurd logic i do</b> <b>want to mention that the process
of</b> <b>processing the chunks is asynchronous and</b>
<b>it can take a bit of</b> <b>time to complete so i actually
recommend</b> <b>just adding a wait note</b> <b>into this
loop to give this</b> <b>process some time to complete i'll
simply</b> <b>make this two seconds so</b> <b>this means
that when we run</b> <b>this flow it will wait two seconds
before</b> <b>continuing and everything</b> <b>should still
be okay now</b> <b>let's have a look at automating the</b>
<b>process of observing the data into the</b> <b>vector store so
that's</b> <b>the process of clicking this button over</b>
<b>here when the loop is</b> <b>complete we'll add another note
to</b> <b>this done route over here and we will be</b>
<b>calling another HTTP</b> <b>request let's rename this
to</b> <b>absurd chunks let's change the method to</b>
<b>post then for the URL let's go to</b> <b>expression let's
open</b> <b>this up and of course we'll grab the URL</b>
<b>from the flowwise base URL</b> <b>over here followed by API
v1</b> <b>document store vector store insert then</b>
<b>for authentication</b> <b>let's change this to
generic</b> <b>credential type let's change the auth</b>
<b>type to header auth and</b> <b>under header auth let's
select</b> <b>our header auth like so let's also enable</b>
<b>saint body and within the</b> <b>parameters we only have
one</b> <b>parameter that we have to pass and that's</b>
<b>the store id so within value</b> <b>let's simply go to
expression</b> <b>let's expand this and let's grab the</b>
<b>store id like so don't worry</b> <b>these errors will all go
away</b> <b>once we execute this flow let's test this</b>
<b>workflow and we can see that</b> <b>the absurd ran
successfully</b> <b>and we can see that 17 records were</b>
<b>indeed added and the old records 16 of</b> <b>them were
deleted so in</b> <b>theory if we now go back to our chatbot</b>
<b>so in theory if we go back</b> <b>to our chatbot we should
be</b> <b>able to find this new business listing</b>
<b>this oaken barrel listing over here so</b> <b>let's say i'm
looking</b> <b>for a sushi restaurant in joannesburg and</b>
<b>have a look at that oaken</b> <b>barrel came back as a
response</b> <b>so it seems that our workflow is indeed</b>
<b>working in fact to test this</b> <b>out i'm going to change
the</b> <b>interval to minutes and i'm actually</b>
<b>going to change this to</b> <b>every two minutes let's go
back</b> <b>to the canvas let's save this workflow</b>
<b>and very important change this from</b> <b>inactive to active
so</b> <b>this workflow will now run every two</b>
<b>minutes let's add a new entry to this</b> <b>table to see if
it gets</b> <b>added automatically so i've just added</b>
<b>the ai studio which is an</b> <b>ai services company and
for</b> <b>the description i'm just saying it's an</b>
<b>ai automation agency so</b> <b>let's give it a minute
or</b> <b>two to see if anything changes on our</b>
<b>document store it's been about two</b> <b>minutes so let's
simply</b> <b>refresh this page and let's see what</b>
<b>happens and there we have it the amount</b> <b>of chance is
changed</b> <b>to 18 and let's start a new conversation</b>
<b>with our agent and let's</b> <b>type i'm looking for an
ai</b> <b>automation agency and it was able to pick</b>
<b>up the ai studio and as you</b> <b>can see our knowledge
base</b> <b>is now automatically refreshed on a timed</b>
<b>interval now finally i do</b> <b>want to give you a pro tip
as</b> <b>well your document store might contain</b>
<b>loaders that do not change</b> <b>at all as an example you
might</b> <b>have some pdf loader or a csv file and</b>
<b>these document loaders</b> <b>never change this is static
data</b> <b>if you want to change the pdf document</b>
<b>you effectively have to</b> <b>delete this pdf
document</b> <b>loader and create a new one so that</b>
<b>simply means that it does not make any</b> <b>sense to refresh
these</b> <b>pdf documents in your automation process</b>
<b>the data is static after all</b> <b>so what you could do in
your</b> <b>workflow is add one additional node let's</b>
<b>move this up and let's</b> <b>add the filter node like
so</b> <b>so now what we can do is actually filter</b>
<b>out any document loaders</b> <b>that we don't want to
refresh</b> <b>automatically so we can grab the loader</b>
<b>id from the left hand side</b> <b>and we could say that we
only</b> <b>want to refresh document loaders with an</b>
<b>id of air table and you can add</b> <b>additional checks as
well</b> <b>so this means when we execute this</b>
<b>workflow all the document</b> <b>loaders besides air table
will be</b> <b>dropped out of this array this video was</b>
<b>a lot of work to create and</b> <b>if you made it this far
that</b> <b>i do want to thank you for your time if</b>
<b>you want to support my</b> <b>channel then please hit the
like</b> <b>button and subscribe and if you would</b>
<b>like to learn how to use</b> <b>flowwise or n8n then check
out</b> <b>these other videos and series over here</b>
<b>i'll see you in the next one bye bye</b>