##Dave Shapiro videos

#OpenAI Agent Swarm: Let's build an autonomous agent swarm Pt 1: AGENTS THAT BUILD
AGENTS
okay check check test hello all right
okay gang so David Shapiro here I'm
actually a little bit nervous because of
how cool this is and how close we are so
a lot of people ask me like Dave what is
your timeline for AGI and I'm like we
are so much closer than everyone
realizes um and the reason is because of
what I call the uh the the automation
Paradox or the Paradox of Automation and
so with automation um as a as an
infrastructure autom engineer who has
automated petabyte scale migrations and
um basically automated like literally
the biggest project that I did was I
automated firmware upgrades for servers
and storage for four data centers and it
did it all overnight so like not to toot
my own horn well I guess I am tooting my
own horn but I'm a damn good automation
engineer um if it has an API it can be
automated and guess what has an API now
open ai's Assistant creation
so what are we going to do well let me
introduce you to this new project so
this is basically an evolution of the
ace framework um and I'll explain it to
you as we go but basically now that we
have the agents that can create the
agents and then we can give them files
and tools and stuff we have the
rudimentary toolbox to create agent
swarms which when you look at what
they're capable of so for instance when
you look at chat Dev which was able to
autonomously create software just with
one single human instruction we are at
the cusp of creating fully-fledged
swarms of autonomous agents so how do we
get started first you need to think of
how do you organize all this stuff how
do you even how do you even orchestrate
uh this you know how do you decide what
to create when and where so that's what
this project is for open AI agent swarm
project hierarchal hierarchical
autonomous agent swarms or
hos so hos is a groundbreaking
initiative blah blah blah blah blah I
had had I had chat GPT help me write
this um so the idea was to I wanted it
to be as explicit so if you just stumble
upon this you read it you'll get the
idea of what it is so um I was being a
little bit tongue and- cheek but
basically the idea is you start with a
supreme oversight Board of Agents so
these are the this is the equivalent of
the aspirational layer of the of the ace
framework where you have a series of
agents that are for instance here let me
zoom in a little bit more um that are
established and uphold the ethical
framework overarching mission of the
agent swarm make high level decisions
and judgments including the creation and
termination of other agents so from a
security perspective because this is
something that a lot of people ask about
how do you ensure that it's secure how
do you ensure that it's stable you have
one set of Agents whose sole
responsibility is the creation and
termination of other agents so rather
than having one massive monolithic agent
that is uh self-correcting you have a
swarm a fleet of Agents with checks and
balances and so the Supreme oversight
board rather than doing any of the work
its primary role is to is to provide
steering um and so that means it will
create sub agents it'll it'll create the
the next layer which is the executive
agents um and then it can also destroy
any agent so if it detects that an agent
is doing something harmful or illegal it
can act that agent and reinstantiate it
with better
instructions and so then uh we we also
give it you know basically this is what
I was just saying monitoring agents uh
to ensure alignment with the values and
objectives uh role-based Access Control
to maintain order and security within
the system and there there will be
checks and balances here like you'll
have a security department that is
responsible for monitoring uh you know
all the permissions and Privileges and
when you think about the fact that uh
the cloud provider platforms like uh
like Google uh uh Azure and AWS all have
API accessible arback ro-based access
control that means you can build an
Azure agent that can build containers
for the rest of your Azure agents um so
this is eminently possible this is why
I'm so excited and so upregulated
because it's like guys you don't know
like or maybe if you're watching this
you do know but like you don't realize
how close we are to just having fleets
of fully autonomous agents so I'm glad
that I was doing the work that I've been
doing um okay so I've in this case the
ace framework is six layers um but I've
only got like kind of the top three
layers here cuz after that you kind of
figure out like Chad Dev already
demonstrated that you can have like a
marketing agent and a coding agent and
an integration agent and a documentation
agent so I'm only documenting the high
the highest layers so the first idea
that I'm adding is the Supreme oversight
board um then the executive agents so
this is like the CEO CTO whatever whose
primary like they this is where you
start to get into uh having specific
domains so the Supreme oversight board
is more about making good judgments and
supervising the whole process whereas
the executive agents these are when they
first when they first start to have uh
specific roles and and and domains of
focus and as I was I was explaining this
to someone last night and I said yeah
like if you if you put uh if you put
like a revenue agent and a marketing
agent in a room together it's entirely
possible they'll come up with stuff that
is unethical illegal or whatever and so
but that's fine right you have you have
the the Tony Stark you know and and and
Tony Robin's you know agent talking
together about how to you know build the
coolest thing and Market it they might
not come up with the best IDE is which
is why you have oversight that's why you
have checks and balances as well as so
there's there's two ways that this is
this this is
defined our back if you're familiar with
that you understand that there's um
there's levels of inheritance so
basically the way that I've got this
structured is that the um is that the
board is tier zero so that's the highest
tier the executive uh agents are tier
one and anyone's that they create are
tier two and so on and so forth and so
that you can very quickly establish
where in the hierarchy where in the rank
an agent is so that you know like okay a
tier 2 agent will never have permissions
to shut down any other tier 2 tier one
or two tier zero agents so that way you
have very clear command structure but
then you can also have silos or vertical
integration so basically like if if the
CTO created an agent then like they have
a bunch of inheritance right so it's
like you you can you can it it it
automatically creates an org chart tldr
okay okay so I think you get the idea
I've got some stories built in here so
that you can kind of read through it and
and get familiar with it but I just
wanted to give that introduction and so
where I was thinking last night like
where do you start where do you start
with this and so I realized duh the
first thing that you do is you create an
assistant to build assistance so let's
try that so I I got an assistant to work
uh last night uh I guess maybe the file
processing was was broken but anyway so
I created captain peard and uh and it
works so like Captain there's cling guns
off the a starboard bow and so let's see
what he says um so you can see it pulls
all the threads and so you know Red
Alert Mr Warf race Shields and prepare
to charge weapons blah blah blah blah
blah okay so this is the reason that I'm
starting with this is because this
demonstrates that the assistants that
the agents that we're creating here can
adopt any Persona or any archetype and
so why this is good is because that
Supreme oversight board you can have
Socrates and Captain peard and you know
Mahatma Gandhi all on your on your uh on
your Supreme oversight board helping to
discuss if a decision is moral or
ethical or if it is in good faith or
whatever and so this is that's why I
started here is because it it's not just
a matter of of having a thing that
you're trying to optimize for right this
is this is one of the biggest
disconnects between purely math and
machine learning engineering and where
we're going because I could give this a
different Mission I could say you know
your mission is to conquer the Klingon
Empire and it'll it'll go and try and do
that um and just no questions asked uh
now but if you say that hey you're part
of a supreme oversight board if you add
this documentation say hey this is what
you're part of you're part of a of an
autonomous agent swarm and this is your
mission then it'll be able to integrate
that information as well and so like
taking a step back what I'm going to try
and do is is next create an agent that
understands this and then we'll go from
there so let's just jump right in um all
right so let's create a new one and so
this will be a um uh autonomous
swarm agent Builder um okay your mission
is to help the user
create agents uh autonomous agents for
the C system all right and so then we'll
do
preview and then we'll do uh files um
let's come back over to agents swarmm
and I'll upload the readme and the open
aai documentation so what I forgot to
show you was that I took all of the most
Salient open aai documentation and saved
it into a markdown file so now I'm
giving it two files I'm giving it the
readme document so that it knows what it
is what it's trying to do and then the
documentation of how to write it so if
this works then I will have just with
uploading you know giving one set of
instructions and two files we should
soon have an agent that can help me
build agents so let's save this and see
if it works that was
fast Okay cool so then we come over to
the playground and I say cool uh let's
get started um uh how do you think we
should build uh the first uh Saab
agents and then let's see what it does
oh my God I'm so excited you guys I'm
losing using my marbles over
here okay so let's see let's look at the
log so what it's doing is it's going to
be going through the
threads um it's pulling a bunch of stuff
okay to create animate with the high
Assurance it looks like it didn't it
yeah it didn't search it correctly okay
let's see I don't think do you know what
the Saab is I don't think it fetched the
the data
correctly so this is disappointing but
this is why we're this is why we're
doing a test so the the most you're
allowed to have is up to up to 20 files
so it might behoove us to let's see
self-organizing Behavior so it's
completely confabulating here that's
really disappointing I gave it the files
it's guessing close but it's it's
doubling down on stuff so well this is
our first problem so I gave it
documentation and it's not using it
correctly um which is really interesting
um let let's see if it understands
function calling
um let's see do you understand function
calling it might be because the the
documents that I gave it are pretty big
so first like this is 57 almost 60,000
characters long so what what we might do
is actually break this down into smaller
documents um where it's like instead of
instead of all of the open aai
documentation in one we'll do um we'll
do uh smaller bits and then this is this
is only 11,000 characters this should be
pretty straightforward like it can read
this all in one
shot um let's see add numbers th add
numbers yeah so it's it's not even it's
not even
reading it it like it's not reading the
function calling stuff this is weird let
me go over to chat and um and plug in
plug it all in here so let's do that let
me just copy paste the documentation
here and see if if just putting it in
here without anything
else
will work all right so now I'm giving it
the the read me and the openi open AI
documentation um hey let's figure out
how to create a Saab agent um
Builder so basically we're going to
start with agents that help you build
other
agents okay now let's see if let's see
if just giving it all the background
okay so now it's Supreme oversight board
it's got it here okay so there's some
disconnect between the way that it's
using um on the other side because this
it very clearly knows like what we're
doing like it news Supreme oversight
board yep by following the steps cool um
let's see can you take a stab at just
writing a python script that will build
the agent for me using the open AI
assistant data all right so it talks
through please note that this script is
blah blah blah okay yep it's all right
so it's going by really fast so I'll
I'll rewind a second and show you but
this clearly is a better way so what
we'll need to do is do some experiments
to figure out why the assistant isn't
referring to the documents cuz literally
all I did did was just copy paste it
into the system window AC crap ton of
documentation and it's able to run with
it so okay so let's see what it said
whoops come
back scrolling up okay so Define it sa
agent Builder um ethical framework so we
gave it an ethical framework which is
great uh Mission reduce suffering
increase Prosperity enhance
understanding Do no harm promote
well-being whoops darn it sorry um
promote well-being and respect autonomy
um so it it it confabulated that which
those are not necessarily
uh bad things but okay cool prioritize
safety ensure fairness and maintain
transparency I wonder if this is some of
open ai's underlying alignment leaking
in which if that's if that's the case
then okay you know um but yeah so it it
came up with it it followed my mission
and then it added principles and
guidelines instructions you're the
Supreme oversight board agent Builder so
it understood the instructions Define
the agent templates data analysis agent
customer service agent that I'm not sure
why it came up with those ideas so we're
we've got some work to do um let's see
open AI assistant create name
description model tools and tools um we
don't have tools yet so it's kind of
confabulating there okay so it looks
like it's it's smart but it's not quite
smart enough to fully Gro what we're
trying to do but I think we're off to a
good start and um so I'm going to pause
the video here because my my goal is not
to overwhelm you with doing a whole lot
at once I'm also not trying to overwhelm
myself so what I do want to do is I'll
come back over here all this is up here
so I've got the open aai documentation
and the readme I've also created a
really simple diagram with the help of
chat GPT to kind of give you a very
simple idea as to how the the the the
structure will work and they're all
colorcoded so like executive agent one
is the purple branch and so then it owns
every other thing under its Branch um
but so this would be tier zero tier one
and tier 2
but yeah so uh other other aspects of
the re repo I've got issues um oh I've
got issues all right no kidding um then
we've also got the discussions tab um so
yeah if you want to jump in uh jump into
the discussions I'll add a contributing
uh document as well just so that way you
can get a little bit of guidance but but
the rule of thumb is start with
discussions go to issues then go to poll
requests if you just open a poll request
without uh proper documentation or
discussing it it's probably going to get
rejected um but yeah so thanks for
watching I hope you like this and uh
guys let's get excited let's let's let's
figure out how to build this um because
we are so close all right
cheers
#The Automation Paradox: Why AGI is closer than you think
hey YouTube I wanted to take a quick
break and address what I call the uh the
Paradox of automation or the automation
Paradox I mentioned it in my uh last
video and so as an automation engineer
and I've confirmed this with uh other
infrastructure engineers and automation
Engineers there's this trend
where nothing happens and I mean nothing
no productivity nothing gets done until
it's until the automation work is done
and then everything happens all at once
and so for instance you might spend days
weeks or months working on an automation
product or project uh or script or
whatever uh some kind of automation
engine and you have a test environment
and nothing happens you know in
production nothing meaningful happens
out in the real world and you test and
you test and you test and then you know
other stakeholders the general public
your managers directors whatever they
don't see anything happening but then
then once the automation is done and
tested and you turn it on all of the
work happens all at once and that's what
we're seeing that's what we're ramping
up to with AGI and I didn't realize that
this was going to be how it played out
because viewing this as fundamentally an
automation problem made it make a lot
more sense in my mind and so as we're
building like fleets of autonomous
agents and and laying all the ground
workor it's going to be like you get all
the you get the you get the swarm set up
you get the autonomous uh swarm set up
you get the hierarchy set up and you
test it and you test it and you test it
and you give it tools you make sure that
they can write tools and use tools and
find apis and document apis and and you
know that the Swarm that the agents can
create each other and deprovision each
other that's going to take a lot of work
but then as soon as it's done you're
literally going to have entire companies
that are fully automated by these swarms
of Agents now as a lot of people are
pointing out in the comments the token
count is cost is still going to be a
little a little bit prohibitive they're
still cheaper than humans so even if it
takes 00 $1,000 an hour to run these
things that's still like they're still
getting as much work done as you know a
thousand humans or 100 humans um and
they're getting it done faster so it's
still going to be cost effective once
these things pan out now yes they are
going to be expensive and I'm sure the
uh open AI servers will have a meltdown
if you try and run a million of these
agents at once or a billion at once
they're not going to be able to keep up
but um we are seeing um what's what's
called javon's Paradox which is
basically that the cheaper that
something gets the more it's used
because of latent or unmet demand so
anyways that's what I wanted to do I
wanted to touch on the the the parad the
Paradox of Automation and say like just
wait like when the other shoe drops in 6
to 12 months it's going to be like oh
yeah this was AGI all along um yeah so
it's this is a good time to
to be paying attention um and I'm very
optimistic that it's going in the right
direction at least from a technological
standpoint um I ran a poll and there was
a lot of people uh that by and large
agreed that the biggest risk is going to
be uh Elite power consolidation so we'll
see if uh see if we can prevent that
from happening um or at least mitigate
it uh the the negative downsides uh
namely through open source so anyway
stay tuned thanks for watching
cheers
#Hierarchical Autonomous Agent Swarm Pt 2: Tool Makers and Agent Builders (oh my!)
good morning it is a good day to be in
AI so let's go ahead and dive right into
today's uh standup meeting or scrum for
the open AI agent swarm so I have an
idea so I want to provide a little bit
of context and I want to let you know
that this is just an idea it's an
experiment we'll see how it goes but the
the uh my hierarchical autonomous agent
swarm idea has garnered a tremendous
amount of attention uh as well as a lot
of very Lively discussion and desire to
contribute so what I'm going to try and
do is steer the ship uh from the
perspective of a just really short uh
cycle and experiment I don't think this
project is going to live too long
because I think you guys are getting it
so basically there's just a few ground
uh points that we want to try and figure
out and I've got two of them defined and
I'll go over them in the video but what
I wanted to do today or first is just
provide a little bit of context so that
you understand why things are organized
the way that they are so first here's
the here's the overall repository in a
day we've had 160 Stars uh and 27 Forks
um and so it's it's popping um so uh
quick recap in case you're new to the
project this is my idea of the
hierarchical autonomous agent swarm uh
from open Ai and so the idea is because
we can now instantiate agents via API
and we kind of have this all-in-one
Source this this single stop single shop
stop to run agents in the cloud uh the
idea is maybe we can use this to create
agents that create other agents but then
how do you know how to do that so
there's a hierarchy so the at the
highest end of the hierarchy is the
Supreme oversight board which is a set
of agents that are going to debate
morality ethics mission that sort of
thing and they are one that's one of
their functions they debate what to do
but then they also have the ability to
create other agents and likewise so on
and so forth down the chain it's all
documented minute here I've got it all
written out and so that's that's the
number one thing next thing that I want
to point you to is contributing so I
updated the contributing document so
before you jump in at all make sure you
take a look at this document I've kept
it short and sweet so you know the
contribution workflow number one watch
the latest video so um I'll probably
have a discussion thread or you'll
probably see it on YouTube so but this
will basically be like the daily update
of where we're at and what we need to do
next so I'm going to try serve as
product owner and scrum Master we'll see
if it works but like I said I don't
expect this project to live too long
because I think once we get the ideas
you guys will will be able to take it
and run with it so I don't need to own
the thing forever like obviously like
the scope of this is way bigger than uh
than I can do myself or that an open
source developed and public team can do
um so like I said the whole idea is just
over the next week or two let's see if
we can get if we can lay the groundwork
and solve some of the basic problems and
then the rest of you can just take it
and go and be free and so on and so
forth um so watch the video as you're
doing right now number two discuss on
the discussion tabs which I'll show you
show you in a second number three if
there's a problem if there's a bug or a
problem create an issue um don't open it
into discussions I already I've seen a
few people um complain about code bugs
don't put that in the discussions that's
not what it's for um put it in put it at
as as an issue which is right up here um
and then number four is submit a PR so
once you've watched the video once
you've read the discussions make sure
you submit a PR or a poll request um you
can also just create a main fork or and
and work on it on your own um I there's
been a few people that have like posted
really long conversations and debugs
please don't do that please post a link
to your own Fork um that will keep the
discussions a little bit cleaner and
easier um so then in terms of uh ground
rules like like code of conduct stay on
topic there's already been a few people
that are trying to change the scope of
the project that are adding uh
Superfluous information so those those
topics have been closed we want to keep
this nice and tight and lean so anything
that is off topic or not relevant will
be closed um and then the other thing is
adhere to the C3PO policy so this is
collaborative culture Community policy
zero tolerance for harm and wasting time
so if you if before you contribute ask
yourself is this harmful or is it
wasting time if I if either is is yes or
maybe don't post it um this is how I
keep how I run a tight ship in the
comment section and everything else uh
and then finally the pr requirements uh
pretty simple clear description limit
limit PRS to one big one per day U make
sure that your your poll request aderes
to the current uh style and structure of
the of the thing um if you're not clear
um then again like make sure you check
because this is this this project is
going to be moving quickly and we're
going to have multiple people uh
contributing in and submitting uh poll
requests I will take a look at any
serious poll request um and I'll provide
you feedback so like if it doesn't
adhere to the right style or if it's
trying to change the scope I'll let you
know and say like hey go change this and
then and then resubmit um and we should
be good there okay so that's the project
at a high level so now let's jump into
the discussions so like I said I'm going
to keep the disc discussions nice and
trim and so we've had a few
conversations uh so far um I've provided
some feedback uh trying to keep
everything aligned so like for instance
this was a really good question how do
you how do you maintain alignment on the
Supreme oversight board I provided some
some feedback there there's like I said
there's been a couple conversations that
were off topic so I close those just
because we want to keep the noise to a
minimum um but yeah so jump in here
there's been no issues submitted yet
we've had a couple of successful poll
requ quests um one they closed on their
own and then uh two have been merged um
so that's that and I think I think uh
we're ready to just dive into the code
so let me show you where we're at so
there's two there's two overarching
functions that we need to figure out
once we can figure out these functions
and bake them into agents the rest is a
matter of experimentation and there and
that's not to say that there aren't
other problems to solve because
communication between agents is going to
be probably the hardest thing uh which
is like how do they talk to each other
how do we host the chat rooms do we use
uh amqp message cues do we use something
else do we use CIS log I don't know do
we spool up containers we'll have to
figure that out um but all the data all
the thoughts are going to need to be
stored and retrieved somewhere um so if
you have any thoughts about that please
jump in the discussions um in the Ace
framework team we realized that that
communication between agents or between
layers is is actually like conceptually
the hardest problem to solve building
agents is
easy that's the easiest part the hardest
part is who's talking to who how much
how often who gets to hear what and and
all those communication channels so if
you have any ideas please let us know in
the discussion um and we'll be happy to
look at your suggestions because like I
said that's the hardest thing so the
first function though is the agent
Builder so we have we have greatly
expanded this I added a read me kind of
telling you a very high level overview
of what the agent Builder is so at the
highest level uh the primary thing we
need to do is figure out agents that
build agents in a structured
hierarchical manner so here's some of
the thoughts that I've had uh a lot of
this is based on feedback from the
comments in yesterday's video as well as
the discussions and uh on on um GitHub
as well as Discord uh so then you know
there's three primary parameters there's
instructions functions and file
retrieval there's there's a lot to
unpack with each of those so I unpack a
little bit like what goes into the
instruction so you need an archetype or
a Persona you need a context and a
mission um at a at a bare minimum and
then the functions there's all kinds of
functions that we can add so this is for
function calling or tools so there's
internal there's two two primary
dispositions for a function there's
internal which is something that the
agent can do internally um such as an
internal data tool or an internal coding
tool um and then there's external tools
which is making calls to other things uh
whether that's you know a cloud platform
or a data broker or a solar thing o I
just had an Insight what if we used like
we8 or or um or pine cone as an external
data broker um in order to organize all
the messages because then you can use
metadata to say who has access to what
oh I like this okay so if we have any
pine cone experts in the audience and I
know some of you people from Pine Cone
watch this
uh let's see if we can figure out using
that as like an external brain or an
external uh way of managing all of the
communication that could be cool because
the vector search is Fast and the
metadata allows for filtering so like
the metadata could include like what
tier it's at as well as what like what
team or whatever uh we could probably
implement some really primitive arbac
with metadata I like that so if you're
not familiar with what arbac is it's
ro-based Access Control
which means that an agent is going to
have uh some some tier so like Supreme
oversight board is tier zero the
executive group is tier one uh their
directors are tier two so on and so
forth and so you can say like okay this
this set of messages is tier zero
messages so that's like only the Supreme
oversight board is allowed to see those
messages obviously if it's just metadata
like that can be compromised but again
we're not trying to this is not trying
to be Enterprise this is just getting
the proof of concept down um so there
you I like that let's see let's see if
that tracks let me know in the comments
either here on YouTube or um or over on
the GitHub repo uh then retrieval so the
last thing is what files do you give the
agent and this the the basically the
file retrieval is its internal KB uh
system knowledge based system so we need
like an Agent Handbook and I realized
that this would be critical is that we
need to give every single agent in a
swarm the same handbook because that's
how that's like that's like the employee
manual or the user manual so every agent
will have the same Baseline
understanding as to what it's doing and
what it needs and standard operating
procedures and morality and ethics and
that sort of thing uh some other stuff
if it's an agent for building software
you're going to need to give it software
specifications definition of done that
sort of thing um and then any other
relevant documentation like uh how to
access the API how to write code code
examples that sort of thing anything
that's gerain to its Mission and the
functions that it needs to use and write
um and then finally chat functions so
these agents are still fundamentally
chat Bots there's the user input so
that's the input which could be an
instruction it could be context it could
be something else and then there's
output so then the question is where
does that user input come from and where
does the output go so the user input
could come from you know a supervisor
agent so like one agent talking to
another agent that should be relatively
easy to set up um and that might be the
most direct way because then you're not
you're not shipping um uh chat logs up
to an external Source it's just
supervisor tells you know another agent
go do this um you might also have group
chats so as we saw with chat Dev and a
few other things where you probably have
the you the you know one agent receiving
multiple inputs from other agents um so
we're going to need to figure out like
how to host a chat room basically which
that's kind of where I I started
thinking that maybe a pine cone or a we8
external uh Vector search search would
be good uh who knows let me know what
you think and then finally Telemetry so
let's say for instance you have an agent
that is monitoring a web server or an
agent that is interacting with um Azure
cloud or something like that it's going
to need to be receiving information
updates from whatever external machine
it's working on um and that could be a
function call as well um so we'll need
to figure out this this this schema this
Paradigm I have no idea how it's going
to work ultimately so if you got some
some ideas um figuring out the
conversation pattern the user input and
output with each agent please jump in
the discussion and let us know one thing
I will say is that um uh comments on
YouTube are great um but people on the
repo won't necessarily see those so if
you've got good ideas please jump over
to the discussions on the repo so that
everyone can see it and then like I said
finally the agent output figuring out
where that goes um is going to be
something that we need to figure out um
how do you route it right like so if you
have a supervisor agent how do you know
that the output goes to um like which
subordinate um or if it goes up to the
cloud or whatever so we'll need some
we'll need some schemas there so that's
what I've done and then the first poll
request um so it starts with a um or the
first uh instantiation so we've got a
basic script this was a member of the
ace framework team um got it working so
come over and check this out um it's
just a really basic agent Builder
there will be improvements there will be
iterative improvements but uh the idea
is that it is getting it's getting
started so check that out and then um
we've got a subfolder that has the
instructions that he figured out that
worked really well um as well as the
files to send up so you can see this is
starting to come together I have a
feeling it's not going to take that long
to figure out how to get the agent
Builder the know the agents that build
other agents well um there's there are a
few functions that we'll need to figure
out like uh IDE ideally we'll have
something that writes the instructions
with the system like a system function
that writes the instructions a system
function that writes the mission a
system function that writes the
archetypes that's what I mean like we
need to parameterize the whole thing so
we'll get there though because these
models are pretty powerful so I think
we're I think we're pretty close on that
so uh to wrap today's video up the last
thing I want to tell you about is the
toolmaker idea so I literally just
started this but basically if we have
something that can instantiate agents
one of the things that it needs to be
able to instantiate is tools so this is
writing the internal tools that it needs
um either finding them like you know
saving them in a library like making a
tool library but honestly like the
ability to write code and integrate it
and use it based on specifications is
going to be critical so basically this
is a this is a function calling function
writer uh so my idea here is that if you
have if you have a function or set of
functions or agents that are purpose
built for making the tools that the rest
of the Swarm needs it's kind of like the
worker bees right like it's like oh hey
The Hive the Swarm needs you know X Y
and Z tools and it goes off to the tool
makers they make the tools and send it
back and you know they tested it and and
that sort of thing that's one of the
things that I think we'll need um so
agent Builder toolmaker we'll probably
need a few other components but again I
think once we nail these like you guys
will be able to take it and run with it
so um yeah I think that's it for the day
um let's keep it going as as much as we
can and uh yeah let's get some stuff
done uh have a good one every
#Milestones to AGI: We'll reach the tipping point when AI can do these three things
all right so what do I mean when I say
AGI this is one of the questions that
comes up a lot
and like out in the out in the world one
of the best defin definitions that I saw
of AGI is we'll know when we see it um
and however I have put a lot of thought
into this and I had an idea as to maybe
not a definition but a milestone so
let's think of it in terms of what what
is the milone and what is a set of
achievements based on our current
trajectory when we could say anything
before this was not AGI and anything
after is Agi so this is when I say AGI
this is kind of the Milestone that I
have in
mind so it is when AGI is achieved when
machines or robots or AI can
collectively and
reliably improve
every element of the full stack and so
what I mean by Stack is Hardware
software and AI models so that includes
data algorithms and that sort of stuff
so uh Anastasia and Tech had uh a couple
videos last week about you know chat GPT
participating in chip
design and uh of course like machine
learning has been used in chip design
for for ages uh but it's getting better
and so for in
uh oh and another thing is uh Ai and
machine learning has been used in the
physical design of robots and other
pieces of
Hardware as well as learning how to make
robots move right like spot and Boston
Dynamics and all those they use neural
networks to figure out movement
now so however it's not there yet it's
still is primarily run by
humans however once AI can take over all
of the hardware design research
fabrication and Improvement that's like
pillar
one and then pillar two is going to be
once Ai and machines can take over the
whole software stack so that that starts
with the operating system you know
kernel level up and that includes
applications that includes managing
entire software repositories and so we
saw the first hints of this with chat
Dev but imagine a year from now
with agent swarms that are capable of
testing the entire life cycle of like
Windows 13 and the next Linux kernel and
the next Unix kernel once I can do that
cuz I mean riding operating systems is
hard as well as creating new programming
languages so once it once it hits that
Milestone where it's like hey we have
the first operating system that is
entirely generated by Ai and the first
Hardware platform like server platform
and robot platform entirely generated by
Ai and then the third pillar is going to
be the data and the training
algorithms uh and the and the and the
deployment of AI models themselves so
like if GPT 4 eventually contributes to
making GPT 5 for instance and then let's
say GPT 5 is multimodal and it also
completely end to end creates GPT 6
that'll be the final
Milestone so like that's kind when I say
AGI that's that's what I mean so before
I get to the rest of the video I do have
a quick favor to ask so I've mentioned
that I have a bunch of books that I'm
working on two are well one is ready I
just need to like go through the final
polishing phase um that's my novel heavy
silver and then my systems thinking book
I just finished draft three
um now before I launch these books I
need to have people on my mailing list
so that means either substack which is
completely free or patreon which has a
free tier and the magic number that
people tell me is I need to get to 3,000
so I need between those two I need at
least 3,000 total people on my mailing
list so I've got a th000 on patreon and
500 on substack so I'm halfway there so
I just need, 1500 of you to either sign
up on the free tier of my patreon or
suback which is also free and that'll
get me to the Tipping Point where I'm
ready to launch my novel and ready to
launch systems thinking once it's out so
all right back back to the talk so if we
hit those three Milestones of of of AI
driven self-improvement that's going to
be the Tipping Point and so I think in
hindsight if if that's the way it plays
out cuz it might not play out that way
but just taking a first principal's view
of okay let's look at the whole stack
right Hardware software and then AIML
and then everything that goes within
those right because we already have data
pipelines we've got data curation
pipelines but it's still humans setting
up those pipelines so like once AI takes
over the building of pipelines which is
why I'm so excited about the autonomous
agent swarm is because each and I and so
here's why I'm making this video is
because some people have commented and I
understand understand their confusion
they're like Dave like you're talking
about autonomous agent swarms this is
not AGI no not right now but it is it is
a major component because autonomy is
one of the characteristics that you'd
expect of AGI full
autonomy and that means self-direction
self-correction and
self-improvement full autonomy has all
three of those and it just doesn't need
humans anymore it doesn't need humans to
tell it what to do it doesn't need
humans for maintenance it doesn't need
humans for you know whatever man I'm
getting toasty cuz I'm talking so
fast so uh right so that's why I'm so
excited about autonomous agent swarms
and uh oh also I just I want to say that
I get a total kick out of all you people
that are like is Dave outside the
funniest comment yet was who decided to
upgrade him to legs really like that was
hilarious I was like anyways
so where we're at right now is these
tiny agents and we're trying to figure
out how to instantiate them and how to
get them communicating with each other
but then of course like there's some
underlying model improvements that need
to be made cuz I suspect that GPT for
Turbo while it can help um with with
algorithm research and Hardware research
and design it probably is just not quite
smart enough to do every step every
level uh uh entirely on its own now
again if it can help and accelerate it
great that is one piece of the puzzle
that's a
keystone uh technology to help make it
go faster so anyways thanks for watching
I'm really excited and uh yeah let me
know what you think you guys seem to
like these kinds of riffing videos so
let me know in the comments like
subscribe follow me on LinkedIn patreon
so on and so forth uh you know the drill
take
care
#Automating Myself Out of a Job: Orchestrating an Autonomous Agent Swarm
I am trying to automate myself out of a
job so back in my days as an IT
infrastructure automation engineer one
of the key uh lessons that I thought was
uh if I automate myself out of a job
that means that one I'm a good automator
but two it also means that I just have
less work to do and I can save my time
for doing other stuff so one of the one
of the key secrets to automation is you
save yourself 5 10 20 minutes every day
uh and then it adds up to 1 2 3 4 hours
every day until eventually you have
automated so many things that you are
getting literally hundreds if not
thousands of hours worth of work done
every single day and every single week
which means that you can either kick up
your heels and just stick around on the
internet while while you're at work or
you can automate even more stuff and
learn and that's basically what I did
and that's how I got to where I'm at so
I uh got access to open ai's new gpts
feature and of course the first thing
thing that I did was build something
that people have been asking for for
literally like years now is a virtual
David chapiro to talk to so Link in the
description it is out there you can go
use it it will I will work on improving
it over time but right now it is
equipped with all of my
transcripts so you can uh you can ask it
all kinds of questions so in this case
uh you know I said what are the
requirements to achieve AGI and it says
a temporal window and axima Alignment uh
nashy equilibrium framework and
Consciousness and learning abilities
again it's not perfect I need to figure
out how to format all the data
underneath but like you can tell that
it's clearly picking up on some of the
things that I have talked about so
that's what I where I wanted to open the
video uh I'll do a real quick plug for
my patreon um so I'm starting to do
monthly Master Class sessions for the
premium tier patreon subscribers so this
is going to be uh a monthly kind of
webinar session um up to 100 people it's
going to be recorded so even if you if
even if you miss uh miss it if you're on
the premium tier you'll get to see the
recording uh but yeah so uh we're going
to talk about uh the state of the AI
industry I'll give you kind of a brief
update from my perspective what I'm
seeing the trends the Technologies the
techniques that sort of thing then uh
the second part of the master class
session will be a lesson in systems
thinking from my upcoming book and then
third finally um we'll do a Mastery
teaching session which is more or less a
Q&A format but it's a it's a Mastery
dialogue uh session to kind of talk
about everything that we've covered and
that is uh the first one is today at 10:
a.m. us Eastern so if you're seeing this
you might have already missed the window
but go ahead and sign up uh I will be
raising the price of my premium patreon
tier uh in early 2024 um as the value of
this content goes up so with all that
being said let's move on over to the
open aai Agent swarm So speaking of
automating myself out of a job uh this
is really exploding we have had so much
engagement um and so what I wanted to do
was uh basically conduct this and so
what I wanted to do was basically
conduct this like uh kind of a scrum
where I just kind of show everyone all
the discussions that have been happening
and tell you kind of what the next steps
are so uh right at the very top um we
have a couple folks jumping in and we're
we're experimenting with what are the
the function s the tools that you need
to to create another agent and so what I
did was I went and tried to create uh
create an agent over here with the
agents um I called it hephestus Hammer
of the the hos project and it didn't it
didn't work really well I really I
really barfed it like cuz I asked like
what is the per the purpose of the
Supreme oversight board so let's see if
it works cuz it often doesn't but
anyways my idea right now is because
it's so easy to to build these gpts
maybe these gpts at the front end should
be the way that we orchestrate the rest
of the the autonomous swarm and so while
this is running I wanted to
differentiate because when I say
autonomous some people say like what do
you mean and some I I I've seen a few
comments I've had a few people reach out
on LinkedIn they say oh well this is a
fully autonomous framework and it's like
if it weights for a human instructions
it is not autonomous that is
orchestration orchestration is where a
human pushes button and then a rubbe
Goldberg machine comes to life and it
goes and does the human's bidding and
then once it's done it stops the purpose
of an autonomous swarm is that you set
up the initial configuration with a
mission with Mission parameters or or
higher order goals and the Swarm keeps
going indefinitely um and it doesn't it
it it might need some help or feedback
at some points but it is designed to not
need human feedback um so in this case
the the hephestus hammer uh say said the
Supreme oversight board within the the
hierarchical autonomous agents SW okay
so in this case it worked like I said
we're still figuring this out so
establishing and upholding ethical
standards high level decision-making
ultimate authority of checks and
balances guiding and Searing the house
okay cool so you are there I'll have a
link to this one as well but again be
warned this is still in beta we're still
figuring this out and we're fudging
through but we you have an agent called
hesus Hammer that can help teach you
about the the hos framework and I also
gave it the the the documentation so
hypothetically it can help you do the
coding um okay so that's pretty cool and
let's uh oops let's come back to where
did it go okay so then one of the other
problems that we're discussing about is
how do you organize all this and so as I
mentioned uh a bunch of people have
really jumped in and it's been super
helpful and so here's here's someone
actually threw together a great diagram
as to how this works so in this case the
the Saab the Supreme oversight board
will be populated with a bunch of
archetypes or personas or whatever they
are the they're going to debate about
what to do how when and why but again
they're given the overarching mission
and the context of what the Swarm is
supposed to do in this case they chose
Captain Kirk King Solomon and Socrates
now that would be an interesting
conversation um and so the Saab is they
create the executive board and then the
executive board um focuses on all the
other actual tasks so one thing that I
did want to point out is that there have
been like I'm really glad that uh a
bunch of people have jumped in um and
I've had to I've had to to to steer the
ship pretty hard because uh despite
despite the enthusiasm uh some people
just didn't quite get it and they wanted
to change the scope of the project
they're like let's do local agents we're
not doing local agents right now because
that's going to take way too long and it
is time to move fast and Break Stuff
also someone tried to like add the idea
of like let's do a dow a decentralized
autonomous organization again that's a
bottom up management this is a top- down
management so uh just wanted to say like
this design is on purpose um and and
that is because in order to orchestrate
a swarm you need to have top- down
command and control whereas you don't
want the the agents at the very bottom
to all be voting one that's a waste of
tokens and two Dows are good for
organizing social movements where where
like equality for all is critical this
is not about equality this is about
getting stuff done as efficiently as
possible so the hos framework is a top
down command and control structure so I
really appreciate people doing this um
but yeah this is a perfect diagram so
tier zero is the Supreme oversight board
tier one is the executive board tier two
and all the all the child agents below
that so basically each agent is able to
create worker agents um as needed in
order to delegate out now we'll probably
have some some general rules where it's
like okay once you get to tier five
you're basically EXP Ed to be a worker
and like do something so one idea that I
had was that in or instead of having one
agent keeping track of multiple tasks
you have one task per agent and so that
might mean you know you might spin up an
agent with the sole purpose of like
write this function or or fix this bug
or send an email to you know Mark
Zuckerberg or whatever right like so one
task for one agent probably at tier four
or five this is where I'm thinking we're
going it's obviously not set in stone so
thanks for jumping in and contributing
that um I think that was it for this
thread yes so okay and it looks like
someone has jumped in and created a tool
Creator and Tool user demo so I'll show
you this this conversation so complete
the working implementation of automated
tool and assistant Builder so this is
great I'll I'll add a comment like I'll
check this out uh recording now um
submit a PR please once it's done so yep
uh let's see moving right along
use AI coder like a sweep so this was a
this was a great idea that that uh uh
Alexander inking um came up with which
so basically this was like a meta
process for probably instantiating
swarms or evolving swarms over time uh
which I really appreciate that idea uh
we're probably not going to focus on
this just yet but basically one swarm is
going to have like one overarching
Mission um but the Swarm you might need
to deprovision the entire swarm and then
start over and so then it's like what if
you treat um all of the different
components of the Swarm like a genetic
algorithm so that that way you can just
say like here's a bunch of variables so
then when you go to instantiate a swarm
you can measure like how well the Swarm
did and then tweak those variables later
so I really appreciate that idea and
here was another idea this one is a
little bit complicated um but it is a
really good workflow of the the actual
formal Logic the workflow of how things
uh work um we're still we're still
workshopping uh exactly how the agent
are going to communicate so uh member of
the ace framework and a few other people
have some ideas about how that will work
uh but yeah so there we go we have
someone who created another uh GPT with
a chat interface so let's see explain
agent swarms this is good but yeah so um
now that you're kind of a little bit
more familiar with where we're at today
I do want to invite people to submit
more um issues and pull requests and
I'll show you exactly what I what I want
and what I mean in just a moment there
we go cool so this this agent swarm um
GPT seems to be working okay so I have
two issues out here so this is where
we're getting into like uh let's let's
have the rubber meet the road so one
idea that I had was I I would like if
someone would create a Discord version
um and so where where you could put that
in the in the repo is just do like
another folder like you know Discord
connector or Discord version just to get
the ball rolling that might be too
complicated um at at first um because
we've also been talking about like how
do we get agents to talk directly to
each other and that sort of thing and
the other Bounty that I have out here is
take a stab at building a toolmaker
agent so uh a few people have already
done that so let's uh as a community
let's look at some of the proposals that
we have there's a few out there um and
um I'll I'll actually create some better
organization in the discussion so by the
time you see it I'll have a few better
few better things here and I'm also
going to create another issue um that is
about doing some of the the
instantiation via the gpts uh rather
than just the API so basically I want
the GPT to be able to use the API in
order to to uh orchestrate all this
stuff uh I hope that makes sense but
yeah so thanks for watching um yeah this
is really exciting and I think I'm
finally getting the hang of how do you
organize and orchestrate open source
development um but yeah so what I'm
going to do right after this and by the
time you see this video is I'm going to
organize the discussions a little bit
better and organize the issues a little
bit better and yes once once we have the
uh a look just a little bit more I'd
like to see some more poll requests
either today or this weekend or next
week um so that we can get some of these
demos up and running and I think that as
fast as you guys are moving we've got
the agent Builder we've got the
toolmaker now we just need to integrate
it and then the last thing is figuring
out communication Cu uh folks definitely
see and understand the the hierarchy so
thanks for watching like subscribe etc
etc hop on over on patreon hop on over
on Discord uh connect with me on
LinkedIn also I've got a free substack
all the links are in the description um
but yeah thanks for thanks for coming
along for the ride this is super
exciting and um yeah I'm just super
jazzed we're we're getting some we're
getting some real work done have a good
one everybody