Newsgroups: comp.robotics
Path: brunix!cat.cis.Brown.EDU!agate!howland.reston.ans.net!swrinde!ihnp4.ucsd.edu!library.ucla.edu!csulb.edu!csus.edu!netcom.com!jfox
From: jfox@netcom.com (Jeff Fox)
Subject: Re: 100 Billion Neurons Nonsense NONSENSE
Message-ID: <jfoxCsH5vM.GLE@netcom.com>
Sender: jfox@netcom.com (Jeff Fox)
Organization: Netcom Online Communications Services (408-241-9760 login: guest)
References: <jfoxCs9v8K.Eu3@netcom.com> <sm6woc1w165w@sfrsa.com>
Date: Tue, 5 Jul 1994 16:02:10 GMT
Lines: 168

In article <sm6woc1w165w@sfrsa.com> bsmall@sfrsa.com (bsmall) writes:
>jfox@netcom.com (Jeff Fox) writes:
>> try comp.ai.philiosophy, this is comp.robotics! :-)
>
>I use the Usenet Waffle provided by the SF Robotics Society of 
>America and this free service only provides the comp.robotics
>group. Other groups such as the comp.ai.neural_nets have also
>been suggested. Since my interest is just robotics I believe that
>the subject matter is appropriate to this group.
> 
> 
>I don't think the group should be limited at all. I have just 
>recently built an autonomous sumo robot with no electronics at
>all. But that doesn't mean I can't think about what real intelligence
>is comprised of.

I have seen the sumo robot, and it does make you think about how to
real things.  I just don't want this group to digress from that kind
of thing into discussing subjects like "Can a computer with 100
billion artificial neurons ever really think? Will it have soul?
etc, etc"  Since no one can definitively answer these questions
at this time it remains philosophy.  You can read all you want
about that in comp.ai.philosophy, and from some people who have
devoted their lives to these questions.  I am merely suggesting
that we limit the discussion here to comp.robotics without
endlessly arguing about the philosophical reasons why this or
that can never happen.  It might be... or might not be ... :-)

>
>> Although there are artificial insect robots with a bunch of these
>> primitive micros enabling reflexes and life like movements there
>> is a BIG difference between a few neurons and billion of neurons
>> with trillions of interconnections!
> 
>Of course. I'm not sure where you came in on this thread but in
>my original post I was trying to say how many authors talk about
>the 100 billion neurons and conclude that we can't match that and
>therefore robots won't every be a competent as humans. But in 
>reality, we can see performance in the flatworm, and other insects,
>that outperforms what we have now in robotics in terms of obstacle
>avoidance and motor control although, and this I must admit is just
>a gut feeling, we have the same amount of computing power on our 
>desktops than these insects.
>

Ah, exactly, "therefore won't ever..." , I prefer to read about
reality (as other people see it) than predictions of what will never
happen.  I consider "therefore won't ever ..." as attempts to disprove
rather than proving by example.  But I really think this is getting
off into philosophy again.

>> For perspective I would suggest that you start with the number of
>> instructions your robot can execute in a millisecond, divide this
>> by ~100 (if your a clever programmer) and you have the number of
>> equivalent neurons.  So most of the robots I have seen are dealing
>> with a couple or a few neurons.  On the high end, if you have a 30 MIP
>> CPU that might be equal to a few hundred.  If your robot has dozens of
>> these high end CPU then maybe it can simulate thousands of neurons.
>> 
>Well I don't know where you are getting these numbers. Suppose we have
>a cell neuron that receives "stimulus" from neighboring neurons.
>Suppose it counts up to 256 and rolls over. Then you attach a table
>to this mathematical neuron that has vectors to, say, 10 other 
>neurons and a "weight factor" to be added into their summation 
>counters. 
>

Well, you know how fast real neurons are, and how many inputs are
needed etc.  All of the simulated neural networks I have seen are
quite different than an index into a table.  Typically they involve 
multiplying the inputs by weighting factors then maybe indexing to
a table.  I have neural network hardware that does just this using
analog op-amps and resistors to do analog multiplications, this is
very fast.  But simulating this on a digital computer is relatively
slow.  Neurons are slow too, so do the math.  How fast can a 3megahertz
micro perform say 32 multiply accumulates and a table lookup.

Now if you can show me how a counter can perform the same function
I would be interested.
 
>What have you got. 1 Instuction to up the counter. 1 instuction to
>decide if it rolled over. 1 instruction to set up the vector to 
>the "attached neuron". 1 instruction to see if that neuron has
>rolled over (or fired). Times 10 attached neurons. That's 22 
>instructions per firing neuron and only 2 if it doesn't fire.
>And let's guess that only 10% of the nuerons are firing at any
>one time. And lets say were using a 3mps micro. Doesn't that mean
>we could have about 1 million neurons. Any do we really need that
>many?
> 
>
Well, can your 3mps micro hold 1 million of these tables?  You are
talking about 256 vectors * 1 million tables of address that are
large enough to point to many other simulated neurons.  I would
disagree with you that you can simulate a neuron with a counter
and a table, but even if it would you are talking about 500 megabytes
or more of data tables anyway.  I would suggest that you are really
talking about alot more computation than you think.  Sure tables
are fast.  How large of a table do you need to simulate say 32
weighted inputs to a neuron? Do you really think that a 3mps micro
can simulate 32 multiply accumulates * 1,000,000 neurons every
milisecond?  

> For a while at least I think discussions of hundreds of billions of
>> neurons is NONSENSE here.  Unless you are using some VERY exotic
>> hardware in your robot lets keep the discussions real! :-)
>> 
>Believe me, my discussions ae always real.
>

Actually I know about your robot vision project, and it is a good
example of how you can do things many many times more efficiently than
the approaches suggested by conventional wisdom.  If you really do
know how to simulate 1M neurons on a 3mps micro, then can I use
this technique?  If so 100 billion simulated neurons is just around
the corner.
 
>> The questions that are more appropriate here are what you can do with
>> a few neurons in a robot, not what 100 billion will do.  Since no one
>> has 100 billion neurons in their robot I think this should remain
>> in comp.ai.philosophy.  Don't get me wrong, I would LOVE to hear
>> about your robot with thousands of high end cpu and neural network
>> chips wired together, and programmed with 20 years of education and
>> trail and error experience at human like activities.
>> 
>> I think the nonsense belongs in .philosophy. :-)  Maybe we should
>> start a comp.robotics.philosophy.nonsense newsgroup.
>> 
> 
>Well here again you have misread my original intent and I'm not
>funded with high end CPU's. An interexting concept though, the 
>training of the robot. Will it take 20 years or can we speed up
>the natural process.

Well who knows, but one of features of neural nets is training.
Humans are born with many neural connects already made, but
more connections are made, and weighting factors are set through
training and experience.  If you experiment with a simulated
network, train it, and record the weighting factors you can
then build it in hardware with some resistors and amps, it is very
simple.  Trainable circuits that can set their own weighting factors
are more involved.  But I would suggest that signifigant training
is involved.  Humans do have 100 billion neurons, and they need LOTS
of training to be able to pass the Turing test.  Even Brook's robots
trash around until they learn to coordinate activites.  But of course
it would be very nice if your robot did not take 20 years of training.
So lots of people are working on that.

>                      I have heard something mentioned at the 
>Homebrew Robotics Club that Rodney Brooks is trying to get funding
>to start such an experiment and try to create a 6 month year old
>robotic child.

That sounds very interesting, and very exciting, I look forward to
learning more about it if the project develops.

>
>> Jeff Fox
>> Ultra Technology
>> 2510 10th St.
>> Berkeley CA 94710
>> (510) 848-0565
>> jfox@netcom.com
>
>Brad Smallridge
>bsmall@sfrsa.com


