11/24/11 Old-school theor is a new force - CNET News
1/6 news.cnet.com/2009-1001-984695.html
CNETNes
18th-century theory is new force in computing
By Michael Kanellos [mailto:[email protected]]
SLuII WrILer, CNET News.com
ebruury 18, zoo, q:oo AM PT
Thomas Bayes, one of the leading mathematical lights in computing today, differs from most of his
colleagues: He has argued that the existence of God can be derived from equations. His most
important paper was published by someone else. And he's been dead for 241 years.
YeL LIe 18LI-cenLury cIergymun's LIeorIes on probubIIILy Iuve become u mujor purL oI LIe muLIemuLIcuI
IounduLIons oI uppIIcuLIon deveIopmenL.
SeurcI gIunL Google [http://www.google.com/corporate/index.html] und Autonomy
[http://www.autonomy.com/] , u compuny LIuL seIIs InIormuLIon reLrIevuI LooIs, boLI empIoy BuyesIun
prIncIpIes Lo provIde IIkeIy (buL LecInIcuIIy never exucL) resuILs Lo duLu seurcIes. ReseurcIers ure uIso usIng
BuyesIun modeIs Lo deLermIne correIuLIons beLween specIIIc sympLoms und dIseuses, creuLe personuI roboLs, und
deveIop urLIIIcIuIIy InLeIIIgenL devIces LIuL "LIInk" by doIng wIuL duLu und experIence LeII LIem Lo do.
DespILe LIe esoLerIc symboIs, LIe Ideu--rougIIy speukIng--Is sImpIe: TIe IIkeIIIood LIuL someLIIng wIII Iuppen
cun be pIuusIbIy esLImuLed by Iow oILen IL occurred In LIe pusL. ReseurcIers ure uppIyIng LIe Ideu Lo everyLIIng
Irom gene sLudIes Lo IIILerIng e-muII.
A deLuIIed muLIemuLIcuI rundown cun be Iound on LIe UnIversILy oI MInnesoLu's Web site
[http://www.mrs.umn.edu/sungurea/introstat/history/w98/Bayes.html] . And u Bayes Rule
Applet [http://www.gametheory.net/Mike/applets/Bayes/] on GumeLIeory.neL IeLs you unswer
quesLIons sucI us "How worrIed sIouId you be II you LesL posILIve Ior some dIseuse?"
One oI LIe more vocuI BuyesIun udvocuLes Is MIcrosoIL. TIe compuny Is empIoyIng Ideus bused on probubIIILy--or
"probubIIIsLIc" prIncIpIes--In ILs NoLIIIcuLIon PIuLIorm. TIe LecInoIogy wIII be embedded In IuLure MIcrosoIL
soILwure und Is InLended Lo IeL compuLers und ceII pIones uuLomuLIcuIIy IIILer messuges, scIeduIe meeLIngs
wILIouL LIeIr owners' IeIp und derIve sLruLegIes Ior geLLIng In LoucI wILI oLIer peopIe.
I successIuI, LIe LecInoIogy wIII gIve rIse Lo "conLexL servers"--eIecLronIc buLIers LIuL wIII InLerpreL peopIe's
duIIy IubILs und orgunIze LIeIr IIves under consLunLIy sIIILIng cIrcumsLunces.
"BuyesIun reseurcI Is used Lo muke LIe besL gumbIes on wIere sIouId IIow wILI compuLuLIon und bundwIdLI,"
suId ErIc HorvILz, senIor reseurcIer und group munuger oI LIe AdupLIve SysLems & nLerucLIon Group uL
Developers are using
Bayes' centuries-old
ideas to take on the
Information Age.
TIey're buIIdIng
progrums desIgned Lo
uuLomuLIcuIIy munuge
LIe deIuge oI duLu LIuL
geLs LIrown uL peopIe
eucI duy vIu e-muII, ceII
pIones, InsLunL
messugIng progrums
und LIe IIke.
SucI u progrum wouId
coIIecL duLu over LIme
und creuLe u modeI oI u
person's pusL beIuvIor
Lo deLermIne Iow besL
Lo deuI wILI u newIy
urrIved messuge.
A sImpIIIIed exumpIe:
1) E-mail arrives
rom: TIe boss
SubjecL: Reud me
SenL: TIu :1o PM
Program accesses
calendar software
"TIursduy, z PM:
11/24/11 Old-school theor is a new force - CNET News
2/6 news.cnet.com/2009-1001-984695.html
said Eric Horvitz, senior researcher and group manager of the Adaptive Systems & Interaction Group at
Microsoft Research. "I personally believe that probability is at the foundation of any intelligence in an uncertain
world where you can't know everything."
Toward the end of the year, Intel will also come out with a toolkit for constructing Bayesian applications. One
experiment deals with cameras that can warn doctors that patients may soon suffer strokes. The company will
discuss these developments later this week at its eveloper Iorom
[http:[[developer.intel.com[idI[index.htm| .
Despite its popularity today, Bayesian theory wasn't always universally accepted: Only a decade ago, Bayesian
researchers dwelled on the fringes of their professions. Since then, however, improved mathematical models,
faster computers and valid results from experiments have given new credibility to the school of thought.
"One of the problems was that it was overhyped," said Omid Moghadam, manager of application software and
technology management in Intel's Microprocessor Lab. "In reality, the power to do anything serious didn't exist.
The real implementation has taken place in the past 10 years."
Buyes Ior dommies
Bayesian theory can roughly be boiled down to one principle: To see the future, one must look at the past. Bayes
theorized that the probability of future events could be calculated by determining their earlier frequency. Will a
flipped coin land heads up? Experimental data assigns it a value of 0.5.
"Bayes said that essentially everything is uncertain, and you have different
distributions on probability," said Ron Howard, a professor in the Department of Management Science and
Engineering at Stanford.
Suppose, for example, that instead of flipping a coin, a researcher tossed a plastic pushpin and wanted to know
what the chances were that it would land flat on its back with the pin pointing up, or, if it landed on its side, what
direction it would be pointing. Shape, imperfections in the molding process, weight distribution and other
factors, along with the greater variety of outcomes, would affect the results.
The appeal of the Bayesian technique is its deceptive simplicity. The predictions are based completely on data
culled from reality--the more data obtained, the better it works. Another advantage is that Bayesian models are
self-correcting, meaning that when data changes, so do the results.
Probabilistic thinking changes the way people interact with computers. "The idea is that the computer seems
more like an aid rather than a final device," said Peter Norvig, director of security quality at Google. "What you
are looking for is some guidance, not a model answer."
Search has benefited substantially from this shift. A few years ago, common use of so-called Boolean search
engines required queries submitted in the "if, and, or but" grammar to find matching words. Now search engines
employ complex algorithms to comb databases and produce likely matches.
As the pushpin example shows, complexity and the need for more data can accelerate rapidly. Harnessing the
results required to transform a good guess into a plausible outcome has become possible through the emergence
"Thursday, 2 PM:
Meeting"
Progrum considers
hubits und corrent
sitoution
When at computer,
usually responds to e-
mails from "The boss"
in 1 to 5 minutes.
When meeting
scheduled for Thursday
afternoon, keyboard
activity usually resumes
1.5 hours from
meeting's listed start
time. (Thus, Thursday
meetings usually last an
hour and a half.)
Progrum druws
conclosions
Meeting started at 2.
E-mail arrived at 3:10.
Recipient probably
away till 3:30.
Usually answers "The
boss" in 1 to 5 minutes.
Progrum ucts
Forwards e-mail to
recipient's cell phone
for immediate retrieval.
) I-muil urrives
From: "The middle
manager"
Subject: Idea
Sent: Thu 3:10 PM
Progrum uccesses
culendur soItwure
"Thursday, 2 PM:
Meeting"
Progrum considers
hubits und corrent
sitoution
When at computer,
usually responds to e-
mails from "The middle
manager" in 25 minutes
to an hour.
Progrum druws
conclosions
11/24/11 Old-school theor is a new force - CNET News
3/6 news.cnet.com/2009-1001-984695.html
results required to transform a good guess into a plausible outcome has become possible through the emergence
of powerful computers.
More importantly, researchers such as Judea Pearl at UCLA have learned how to make Bayesian models that
better home in on the conditional relationships between different phenomena, which greatly reduces the number
of calculations.
A quest in the population at large for the causes of lung cancer would reveal it to be a minor disease, for
instance, but research confined to smokers would show some correlation. Examining lung cancer victims can
then help draw a hypothesis on causation between the disease and the habit.
"Every individual attribute or symptom can depend on a lot of different things, but it depends directly only on a
small number of things," said Daphne Koller, an assistant professor in the computer science department at
Stanford. "In the past 15 years or so, there has been a revolution in tools that will allow you to represent large
populations."
Among other projects, Koller is using probabilistic techniques to better match
symptoms to diseases and to link genes to specific cell phenomena.
Speaking to numbers
A related technique, called Hidden Markov models, allows probability to anticipate sequences. A speech
recognition application, for example, knows that the sound most likely to follow "q" is "u." Along those lines, the
software can also calculate the possible utterance of the word Qagga, an extinct zebra.
Probabilistic techniques are already embedded in Microsoft's products. Outlook Mobile Manager, which
determines when to send a deskbound e-mail to a mobile device, grew out of Priorities, an experimental system
unveiled at Microsoft in 1998. The troubleshooting engine in Windows XP also relies on probabilistic
calculations.
More applications will trickle out over the coming years as the company's Notification Platform becomes
embedded in products, Microsoft's Horvitz said.
An application named Coordinate, a major element of the Notification Platform, gathers data from personal
calendars, keyboards, sensor cameras and other sources to create a mosaic of a person's life and habits. The data
gathered can include arrival schedules, typical time and length of lunches, what types of phone and e-mail
messages are kept or discarded, how frequently the keyboard is in use at given times of the day, and so on.
Such data can be used to manage the flow of messages and other information to people who use the application.
If a manager sent an e-mail to a worker's computer at 2:40 p.m., for example, Coordinate could check that
worker's calendar program and find that a meeting was listed for 2:00 p.m. The program could also scan data
about the worker's habits and discover, say, that the person usually resumed keyboard activity about an hour
after the listed start times of meetings. The program might also find that the worker typically responded to e-
mails from this manager within five minutes. Based on all that data, and given that the worker probably wouldn't
return to the computer for at least 20 minutes, the program could decide to forward the message to the worker's
cell phone. Meanwhile, the program might decide not to forward e-mails from other people.
"We're balancing the value of information that is coming in with the cost of interrupting you," Horvitz said. With
these applications, he maintained, "there will be a lot more people keeping up with things and not drowning in
information."
conclusions
Meeting started at 2.
E-mail arrived at 3.
Recipient probably
away till 3:30.
Usually answers "The
middle manager" in 20
minutes to an hour.
Program acts
Decides not to forward.
OK for recipient to get
it upon return.
Editors: Mike
Yamamoto, Edward
Moyer
Cop editor: Lisa
Denenmark
Design: Pam Dore
Production: Meghan
McDowell
11/24/11 Old-school theor is a new force - CNET News
4/6 news.cnet.com/2009-1001-984695.html
PrIvucy und user conLroI over LIese IuncLIons, HorvILz udded, Is ussured. CuIIers don'L know wIy u messuge muy
Iuve been prIorILIzed or pusIed buck.
OLIer MIcrosoIL BuyesIun proLoLypes IncIude DeepIsLener und QuurLeL (voIce ucLIvuLIon), und SmurLOO und
TImeWuve (conLucL conLroI). Consumer muILImedIu uppIIcuLIons wIII uIso beneIIL, HorvILz suId.
BuyesIun LecInIques wIII uIso go beyond LIe PC. AL LIe UnIversILy oI RocIesLer, reseurcIers Iuve deLermIned
LIuL u person's guIL cun cIunge beIore u sLroke. WIIIe LIe cIunges ure Loo subLIe Ior Iumuns Lo Lruck, u cumeru
IeedIng duLu Lo u PC cun cupLure und Lruck movemenLs. TIe compuLer cun LIen send un uIerL II wuIkIng unomuIIes
occur.
An experImenLuI securILy cumeru uses LIe sume prIncIpIe: MosL uIrporL puLrons go sLruIgIL Lo LIe LermInuI uILer
purkIng, so someone wIo purks und LIen goes Lo unoLIer cur Is ouL oI LIe ordInury und cun LrIgger un uIerL. A
busIc engIne Ior creuLIng u BuyesIun modeI und LecInIcuI InIormuLIon wIII be posLed Lo nLeI's deveIoper sILes LIIs
IuII.
Clah of he ned
AILIougI LIe LecInIques sound sLruIgILIorwurd, LIe compuLIng worId Ius been sIow Lo embruce LIem. HorvILz
recuIIed beIng onIy one oI Lwo gruduuLe sLudenLs uL SLunIord workIng on probubIIILy und urLIIIcIuI InLeIIIgence In
LIe 1q8os. Everyone eIse wus sLudyIng IogIcuI sysLems, LIose LIuL InLerucLed wILI LIe worId LIrougI "II und LIen"
sLuLemenLs.
"ProbubIIILIes were deIInILeIy ouL oI IusIIon," HorvILz suId. TIe LIde Lurned us IL
becume uppurenL LIuL IogIcuI sysLems couIdn'L unLIcIpuLe uII unexpecLed cIrcumsLunces.
Muny reseurcIers uIso begun Lo ucknowIedge LIuL Iumun decIsIon-mukIng Is Iur more mysLerIous LIun orIgInuIIy
beIIeved. "TIere wus u cuILuruI bIus wILIIn LIe urLIIIcIuI-InLeIIIgence communILy uguInsL numbers," KoIIer suId.
"PeopIe now recognIze LIuL peopIe don'L reuIIze wIuL LIey do In LIeIr Ieuds."
Even In IIs duy, Buyes Iound IImseII ouLsIde LIe muInsLreum. Born In 1;oz In ondon, Ie becume u PresbyLerIun
mInIsLer. AILIougI Ie suw Lwo oI IIs pupers pubIIsIed, IIs prIncIpuI work, "Essuy Towurd SoIvIng u ProbIem In
LIe DocLrIne oI CIunces," wusn'L pubIIsIed unLII 1;6q, LIree yeurs uILer Ie dIed.
HIs membersIIp In LIe presLIgIous RoyuI SocIeLy wus someLIIng oI u mysLery unLII recenL yeurs, wIen newIy
dIscovered IeLLers sIowed LIuL Ie prIvuLeIy corresponded wILI LIe oLIer IeudIng LIInkers oI EngIund.
"He never, us Iur us cun LeII, wroLe down Buyes' LIeorem," Howurd suId oI LIe IormuI muLIemuLIcuI IormuIu.
TIeoIogIun RIcIurd PrIce und rencI muLIemuLIcIun PIerre SImon uPIuce becume eurIy cIumpIons. TIe Ideus,
LIougI, run counLer Lo LIose seL ouL IuLer by George BooIe, IuLIer oI BooIeun muLI, wIIcI Is bused on uIgebruIc-
IIke IogIc und evenLuuIIy guve bIrLI Lo LIe bInury sysLem. BooIe, uIso u member oI LIe RoyuI eIIows, dIed In
11/24/11 Old-school theor is a new force - CNET News
5/6 news.cnet.com/2009-1001-984695.html
like logic and eventually gave birth to the binary system. Boole, also a member of the Royal Fellows, died in
1864.
While few discount the importance of probability, debate on its uses lingers. Critics periodically assert that
Bayesian models depend on inherently subjective data, leaving humans to judge whether an answer is correct.
And probabilistic models do not completely account for the nuances in the human thought process.
"It's not exactly clear how children learn," said Alfred Spector, vice president of services and software at IBM's
Research Division, who proposes mixing statistical methods with logical systems in a so-called Combination
Hpothesis [http://www.cnet.com/IBM-aims-to-get-smart-about-AI/2100-1008_3-981256.html] .
"I'm convinced it's statistical initially, but then at a certain point, you will see at three it is not just statistical."
Yet, probability is, in all probability, here to stay.
"This is a foundation," Horvitz said. "It was overlooked for a while, but it is a foundation for reasoning."
[]
2 comments
Join the conersation! [] Add our comment [#addcomm] (Log in or register [#addcomm] )
[]
Heuristicsisreallabaesianapproach?
I would like to suggest the negative answer.
In most heuristic techniques is rather the gradual approach rather than the probabilistic one shot and harvest. You normaly seed then
crop followed by evaluation of result and correction of the seed until a reasonable solution is reached.
I am referring to neural networks and genetic algorithms for instance. I do not see probability driven steering or bayesian approach
either.
When solving searches by recurring on what it used to be is pure bayesian but such method can hardly be called heuristics.
Heuristic (by the way the "h" adds emphasis but is in excess in its greek etimology and as vicious as Marco Polos invention of the word
China just because in such country language sounded as chin chin) definition should be restricted to a methodology based in the
sistematic approach towards a solution. Only the convergence of succesive results confirm the path was right, not the probability of
success of certain coordinates at a certain point in the process towards the unknown.
Postedb[](1comment[http://www.cnet.com/87054_10.html?username=])
November 17, 2004 6:24 AM (PST)
Like []
Repl []
Link [http://www.cnet.com/8618-1001_3-984695.html?assetTpeId=12&messageId=355575]
Flag []
E-mail []
[]
Anewcomertobaesiantheor
I read with your post with interest.
I am a newcomer to bayesian theory, in that I have only become aware of the theory today.
I am a deconstructionist by nature and would welcome any perspective in this area.
At first glance I would put the issue in "the chicken or egg" category.
Postedbbrianwalkden[http://www.cnet.com/profile/brianwalkden/](
1comment[http://www.cnet.com/87054_10.html?username=brianwalkden])
December 15, 2006 10:06 AM (PST)
Like []
Link [http://www.cnet.com/8618-1001_3-984695.html?assetTpeId=12&messageId=355565]
Flag []
E-mail []
[]
Join the conversation
Log in or create an account [] to post a comment,orquickly sign in with:
11/24/11 Old-school theor is a new force - CNET News
6/6 news.cnet.com/2009-1001-984695.html
Comment Add our comment []
The posting of advertisements, profanit, or personal attacks is prohibited. Click here to review our Terms of Use
[http://www.cnet.com/legal/terms-of-use/] .
CBSInteractive.Allrightsreserved.
