h t t p : / / w w w. a n a ly t i c s m a g a z i n e .

c o m

Driving Better Business Decisions Ja n ua ry / febr ua rY 2011

Risk(y) Analysis
Lessons learned from
real-world catastrophes:
Eight common mistakes to avoid
when utilizing risk analysis
and risk management
ALSO INSIDE:
• SOFTER SIDE OF CONSULTING: Six “soft” skills every analyst needs to know
• BUSINESS ANALYTICS: Preview of INFORMS spring conference in Chicago
Executive Edge
• DATA MINING: Deriving meaning, purging chaos from deluge of documents V.P. Chuck Holland:
The three critical
• BEYOND BUSINESS INTELLIGENCE: Journey from what to who, when and why changes that saved
O.R. at UPS
I n s i de sto ry

A ‘home’ for analysts

When the Institute for Operations jump on the analytics bandwagon, but to
Research and the Management Scienc- lead it through the creation of new pro-
es (INFORMS) and partner Lionheart grams and initiatives specifically aimed
Publishing first discussed publishing the at the hundreds of thousands of analysts
online magazine you’re now reading, it around the world looking for a profes-
took about 10 seconds to come up with sional “home” to enhance their knowl-
a name: Analytics. It was toward the end edge, skills, networking and careers.
of 2007, and Tom Davenport and Jeanne Along those lines, INFORMS recently
Harris’ book, “Competing on Analytics: renamed its spring “practice” conference
The New Science of Winning,” had seem- the “INFORMS Conference on Business
ingly convinced every CEO in the world Analytics and Operations Research” and
that their organization needed “analytics” revamped the program accordingly (see
in order to gain a competitive advantage. page 35). The line-up for the 2011 con-
Organizations were swimming in data, ference, set for April 10-12 in Chicago,
but most of them didn’t know what to do features tracks devoted to the “analytics
with it. “Analytics” promised to turn data process” and “public sector analytics,” as
into profits, and quickly the concept in well as “supply chain management.” “An-
its various forms (“Business Analytics,” alytics Connect,” a new, professional job
“Advanced Analytics,” etc.) showed up at event held in conjunction with the confer-
or near the top of every business pub- ence and sponsored by Analytics maga-
lication’s or business consulting group’s zine, will give employers and potential job
“Top 10 Business Trends/Strategic Tech- seekers an opportunity to meet face-to-
nologies/etc. of the Year.” face in a casual business environment.
With the launch of Analytics maga- The conference will also offer many
zine in spring 2008, INFORMS made a networking opportunities and “special
statement that it intended to be a serious programs,” including a soft skills work-
player in the worldwide analytics move- shop that any analyst interested in sharp-
ment. Three years later, the analytics ening his or her selling, communication,
bandwagon continues to gather momen- problem framing and team-building skills
tum as does INFORMS’ commitment to will not want to miss (see page 10).
it. A recent market study conducted by
Capgemini Consulting on behalf of IN- – Peter Horner, editor
FORMS convinced the Institute to not just horner@ lionhrtpub.com
1 | a n a ly t i c s m a g a z i n e . c o m a n a ly t i c s | J AN U A R Y / F E B R U A R Y 2 011
C o n t e n t s

DRIVING BETTER BUSINESS DECISIONS JANUARY/FEBRUARY 2 011

Features
10 Six ‘soft’ skills every analyst needs to know
By F. Freeman Marvin and William K. Klimack
The “art” of communication, team-building and other soft skills are
often critical to a project’s success.

16 Why risk analysis fails

By Douglas A. Samuelson
Catastrophes as diverse as Hurricane Katrina and the financial crisis
10
point to a series of common mistakes in utilizing risk analysis and risk
management.

21 Text Analytics
By Fiona McNeill
Deriving meaning from the deluge of documents and
purging content chaos.

23 HP’s world-class e-marketing business

By Prithvijit Roy, Arnab Chakraborty, Pritam Kanti Paul and
Girish Srinivasan
16 How Hewlett-Packard uses advanced analytics to drive its digital
marketing strategies.

29 From business intelligence to analytics

By Talha Omer
The journey from what to who, when and why.

32 ROI for RFID

By Jerry Banks and Federico Trigos
Different definitions, assumptions make determining return on
23 investment for Radio Frequency Identification tricky business.

2 | a n a ly t i c s m a g a z i n e . c o m a n a ly t i c s | Winter 2008
 DRIVING BETTER BUSINESS DECISIONS

Register for a free subscription:

4 http://analytics.informs.org

INFORMS Board of Directors

President Rina Schneur, Verizon Network & Tech.
President-Elect Terry P. Harrison, Penn State University
Past President Susan L. Albin, Rutgers University
Secretary Anton J. Kleywegt, Georgia Tech
Treasurer Nicholas G. Hall, The Ohio State University
Vice President-Meetings William Klimack, Decision Strategies, Inc.
Vice President-Publications Linda Argote, Carnegie Mellon University
Vice President-
Sections and Societies Ariela Sofer, George Mason University
42 Vice President-
Information Technology Bjarni Kristjansson, Maximal Software
Vice President-Practice Activities Jack Levis, UPS
Vice President-International Activities Jionghua “Judy” Jin, Univ. of Michigan
Vice President-Membership

Departments


and Professional Recognition Pinar Keskinocak, Georgia Tech
Vice President-Education Joel Sokol, Georgia Tech
Vice President-
Marketing and Outreach Anne G. Robinson, Cisco Systems Inc.
1 Inside Story Vice President-Chapters/Fora Stefan Karisch, Jeppesen

INFORMS Offices
4 Executive Edge www.informs.org • Tel: 1-800-4INFORMS

Interim Executive Director Terry Cryan
6 Profit Center Marketing Director Gary Bennett
Communications Director Barry List

Corporate, Member, INFORMS (Maryland)
8 Analyze This! Publications and 7240 Parkway Drive, Suite 300
Subdivision Services Hanover, MD 21076 USA
Tel.: 443.757.3500
35 Conference: Preview E-mail: [email protected]
Meetings Services INFORMS (Rhode Island)
37 Conference: IPC 12 Breakneck Hill Road, Suite 102
Lincoln, RI 02865 USA
Tel.: 401.722.2595
E-mail: [email protected]
39 Learning Resources
Analytics Editorial and Advertising

41 Data Mining Lionheart Publishing Inc., 506 Roswell Street, Suite 220, Marietta, GA 30060 USA
Tel.: 770.431.0867 • Fax: 770.432.6969
President & Advertising Sales John Llewellyn
42 Thinking Analytically [email protected]
Tel.: 770.431.0867, ext.209
Editor Peter R. Horner
[email protected]
Analytics (ISSN 1938-1697) is published six times a year by Tel.: 770.587.3172
the Institute for Operations Research and the Management Art Director Alan Brubaker
Sciences (INFORMS). For a free subscription, register at http:// [email protected]
analytics.informs.org. Address other correspondence to the Tel.: 770.431.0867, ext.218
editor, Peter Horner, [email protected]. The opinions Assistant Art Director Lindsay Sport
expressed in Analytics are those of the authors, and do not [email protected]
necessarily reflect the opinions of INFORMS, its officers, Tel.: 770.431.0867, ext.223
Lionheart Publishing Inc. or the editorial staff of Analytics. Advertising Sales Sharon Baker
Analytics copyright ©2011 by the Institute for Operations [email protected]
Research and the Management Sciences. All rights reserved. Tel.: 813.852.9942

3 | a n a ly t i c s m a g a z i n e . c o m
E x e c u t i v e Ed g e

A near-death experience faced outweighed the cost concerns, and op-

erations research was given a second chance
IE’s can’t write the sophisticated algorithms
associated with O.R., they understand the
at life, but clearly some changes were needed power and limitations of O.R.
in order to properly tap its potential.
Change No. 2: The approach. Along with
Change No. 1: Organization. Many compa- consolidating the O.R. expertise and moving
nies over the years have debated where O.R. it to the IE department, UPS also changed its
Three critical changes that saved operations research at UPS.
should reside in an organization. Should O.R. approach to O.R. Previously, the company’s
report to IT? Should O.R. report to the busi- approach went something like this: “O.R. is all
ness side? If on the business side, where? about optimization, right? All the O.R. groups
Before changes were implemented, opera- need to do is push a button and out pops the
At the time, UPS was Many of us have heard stories about near- tions research at UPS reported to two groups: optimized answer.” This flawed approach was
looking at areas to reduce death experiences. A person involved in a se- UPS Airline and UPS Transportation. The one of the major reasons that O.R. spent
costs, and O.R. was, quite rious accident or a person gravely ill recovers O.R. team associated with UPS Airlines was years with no tangible deliverables, resulting
frankly, a cost bucket with and tells their account of leaving this Earth and located in Louisville, Ky., worked almost ex- in frustration by both the O.R. groups and the
no apparent return on walking toward the bright light, outstretched clusively on airline projects and reported to IT. business owners demanding the answer. Les-
investment despite being arms awaiting them. Miraculously, they come The second O.R. group, located in Timonium, sons learned: The managing group needs to
back to life – functioning, contributing and tak- Md., worked on opportunities associated with understand the power of operations research.
surrounded by plenty
ing advantage of a second chance – and ea- the tractor-trailer and rail movements between Just as important, it needs to understand the
of problems ripe for
ger to tell their story. UPS facilities. The two O.R. teams – separat- limitations of O.R.
O.R. approaches and
A near-death experience is a good descrip- ed geographically and by the nature of their The new approach brought to the surface
algorithms.
tion of what happened to operations research projects – had little interaction. three key issues: 1. data, 2. assistance vs. op-
(O.R.) – an advanced form of analytics some- In the first major change, the two O.R. timization, and 3. roadmaps.
times referred to as “prescriptive analytics” groups were placed together in the Industrial A longstanding dilemma with O.R. can be
– at UPS. In the mid-1990s, the O.R. group Engineering department (IE). IE is responsible summed up as follows: When a model produces
at UPS was walking toward the bright light, for optimizing UPS operations, from the vision a bad answer, is the problem with the data or the
voices calling, ready to leave this world as we to the execution of the plan. IE develops “road- algorithm? As this question was being addressed
know it. At the time, UPS was looking at areas maps” for processes and technology and prior- at UPS, the assistance vs. the optimization op-
to reduce costs, and O.R. was, quite frankly, itizes projects to deliver against the roadmaps. portunity became clearer. Once the data require-
a cost bucket with no apparent return on in- As these roadmaps are developed, optimiza- ments were identified and obtained, in many
vestment despite being surrounded by plen- tion components are included in the vision. cases the organized data (presented in a simple
ty of problems ripe for O.R. approaches and The move to IE proved vital to O.R.’s even- tool such as Excel or Access) became a valu-
By Chuck Holland algorithms. tual success at UPS in part because O.R. is a able aid to the employee making planning and
Ultimately, O.R.’s potential to positively ad- component in many industrial engineering de- execution decisions. Although the human could
dress the complex operational problems UPS gree programs at universities. Although most not determine the optimum with the assistance,

4 | a n a ly t i c s m a g a z i n e . c o m a n a ly t i c s | J AN U A R Y / F E B R U A R Y 2 011
E x e c u t i v e Ed g e

the orga- without the algorithms. The O.R. applica- of the project. Corrective action can be MS and advanced analytics practice in the
nized data tion simply made the solution optimum. taken to make sure a project stays on world. Both the INFORMS Prize and the
allowed him This approach allows UPS to make im- track. Edelman Award are presented by the Insti-
or her to provements while the O.R. applications Research progress is much harder to tute for Operations Research and the Man-
make better are developed. gauge. Seemingly large advances are of- agement Sciences (www.informs.org).
decisions. These improvements, usually in the ten followed by long periods spent discov- Along with its active engagement with
While the form of cost reductions, help fund addi- ering what doesn’t work. At UPS, in many INFORMS, including contributions to IN-
algorithms tional research while bringing immediate cases, if the research hits a plateau for an FORMS journals, UPS O.R. collaborates
were being de- benefit to the bottom line. Because the extended period of time, management de- (or has collaborated) with Georgia Tech,
veloped and im- properly integrated systems and process- termines a decision date. If significant prog- University of Iowa, University of Michigan,
proved in research, es are not dependant on the O.R. “black ress is not made by the decision date, the MIT, Northwestern, Princeton and Univer-
the employee was making better deci- box,” UPS does not have O.R. roadmaps; research is considered a sunk cost, the re- sity of Southern California.
sions, bringing immediate benefit to UPS. instead, UPS has roadmaps with O.R. search is halted and the O.R. group moves The O.R. group has expanded its
And once the new algorithm was available, on to the next “black box” identified in the reach to many components of UPS op-
the optimization process benefited from Change No. 3: Accountability. A near- roadmap. Research progress is reviewed erations, including pickup and delivery,
the validated data. death experience means you survived, by mid-management at least every month air and ground transportation and facility
The final lesson learned through the but it does not guarantee how long you and by senior management at least twice utilization. Operations research has been
change in approach concerned the “road- will live. Everyone involved with the a year. Just because it’s research doesn’t recognized by UPS’ executive manage-
maps.” Many of the previous O.R. appli- O.R. groups within UPS knew this sec- mean it can’t be held accountable. ment for its contributions to the company,
cations were standalone applications, ond chance was in all likelihood the last and C-level executives continue to chal-
creating a lack of continuous “clean” chance. Today, UPS truly has a world-class op- lenge the O.R. group to deliver increased
data, as well as the need for special- Not every research attempt will de- erations research group. UPS received the benefits in the area of service and cost.
ized runs (often conducted by the O.R. liver. Research is not the same as a INFORMS Prize in 2003 and placed sec- O.R. faced a near-death experience
person). By having a vision for the pro- project. A project has tools that allow the ond in the Franz Edelman Award competi- at UPS and took full advantage of its sec-
cesses and technology (the roadmap), measurement of the triple constraints of tion that same year for its work on “Planning ond chance at life. Today, 15 years later,
the optimizations became “black box” functionality, schedule and costs. Earned the UPS Air Network.” The INFORMS Prize it’s hard to image UPS without O.R.
O.R. applications that were part of the value and other measures can continu- recognizes organizations that “effectively Chuck Holland ([email protected]) is vice president
overall process. The systems and pro- ally keep the project manager and senior and repeatedly integrate the principles of of Industrial Engineering at UPS, headquartered in
Sandy Springs, Ga. He is responsible for defining and
cesses were integrated to work with or level managers informed on the progress operations research and management sci- executing a vision for the optimization of UPS’ operations,
ence in pioneering, varied, novel and last- including anticipating and satisfying customer and market
demands. Prior to his current role, Holland held multiple
Subscribe to Analytics ing ways.” The Edelman, considered the operations, engineering and technology assignments at
It’s fast, it’s easy and it’s FREE! Just visit: http://analytics.informs.org/ “Super Bowl” of O.R., is an annual compe- UPS, including vice president within UPS’ Supply Chain
Solutions subsidiary.
tition that honors the best example of OR/

5 | a n a ly t i c s m a g a z i n e . c o m a n a ly t i c s | J AN U A R Y / F E B R U A R Y 2 011
Profit Center

An analytics education program in predictive analytics in fall 2011.

DePaul draws heavily upon traditional
label). Setting aside the name, there are well-
established university programs that intersect
courses such as those offered through the with analytics.
math department. Students at N.C. State take One area is statistics. Forecasting, the de-
courses specifically designed for the analytics sign and analysis of experiments and data min-
Analytics is neither A young man recently contacted me about
program, as do students at Northwestern. It’s ing are fundamental components of analytics,
well defined nor well finding a good graduate program in analytics.
reasonable to assume DePaul graduates will and statisticians come ideally trained for these
established as a stand- I was happy to oblige, but couldn’t immedi-
be proficient with IBM’s analytics software and activities. Unfortunately, many aspects of ana-
alone university discipline. ately answer him. Had he asked about gradu-
N.C. State graduates with software from SAS. lytics don’t easily fit within a statistics program,
ate programs in economics, mathematics or
Additional programs of study can be found among them, optimization and non-statistical
some other well-defined, well-established dis-
under the name “business intelligence.” Nei- problem solving.
cipline, I could have responded without pause.
ther analytics nor business intelligence can be Another area that isn’t as commonly known
Analytics, however, is neither well defined nor
unambiguously defined, though they overlap as statistics is operations research (O.R.) O.R.
well established as a stand-alone university
extensively and some consider them essen- has its roots in operational problem solving for
discipline. This isn’t to say universities aren’t
tially identical. As a catch phrase, “analytics” the British and U.S. armed forces during the
busy training students in the tools of analytics.
appears to be gaining the upper hand. Saint Second World War. Early O.R. practitioners
Schools simply haven’t offered degrees bear-
Joseph’s University offers a master’s of sci- thought of O.R. in the same, somewhat amor-
ing the name “analytics.”
ence in business intelligence through its busi- phous way people now think of analytics: the
This state of affairs, however, is chang-
ness school, and the University of Denver use of models, data, mathematics and good
ing. North Carolina State University lays claim
offers a similar degree, also through its busi- sense to solve problems. Over time, the study
to the first Master’s of Science in analytics
ness school. of optimization and stochastic processes came
through the affiliated Institute for Advanced
A look at the various curricula shows com- to form the core of O.R. curricula, together with
Analytics. The thoughtfully developed degree
mon themes: mathematics, computers, data some introduction to statistics and simulation.
program was first offered in the 2007-2008 ac-
and modeling. Many programs place an em- More recently, O.R. programs have broad-
ademic year and is growing and thriving. SAS,
phasis on marketing. While marketing is a ened their requirements, including coursework
an established software vendor and consultan-
fertile area for the application of analytics, it’s in such areas as computers and non-textbook
cy located near N.C. State’s Raleigh campus,
no more a part of analytics than finance, op- problem solving.
was instrumental in spearheading the effort. In
erations or sales. Notable exceptions to the O.R. programs are housed in various lo-
June 2010, DePaul announced a master’s of
marketing emphasis are the programs at N.C. cations within universities. The Industrial and
science in predictive analytics through its Col-
State and Northwestern. Systems Engineering Department at Georgia
lege of Computing and Digital Media and the
The emergence of these new programs Tech, for example, offers master’s and Ph.D.
Center for Data Mining and Predictive Analyt-
attests to the growing importance of analyt- degrees in O.R. The Web site of GradSchools.
ics. The program was developed in coopera-
By E. Andrew Boyd ics, but it also raises the question of whether com lists more than 150 degree programs
tion with sponsor IBM. Northwestern’s School
such programs must actually bear the name in O.R., though the list is far from complete
of Continuing Studies will begin offering a
analytics (or business intelligence, or a similar (www.gradschools.com/search-programs/

6 | a n a ly t i c s m a g a z i n e . c o m a n a ly t i c s | J AN U A R Y / F E B R U A R Y 2 011
 Profit Center

All things considered, operations-research). At present, most such specific set of analytics tools, not analytics Will we one day see analytics recognized
O.R. programs are an programs emphasize optimization and sto- broadly defined. in universities alongside economics and math-
excellent starting point chastic processes more extensively than might All things considered, O.R. programs are an ematics departments? Probably not. Will we see
for individuals seeking be considered ideal for an analytics curriculum. excellent starting point for individuals seeking more analytics programs and concentrations
an analytics education This emphasis comes at the expense of more an analytics education (or employers seeking to emerge? Without question. The extent to which
(or employers seeking to thorough training in areas such as statistics. hire students trained in analytics). With the many “analytics” embeds itself in the academic lexicon
However, many of the departments that house different types of O.R. programs being offered, remains to be seen. And given the breadth of
hire students trained in
O.R. programs provide sufficient flexibility for it’s necessary to look into the details to find a territory it covers, it’s unclear that a uniform cur-
analytics).
students to fashion a well-rounded “analytics” program that best meets an individual’s needs. riculum will ever materialize. However the details
course of study. The INFORMS Conference on Business Analyt- unfold, the analytics skill set will be taught for the
It’s likely that in the near future some depart- ics and Operations Research is an ideal oppor- simple reason that the market demands it.
ments housing O.R. programs will offer mas- tunity to talk to academicians about educational Andrew Boyd served as executive and chief scientist at
ter’s degrees or concentrations in analytics. programs and to ask people in industry what an analytics firm for many years. He can be reached at
[email protected].
INFORMS, the leading U.S. professional so- specific analytics skills they’re looking for.
ciety for academics and practitioners of O.R.,
appears on the verge of embracing analytics.
INFORMS underwrites the publication of this
magazine, and in 2011 it’s changing the name
of its annual practitioner conference from the
INFORMS Conference on O.R. Practice to the
INFORMS Conference on Business Analytics
and Operations Research.
So where does this leave the student look-
ing for a professional degree in analytics? In
fact, there are already some good choices,
and we’re sure to see more emerge this year
Other analytics programs and for years to come.
Are you aware of any Of course, getting an analytics educa-
programs that students tion isn’t about the name on the degree. It’s
wanting training in analytics
about learning a set of skills – even though
should consider? If so, send
your recommendations to the skills that comprise analytics aren’t en-
[email protected]. tirely well defined. An individual who is in-
terested in data mining might do very well
Be sure to include Web links
about the programs. in a statistics program. The downside is that
statistics programs cover an important but

7 | a n a ly t i c s m a g a z i n e . c o m a n a ly t i c s | J AN U A R Y / F E B R U A R Y 2 011
A n a ly z e Th i s

When geeks and Ben Alamar is the Michael Lewis of the

academic world. He has been publishing
for most of these “enlightened” teams, the con-
nection between data, analysis and decision-

jocks collide scholarly papers on professional baseball,

football and basketball topics for a decade,
and some of his research is presented by
making is somewhere between tenuous and
non-existent.
Like Lewis in “Moneyball,” Alamar attri-
Some teams take great pride During my first week as a professor, I re- Lewis in his football book, “The Blind Side: butes much of the resistance to the cultural
in not utilizing data analysis ceived an e-mail from a San Francisco State Evolution of a Game” [4]. An economist by clash between the “insiders” (who have spent
for any meaningful decision- undergraduate named Kevin Mello. “I have training, Alamar is the founder and editor in their life immersed in a game’s customs and
making. On the other end of just completed a summer internship with the chief of the Journal of Quantitative Analysis intricacies) and the “outsiders” (with their
Oakland A’s,” he wrote, “and though I’ve taken in Sports [5]. In addition to his day job as a laptop computers who are often perceived
the spectrum, the Houston
a course in statistics, I’m interested in learning professor at Menlo College, he also serves as irrelevant). Still, there appears to be no
Rockets have an MIT MBA
more.” In the world of sports, the Oakland A’s as the team statistician for the NBA’s Okla- turning back. Computers keep getting faster
as their general manager,
are viewed as one of its first analytics success homa City Thunder. and cheaper; analysis and visualization tools
as well as two full-time
stories, thanks in large part to Michael Lewis’ From chatting with Alamar, my sense is keep improving; and many companies are
statisticians and a data 2003 book, “Moneyball: The Art of Winning an that the adoption of analytical techniques in now dedicated to extracting and delivering
specialist on their payroll. Unfair Game” [1]. I wrote back enthusiastical- professional sports is evolving in a strange structured and increasingly detailed historical
ly, offering to do an independent study course and disparate way. Some teams – for ex- data for professional sports to young, data-
with him. Our semester studying Jim Albert’s ample, the Detroit Pistons and the California literate professionals who are trying to figure
book, “Teaching Statistics Through Base- Angels – take great pride in not utilizing data out how to effectively make use of it. As the
ball,” [2] ultimately led to a research paper [3], analysis for any meaningful decision-mak- business of sport grows bigger, the constant
while also helping Kevin earn some statistical ing. On the other end of the spectrum, the quest for a competitive edge will continue to
credibility with the Oakland A’s. He has since Houston Rockets have an MIT MBA as their lead some coaches and executives to grasp
embarked on a career with the A’s as a profes- general manager, as well as two full-time for whatever advantages might possibly be
sional scout. statisticians and a data specialist on their obtained from data and algorithms.
A brilliant and eclectic writer with a finance payroll. In a 2009 New York Times Maga- As evidence, Alamar cites the growth of
background and a passion for sports, Lewis zine article [6], Lewis revealed some of the the MIT Sports Analytics Conference [7].
has a keen eye for situations in which fresh fruits of their analysis actually make it into Launched in spring 2007 with less than 300
paradigms and smart analysis are combined the team’s game strategies. attendees, its 2011 organizers expect nearly
to create significant competitive advantages But the Rockets remain something of an 3,000 people to attend, with keynote presen-
that conventional observers often simply can- outlier. By Alamar’s informal count, roughly tations by team owners, general managers
not grasp. In “Moneyball,” he eloquently de- 15 (about half of) NBA teams have somebody and player personnel directors. One of the
scribes the analytic innovations deployed by spending at least some of their time dealing feature panels from last year’s conference,
By Vijay Mehrotra the A’s front office to compete successfully with statistical models, though several of these moderated by Lewis, was entitled “What
with a payroll less than 25 percent of some of are merely part-time folks or interns noodling Geeks Don’t Get: the Limits of Moneyball”
its competitors. around with some datasets. Moreover, even [8].

8 | a n a ly t i c s m a g a z i n e . c o m a n a ly t i c s | J AN U A R Y / F E B R U A R Y 2 011
A n a ly z e Th i s

The challenges and opportunities and coaches. Through years of hard

associated with the successful applica- work, Belichick ultimately finds that he
tion of analytics in professional sports can win trust, respect and loyalty in the
seem strangely typical. Analytics pro- locker room only by providing wisdom
fessionals are almost always more fa- and insights that help his teams win on
miliar with their tools and techniques the field. For those who know Belichick
(what statistician Rob Easterly calls as the Super Bowl-winning head coach,
“the verbs”) than with the businesses in it is perhaps comforting to know that he
which analysts seek to apply them (“the started out just where we so often do:
nouns”). This too often limits our effec- on the outside looking in, just wanting
tiveness by causing industry profes- to find a way to use his talents to help.
sionals, with their hard-earned domain Vijay Mehrotra ([email protected]) is an
knowledge, to distrust us. associate professor, Department of Finance and
Quantitative Analytics, School of Business and
What to do? First of all, if you are Professional Studies, University of San Francisco.
reading this column, I would encour- He is also an experienced analytics consultant
and entrepreneur and an angel investor in several
age you attend the upcoming INFORMS successful analytics companies.
Conference on Business Analytics and
Operations Research, April 10-12 in Chi- References

cago (http://meetings2.informs.org/Ana- 1. www.amazon.com/Moneyball-Art-Winning-

Unfair-Game/dp/0393057658
lytics2011). Conference organizers have 2. www.amazon.com/Teaching-Statistics-Using-
worked hard to create a forum to bring Baseball-Albert/dp/0883857278
3. Mehrotra, V. and K. Mello, “A Different Game?
together business professionals seek- Analyzing Management Strategies In Baseball’s
ing an edge with analytics professionals World Series Using A Markov Chain Model,”
California Journal of Operations Management,
who might have the tools and tenacity to Vol. 3, No. 1, February 2005, pp. 92-98.
4. www.amazon.com/Blind-Side-Evolution-
help them address vexing challenges. It Game/dp/039306123X
is a great opportunity for all of us to learn 5. www.bepress.com/jqas/
more about what’s out there. 6. www.nytimes.com/2009/02/15/
magazine/15Battier-t.html
Secondly, a lesson from the world of
7. www.sloansportsconference.com/
sports. In his fine book “The Education 8. www.sloansportsconference.com/panels-
of a Coach” [9], the late David Halber- 2/2010-2/what-geeks-dont-get-the-limits-of-
moneyball/
stam describes Bill Belichick as lowly 9. www.amazon.com/Education-Coach-David-
Halberstam/dp/1401301541
assistant coach in his early 20s with
no credibility with established players

9 | a n a ly t i c s m a g a z i n e . c o m
P eo p l e to p eo p l e Sp e c i a l C o n f e r e n c e s e c t i o n

In a recent article in Interfaces, Based on the many valuable lessons

I the authors (Liberatore and

Luo) point to four driving factors
behind the analytics movement
we’ve learned while preparing and teaching
the workshop, here are six soft skills every
analyst should know:
[1]. One factor seems to be that a “focus on
processes in many organizations has shifted
demand for decision support and problem
solving away from human decision-makers
to processes.” And yet, Liberatore and Luo
note, competing on analytics still requires
well-rounded analysts: “In analyzing more
than a thousand O.R. [operations research]

Six ‘soft’ skills

job advertisements, Sodhi and Son (2008)
find that employers value strong technical
skills but also seek an array of soft skills.”
As practicing decision analysts, we

every analyst have learned from experience that the

lack of interpersonal “soft” skills can be
a significant barrier to project success

needs to know
and the adoption of analytics within or-
ganizations. In order to help improve the Figure 1: Six soft skills every analyst needs to know.
interpersonal skills of analysts, we de-
veloped the Soft Skills Workshop in 2008 1. How to turn a client into a partner.
with a team of other O.R. practitioners The first soft skill an analyst needs to
The art of communication, team-building and other soft to gather best practices and provide re- know is how to create and manage a win-
skills are often critical to a project’s success. alistic training. Our team has conducted ning relationship with the project client. Most
the workshop for the past three years analysts can’t wait to dive into the analyti-
prior to the annual INFORMS Confer- cal complexity of a new project. However,
ence on O.R. Practice (see sidebar sto- analysts must also take into account the or-
ry on page 13 as well as page 35). The ganizational complexity facing their clients
workshop has received positive reviews – office politics, differing motivations, per-
By F. Freeman Marvin (LEft) and from participants as a valuable devel- sonality conflicts and group dynamics. The
William K. Klimack (Right) opmental experience. analyst and client must work as partners.

10 | a n a ly t i c s m a g a z i n e . c o m a n a ly t i c s | J AN U A R Y / F E B R U A R Y 2 011
Soft Skills Sp e c i a l C o n f e r e n c e s e c t i o n

Discover the problem together. A cli- a good client. The effective client is not Plan your time. Teams can be more 3. How to frame a problem.
ent often begins a project by telling you only a customer for your analysis but is effective if they use their meeting time The third soft skill an analyst needs to
what the problem is: “We need a tomato the champion for the project within the or- well. We use the acronym GRASP to know involves collaborating with stake-
harvester that will not damage the toma- ganization. The champion must be able remember everything to consider when holders – the people, inside or outside of
toes.” Don’t believe it. Instead, acknowl- to secure access to data, win funding for planning a meeting: Goals and out- the organization, who have the ability to
edge that perspective and engage the the project and manage the expectations comes, Room and logistics, Agenda influence the project or are impacted by
client on a journey to explore the problem of decision-makers. and time available, Support team, tech- its outcome. Stakeholders may bring very
space together. We recommend using a niques and tools, and the Participants different perspectives on the problem and
structured dialogue process, an approach 2. How to turbocharge a project team. and observers. often introduce conflicting objectives.
developed in the health care industry to Many projects require a multi-disci- Manage conflict. Many teams freeze Talk to stakeholders. Stakeholders
improve understanding among physicians plinary team with members from a variety up in the face of interpersonal and politi- may include managers, employees, inter-
and hospitals about clinical priorities. The of professional backgrounds. People who cal conflict. Analysts can help the team nal and external partners, customers, sup-
solution to damaged tomatoes, as you work in different fields bring unique talents move forward by using techniques that pliers and the general public. A stakeholder
may know, was not a new harvester at all, but often speak with their own “lingo” and keep the team focused on the problem. analysis can help you select the appropri-
but a thicker-skinned tomato. approach problem solving differently. The For example, the Nominal Group Tech- ate set of people whose interests are key
Learn from each other. Analysis team can get bogged down from the start. nique (NGT) is a well-known group pro- to the success of the project. Structured
projects involve learning as much about The second soft skill an analyst needs to cess where each individual writes down interviews with stakeholders will identify
a client’s problem as you can. Share know is how to energize a project team his or her ideas before sharing and dis- their goals and objectives, alternative so-
that learning opportunity with the cli- and help it move forward. cussing them. lutions and possible sources of data and
ent. Help your client find a niche on the Follow a process. All successful teams Practice divergent and convergent information [3].
project where he or she can add value go through similar developmental stages: thinking. Divergent activities such as Build a decision framework. The de-
while learning as much as possible from forming, storming, norming and perform- brainstorming help a team identify new cision framework is used to define the
you. This co-learning will pay off in in- ing [2]. Inexperienced teams often try to options or important considerations. Con- scope of the problem solution. Some
creased communication and a better skip the storming and norming stages and vergent activities narrow the range of op- aspects of a problem might only be re-
project outcome. head straight to performing. These teams tions and lead to choices. When these solved by changing policies or allocating
Make the client your champion. You fail to build the common trust, mean- activities are used together, they create resources that are above the level of the
have been trained in how to be a good ing and purpose necessary in a winning a natural “pulse.” If a team is pulsing too organization. Other parts of the problem
analyst, but your counterpart probably team. Don’t try to take shortcuts or you fast, it risks reaching premature conclu- may be troubling, but they can be de-
has had no formal training in how to be will end up in a ditch. sions – groupthink. If the pulses are too layed until later, simplifying the analysis.
slow, the team risks becoming nothing Verify the decisions that remain in scope
Subscribe to Analytics more than a debating society. Find a with the stakeholders to align everyone’s
It’s fast, it’s easy and it’s FREE! Just visit: http://analytics.informs.org/ healthy pulse for your team. expectations.
(Continued on page 13).

11 | a n a ly t i c s m a g a z i n e . c o m a n a ly t i c s | J AN U A R Y / F E B R U A R Y 2 011
Smarter technology for a Smarter Planet:

The anatomy of a decision.

Our increasingly connected world is generating 15 petabytes of data a
day. What if you could harness all that information to test new strategies
and make better decisions? On a smarter planet, you can.
Today, with computation times compressed from days to seconds,
Operations Research is becoming accessible to not only scientists and
engineers, but business leaders as well—fundamentally redefining how
real-world companies meet their performance goals.
Together with the sophisticated analytics and consulting expertise of IBM
Research and Global Business Services®, IBM’s industry-leading ILOG®
Optimization technology is allowing businesses to model any number of
“what-if” scenarios in real time. Already, over 1,300 corporations and
government organizations are using IBM ILOG CPLEX® to support their
mission-critical decisions. And to train the next generation of business
and technology leaders, the IBM Academic Initiative is providing ILOG
Optimization products to universities for free.
A smarter business needs smarter software, systems and services.
Let’s build a smarter planet. ibm.com/optimize

IBM, the IBM logo, ibm.com, Smarter Planet and the planet icon are trademarks of International Business Machines Corp., registered in many jurisdictions worldwide. Other product and service names
might be trademarks of IBM or other companies. A current list of IBM trademarks is available on the Web at www.ibm.com/legal/copytrade.shtml. © International Business Machines Corporation 2010.
Soft Skills Sp e c i a l C o n f e r e n c e s e c t i o n

To “de-anchor” an expert, suggest (Continued from page 11). know is how to elicit subjective judgments expert on the importance of the task, 2)
data at both extremes first before Create a value hierarchy. People are from subject matter experts when objective clearly define the variables to be elic-
asking for an estimate. Many naturally “alternative focused.” When fac- data are not adequate for an estimate or ited, 3) condition the expert’s respons-
motivational biases, such as having ing a choice, they only look at the alter- prediction. Expert judgment is especially im- es with as much relevant information
a personal stake in the outcome of natives before them and select the most portant when historical or empirical data are as you can, 4) ask the questions, and
an event, are subtle and can even attractive. By creating a value hierarchy unreliable, ambiguous or conflicting. The 5) verify the responses – ask, “Would
be subconscious. that includes all stakeholder goals and meaning of even the best data may require you make a decision based upon these
objectives, you will be able to use it to interpretation by a knowledgeable person. judgments?” [5]
generate and evaluate new and better al- Plan and conduct interviews. Counteract biases. Experts are
ternatives. This “value focused thinking” Planning expert interviews involves known to be susceptible to cognitive and
approach is also useful for communicating being clear about your purpose, iden- motivational biases – just like the rest of us.
the rationale for your recommendations to tifying experts who are acceptable to For example, anchoring is a common cog-
decision-makers [4]. stakeholders and decision-makers, and nitive bias where we tend to “anchor” our
scheduling an adequate time and place. judgment on an initial estimate, but then in-
4. How to elicit data from experts. When you conduct the interview, follow adequately “adjust” for different scenarios or
The fourth soft skill an analyst needs to a structured process: 1) motivate the range of uncertainty. To de-anchor an expert,

Soft Skills Workshop: Real-world skills for analysts

A team of operations research prac- The Soft Skills Workshop is a one-day, day. Course workbooks contain the lesson includes: William K. Klimack (senior con-
titioners developed the Soft Skills Work- hands-on seminar for new and mid-career materials, a case study and practical exer- sultant, Kromite, LLC), F. Freeman Marvin
shop in 2008 as a professional forum analysts. The workshop teaches the criti- cises. This past year’s case study looked (executive principal, Innovative Decisions,
to help improve the interpersonal skills cal people-to-people skills necessary to a at whether or not INFORMS should spon- Inc.), Paul Wicker (senior decision analyst,
of operations research (O.R.) and deci- successful analysis project. There are no sor a certification program for analysts. Decision Strategies, Inc.), Don Buckshaw
sion analysts. For three years, the work- pre-requisites needed to attend. Each table team is assigned a facilita- (senior principal analyst, Innovative Deci-
shop has been held prior to the annual The instructor team organizes the day tor, who is either an instructor or an in- sions, Inc.), Jack Kloeber, (partner, Kro-
INFORMS Conference on O.R. Practice. around six categories of people who ana- vited expert practitioner. The facilitators mite, LLC) and David Leonhardi (Boeing
The next Soft Skills Workshop will be held lysts must interact with during the course introduce the case study and coach the Commercial Airplanes).
in Chicago on April 10, prior to the 2011 of a typical project: clients, project team table teams during the exercises. For more information on the workshop,
INFORMS Business Analytics and Opera- members, stakeholders, subject matter At the end of the day, the teams must visit: http://meetings2.informs.org/Analyt-
tions Research Conference. This year, the experts, working groups and decision- brief a “decision-maker” on the results ics2011/softskills.html .
conference will also include a one-day makers. of their analysis. A special guest lecturer For more information on the confer-
track on “Soft Skills for Analysts” as part Workshop participants are assigned to plays the role of decision-maker. ence, visit: http://meetings2.informs.org/
of the regular conference program. a table where they work as team during the The Soft Skills Workshop team Analytics2011/.

13 | a n a ly t i c s m a g a z i n e . c o m a n a ly t i c s | J AN U A R Y / F E B R U A R Y 2 011
Soft Skills Sp e c i a l C o n f e r e n c e s e c t i o n

suggest data at both extremes first before Use swing weight elicitation.
asking for an estimate. Many motivational Groups are often tasked with providing
biases, such as having a personal stake criteria weights for an analysis. Swing
in the outcome of an event, are subtle and weights are the appropriate type of
can even be subconscious. To help iden- weight to elicit for a multi-objective
tify these biases, consider bringing two model because they account for both
experts together to discuss their interview the relative importance and the vari-
responses. ability of the criteria being assessed.
There are at least a dozen elicitation
5. How to collect data from groups. techniques you can use. Each tech-
The fifth soft skill an analyst needs nique has advantages and disadvan-
to know is how to elicit and combine tages. Learn several techniques and
assessments from groups of decision- practice them until you feel confident
makers, stakeholders or subject matter in front of a group.
experts. Many projects involve a com- Bring the group to convergence.
mittee, panel or other type of working The analyst can choose from a num-
group that is tasked with providing in- ber of methods to combine individual
put to the analysis in the form of rela- assessments into a group assessment.
tive values or priorities. These include averaging, voting, con-
Group assessment approaches. sensus and melding. Regardless of the
There are three effective approaches to method you choose, always be pre-
group assessment. The Delphi Method pared to perform sensitivity analysis to
elicits individual, but anonymous, as- show how your analysis results would
sessments from a group and then com- vary across the range of individual
bines them mathematically. Consensus assessments.
groups, on the other hand, use free and
open discussion to arrive at a group as- 6. How to get a decision-maker to
sessment with no attempt at eliciting indi- say yes.
vidual assessments. The NGT, discussed The sixth soft skill an analyst
above, is a balanced approach – each needs to know is how to communi-
group member makes an individual as- cate results to executives and other
sessment, followed by open discussion decision-makers in an organization.
and consensus on a group position. A project can be a technical success

14 | a n a ly t i c s m a g a z i n e . c o m
Soft Skills Sp e c i a l C o n f e r e n c e s e c t i o n

Get access to senior leaders early but have no impact if the results are not the problem and that you followed a logi- collaboration technologies. He has worked as an
operations research analyst for Rockwell International and
in the analysis and at key decision understood and accepted. cal path to arrive at the results. Decision Science Consortium, and was a senior member
points. This allows you to clearly Understand the decision-maker’s ob- of the technical staff for Northrop-Grumman TASC. He
is now an executive principal with Innovative Decisions,
understand their goals and jectives. The decision-maker may not be How to Have a Lasting Impact
Inc. He holds a master’s degree in Public Policy from the
objectives before you begin your your client. Get access to senior leaders Mastering these six soft skills will allow Kennedy School of Government at Harvard University and
a B.Sci. from the U.S. Military Academy at West Point.
analysis and to check your progress early in the analysis and at key decision a project analyst to integrate the technical
points. This allows you to clearly under- analysis with the social and political fabric Bill Klimack ([email protected]) ) has a wide
along the way. variety of operational and analytic experience. As a senior
stand their goals and objectives before of the organization. In the end, the value consultant for Decision Strategies, Inc., he uses decision
you begin your analysis and to check of any analysis project is measured by analysis, simulation and other operations research tools
for clients engaged in exploration and development of
your progress along the way. the lives of the people it affects. Seek to hydrocarbon deposits. He previously developed and
Tell the story. Tell a compact, com- have a lasting impact by following these implemented O.R. tools for management of portfolios of
programs in government and the private sector. A retired
pelling story. Don’t drag the decision- final guidelines: Army colonel, Klimack’s assignments included serving on
maker through the entire chronology of • Be a good listener. Practice the skills the faculty of the Department of Systems Engineering,
U.S. Military Academy, where he was director of the
your analysis. Like any good story, your of a good listener. Ask open-ended Operations Research Center of Excellence. A co-recipient
story should have a beginning, middle questions, paraphrase discussions of the 2003 Decision Analysis Society Practice Award, he
holds a Ph.D. in operations research from the Air Force
and end. Draw your conclusions on a and show empathy for people who are Institute of Technology.
single chart. Create a rough storyboard touched by the project.
of your final presentation at the begin- • Seek to build consensus. Projects
ning of the project. This exercise will that reach conclusions by consensus References

help check your assumptions. It can have a better chance of success. 1. Liberatore, Matthew J. and Luo, Wenhong, “The
Analytics Movement: Implications for Operations
also serve as a guide in planning your Consensus ensures that everyone Research,” Interfaces, Vol. 40, No. 4, July-August
2010, pp. 313–324.
analysis. involved has had their say and
2. Tuckman, Bruce W., “Developmental Sequence
Provide insight. Presenting complex encourages commitment to action. in Small Groups,” Psychological Bulletin, Vol. 63,
No. 6, 1965.
quantitative information is a challenge. • Transfer soft skills to the
3. Gregory, Robin, and Keeney, Ralph L., “Creating
Your goal is to allow the decision-maker organization. Teach what you have Policy Alternatives Using Stakeholder Values” in
“Strategic Development: Methods and Models,” ed.
to discern the important aspects of the learned to others. Build organizational Robert G. Dyson and Frances A. O’Brien, 1998, pp.
19-38.
data. Different types of visuals are better competency as you conduct the
4. Keeney, Ralph L., “Value-Focused Thinking:
at delivering different messages. For sim- project and instill the attributes of a A Path to Creative Decisionmaking,” Harvard
University Press: Cambridge, Mass., 1992.
ple results, a table is best if you want the high-performing organization. 5. Clemen, Robert T. and Terence Reilly, “Making
viewer to recall the data. A graph of the Freeman Marvin ([email protected]) has
Hard Decisions with Decision Tools,” Pacific
Grove, Calif.: Duxbury Press, 2001.
same data may better convey a trend if more than 25 years of experience as a group facilitator,
6. Tufte, Edward R., “The Visual Display of
decision analyst and process consultant. His expertise
that is the message [6]. Always include a is in the integration and application of organizational
Quantitative Information,” Graphics Press, 2001.

process chart to show that you understood development, operations research and electronic

15 | a n a ly t i c s m a g a z i n e . c o m a n a ly t i c s | J AN U A R Y / F E B R U A R Y 2 011
“Kill the Quants?”

risks and under-prepared. There were

some genuine modeling errors and ques-
tionable assessment of predictions the
models did make.
Katrina and its aftermath showed a

Why risk analysis fails similar pattern. The state and federal
governments repeatedly postponed
repairs to the levees around New Or-
leans, overruling strong recommenda-
tions from their engineers. After 9/11,
Catastrophes as diverse as Hurricane Katrina and the financial crisis point to a series of the federal government shifted much of
common mistakes in utilizing risk analysis and risk management. its in-house expertise from natural di-
sasters to terrorism. The analyses that
were conducted focused more on the
danger from a storm surge up the Mis-
sissippi River than on the danger of a
“back-side” surge from Lake Pontchar-
train, on the other side of the city – ne-
By Douglas A. Samuelson glecting the critical fact that repair ships
can navigate the river but could not
Should we “kill all the hit, raising the most intriguing question of questions about the uses and limitations traverse the lake, rendering the main

S quants?”
The financial crisis, Hur-
ricane Katrina and the BP
why modelers don’t usually apply quanti-
tative measures to how well their models
worked. (He and this reporter followed up
of risk analyses, and about the ways in
which managers expect to use them.
First of all, we can surely agree that the
repair plans ineffective for the actual
event. The problems were greatly com-
pounded by uncertainty about who was
oil spill have aroused many claims that on his main points in an article for OR/MS examples cited here, among others, do responsible for what.
quantitative risk analysis failed. Even Today [Hubbard and Samuelson, 2010]). indicate major deficiencies in predicting The financial crisis featured both mod-
fairly sober scientific reviews seriously Even more popular and more trenchant is and managing risk. The BP spill left many els that were wrong and models that
criticized models in general [Scientific Nassim Taleb’s “The Black Swan,” based people asking, “Why didn’t we know what were right but were ignored by managers.
American, 2008] or called for a major on the assertion that the most important could happen?” The responsible state Modeling errors vied with deliberate mis-
change in their use [Lo, 2009]. Douglas events are the “unknown unknown” ones, and federal agencies lacked in-house ex- statements. Federal agencies deferred to
W. Hubbard’s book on failures of risk beyond any modeling method’s scope pertise and apparently ignored a pattern the expertise and presumed non-malevo-
management [2009] has been a popular [Taleb, 2007]. These critiques raise major of risky practice. BP underestimated the lence of private firms.

16 | a n a ly t i c s m a g a z i n e . c o m a n a ly t i c s | J AN U A R Y / F E B R U A R Y 2 011
R i s k a n a ly s i s

In all these cases and a number consumer credit and home equity lines
of others, some common elements as ready reserve for small businesses
emerge: got squeezed, delaying payments to
1. unduly limiting assumptions in the other small business who in turn got
analyses; squeezed – and suddenly a large pro-
2. over-reliance on inappropriate portion of the populace looked high-risk
theory; to the scoring models.
3. underestimating probabilities and This is one example of a gen-
effects of rare events; eral principle: When circumstances
4. excessive trust in markets; change substantially, models are li-
5. over-specialization and resistance able to deteriorate not only because
among disciplines; variables take on values different from
6. insufficient empiricism about the observations on which the model
assessing quantitative methods; was based, but also because the re-
7. insufficient attention to availability lationships among variables change.
and quality of data; and For instance, a model of crowd be-
8. unclear responsibility and havior that works very well in normal
accountability for failures. situations may fail badly when, in a
crisis, everyone wants to rush to a
1. Unduly limiting assumptions. Most limited set of exits. In a crisis, usually
creditors have relied for many years on uncorrelated behaviors become cor-
statistical credit scores, most famously related – in the case of the financial
the FICO score almost universally used system, urgent selling.
for residential mortgages. For obvious In the BP case, it is not clear that
reasons, credit scores rely entirely on anyone potentially responsible for
borrowers’ recent experience. Hence, problems asked whether the geologic
in usual pre-2008 times, which saw structure was unusual or how much dif-
no downturns, a borrower with high ference it made that the well was 5,000
utilization of available credit probably feet under water. In the planning for Ka-
was someone who tended to spend trina, evidently no one considered how
too much and kept too little reserve. people without cars would heed an or-
As the housing market spiraled down- der to evacuate after the buses stopped
ward, however, people who had used running.

17 | a n a ly t i c s m a g a z i n e . c o m
R i s k a n a ly s i s

The financial crisis featured both 2. Over-reliance on inappropriate the- regulatory lapses, again relying on mine on data, the temptation to downplay rare
models that were wrong and models ory. Some years ago, as a relatively ju- operators’ desire to keep their mines op- events is, if anything, accentuated, as the
that were right but were ignored nior federal policy analyst, this reporter erating long-term, in the West Virginia presence of some solid data discourages
by managers. Modeling errors was one of several reviewers of a study coal mine collapse early last year. speculation about the unknown. Comput-
vied with deliberate misstatements. of estimating oil spill risks from tankers. ing confidence intervals usually relies on
The researchers who had done the study 3. Excessive trust in markets. A special distributional assumptions that may not
Federal agencies deferred to the
recommended having the Coast Guard and important case of over-reliance on be applicable and definitely relies on hav-
expertise and presumed
require reports of all oil spills, no matter theory is over-reliance on markets. Wil- ing accounted for all sources of variation
non-malevolence of private firms.
how small, for three years, to enable more liam Kahn, who was director of the risk – which usually has not happened. Con-
precise estimation of the distribution. (As management modeling group at FNMA fidence intervals computed only on sam-
it was, spills under five gallons were ex- (Fannie Mae) in 2008, told of how the pling variation without other sources of
empted.) I pointed out the difference in CEO believed the market price, not the randomness overstate precision and can
what causes different-sized spills and model’s, until FNMA was $50 billion “un- easily mislead even the knowledgeable
asked, “How much do small spills tell you derwater” in its risk pricing of residential about the risk from rare events.
about large spills?” The lead researcher mortgages. Economists readily concede Hubbard [2009], in a survey of 60
replied in a patronizing tone, “You don’t that markets are myopic and tend not to firms that used some form of quantitative
understand statistics.” Obviously he was give sufficient weight to sketchy informa- models, found that most experts asked to
oblivious to the assumptions implicit in tion. Also, the assumption of approxi- produce 90 percent confidence intervals
treating the data as part of one data set mately equal information among parties, came nowhere near getting 90 percent
– and prepared to follow those assump- critical to market equilibrium theory, may of the correct values within these inter-
tions to any conclusion whatsoever. not hold in practice, and it is hard to pin vals. Training and feedback can greatly
The researcher in this story was a bi- down whether it does. improve these experts’ assessments of
ologist, but economists are prone to the uncertainty – particularly when the train-
same fallacious thinking. Alan Greenspan 4. Underestimating probabilities and ing includes requiring the experts to bet,
admitted that one cause of the financial effects of rare events. It is well known with real money, on their predictions. “If
crisis was his assumption that private that known rare events tend to be over- you won’t bet on it at 9-to-1 odds, it’s not
firms would not sacrifice their own long- looked entirely and under-predicted when a 90 percent interval” has a profound ed-
term interests and those of the country they are considered. Some locations have ucational effect.
for short-term profits. It would not be sur- had three or four “hundred-year floods” in
prising if federal regulatory officials made the past century, and some financial mar- 5. Insufficient empiricism about as-
similar assumptions about BP’s likelihood kets have experienced “once-in-a-gen- sessing quantitative methods. In his
of sacrificing safety for short-term profit. eration” swoops and drops every five to survey of users of risk analysis meth-
Some litigation now in process alleges 10 years or so. In models heavily reliant ods, Hubbard found that several popular

18 | a n a ly t i c s m a g a z i n e . c o m a n a ly t i c s | J AN U A R Y / F E B R U A R Y 2 011
R i s k a n a ly s i s

techniques increase comfort far more are available sufficiently ahead of time
than they improve actual results. Among to make forecasts. This reporter remem-
the more prominent examples, he cited bers with regret a contractor’s fine model
balanced scorecards and the analytical of the price and supply of chromium that
hierarchy process (AHP). turned out to be critically dependent on the
This is not the occasion for argu- amounts purchased under a few federal
ments about whether these methods contracts. These purchase figures were
were applied as designed and still readily available for past years, but future
produced the disappointing results he such purchases were classified, as they
found. In any event, he urges persua- indicated intended construction of certain
sively, both producers and users of high-performance military aircraft.
models should insist on assessing how Modeling often demonstrates a
accurate the models’ predictions were need for more data, but additional data
and evaluating subsequent efforts by collection after model development is
the same modelers accordingly. underway seems to be done rather infre-
quently. When it is done, modelers tend
6. Insufficient attention to availabil- to collect more of the kinds of data they
ity and quality of data. Good mod- can get easily, which often means more
els require good data. Nevertheless, of what they already have. Hubbard sug-
it is not at all difficult to find instances gests that collection should be guided by
in which the data required were either the expected value of perfect information
unavailable or not good. One example (EVPI): how much better could we do if
from this reporter’s experience is hos- we had this measurement precisely? Of-
pital emergency department diagnostic ten the most useful data to collect will be
codes, which tended to have little as- a very few sketchy observations about a
sociation with the patients’ eventual di- variable we know next to nothing about.
agnosis and treatment.
In assessing a model, it is useful to 7. Over-specialization and resis-
ask how much effort was spent on data tance among disciplines. Scientific
quality and whose responsibility that was. specialization has become a major
Even when good data were available for and growing problem. In risk analysis,
developing the model, it is also impor- it appears to be the driving factor be-
tant to ascertain whether key data series hind many of the other shortcomings.

19 | a n a ly t i c s m a g a z i n e . c o m
R i s k a n a ly s i s

Central to all these stories of In the FNMA example, for instance, the of clarity about who is accountable for short, risk analyses remain useful, but
shortcomings, as well, is lack of CEO trusted conventional economics over failures, whether of analysis, planning, we can all do better, and the best way
clarity about who is accountable risk models. The BP crisis highlighted the reaction to changing events or carry- forward is to help each other improve.
for failures, whether of analysis, long-established reluctance of economists, ing out needed actions. Both managers Douglas A. Samuelson ([email protected])
planning, reaction to changing events geologists and risk analysts to talk to each and analysts could have done better. is president and chief scientist of InfoLogix, Inc., an
R&D and consulting company in Annandale, Va. He is a
or carrying out needed actions. other, and the propensity to talk past each Redesigning organizational structures, frequent contributor to Analytics and OR/MS Today.
other when conversations do take place. incentives and information handling to
This is not new: the Forrester Meadows do better in crises is another topic for
World III model, the basis for the book “The another occasion, but it is critical to fu- References
Limits to Growth” that ignited a hot policy ture improvement. 1. Douglas W. Hubbard, “The Failure of Risk
debate in the mid-1970s, omitted all price Everyone has blind spots, from indi- Management: Why It’s Broken and How to Fix It,”
John Wiley & Sons, 2009.
effects, so economists simply dismissed it, viduals to large organizations and entire 2. Douglas W. Hubbard, “How to Measure Anything:
while geologists discussing resource limita- professions. Over-specialization exac- Finding the Value of Intangibles in Business,” John
Wiley & Sons, 2007.
tions dismiss econometric models that as- erbates the problem. In trying to antici- 3. Douglas W. Hubbard and Douglas A. Samuelson,
“Modeling Without Measurements: How the
sume prices dominate. pate “unknown unknowns,” breadth of Decision Analysis Culture’s Lack of Empiricism
In the early days of credit scoring, in vision and consideration becomes criti- Reduces its Effectiveness,” OR/MS Today, October
2010.
the 1960s, pioneering Fair, Isaac and cal. One expert group [NRC, 2008] sug- 4. William Kahn, press conference at SAS Global
Forum, 2009
Company (now FICO) frequently en- gested that all models of complex social
5. S. Lichtenstein, B. Fischhoff, and L. Phillips,
countered resistance to letting models phenomena should be checked by sce- “Calibration of Probabilities: The State of the Art to
1980,” in “Judgment under Uncertainty: Heuristics
override loan officers’ judgment. Be- nario experts, domain experts, model- and Biases,” eds. D. Kahneman, P. Slovic, and A.
Tversky, Cambridge University Press, Cambridge,
fore that, they first tried offering, free ers and users to minimize the chance of 1982, pp. 306-334.
of charge, to model risks of medical ail- something known to anyone available 6. Andrew Lo, “Kill All the Quants?: Models vs.
Risk Management Videos Mania in the Current Financial Crisis,” NYU Stern
ments, such as heart attacks, and got being overlooked by the other people School Conference on Volatilities and Correlations
in Stressed Markets, April 3, 2009.
For more on risk management in this no takers at all from the doctors they involved in making the decisions. Stra-
context, click on the following streaming approached. Most OR/MS practitioners tegic gaming [Samuelson, December
7. Donella Meadows et. al., “The Limits to Growth,”
Universe Books, New York, 1972.
videos filmed during presentations at the
2010 INFORMS Annual Meeting in Austin, have run into this kind of interdisci- 2009] is one of the most effective ways 8. National Research Council, Behavioral Modeling
and Simulation, 2008.
Texas (brief login required): plinary resistance many times; some, to elicit alternative assumptions and
9. Douglas A. Samuelson, “Playing for High Stakes:
• John Birge, University of Chicago, regrettably, provide such resistance sketch their likely effects. More empiri- Wargamers and Cognitive Scientists Seek to Avoid
“ORMS and Risk Management ‘Strategic Surprise,’ ” OR/MS Today, December
themselves. cism in evaluating models and the infor- 2009.
Failures: What are We Doing Wrong?”
• Gerald G. Brown, Naval Postgraduate mation on which they are based is also 10. “SciAm Perspective: The Quants Did It,”
Scientific American, December 2008.
School, “Rethinking Risk 8. Unclear responsibility and account- clearly warranted. Both analysts and 11. Nassim Taleb, “The Black Swan: The Impact of
Assessment and Management for ability for failures. Central to all these managers need to challenge modeling the Highly improbable,” Random House, 2007, 2nd
Homeland Security and Defense” Ed. 2010.
stories of shortcomings, as well, is lack assumptions much more diligently. In

20 | a n a ly t i c s m a g a z i n e . c o m a n a ly t i c s | J AN U A R Y / F E B R U A R Y 2 011
Mining for Intelligence

on all the available information. That avails Text analytics technologies allow new,
organizations a whole new lease on the de- numeric representations of text to be
rived intelligence. embedded into traditional statistical and
You no longer have to encode the char- forecasting models. From there, results
acteristics into discrete categories, such of this new, previously unknown insight
as customer complaints or bad reviews on are delivered to the end user/information
blogs. With today’s text analytics technol- consumer within the reporting capabilities
ogy, this data can be input as unstructured of the framework.

Text analytics data, directly and consistently classifying

the essence of the material. This puts con-
text back into the numbers.
I have a condition I call “formaphobia”;
All of this can be invisible to the infor-
mation consumer; they could be looking
at a spreadsheet or their corporate dash-
board and see the forecasted volume of
I have a really hard time filling in boxes to positive and negative sentiment being ex-
Deriving meaning from the deluge of documents and describe me, my life, feelings and needs. I’d pressed across different Web channels,
purging content chaos. be glad if I never had to check off another for example.
box! Creating inputs as boxes was the result Effective text analytics solutions – in
of limitations in recording information de- fact, the only sustainable, semantic-driven
fined by the relational systems that we are “smart” ones – are most successful when
all familiar with. But no longer does the tech- you don’t even know they are there. What
nology have to drive how you receive input you do know is that your searches find rel-
from your customers, constituents and op- evant, accurate information, and you see
By Fiona McNeill erations. Now you can examine, utilize and reductions in warranty claims, workplace
decipher from the text what you need, and in injuries and customer complaints. Perfor-

E
Experience is a valuable asset, 30 percent of the available data. There’s the case of text mining, find new items that mance is as predicted, and your subject
but how do organizations cap- great opportunity for improvement, then, you never even knew existed. matter experts are freed to do even more
ture and organize such assets when organizations include the intelli- intelligent tasks for your organization.
for use in decision-making? gence buried in unstructured data assets How Organizations Turn Text Keep in mind that circumstances
Given that 70 percent to 80 percent of in the decision-making process. into Gold change over time. In fact, customer in-
all data is unstructured, one can reason- This means identifying any unique at- Technology plays a strategic role in telligence marketing is designed to do
ably assume that organizational decision tributes in text collections and harmonizing decision-making. A business analytics exactly that – change behavior. If your
making – based on data warehouses, them with the structured data, including that framework integrates all the required ele- methods are effective, then sentiment
business reporting and the like – has which is generated by operational systems. ments needed to make decisions and in- will adjust, new words will be used to de-
often been based on only 20 percent to It means ensuring that decisions are based telligently improve operations. scribe you relative to your competitors,

21 | a n a ly t i c s m a g a z i n e . c o m a n a ly t i c s | J AN U A R Y / F E B R U A R Y 2 011
T e x t A n a ly t i c s

and public perception will change. As we building little used “field of dreams numbers across four different brands – via machine learning (semantic tech-
such, these systems are not static. They warehouses.” These are intelligent sys- they had been through a lot of change. nologies) are advancements associated
must be dynamic, editable and flexible to tems, implemented in operational re- For them, getting through that material with text analytics. But it is not mining.
change with your business. porting and activity systems. was like reading 500 copies of “War Finding things that you already know
and Peace.” Text analytics helped them is not discovery. Identifying completely
Three Keys to Success 2. Semantics is fundamentally about make sense of the data – defining well- new concepts or previously unknown
Text analytics is designed to extract and encoding knowledge, recording it and organized concepts, capturing key iden- associations by examining collections of
decipher the meaning held within documents processing it in a meaningful way. In tifiers and predicting which categories documents is what mining is all about.
from repositories, blogs, tweets, customer that codification, the text analytics tech- any particular product belongs to with Identifying new patterns and word com-
communication systems, claims and service nology documents how people think 95 percent accuracy. binations and isolating emerging issues
notes, to name a few. The extraction associ- about and interpret the written word. It’s In the medical field, examining – now that is mining.
ated with content categorization and ontolo- not Y2K encoding – we have all learned across a patient’s entire unit of care and There is an art and a science to under-
gies is a historically-focused activity – looking from that. This uses dynamic, learning across collections of patient records, standing text, in deriving meaning from the
at what was already written and transcribed, structures designed to identify concepts text data research has revealed that the deluge of materials and, by consequence,
and deciding what the core meaning is within and topics and change over time. frequently asserted link between diabe- purging the content chaos. Because the
that particular text. When we extend to senti- tes and obesity may not hold true per goal is to mirror the human rationalizing
ment analysis and text mining, we discover 3. Organizations need to take a stra- se. It is at the threshold between normal process, how we interpret materials us-
completely new things that never would be tegic approach. Semantic technology and obese where the peak of diabetes ing machine learning and statistical tech-
identified by looking at each document in iso- must be open to new rule definitions occurrence is happening – not simply niques requires refinements from human
lation. Semantics are important. and adapt to new knowledge the orga- in the obese classification of the body subject-matter experts. Experience is a
Three keys to success in this regard: nization derives – refining rules, includ- mass index (BMI) spectrum. valuable asset, and dynamically encod-
ing new topics. It needs to change with A large insurance company used text ing it so the entire organization can sys-
1. With smart organizational cultures, the organization. This means more than analytics in evaluating workplace injury tematically and consistently draw upon it
semantic-driven implementations are just getting through the materials faster. claims and found 600 completely new can produce an army of experts.
successful when end-user employees While that is part of the challenge, re- cases of threatening situations (new As global product marketing manager, Fiona McNeill
don’t even know the technologies are view of each document in and of itself concepts or categories, if you will) from oversees the product marketing of text analytics at SAS
(www.sas.com). During her 12-year tenure, she has
there – the capabilities are embedded will not give the insight that you get from free-form, handwritten text that would helped organizations derive tangible benefit from their
into business processes. No longer are examining the entire collection and min- not have been discovered by using code strategic use of technology. In addition to working with
a wide range of industries, McNeill has defined product
ing it for new discovery. matching. strategy and corporate relationships at SAS. Before
Subscribe to Analytics These examples also help explain joining SAS, McNeill was a member of IBM Global
Services.
Text Analytics Yield Big Wins the difference between mining and ex-
It’s fast, it’s easy and it’s FREE!
A manufacturer with a rich acquisi- traction. Looking for known items and A version of this article was published in Business
Just visit: http://analytics.informs.org/ Intelligence Network. Reprinted with permission.
tion history had over a million product concepts and extracting them from text

22 | a n a ly t i c s m a g a z i n e . c o m a n a ly t i c s | J AN U A R Y / F E B R U A R Y 2 011
D i g i ta l M e d i a

HP’s world-class e-marketing business

How Hewlett-Packard uses advanced analytics to drive its digital marketing strategies.

By Prithvijit Roy, Arnab Chakraborty, Pritam Kanti Paul and Girish Srinivasan (left to right)

Compelling benefits The growing effectiveness of most out of their digital marketing strate- applications to drive its digital marketing
encourage organizations to
move their marketing dollars
from traditional to digital
T digital media has changed the
way marketers interact with
customers. New age custom-
gies is the ability to capture and mine data
exhaustively and cost effectively through
advanced analytics techniques. These
strategies and overcome business chal-
lenges. HP’s in-house analytics team,
Global Business Services Analytics (GBS
channels. ers are informed and empowered. The techniques shed light on the effectiveness Analytics), directs this enterprise-wide
challenge for organizations is to be a part of all marketing activities, thus helping mar- analytics effort and plays a critical role
of the customers’ conversations involving keters fine-tune their strategy. In the long in developing various advanced analyt-
their product and to influence their choic- run, this enhances customer engagement, ics solutions. These solutions leverage
es. Hence, the adoption of digital channels optimizes digital “marketing spend” [1] and structured and unstructured data across
such as social media, e-mail marketing has a direct impact on revenues. billions of customers worldwide to im-
and online search and display ads is grow- Hewlett-Packard uses advanced an- prove the effectiveness of the e-market-
ing steadily. What helps marketers get the alytics and operations research (O.R.) ing model across HP [2].

23 | a n a ly t i c s m a g a z i n e . c o m a n a ly t i c s | J AN U A R Y / F E B R U A R Y 2 011
E - M a r k e t i n g at H P

The Era of Digital Marketing traditional media, such as newspapers, Multiple levers are available to grow the HP deploys various analytical solu-
Compelling benefits encourage orga- magazines, television and direct mail [3]. online business. These levers include: tions to drive online commerce. This has
nizations to move their marketing dollars In contrast, more than 60 percent of them • driving traffic by guiding customers to proven valuable in three broad areas:
from traditional to digital channels. For are expected to increase spend in digi- the right content; 1. Enhanced online store effectiveness by
starters, digital channels assure better tal media, such as e-mail marketing, paid • increasing conversion by offering driving quality traffic, analysis of marketing
and cheaper access to a broader cus- search and social media. relevant products at appropriate price effectiveness and inventory planning
tomer base. Additionally, these channels points; 2. Proactive customer engagement to
offer the ability to mass-customize mar- E-Marketing at HP • cross-sell and up-sell to increase identify opportunity and target customers
keting messages and speed up the cap- HP is at the center stage of this digi- basket size and improve margins; and with right products
ture and analysis of customer responses. tal revolution and has been making sig- • increasing customer engagement to 3. Increased profitability of digital media
All this knowledge can be used to quickly nificant investments on digital channels increase loyalty and repeat purchase. investment by intelligent allocation of
refine marketing and sales tactics. to generate demand, engage customers budget across marketing vehicles
According to Forrester Research Inc., and provide technical support. E-market-
less than 20 percent of organizations are ing at HP can be summarized into three
expected to increase marketing spend in broad themes – the three “Cs” of the e-
marketing ecosystem:

1. Community deals with

the aspect of engaging
customers and creating
awareness about HP
products and services,
leading to effective lead
generation.
2. Content deals with
the aspect of providing
customized relevant content
to customers and improving
user experience.
3. Commerce deals with
aspects of generating
Figure 1: Three Cs of E-marketing ecosystem. revenue through online
commerce. (See Figure 1.)

24 | a n a ly t i c s m a g a z i n e . c o m a n a ly t i c s | J AN U A R Y / F E B R U A R Y 2 011
E - M a r k e t i n g at H P

In the digital age, customers no Analytics Solutions such as click-stream analysis, Proactive Customer Engagement
longer just experience the brand; Online stores with high volumes of cart abandonment analysis and In the digital age, customers no longer
they have also started impacting customer transactions and behavioral experimental design to help increase just experience the brand; they have also
brand perception. data create the perfect environment to click-through. started impacting brand perception. Mar-
apply advanced analytical solutions. This • Supply chain operations – order keters who can proactively engage cus-
helps elevate performance of key store fulfillment and inventory optimization. tomers can expect to increase the circle
functions, such as: Analytics solutions could include of influence many times over by making
• Demand generation – driving quality standard performance measurement these “engaged customers” their brand
traffic to the store. Data analysis framework and managerial dashboards advocates.
helps understand the key marketing to manage daily store operations. An organization needs to keep its cus-
activities driving traffic to the Web • Customer support – customer tomers engaged to achieve long-term
store and accurately forecast Web tech and warranty support center. profitability and sustainability. It is critical
traffic. Customer surveys and call-center that customers not only continue to use
• Web store experience – online analytics can help identify problem a specific product/service but also try out
customization for enhanced user areas, drive down costs and build other offerings from the same brand. That
experience. Analytics helps in areas customer relations. can only be achieved if customers are

Demand generation: Forecasting traffic for HP’s online store

Analytics helped HP’s online longer forecast duration for better integrated with the result of the time up to 90 percent of traffic to the on-
store get a better understanding of financial and inventory planning. series model. line store and enhanced weekly lon-
the key drivers of Web traffic and The tool works on a combina- The solution led to identification ger term forecast accuracy up to 95
ways to increase range and accura- tion of multiple linear regression of key marketing activities that drive percent.
cy of traffic forecast. and time series analysis to generate
The Web analytics team devel- a forecast. Marketing performance
oped a Web Traffic Forecasting tool from historical promotional activi-
to enable the store to understand ties across online channels (Web
the impact of multiple marketing ac- search, affiliate Web sites, E-mail,
tivities (paid search, organic search, etc.) were considered to arrive at the
e-mail campaigns, online affiliate final regression model coefficients
campaigns, etc.) on consumer Web for each product category. A time
traffic to the product category pag- series model was used to generate
es (notebook, desktop, printers and traffic forecasts to the main prod-
printing supplies) and to forecast uct category Web pages of HP Shop- Figure 2: Web traffic forecasting methodology.
accurate weekly Web traffic over a ping. The result of the model is then

25 | a n a ly t i c s m a g a z i n e . c o m a n a ly t i c s | J AN U A R Y / F E B R U A R Y 2 011
E - M a r k e t i n g at H P

satisfied with their previous experiences. customer sentiments and the level
Studies have proven that word-of-mouth is of engagement. Conversations
a highly effective way of attracting new cus- are un-bounded and customers
tomers. Satisfied customers act as a com- can express their true feelings.
pany’s covert sales force and significantly Understanding these conversations
reduce the cost of acquiring new customers. will enable marketers to take positive
However, influencing customers enough for actions. Analytical tools (e.g., text
them to champion a product or service re- mining) help provide a structure to
quires deep behavioral insights. this unstructured, conversational
Analytical techniques can be deployed
to tap these insights at various stages of
data and aid in the discovery of key
take away.
Informs #1
customer engagement: • Engagement optimization:
• Customer need identification:
Marketers can understand
Knowledge gained in the first three
stages are brought together in this
Full Page Advertisement
differences in media consumption stage to optimize customer value
behavior by capturing the unique by creating an intelligent cube that
needs and expectations of helps marketers answer the “who, New Artwork
customers and then creating a what, when and how” questions.
logical segmentation the existing The intelligent cube helps create an
customer base.
• Engagement-level assessment:
effective targeting engine to offer the
most relevant products to customers
Page 26
Marketers can reach out to and increase conversion several
their customers through various folds.
touch points to review the level
of engagement and determine
the drivers of loyalty – rational,
emotional or brand-image related.
Organizations can use their internal
data (length of relationship, recent
activity and frequency of purchase)
to assess customer loyalty.
• Active listening: Social media
Figure 3: The customer engagement journey.
is a great platform to assess

26 | a n a ly t i c s m a g a z i n e . c o m
E - M a r k e t i n g at H P

HP’s relevant “digital” presence on Boosting Profitability of Digital spend across vehicles and evaluate the lower bound on the number of e-mails
popular social networks provides an op- Media Investment resulting ROI of the campaign. per campaign) and product-specific con-
portunity to actively listen to customers’ To reach different customer segments The first phase of the project used a straints (e.g., Notebooks have to be mar-
likes and dislikes. Consumer buzz is cap- with different communication and buy- linear programming approach to deter- keted on affiliate sites and the HP Web
tured and fed into the enterprise data ing preference, marketers use a mix of mine the budget allocation across dif- site and search engine). The solution,
warehouse, and then analyzed along with marketing vehicles. Optimizing spend ferent marketing channel and product when tested on some historical campaign
other relevant data by skilled analytics on various marketing vehicles is critical category combinations, with the objective scenario and compared with the actual
professionals to gauge consumer senti- to enhancing demand generation efforts. of maximizing the overall revenue. spend allocation results, was found to
ments and satisfaction level, new product However, determining which marketing The team developed a user-friendly provide an overall ROI increase of up to
ideas and selling opportunities. mix is most beneficial to the business re- optimization tool that allows HP to maxi- 25 percent.
quires not only a scientific mize revenue (or ROI) subject to channel In the second phase of the project, the
HP’s effective approach to analyzing spend specific constraints (e.g., an upper and impact of interaction effects on ROI and

offer engine and revenue, but also a test-

learn-optimize culture.
HP utilizes information on customer past The GBS Analytics team
purchases to accurately predict the future pur- has developed an optimiza-
chase behavior of customers using Bayesian
modeling technique. The marketing department tion framework that helps HP
knows the best time to market product to a par- decide how best to allocate
ticular customer belonging to a given segment. marketing spend based on
Markov chain modeling of product purchase
sequence is used to predict the next most logi- historical performance. The
cal product that a customer would buy. This framework:
helped build effective offers and has resulted
in about a 25 percent increase in average order
• provides a mathemati-
size across multiple campaigns in 2010. cal framework to a business
decision of selecting a spe-
cific marketing vehicle for a
campaign;
• helps determine the
business justification for ap-
propriate allocation of spend
on different marketing vehi-
Figure 4: Markov chain modeling cles; and
helped boost average order size. • provides the user with the
ability to tweak the marketing

27 | a n a ly t i c s m a g a z i n e . c o m a n a ly t i c s | J AN U A R Y / F E B R U A R Y 2 011
E - M a r k e t i n g at H P

the non-linear nature of ROI by online Conclusion

channel will be evaluated through non- As digital marketing and social me-
linear programming. The tool will also dia grow as influencers, it becomes im-
provide the ability to perform simula- perative that the insights gained and
tions using a Monte-Carlo simulation actions taken based on the measure-
approach to allow the business to test ment of marketing effectiveness are
multiple marketing spend allocation disseminated across the enterprise
scenarios and pick the one that suits in an integrated manner. While digital
them the best. The traffic generated channels enhance an organization’s
by marketing activities, the conversion ability to reach its target customer in
rate and order size are the drivers of the most effective and efficient man-
revenue in the next phase of model ner, analytics helps companies mea-
formulation. Traffic generated by each sure the previously immeasurable and
vehicle, conversion rate and average provides the means to evaluate data
order size are variable. Given a particu- and turn it into actionable insights.
lar budget allocation, the tool allows the Prithvijit Roy ([email protected]) is a senior
user to simulating web-store traffic and director, Arnab Chakraborty (Arnab.Chakraborty@
hp.com) and Pritam Kanti Paul (Pritam-kanti.Paul@
predicts the total revenue. hp.com) are directors and Girish Srinivasan (Girish.
[email protected]) is a senior manager at HP
Global Business Services. Copyright Hewlett-Packard
Development Company, L.P. All rights reserved.

References

1. “Marketing spend”’ refers to the marketing

budget allocated to such activities as search
marketing, e-mail marketing, affiliate marketing and
comparative shopping.
2. http://viewer.zmags.com/publication/07afc96c#/07
afc96c/29.
Figure 5: Demand generation optimization process. 3. “U.S. Interactive Marketing Forecast, 2009 to
2014,” Forrester Research Inc.

Subscribe to Analytics
It’s fast, it’s easy and it’s FREE! Just visit: http://analytics.informs.org/

28 | a n a ly t i c s m a g a z i n e . c o m
T e l e c o mm u n i c a t i o n s M a r k e t i n g

From business intelligence to analytics

The journey from what to who , when and why .

By Talha Omer

T
The other day, my analyti- time you called and whether your call was However, no matter what tool you use, • Data: petabytes
cal team at a major telecom- successful or if it was dropped. When you the best that traditional BI can help you un- • Reports: terabytes
munications company and I use your cell phone, the cell phone provid- derstand is what happened. It cannot, no • Excel: gigabytes
were presenting an all-new er knows where you are, where you make matter how much you torture the data, tell • PowerPoint: megabytes
behavioral segmentation to one of our se- most of you calls from, which promotion you when something happened, who will • Analytics: bytes
nior executives. Seeing the first slide, the you are responding to, how many times it happen to or why something happened. • One business decision based on
executive jumped out of his seat in ex- you have bought before and so on. In many This is precisely the reason that advanced analytics: priceless
citement and exclaimed, “Some of these cases, a cell phone is the only thing people analytics is so important; it’s the difference
insights took me years to uncover after carry with them at all times. between the 99 percent of effort that typi- Analytics is not always easy – peta-
repeated trial and error and hundreds of For a BI analyst trying to make sense cally yields very little insight, and the 1 per- bytes of data everywhere and a priceless
BI (business intelligence) reports.” of all of this data, the task becomes man- cent that provides a window into the mind insight difficult to find. Creating terabytes
Using a cell phone to make a voice call ageable thanks to hundreds of attractive- of a customer. of reports, engineering gigabytes of excel
leaves behind a significant amount of data. looking tools that instantly create reports Borrowing an analogy from the popu- sheets by writing killer macros and pre-
The cell phone provider knows every per- presenting data in every conceivable slice, lar MasterCard commercial, the distinc- senting them in flashy slides to top man-
son you called, how long you talked, what graph, table, pivot or dump imaginable. tion can also be seen as: agement in the hope that all this effort will

29 | a n a ly t i c s m a g a z i n e . c o m a n a ly t i c s | J AN U A R Y / F E B R U A R Y 2 011
B I & A n a ly t i c s

tell you something and result in action influence of social network on your
rarely occurs. customer’s decisions or suggesting the
On the other hand, combining the next best offer.
what with the when, who and why will If we measure any of the following it
provide a company with a long-term is likely that we live in the world of BI:
strategic competitive advantage. • daily revenue
Many different methodologies an- • churn rate
swer the when, who and why questions • daily unique subscriptions
including: • top international calling
destinations
• Predictive analysis (to answer the • number of new customers
when question – identify customers
that will take a certain action, e.g. We are now heading toward a world
churn) where we don’t simply measure the
• Descriptive analysis (to answer the what question but rather do a rigor-
who question – profile customers, ous analysis of the what question in
e.g. segmentation) order to determine the when, who and
• Qualitative analysis (to answer why questions such as customers’ pri-
the why question– identify the mary purposes and product/service
reasons for the when and who, e.g. affinities.
surveying) No matter how you slice and dice the
data using traditional BI methodologies
If you are new to this world, the last and tools, analytics is always big step
bulleted item – the why question – is a ahead of it. Analytics requires a business-
great way to start your foray because it and data-driven exploratory process. Any
is easy to implement and full of insights attempt to limit either of these dimensions
that will be very action-oriented. will inhibit its effectiveness.
Though traditional BI has several Absence of thorough analysis
benefits and is easy to implement, it is can lead to frail conclusions and in-
confined in its ability to predict, iden- correct decision-making. The data
tify and give a full customer view, e.g. for the analysis must be pulled from
predicting customer lifetime, identify- a number of subject areas: voice,
ing triggers for churn, measuring the sms (short message service, the text

30 | a n a ly t i c s m a g a z i n e . c o m
B I & A n a ly t i c s

communication service component of analytical techniques such as linear time

phone, Web or mobile communication variant would help identify the most prof-
systems), mms (multimedia messag- itable and loyal customers over their life-
ing service, a standard way to send time, which is of utmost importance to
messages that include multimedia strategists these days.
content to and from mobile phones), When analysis is done in an isolated
bill payments, etc. Simply sending fashion it also leads to the types of mis-
all customers a special discount to takes that drive customers away. One
remain loyal isn’t an effective use of company sent out campaigns based on
marketing funds. the analysis done on a handful of vari-
Consider the example of a telecom- ables. This was sub-optimal since it did
munication company. For marketing not depict the “whole” behavior of a cus-
analysts to optimize their campaigns, tomer. Segmentation and profiling across
they need to take advantage of ana- more than 1,500 variables was neces-
lytics techniques such as segmenta- sary to depict the true persona of the
tion analysis to draw a line within their customer. The result was an increase in
consumer base. By knowing what each marketing return on investment.
segment of customer wants, market- We live in a world of analytics when we
ing and campaign management ex- realize that every piece of data we look
perts can optimize the campaigns for at drives action that adds the value we
maximum efficiency. Hitting the loyal are trying to achieve for our customers.
customer with the wrong offer may Without analytics, BI alone can no longer
put his loyalty at risk. keep a business competitive – only an-
Telecommunication companies are swering the what question is no longer
interested in identifying their most prof- sufficient. The world of analytics that an-
itable customers and rewarding them. swers the when, who and why questions
A company needs a 360-degree view takes time to move into, but once you get
of its customers’ spending and behav- comfortable in it you will have achieved a
ioral patterns across all properties. Two long-term strategic advantage.
customers with the same revenue may Talha Omer is an analytics professional and
not necessarily be similar in profit, and researcher. He currently serves as an analyst at
a major telecommunication company. He holds a
each customer’s future lifetime with the Master’s degree from Cornell University in operations
company may vary. Applying advanced research. He can be reached at [email protected].

31 | a n a ly t i c s m a g a z i n e . c o m
E va l u a t i n g I n v e s t m e n t P r o j e c t s

ROI for RFID

Different definitions, assumptions make determining return on investment for Radio Frequency Identification
tricky business. Here are two performance measures to focus on.

By Jerry Banks (left) and Federico Trigos (Right)

W
We learned that there was little in While return on investment wondered whether RFID has a ROI. income/loss, while the money invested
the way of standardization of the (ROI) is a long used measure, The appearance of ROI and RFID may be referred to as the asset, capital,
computation of ROI. Radio Frequency Identifica- in the same sentence led us to search principal or cost basis of the investment.
tion (RFID) is a technology what is the definition of ROI when used ROI is sometimes also known as ‘rate of
that has re-emerged since Wal-Mart start- in the context of RFID. The RFID Jour- profit’ or ‘rate of return.’ ”
ed pushing it on its suppliers in 2003. More nal (2009) says that ROI is: “The ratio of If this definition were unique, that
recently, Wal-Mart announced that it is go- money gained or lost on an investment would be the end of it. However, at least
ing to use RFID intensively in the sale of relative to the amount invested. The 14 financial performance measures sat-
merchandise (Wailgum, 2010). Taken to- amount gained or lost may be referred isfy this definition. Each has its own as-
gether, many realists and doubters have to as interest, profit/loss, gain/loss or net sumptions, definitions and interpretations.

32 | a n a ly t i c s m a g a z i n e . c o m a n a ly t i c s | J AN U A R Y / F E B R U A R Y 2 011
ROI f o r RFI D

Number Name Description Reference Table 1: Chart Key Performance Frequency Percentage
1 Future Value After-tax cash flow available at the end of a project T/F2001 T/F2001 is (Thuesen and Fabrycky, 2001) Measure Used of Total
P/SB1990 is (Park and Sharp-Bette, 1990) (%)
after honoring all financial commitments
E1958 is (Eckstein, 1958) Net Present Value 10 25.0
2 Net Present Value Equivalent amount that the project will be worth at T/F2001 A(1959) is (Avery, 1959)
the beginning of the project Internal Rate of Return 5 12.5
NT1956 is (Nyland and Towle, 1956)
Pay Back Period 5 12.5
3 Periodic Equivalent Constant periodic amount to be realized over the T/F2001 B1957 is (Bierman, 1957)
Return on Equity 3 7.5
time horizon
Return on Assets 1 2.5
4 Perpetuity Permanent amount of money that a project will T/F2001 Therefore, we say that the above defini- Economic Value Added 1 2.5
return if its value is invested at time 0
tion is not “discriminatory.” No Specific ROI 15 37.5
5 Payback Period First time at which a project has no negative project T/F2001 Reported
balance In this article, we will examine defi-
nitions of ROI as well as other articles Total 40 100
6 Modified Internal Compares the future equivalent of the positive cash P/SB1990
Rate of Return flows versus the net present value of the negative that discuss RFID and ROI. Then, we Table 2: ROI measure used in RFID-related articles.
cash flows will give our recommendation of how to
7 Salmon’s Average Interest that equates the original outflow taken to P/SB1990 proceed.
Rate of Return the end of the planning horizon with the final value
Table 1 shows 14 financial perfor- Their frequency of use is shown in the sec-
of the project considering after-tax cash flows at the
end of each period mance measures. Each of them has a ond column. There are articles that indicate
8 Eckstein’s Ratio of present value of project income to net E1958 mathematical formulation that can be more than one measure of ROI. That is why
Aggregate B/C present value of project outflows (both during and found in the reference indicated. The the column labeled “Frequency” sums to
Ratio after the initial investment period) first five are absolute measures. Ab- 40 measures. We don’t call these 32 RFID
9 Netted B/C Ratio The benefits used in computing this ratio represent P/SB1990
solute measures are solely monetary. articles a sample; they are the result of a
the savings in user costs between the existing and
the improved alternatives minus the increase in the Numbers 6 through 11 are relative search. Table 2 shows further that there are
maintenance and operation costs with the costs measures. These measures compare at least six different measures of ROI used
representing the project investment costs cash inflows to cash outflows. The last in the RFID literature. In addition, some 15
10 Lorrie-Savage Ratio of the investor’s gain at time zero to the net P/SB1990 three are corporate measures. Corpo- out of 32 articles mention ROI, but do not
Value present value of outflows during the investment
period
rate measures assess the performance give enough information to determine the
11 Return on Invested Measures how well a company is using its money to P/SB1990 of a business unit for a specified time exact performance measure used.
Capital generate returns period. We learned that there was little in the
12 Economic Value The amount of money made beyond the capital A1959 Table 2 shows the financial perfor- way of standardization of the computation
Added invested in the project mance method, all called ROI in the of ROI. We use the term ROI to represent
13 Return on Assets Profitability of a firm relative to its total assets NT1956 source material. the many measures – 14 mentioned in
14 Return on Equity The ratio of net earnings at the end of the study B1957 The 32 articles used to construct Table 2 this article – that indicate the return from
period to the amount of capital supplied by investors
at the outset range from informative to scholarly. The first a capital investment.
column of Table 2 shows the different per- We checked the consistency of these
Table 1: Fourteen financial performance measures.
formance measures that were mentioned. 14 measures. That is, from a mathematical

33 | a n a ly t i c s m a g a z i n e . c o m a n a ly t i c s | J AN U A R Y / F E B R U A R Y 2 011
ROI f o r RFI D

We recommend the use of net standpoint, if one of the measures indi- key variables: return and investment. … of net present value as an absolute mea-
present value as an absolute cates that a project should be undertaken, Return is usually considered to be the net sure of ROI, and return on invested capital
measure of ROI and return on then all of them do the same, under some profit resulting from RFID projects in a as a relative measure of ROI in RFID in-
invested capital as a relative reasonable assumptions that hold for many certain amount of time. This is not always vestment projects evaluation. These two
measure of ROI. projects. If one measure indicates that a easy to determine since you may have re- measures are the most meaningful in de-
project should be rejected, all of them do turns that are not necessarily a matter of termining the worth of an RFID project.
the same, again, under some reasonable financial profit. Some RFID projects may Jerry Banks ([email protected]) holds the title
assumptions that hold for many projects. not yield significant cost reduction nor Academic Leader at Tecnológico de Monterrey in
Monterrey, Nuevo León, México. Previously, he retired
Yes, the measures are consistent. increase in revenue – at least not right from the faculty of the School of Industrial and Systems
Still, there are problems with the use away. However, they may provide sig- Engineering at Georgia Tech in Atlanta and then he
was Senior Simulation Technology Advisor at Brooks
of ROI. Consider the article that was men- nificant benefits like increase in customer Automation (now part of Applied Materials, Inc.).
tioned in the first paragraph (Wailgum, satisfaction or risk reduction.” Federico Trigos ([email protected]) is a professor at the
EGADE Business School at Tecnológico de Monterrey
2010). The article appeared in the July 28 in Monterrey, Nuevo León, México. He is the chair of the
issue of Bloomberg Businessweek and dis- Recommendation Leaders for Manufacturing Program on that faculty.
cussed Wal-Mart’s push to introduce RFID Financial performance measures, col-
References
tags at the retail level. The article says, “… lectively called ROI, used to evaluate RFID
presumably, ROI can be attained.” But, as investment projects, have different mean- 1. Avery, H.G., 1959, “Economic values vs. original
cost – A discussion of bases for calculating
in many (15) cases indicated in Table 2, no ings and assumptions. It is important to earnings,” N.A.A. Bulletin, (February), pp. 5-14.

definition of ROI is given. consider each of them in order to make the 2. Bierman, H. J., 1957, “Some problems in
computation and use of return on investment
In an RFID software advertisement (not appropriate interpretation. We also men- N.A.A. Bulletin, (December), pp. 75-82.

considered an article for Table 2) appearing tioned that these performance measures 3. Eckstein, O., 1958, “Water Resource
Development: The Economics of Project
online (http://blog.deploysoft.com/index. are consistent, under some reasonable Evaluation,” Harvard University Press, Cambridge,
Mass.
php/rfid/ accessed Aug. 17), whoever wrote assumptions that hold for many projects, 4. Nyland, H.V. and G.R. Towle, 1956, “How we
the copy made some excellent points: in that they concur in the decision to ap- evaluate return on investment – Experience of an
oil company,” N.A.C.A. Bulletin, (May), pp. 1,092-
“If you have been following the RFID prove or reject a project. In general, not 1,099.
5. Park, C.S. and G.O. Sharp-Bette, 1990,
market for a while, chances are that you all of them give the same information and “Advanced Engineering Economics,” John Wiley,
have heard the term ROI (Return of In- interpretation. The corporate measures New York, N.Y.
6. RFID Explorer, 2010, http://blog.deploysoft.com/
vestment) quite often. Although ROI cal- are not created to evaluate projects but to index.php/rfid [accessed Aug. 17] URL dated July
17.
culation may be relatively straightforward measure the performance of firms in a par-
7. Thuesen, G.J. and W.J. Fabrycky, 2001,
for many projects, this is not the case ticular period of time. Therefore, we do not “Engineering Economy” (9th ed.), Prentice Hall,
Upper Saddle River, N.J.
with RFID projects. … The ROI calcula- recommend corporate measures to evalu-
The list of the 32 articles used for analysis in this
tion for RFID projects requires that we ate investment projects. article is available upon request to the first author.

have a very good understanding of two In summary, we recommend the use

34 | a n a ly t i c s m a g a z i n e . c o m a n a ly t i c s | J AN U A R Y / F E B R U A R Y 2 011
B u s i n e s s A n a ly t i c s Sp e c i a l C o n f e r e n c e s e c t i o n

Spring conference setting (see sidebar on page 36). The Ex-

ecutive Forum, introduced last year, will of-
focus on public sector analytics, covering
federal and state government topics as well

broadens scope fer an expanded program to provide senior

executives with a bottom-line understanding
of how analytics is being used to drive better
as military talks. This track has emerged
from outreach efforts to government initiated
by INFORMS President-Elect Rina Schneur
business decisions. of Verizon. Schneur co-chairs the public sec-
tor track with Arnie Greenland of IBM and
The supply chain The Institute for Operations Research Doug Matty, U.S. Army. Confirmed speakers
management track will and the Management Sciences’ (INFORMS) Analytics Process, SCM, Public Sector include professionals from the U.S. Depart-
emphasize integration spring “Practice Conference” has been re- Enhancements to the educational program ment of Defense, Department of Homeland
from strategy to execution, branded and re-configured to encompass itself include a more vertical focus to many Security and Carnegie Mellon University.
with talks on procurement the emerging field of business analytics topics, including tracks that offer 10 sessions
transformation, total while keeping the core competencies of on supply chain management, the analytics
operations research (O.R.) intact. Renamed process and public sector analytics. Michael Soft Skills, DA and Forecasting
supply chain visibility,
the “INFORMS Conference on Business Rappa, director of the Institute for Advanced Soft skills for analysts, which have been
integrated transportation
Analytics and Operations Research,” the Analytics at North Carolina State University, the focus of a pre-conference workshop over
and other key issues.
2011 meeting will take place April 10-12 at has designed the analytics process track, along the past four years, will now be integrated into
the Marriott Downtown, located in the heart with co-chairs Stephen Powell of Dartmouth’s the regular program. Freeman Marvin and
of Chicago’s Magnificent Mile. Tuck School and James Williams, O.R. man- Don Buckshaw of Innovative Decisions, who
Real-world applications of O.R., imple- ager at Land O’Lakes. The preliminary list of helped design the soft-skills workshop, are
mented by leading companies, nonprofit speakers includes professionals from Walmart/ leading this track. The five sessions include
and government organizations, have always Sam’s Club, Teradata, Deluxe Corp., IBM, De- talks on executive decision makers and cogni-
been the focus of the spring meeting. This loitte, SAS and other companies that lead in tive styles, project team building and problem
year, the conference will provide enhanced analytics implementation. framing (see page 10).
services in line with findings from the analyt- The supply chain management track will A track on forecasting, being developed by
ics market study conducted by CapGemini emphasize integration from strategy to ex- Peter Buczkowski of Disney and Michael Gil-
Consulting for INFORMS. A new premier job ecution, with talks on procurement transfor- liland from SAS, feature talks by analytics pro-
placement service, called Analytics Connect, mation, total supply chain visibility, integrated fessionals at McDonalds, Bank of America and
will help employers and experienced analytic transportation and other key issues. Co- Accenture. Three other committee members
By David R. Leonhardi professionals connect in a casual recruitment chairs of the SCM track are Grace Lin of are developing a five-session track on deci-
World Resource Optimization, Eva Lee of sion analysis: Frank Koch of Chevron, Stefan
Georgia Tech and Doug Meiser of The Krog- Karisch from Jeppesen and Paul Wicker from
Help Promote Analytics
er Company. Decision Strategies.
It’s fast and it’s easy! Visit: http://analytics.informs.org/button.html
Another expanded 10-session track will

35 | a n a ly t i c s m a g a z i n e . c o m a n a ly t i c s | J AN U A R Y / F E B R U A R Y 2 011
B u s i n e s s A n a ly t i c s Sp e c i a l C o n f e r e n c e s e c t i o n

Poster Submission Deadline Jan. 15 Midwest Independent Transmission • Richard E. Rosenthal Young
Introducing Analytics While many conference speakers System Operator Inc. (“Operations Re- Researcher Connection: Exclusive
Connect are invited by the committee, interested search Ramps Up the Power Market in networking program for junior faculty
professionals can submit a poster pre- Mid-West”), CSAV (“A Strategic Empty and young industry practitioners.
Analytics Connect is a new, premier,
professional job event to be held at the
sentation and receive a steep registra- Container Logistics Optimization in a Ma- Reduced conference registration rate
2011 INFORMS Conference on Business tion discount. Poster sessions provide jor Shipping Company”), Fluor Corpora- $375; nominations due March 25.
Analytics and Operations Research, set a visual, brief-presentation option espe- tion (“Project Management and Change • INFORMS Professional Colloquium:
for April 10-12 in Chicago. The job event cially suited to analytics work that may ¬ A Simulation Based System for Better Intensive career guidance for master’s
is designed to provide analytics execu-
tives a venue to help feed their talent not be extensive enough for a 50-minute Performance”), Intercontinental Hotels and Ph.D. students interested in
pipeline and attract top professionals to talk. This includes projects that are still in Group (“Retail Price Optimization at Inter- practice, held on April 10. Reduced
their organizations. progress, allowing presenters to benefit national Hotels Group”), Industrial and conference registration rate $375;
Analytics Connect takes professional
networking to a whole new level. With An-
from direct feedback from colleagues. Commercial Bank of China (“Branch nominations due March 25.
alytics Connect, employers – under one The poster format also lends itself to Reconfiguration Practice through Opera- • Soft Skills Workshop: Half- and
roof – can provide their analytic recruit- corporate or consulting work subject to tions Research in ICBC”) and State of full-day workshops on the “soft” skills
ment materials in a casual setting and ar- nondisclosure restrictions that can still be New York (“Tax Collection Optimization needed to partner with decision-
range discreet one-on-one interviews in
private booths with the more than 500 an- presented at a high level. In addition, the for the State of New York”). The winner makers and users.
alytics professionals expected to attend. committee is interested in practitioners’ will be announced at a gala banquet April • Technology Workshops: In-
Job seekers, whether casual or highly advice on “soft topics,” such as selling 11. depth training from leading solution
motivated, will be able to browse recruit-
ment materials from top industry employ-
analytics to internal or client decision- Winners of two other applications-ori- providers. Free to conference
ers and decide whether to seek additional makers, working with non-technical us- ented honors are also announced at the registrations.
information and connections. ers, etc. gala: the INFORMS Prize for effective in- • Birds-of-a-Feather: Facilitated
Go to http://jps.informs.org/ to regis- tegration of analytics into organizational discussion groups that provide
ter for Analytics Connect as an employer
or applicant. decision-making and the Wagner Prize the opportunity to share common
Analytics Connect is brought to you Keynotes: UPS Vice President, for outstanding quality and coherence of challenges and solutions.
by Analytics magazine, the only informa- Edelman Winner analysis. Admission to the Gala is includ-
tion source that completely covers the
analytics profession.
Chuck Holland, VP of Industrial Engi- ed in the conference registration fee. For complete conference informa-
neering at UPS, will headline the confer- tion, go to: http://meetings2.informs.
ence in a special keynote presentation org/Analytics2011/
(see page 4). On April 12, the winner of the Special Programs
David R. Leonhardi ([email protected]) is
Subscribe to Analytics prestigious Edelman Award will reprise its The conference also offers special pro- the chairperson of the 2011 INFORMS Conference on
Business Analytics and Operations Research.
presentation. The annual Edelman Com- grams for graduate students and young
It’s fast, it’s easy and it’s FREE!
petition showcases the high-impact ana- researchers, as well as learning and net-
Just visit: http://analytics.informs.org/
lytics work of six finalist organizations: working events for all attendees.

36 | a n a ly t i c s m a g a z i n e . c o m a n a ly t i c s | J AN U A R Y / F E B R U A R Y 2 011
P r o f e s s i o n a l C o l lo q u i u m Sp e c i a l C o n f e r e n c e s e c t i o n

Career guidance Other elements focus on the soft skills nec-

essary to add value in the workplace. For
offered to help defray travel costs to the IN-
FORMS spring conference. Through the IPC,
example, for many attendees, the pre-col- students gain an understanding at an early,
loquium telecon will be the first large work- sensitive stage in their career of the role IN-
ing teleconference they will participate in. FORMS and the INFORMS community can
Other activities aim to kick start the profes- play in their professional life. The IPC allows
IPC prepares analytics and OR/MS students for professional practice.
sional network of the participants through them to begin to build professional connec-
INFORMS. tions that will help them to flourish within and
While the goal of providing career educa- contribute to the larger community of OR/MS
tion and networking opportunities for students practice.
While the goal of The INFORMS Professional Colloquium may seem altruistic, the IPC serves to main- Colloquia participants are nominated by
providing career (IPC) offers a unique opportunity to select tain and increase the vitality of the INFORMS their university departments and selected by
education and networking graduate-level analytics and operations re- practice community. Many of today’s industry the IPC committee. Historically, more than
opportunities for students search/management science (OR/MS) stu- leaders in the analytics field matriculated with one-quarter of the participants have been fe-
may seem altruistic, the dents. Held in conjunction with the INFORMS terminal masters degrees. For this reason, un- male and nearly half international (non-U.S.
Conference on Business Analytics and Op- like other programs that are only available to origin). Roughly half of the attendees have
IPC serves to maintain
erations Research Conference in Chicago in Ph.D.-level students, the IPC is open to mas- been master’s students; the remainder has
and increase the vitality
April, the IPC is a one-day program that pro- ter’s-level students as well as Ph.D. students been a mix of Ph.D. and MBA students. In past
of the INFORMS practice
vides insight from the leading practitioners in and recent graduates. years, academic institutions represented have
community.
the field on how to manage the transition from Unfortunately, university funding for stu- included the University of California at Berke-
academia to industry. dent conference travel is often in short sup- ley, Carnegie Mellon, Massachusetts Institute
This year marks the seventh consecutive ply, especially for students who are not doing of Technology, University of Colorado at Den-
year of the colloquium. More than 100 students original research. Because master’s-level ver, Naval Postgraduate School, University of
pursuing master’s and Ph.D.s from more than students rarely receive university funding Wisconsin and Texas A&M.
25 schools have participated in the program for conference travel, their exposure to IN- The IPC is organized and run by a group
since it began. The 2011 event will be held FORMS conferences and other INFORMS of active, experienced OR/MS professionals,
April 10 at the Marriott Chicago Downtown services is limited. For many IPC attendees, along with a group of past IPC participants.
Magnificent Mile. their first exposure to an INFORMS meeting This year’s organizing committee includes
The IPC program combines a variety of is the INFORMS analytics conference held representatives from a variety of public, pri-
activities. Some program elements are de- in conjunction with the IPC. vate and academic organizations, including
By Paul M. Thompson, signed to give practice-oriented students an IPC participants receive an INFORMS Northrop Grumman, the University of Wash-
Theresa Barker and understanding of what a typical “day in the membership, and to the extent that corporate ington, PROS Pricing, IBM ILOG, SAS, IBM
Robin Lougee life” is like for the many different kinds of jobs sponsorship fundraising by the IPC organizing Research, Booz Allen Hamilton, Heinz, UPS,
they could hold as an analytics professional. committee permits, limited financial support is Monsanto and Lockheed-Martin.

37 | a n a ly t i c s m a g a z i n e . c o m a n a ly t i c s | J AN U A R Y / F E B R U A R Y 2 011
P r o f e s s i o n a l C o l lo q u i u m

From the participant perspective, will find the IPC especially valuable.
the IPC has been highly successful. The committee encourages gradu-
As one participant put it, “Attending the ate students who have a background
[IPC] was one of the top-10 experienc- in analytics or OR/MS, and who have
es I have encountered during my first an interest in the world of analytics/
year as an MBA student. Not only was I OR/MS practice in any public or private
a member of a select group of students sector area, to seek a nomination to the
invited to the conference, but I was IPC from their professor or department
also sought out by professionals in at- head. The committee also strongly en-
tendance. During the three-day event I courages professors and department
was invited to interview with two dozen heads of analytics, OR/MS and relat-
organizations. I was even invited to fly ed fields to consider nominating top,
out to a corporate campus immediately practice-oriented students to attend the
following the event.” 2011 colloquium.
The IPC Planning Committee is so- The committee also welcomes spon-
liciting nominations for IPC 2011. Par- sorship from companies; 100 percent
ticipants should expect to complete a of sponsorship funds subsidize student
post-graduate degree between Octo- travel. Further information about IPC
ber 2011 and August 2012, or be re- 2011 is available on the 2011 INFORMS
cently graduated. They should have Practice Conference Web site, http://
a serious interest in and be qualified meetings2.informs.org/Analytics2011/
for entry-level work as an analytics or profcolloquium.html.
OR/MS professional. The committee Paul M. Thompson ([email protected]),
encourages master’s-level students chair of the 2011 IPC Planning Committee, currently
works for Northrop Grumman Corporation in Northern
who have not yet decided between Virginia. Robin Lougee ([email protected]),
pursuing Ph.D.-level studies and en- co-founder of the IPC and member of the planning
committee, works for IBM Research in New York.
tering the working world of OR/MS Theresa Barker ([email protected]), a
practice to attend the colloquium. former participant and current member of the IPC
planning committee, is at the University of Washington.
Students with prior work experience

Subscribe to Analytics
It’s fast, it’s easy and it’s FREE! Just visit: http://analytics.informs.org/

38 | a n a ly t i c s m a g a z i n e . c o m
Learning Resources
Mary Benner, Wharton School Mary Grace Crissey, SAS

INFORMS’ library Do Financial Analysts Sabotage New

Product Innovation
The Nitti-Gritty of Working with
O.R. Providers

of audio and Recorded June 11, 2010 Recorded Feb. 5, 2010

video presentations
David Alderson, Dina Mayzlin, Yale University &
Barry List Naval Postgraduate School David Godes, University of Maryland
Danger: Infrastructure Under Attack Word of Mouth Marketing
Recorded May 28, 2010 Recorded Jan. 22, 2010
Science of Better: Podcasts Claudia Perlich, Media6Degrees
Gain insights from experts on how Mining for Gold Peter Kolesar, Columbia University Pinar Keskinocak & Julie Swann,
math, analytics, and operations research Recorded Oct. 8, 2010 Ending the Water War Surrounding Georgia Tech
affect organizations like yours in these New York’s Reservoir Haiti: Humanitarian Logistics
20-30 minute podcasts conducted by Maurice Levi, University of British Recorded May 14, 2010 Recorded Jan. 14, 2010
INFORMS Director of Communications Columbia
Barry List. Visit www.scienceofbetter.org/ Deal or No Deal: Hormones Impact Eva K. Lee, Georgia Tech Doug Samuelson, Infologix
podcast. Business Analytics Goes to War - Winning Elections with O.R.
Recorded Sept. 3, 2010 Against Cancer Recorded Jan. 12, 2010
Jeanne G. Harris, Accenture Recorded April 30, 2010
Talent Analytics Scott Patterson, author of Ron Howard, Stanford University
Recorded Dec. 3, 2010 “The Quants” Warren Lieberman & Michael Raskin Master Decider
The Quants Forecasting Consumer Behavior Recorded Dec. 18, 2009
Joan Woodard, Sandia National Labs Recorded Aug. 19, 2010 Recorded April 8, 2010
Needed: Analytics Pros in Energy John Sterman, MIT
Recorded Nov. 19, 2010 Admiral Mike Mullen, Michael Schrage, MIT Climate Change: On to Copenhagen
Chairman, Joint Chiefs of Staff Experimenting at the Workplace Recorded Dec. 4, 2009
Bill Hostmann, Gartner Chairman and Analytics Champion Recorded March 19, 2010,
Gartner on Business Intelligence Recorded July 21, 2010 Anna Nagurney, University of
Recorded Nov. 3, 2010 Arnold Barnett, MIT Massachusetts, Amherst
Ed Kaplan, Yale University How safe are our airports? Supernetworks: Building Better Real
Allan Lichtman, American University Terror Queues Recorded March 4, 2010 and Virtual Highways
Obama in 2012 Recorded July 9, 2010 Recorded Nov. 6, 2009
Recorded Oct. 22, 2010 Brian Lewis, Vanguard Software
Andy Boyd, University of Houston Advice to Execs on Working with Sam L. Savage, Stanford University
Pricing and Gumballs in Airline Reservations Operations Researchers The Flaw of Averages
Recorded June 24, 2010 Recorded Feb. 18, 2010 Recorded Nov. 6, 2009

39 | a n a ly t i c s m a g a z i n e . c o m a n a ly t i c s | J a n u a r y / f e bru a r y 2 011
Learning Resources

INFORMS Video Learning Center Pinar Keskinocak, Georgia Institute Arturo Palacios, Daniel Myers, Proctor & Gamble,
View these on-demand presentations, of Technology, Hannah K. Smalley, INDEVAL Glenn Wegryn, Proctor & Gamble,
complete with slides, from INFORMS Georgia Institute of Technology INDEVAL Develops a William Tarlton, Proctor & Gamble,
renowned meetings and conferences. Universal Tool for Vaccine Scheduling: New Operating and Sean Willems, Boston University
Visit http://livewebcast.net/INFORMS_ Applications for Children and Adults and Settlement Inventory Optimization at Proctor &
Video_learning_Center. (A short log in (Georgia Tech and CDC) System Using Gamble: Achieving Real Benefits
is required.) Hector Anaya Operations Research Through User Adoption of Inventory
Germán Riaño, Kimberly Clark Latin Tools
Annual Meeting 2010 American Operations Robert Tudor, Delaware
Plenary Presentation: Kimberly-Clark Latin America Builds River Basin Commission, Hylton Robinson, Sasol Technology,
John Birge, The University of an Optimal Scheduling System James Serio, The Delaware Marlize Meyer, Sasol Technology,
Chicago River Foundation, and Peter Michele Fisher, Sasol Technology,
ORMS and Risk Management Diavakar Rajamani, The University of Kolesar, Columbia University and Willem Louw, Sasol Technology
Failures: What are we doing wrong? Texas at Dallas Delaware River Basin Commission – Sasol – Innovative Decision Support
(Omega Rho Distinguished Lecture) A Short-Range Scheduling Model Breaking the Deadlock: in a Petochemical Production
for Blockbuster’s Order Processing Improving Water Release Policies Environment
Keynote Presentation: Operation on the Delaware River through
Gerald Brown, Naval Postgraduate Operations Research
School Ramasubramanian Sundararajan, Interview with
Rethinking Risk Assessment and GE Global Research, India Lori Folts, Deutsche Post DHL, Admiral Mike Mullen
Management for Homeland Security Marketing Optimization in Retail Marc Fischer, University of Passau, Admiral Mike Mullen,
and Defense Banking (GE) and Tjark Freundt, McKinsey & Chairman, Joint
Company Chiefs of Staff
2010 Wagner Prize Presentations: Practice Conference 2010 Deutsche Post DHL – Managing Chairman and
Mark Squillante, IBM T. J. Watson 2010 Edelman Global Brand Investments at DHL Analytics Champion
Research Center Award Presentations: (video companion to
OnTheMark: Integrated Stochastic Hector Anaya, INDEVAL, Dale Wilson, New Brunswick DoT, Admiral Mike podcast) July 21, 2010
Mullen
Resource Planning of Human Capital Jaime Villasenor, INDEVAL, Kim Mathisen, New Brunswick DoT,
Supply Chains (IBM) Francisco Solis, Banco de Mexico, Ugo Feunekes, Remsoft, and John
Miguel de Lascurain, ITAM, and MacNaughton, New Brunswick DoT
New Brunswick (Canada) Dept. of
Transportation – Taking the Politics out
Subscribe to Analytics of Paving: Achieving Transportation
It’s fast, it’s easy and it’s FREE! Just visit: http://analytics.informs.org/ Asset Management through O.R.

40 | a n a ly t i c s m a g a z i n e . c o m a n a ly t i c s | J a n u a r y / f e bru a r y 2 011
Data M i n i n g

Predicting stock price can use them to achieve a “best execution”

of fund managers’ buy/sell orders, while day
with the best performance is implemented.
The test database generally represents 30

movements traders can use them to realize fast profits

over short periods of time.
The predictive analysis solutions produce
percent of the database.
When the implementation of the data min-
ing techniques is complete, one code runs
valuable business results in the real world each five minutes. The code collects, from the
applications by building recommendations database, the different information needed,
INFORMS data mining contest attracts 894 participants representing
147 teams from 27 countries. systems. applies the data mining algorithm, predicts the
Here’s how it works: studied stock price will increase or decrease in
First, the data miners develop the data min- 60 minutes and recommends the appropriate
ing techniques by: action to maximize the profit.
In the INFORMS Cole Harris of Exagen Diagnostics (www. • making the different databases In the INFORMS Data Mining Contest, par-
Data Mining Contest, exagen.com) won the 2010 Data Mining Con- communicate with each other to collect ticipants were provided with a set of macro-eco-
participants were test that required participants to develop a all the information on the back office nomic and high frequency financial data to build
provided with a set of predictive analysis solution to predict stock databases; their predictive analysis solutions. The data were
macro-economic and high price movement (increase or decrease) in “the • extracting the needed data and dealing with composed of stock prices, sector indexes, eco-
frequency financial data to next 60 minutes” in five-minute intervals. (For missing data and outliers; nomic indicators and expert predictions on eco-
example, at 9:30 a.m. will XYZ’s stock price • preparing the data for the data mining nomic indicators. The database was separated
build their predictive
increase or decrease at 10:30 a.m. based on algorithm; into two data sets: the training set for building
analysis solutions.
historical data?) • writing the code relative to the data mining predictive analysis model(s) and the test set (in
Christopher Hefele of AT&T finished sec- algorithm in the company systems; which the target variable has been excluded) for
ond and Nan Zhou from the University of Pitts- • predicting the data from the test database, evaluating participants’ predictions. The partici-
burgh placed third in the contest organized and to validate the performance of the data pants built their predictive analysis solutions us-
sponsored by the INFORMS Data Mining Sec- mining techniques; and ing the training set and implemented it on the
tion. The contest drew 894 participants from • writing the code relative to the test set by predicting target variable.
147 teams representing 27 countries, making implementation of all the processes on the The top three finishers in the contest pre-
it the largest event of its kind in the world. company systems. sented their methods at the 2010 INFORMS
Day traders, mutual fund traders and To verify that the data mining algorithm Annual Meeting in Austin, Texas, in November.
hedge funds have always tried to predict will work when they implement it, the “min- To view slides of their solutions, see: www.
the direction of stock prices in the next few ers” predict (score) the test database obser- kaggle.com/informs2010.
hours. Predictive analysis solutions devel- vations and validate the performance. The Durai Sundaramoorthi is an assistant professor at Missouri
oped in the contest aim to pursue this ob- data mining algorithm should have similar Western State University, Philippe Belanger of Laval University
By Durai Sundaramoorthi, jective. Hedge funds can use these types of performance in the training database and in served as co-chair of the 2010 Data Mining Contest, and Louis
Duclos-Gosselin ([email protected]) of Sinapse
Philippe Belanger and solutions to build complex strategies to be the test database to be considered a “supe- chaired the contest, a role he will continue in 2011.
Louis Duclos-Gosselin
executed automatically. Mutual fund traders rior performer.” The data mining algorithm

41 | a n a ly t i c s m a g a z i n e . c o m a n a ly t i c s | J AN U A R Y / F E B R U A R Y 2 011
 T H INKING ANALYTICALLY

Planet colonization
Colonizing a new planet where conditions
are unpredictable and harsh is never easy. The
primary concern when selecting a landing site
for the first colony is the proximity to natural re-
sources. Access to these valuable resources de-
termines whether or not a new colony thrives or
dies.
Figure 1: Front (left) and rear (right) Figure 1 shows the front and rear views of a
view of planet.
newly discovered planet. The planet has been
divided into 20 areas, each of which is a poten-
tial site to establish a new colony. Some areas
contain value resources (represented by colored
icons) that are needed in order for the new colony
to survive. Food is represented by the green ear
of corn, oxygen by the red O2, water by the blue
drop and energy by the orange lightning bolt.
When choosing a landing site, it is best to
minimize the distance between that site and the
four needed resources. Distance is calculated
by the number of units it takes to get to the re-
source. For example, if area 9 was selected as
the landing site, the total travel distance required
to reach all four resources would be six units
(two for energy, one for good, one for water and
two for oxygen).

Question: Which of the 20 areas is the best

landing site to minimize the total distance you
By John Toczek should have to travel to all four resources?
John Toczek ([email protected]) is the Send your answer to ThinkingAnalytics@gmail.
senior decision support analyst for
ARAMARK Corporation in the Global com by March 15. The winner, chosen randomly
Risk Management group from the correct scores, will receive an “Analytics:
Driving Better Business Decisions” T-shirt. ❙

42 | a n a ly t i c s m a g a z i n e . c o m a n a ly t i c s | S e p t e m b e r / O c t o b e r 2 010