Greg Satell (http://www.forbes.

com/sites/gregsatell/) Contributor
Opinions expressed by Forbes Contributors are their own.

FOLLOW

OPINION (/OPINION)

12/03/2013 @ 1:45AM

11,825 views

Yes, Big Data Can Solve Real

World Problems
Comment Now

Follow Comments

Working with IBM, the Memphis Police Dept.

managed to reduce crime by 30% using big data
analytics. (Photo credit: Wikipedia)

Most people in the tech world have become familiar with the hype cycle
(http://www.digitaltonto.com/2011/managing-the-hype-cycle/). A new technology enters the
marketplace amid great expectations. Inevitably, disappointment sets in and a retrenchment
period begins, practice and process catch up to expectations and new value is unleashed.
Right now, there is probably no area more hyped than big data
(http://www.digitaltonto.com/2013/what-is-big-data-and-what-do-we-do-with-it/) and theres
already no shortage of self-proclaimed experts. Yet most big data efforts fail
(http://www.itmanagerdaily.com/big-data-projects-fail/) and there is a growing divide
(http://www.digitaltonto.com/2013/the-data-divide/) between enterprises that are benefiting from
its use and those who are not.
There are a variety of reasons for thisa lack of qualified data scientists, poor integration across
departments and a failure to manage expectations all play a part. Yet for those who have built a big
data culture, the investment is already paying off. They key lies not with fancy algorithms or
buzzwords, but by focusing on real world problems.
Replacing Routine Maintenance With Predictive Analytics
UPS runs one of the largest logistics operations in the world, delivering millions of packages every
day. If one of their trucks has even a minor breakdown, it can be a big deal, resulting in driver
downtime, late packages and angry customers.

So UPS used to replace important parts every few years to ensure that its vehicles stayed in good
working order. Now, however, they collect data from hundreds of sensors in each vehicle
(http://www.fieldtechnologies.com/gps-tracking-systems-installed-in-ups-trucks-driverefficiency/). Then algorithms analyze that data from thousands of trucks to predict when a part is
likely to break down, allowing UPS to save millions in maintenance costs.
IBM used similar algorithms (http://www-03.ibm.com/press/us/en/pressrelease/40582.wss) to
help the city of Boston to reduce municipal costs by predicting when repairs are likely to be needed.
They even went a step further in Rio. They even went a step further in Rio, where big data systems
not only anticipate when deadly landslides are likely to happen, but are integrated with a nextgeneration operations center (http://www.nytimes.com/2012/03/04/business/ibm-takes-smartercities-concept-to-rio-de-janeiro.html?pagewanted=all) that helps coordinate response.
This is all possible because, with enough data, patterns begin to emerge that can detect anomalies.
By combining smart algorithms with the Web of Things (http://www.digitaltonto.com/2012/cocreation-and-the-new-web-of-things/), we can both cut costs and increase operational
effectiveness.
The New Consumer Conversation
Marketers have long known how important it is to have your finger on the pulse of the consumer.
However, in the past, its been a fairly crude process. You either performed basic surveys with precanned answers that customers would check off or had to read genuine responses one by one.
Now big data is enabling a new consumer conversation (http://www.digitaltonto.com/2012/thenew-consumer-conversation/) that uses natural language processing
(http://en.wikipedia.org/wiki/Natural_language_processing) to read and evaluate consumer
responses. For example, Semantria worked with Schwans frozen foods
(http://img.en25.com/Web/LexalyticsInc/%7b2ed607db-ddb2-435f-bb0d2726ae16e980%7d_Schwans_Case_Study.pdf) to evaluate thousands of responses and understand
what their customers really thought about them.
The USTA took an altogether different approach to speak to their audience at the recent US Open.
They used big data analytics to analyze 41 million data points over 8 years of Grand Slam play and
created a Slam Tracker (http://2013.usopen.org/en_US/slamtracker/index.html) that would
instantly generate keys to the match so that fans could see what their favorite player had to do to
win.
And the NBA recently launched a similar effort (http://www.fastcolabs.com/3022190/how-nbasbig-data-strategy-will-change-the-way-you-watch-basketball), opening up its statistical database
and corresponding video library to all of their fans for free. We often think about big data as a
purely analytical effort, but sometimes it can be most useful improving communication and
transparency.
Improving Public Safety
We like to see police out on the street. They make us feel safe by keeping watch, responding quickly
and intervening when trouble comes along. Yet they often get stuck doing grunt work and shuffling
papers. Although important, these things dont make us feel all that much safer. Big data is making
an impact here as well.
One of the earliest systems was the Memphis Polices Blue CRUSH system
(http://www.ibm.com/smarterplanet/us/en/leadership/memphispd/), which reduced crime by
more than 30% and was so effective that when funding was cut, it created a controversy

(http://www.commercialappeal.com/news/2013/jan/27/blue-crush-controversy/). More recently,

IBM started a research partnership in Ft. Lauderdale, FL (http://www.govtech.com/data/FortLauderdale-Fla-PD-Analytics.html) that will help analyze data across city departments.
This opens up a whole new world of possibilities. Do new building permitsand the construction
that followsincrease theft? How will changing demographics in different areas of the city affect
crime rates? Does an increase in requests for public assistance correlate with an increase in crime?
And were only starting to scratch the surface. Semantria (https://semantria.com/), recently tuned
its language processing algorithms to recognize street slang in Detroit. By combining chatter
gleaned from social media with predictive crime analytics, police departments can become even
more effective.
How To Big Data Work
While all of the success stories are impressive, what we dont hear much about are the big data
failuresthe big initiatives with big budgets that amount to little or nothing at all. Big data is no
panacea, its a tool and much like any other, it needs to be used intelligently. Heres how you do
that:
Start With A Problem: Probably the worst thing you can do is to create a highly publicised big
data initiative. Much like any other technology, big data works best when its invisible to the end
users, but helps them do their jobs nonetheless.
One thing that all of the projects described above have in common is that they started with a
problem to solve. UPS and Boston wanted to lower maintenance costs, Rio and the other cities
wanted to improve public safety, Schwan and the USTA wanted to talk to consumers. Every good
project starts with a problem, not a solution.
Focus On The Core: Humans are very bad information processors. Nevertheless, we spend a lot
of our time performing informational tasks. Poring over documents and shuffling papers is simply
not a good use of peoples time. Big data can free up those tasks so that front line personnel can
focus on their core mission.
As Mark Cleverley, Global Director of Public Safety at IBM puts it, We are moving into an era
where we can be roughly right more frequently and precisely wrong less. Narrowing down
possibilities helps us to make the data to be actionable.
In other words, big data is most effective when it is aimed at improving peoples ability to do their
jobs.
Increase Visibility Across The Enterprise: Another advantage to big data is that it can help
the entire enterprise work as one functional unit. There is no longer any need for data silos for
different functions such as marketing, finance, logistics, etc. Big data techniques allow us to all
work from the same data set and pull out what we need.
Build Collaboration: In most organizations, there are subject matter experts who are able to
generate insights and front line personnel who are responsible for delivering a product or service.
One of the great management challenges is helping one group communicate effectively with the
other.
Thats why IBMs Cleverley sees a big part of his job as bringing analytical capabilities to much
broader class of people who are not specialists. By letting giving people at the point of service
access to expert level insights, the entire organization benefits.

And thats the secret to success. The best way to approach big data is not to try to build a better
system, but to build a better enterprise.
***
5 comments, 5 called-out

Comment Now

Follow Comments

Greg Satell (http://www.forbes.com/sites/gregsatell/) Contributor

Opinions expressed by Forbes Contributors are their own.

FOLLOW

OPINION (/OPINION)

10/11/2013 @ 11:30PM

21,598 views

Why Big Data Matters

Comment Now

Follow Comments

Terabytes, Petabytes, Exabytes. Who can keep track? These strange terms have just begun to enter
the business lexicon, but the hype surrounding them has reached a fever pitch. We have
undoubtedly entered the age of big data (http://www.digitaltonto.com/2013/what-is-big-data-andwhat-do-we-do-with-it/).

Photo credit: Kevin Krejci

Yet its hard for many to take it seriously. While the blogosphere buzzes and millennials preach,
most serious business people are focused on their jobs. They have partners, customers and
employees to keep happy and all the techy mumbo jumbo just doesnt seem relevant.
Yet big data is important because it will transform how we manage our enterprises. For most of the
20th century, business leaders relied on scientific studies and statistical significance to
determine what information they could trust. Now, technology is making those assumptions
obsolete and the practice of management will never be the same.

A Big Discovery of a Small Planet

In 1801, the astronomer Giuseppe Piazzi (http://en.wikipedia.org/wiki/Giuseppe_Piazzi) noticed a
small body in the night sky. At first, he thought it was a comet, but it soon became clear that it was,
in actuality, the dwarf planet now known as Ceres (http://en.wikipedia.org/wiki/Ceres_(asteroid)).
He tracked it for 40 days before it disappeared behind the sun and astronomers were unsure how
to find it again.

It was then that a young man named Carl Friedrich Gauss

(http://en.wikipedia.org/wiki/Carl_Friedrich_Gauss) applied his method of least squares
(http://en.wikipedia.org/wiki/Least_squares) and was able to predict the orbit of Ceres
(/companies/ceres/) . Shortly after, as if by magic, it appeared exactly where Gauss said it would
be. He went on to become the most influential mathematician of his time.
In business, we often run into very similar problems. We need to make decisions based on
incomplete information in a rapidly changing context. So not surprisingly, Gausss work has
formed the basis of many of the statistical techniques that modern day management employs, such
as regression analysis (http://www.digitaltonto.com/2012/its-the-math-stupid/), to make sense of
a messy world.

Rules For Control

For nearly a century, Gausss ideas were mainly a subject for academics, but in the 1920s Ronald A.
Fisher (http://en.wikipedia.org/wiki/Ronald_Fisher) burst onto the scene. He set out rules for the
design of experiments (http://en.wikipedia.org/wiki/Design_of_experiments), confidence
intervals (http://en.wikipedia.org/wiki/Confidence_interval) and statistical significance
(http://en.wikipedia.org/wiki/Confidence_interval), among other things. Underlying his methods
was an emphasis on controls. Put good data in and you would get good answers out.
Before long Fishers methods were adopted by business, culminating in the Six Sigma movement
(http://en.wikipedia.org/wiki/Six_Sigma) that purported to achieve stable and predictable results.
Much like Fishers earlier efforts, it was thought that by controlling every aspect of the process,
uncertainty could be tamed and management could be transformed from an art into a science.
Yet all was not well. Many, Nassim Taleb (http://en.wikipedia.org/wiki/Nassim_taleb) in
particular, argued that control was a dangerous illusion. Anything that met a basic standard of
statistical significance (usually 95% confidence) was treated as fact. False certainty led managers to
discard inconvenient information as outliers, often with disastrous results.

An Alternative Approach
Hovering in the background all this time was an alternative approach called Bayesian inference
(http://en.wikipedia.org/wiki/Bayesian_inference), which allowed you to simply make a guess and
then revise your judgement as new information came in. It was, in many ways, the polar opposite
of Fishers approach. No specific controls, no rules about significance, just an updating of
probabilities.
Although Bayesian methods were successful in some important cases where controlled studies
werent an option, such as hunting German subs
(http://en.wikipedia.org/wiki/Cryptanalysis_of_the_Enigma) during World War II, they werent
widely deployed. Part of the reason was that Fisher and his followers fought hard against them, but
mostly it was because they were impractical. It was hard to gather enough data to make them work.
Thats what big data is starting to change. The combination of accelerating returns
(http://www.digitaltonto.com/2012/the-new-new-economy-of-accelerating-returns/) in storage
and processing power, along with a sea of data from the Web of Things
(http://www.digitaltonto.com/2012/co-creation-and-the-new-web-of-things/) and increasingly
efficient algorithms, are making Bayesian methods not only practical, but faster, cheaper and more
accurate than conventional methods.

A Fundamental Shift

Like any new technology, there is a lot of confusion surrounding big data. There are endless debates
about what is and isnt big data, armies of consultants who are eager to muddy the waters in return
for a hefty retainer fee and the usual amount of hype and alphabet soup of acronyms and
buzzwords.
But what you really need to know about big data is this: It represents a fundamental shift in how
we do things. In effect, big data opens the door to a Bayesian approach to strategy
(http://www.digitaltonto.com/2013/bayesian-strategy/) where we no longer try to be right based
on controlled research and small samples, but rather become less wrong over time
(http://www.digitaltonto.com/2012/why-our-numbers-are-always-wrong/) as real world
information floods in.
The truth is even the old mantra of failing fast and cheap is becoming too slow and expensive.
Thats why there is a growing divide between businesses who use data effectively and those who
don (http://www.digitaltonto.com/2013/the-data-divide/)t. Big data means much more than a
change in technology, it represents a structural transformation is how we will manage our
enterprises.
***
4 comments, 2 called-out

Comment Now

Follow Comments