THE BIG DATA REVOLUTION

OTHER GREAT READS

COPYRIGHT

A business running without accurate data is running blind.

- Ash Mahmud
In the third century B.C., Egypts ruling Ptolemaic dynasty erected the Library
of Alexandria. A testimony to civilization and learning - and the dynastys own
wealth and influence - the library was reputed to hold the sum of humankinds
knowledge. For the next twenty-three centuries, this body of knowledge grew
at a steady pace and libraries were constructed all over the globe to contain it.
Then in the late twentieth century the Internet came of age. This historic
advance revolutionized information gathering and storage capabilities. Today,
there is approximately 2.5 trillion times as much information in the world as
was held in the Library of Alexandria - and the amount is doubling every three
years. The vast majority of it is stored not in libraries but on servers. In 2000,
three-quarters of the worlds stored information was in analog form; now, that
figure is less than 2 percent. In other words, to explore the worlds knowledge
a person no longer needs a library - a laptop or tablet or smartphone will do
just fine. This vast digital storehouse is, in essence, big data.
Big data means big profits. Research from the Harvard Business School, the
Massachusetts Institute of Technologys Sloan School of Management,
McKinsey & Companys business technology office, and the University of
Pennsylvanias Wharton School shows that businesses that make the most of
big data are fully 6 percent more profitable than those that dont. In times like
these, thats an uncommonly large number.
Heres what else you need to know about big data.

The Business Case

The Harvard Business Review concluded that the companies that significantly
outperform their peers are more likely to collect multiple types of data - from
that generated by RFID tags, for instance, to data from Web-tracking
technologies. Furthermore, high-performing companies are more in touch with
data than their less-successful rivals. When asked about marketing and
communications, 59 percent of executives from top-rated companies called
data extremely important to their businesses as compared to a 39-percent
ranking from lower-rated concerns.
As for the technology itself, executives of companies that are ahead of their
peers in analyzing data gave it high marks. Forty percent of high-performing
companies said the speed at which their organization processes data has
increased significantly over the past twelve months, and that speed is
producing dividends.

A joint study conducted by IBM and MIT and published under the title, The
New Intelligent Enterprise concluded that the number of businesses using
analytics to build a competitive advantage had jumped by almost 60 percent
from a year earlier. The study said that nearly six out of ten organizations are
using analytics to differentiate themselves from competitors.
Case studies back up the results.
Picture a customer call center that has the technology to detect a change in
tone as a frustrated customer raises his voice to say: This is the third power
outage Ive had in one week! A big-data solution would both identify the
words third and outage as negative terms affecting the consumer, and the
tonal change would be another indicator that he or she might choose another
provider. These insights can be gleaned from unstructured data. But what if
that unstructured data could be combined with the customers record data and
transaction history? Now the company has a personalized model of the
consumers value and a sense of how tenuous the relationship has become.
The demand for big-data insights has risen so quickly that there are now
numerous companies whose product itself is big data. Cases in point:
Anyone who flies regularly knows it isnt always a pleasant experience. And one of
the most frustrating parts is the all-too-common wait after landing for a gate and
ground crew to be ready for deplaning. This happens when a plane lands early. If a
plane lands late, its less annoying for the passengers, but the ground crew has
been waiting, costing the airline money. Its not only the carriers that suffer;
airports with lots of late and early arrivals develop bad reputations with passengers.
Enter PASSUR Aerospace, a big-data company that tracks plane arrivals with
unprecedented accuracy. Its rightETA program relies on patented algorithms
backed by over 150 passive radar sensors and integrated with real-time
mining of multiple databases. Every 4.5 seconds, PASSUR collects data on
every plane tracks. It also uses its vast storehouse of past arrivals to compare
previous landings under the same conditions at the same airport. PASSUR
currently has contracts with the top eight North American carriers, sixty U.S.
airports, 200 corporate aviation departments, and the U.S. government.
IBM has a patent on technology for securing premises using surface-based
computing technology. The surface in question is the floor, and the technology
identifies what or who is on the floor - i.e. furniture or people, what the object or
person weighs, and when and where they move. In other words, IBM has developed
smart floors. Walk into a room and the lights come on, and the latest episode of
Mad Men appears on your flat-screen. If an elderly person falls, Lifeline will receive
a signal. Stores, casinos, hotels, and government agencies could use the floors to

gather data on foot-traffic patterns and use it to redesign their spaces. And it could
be a potent antitheft device, alerting homeowners and businesses to an invader
with unprecedented precision.
New York City is using big data in innovative and effective ways. It was a pioneer in
the use of proactive policing, which uses computers to track crime as it takes place
in the citys neighborhoods. From experience, police know that a spike in minor
crimes in a neighborhood is the precursor to a wave of more serious offenses. When
the police departments big-data modeling shows this happening, officers flood the
area. The citys crime rate has fallen to historic lows since this system was
instituted. It works.
So does the citys use of big data to cut down both on fires and violations of
the housing code.
With affordable housing at a premium in New York, many buildings and
apartments are illegally subdivided into small rooms. Often these rooms are
shared by groups of immigrants from Asia or Latin America who work at lowpaying jobs. Fire-department data showed that fires were far more likely to
occur in buildings that had been illegally subdivided. But with approximately
25,000 complaints a year about overcrowded buildings and only 200
inspectors, the city was overwhelmed. Big-data analysts in then-Mayor Michael
Bloombergs office searched for ways to identify which of the 25,000
complaints were most egregious. They created a database of all 900,000
buildings in New York, along with information from nineteen city agencies that
included ambulance visits, rodent infestations, calls to 911, unpaid taxes,
unexplained spikes in utility usage, building age, neighborhood crime rates,
and more. They then cross-referenced this with five years of fire-department
information that ranked fires by their size and cause. The data quickly
demonstrated that certain combinations of complaints, fines, and other
violations were a reliable predictor of where fires would occur. Inspectors could
now triage their visits. The results speak for themselves: Before the big-data
initiative, inspectors issued emergency vacate orders to 13 percent of the
buildings they visited; today that figure is 70 percent. And the numbers of
fires in the city has decreased dramatically.
Big data can not only prevent fires, it can save lives. Canadian medical researchers
have developed a series of big-data measures to improve the survival rate of
premature babies. The goal is to determine which babies are most likely to develop
life-threatening infections. The process continuously measures sixteen vital signs,
including breathing, blood-oxygen levels, organ function, blood pressure, and
temperature. This information is aggregated into a 1,000-data-points-per second
flow that alerts doctors to subtle changes that are often the precursor of infection.
Prophylactic antibiotics can then be administered. The system has increased the
survival rate of premature infants at the hospitals where it is used.

In Baltimore, John Hopkins Medical School researchers have found that data
from Google Flu Trends - a free aggregator of flu-related search terms
predicts increases in flu-related emergency room visits a full week before the
Centers for Disease Control. Flu outbreaks are tracked nationwide, and local
hospitals are able to prepare for the onslaught. Twitter and Google as
diagnostic tools this is what the new digital democracy looks like.

What Big Data Isnt

Big data is not the same as analytics. It is true that both seek to glean
intelligence from data and translate the findings into competitive advantage,
but it is the volume, speed, and advanced technology involved that puts big
data on a higher plane.
For example, in 2012, about 2.5 exabytes of data were created every day. For
perspective, one exabyte is equal to one quintillion bytes. Put another way,
the data that now floods the Internet every second is equivalent to the data
stored on the entire Internet twenty years ago. And now, in 2014, companies
are working with petabytes, i.e., one quadrillion bytes of data in a single data
set. Wal-Mart alone collects more than 2.5 petabytes of data every hour from
its customer transactions - or about 20 million, four-drawer filing cabinets
worth of text.
But speed of data creation can trump volume in certain cases, as an MIT group
discovered one recent Black Friday, the kick-off of the holiday shopping season
in the United States. Using location data from mobile phones to deduce the
number of people in a Macys parking lot, the researchers were able to
estimate the retailers sales on that day before the retailer itself knew its
numbers. Accessing real or nearly real-time information can make it possible
for a company to be much more agile than its competitors.
Even a bull named Badger-Bluff Fannie Freddie figures in the big-data
revolution. There are 8 million-plus Holstein cows in the United States, but
only one bull merits mention based on scientific data. Dairy cattle sired by him
346 daughters and counting - produce more milk than cows fathered by
more ordinary bulls.
The dairy industry, long cursed by skimpy margins and bulging costs, is
profiting from knowledge acquired from big data. Dairy breeding, it turns out,
is ideal for quantitative analysis, because breeders keep exacting pedigree
records and artificial insemination makes it easier to trace genetic information
about the handful of top-rated animals. The producers can also track overall
milk output, fat and protein content, and udder quality. Since farmers pay
thousands of dollars for their breeding stock, they dont mind paying top dollar

for the best genetic material.

So whether we are talking about airlines or fires or animal husbandry, it all
adds up to an almost unfathomable amount of information. Learning to master
and manipulate it is the defining managerial competence of the twenty-first
century.

Where Big Data Comes From

Technological advances, coupled with changes in consumer behavior, now
allow managers, marketers, and leaders to see every online step a customer
makes. Add up the oceans of data from digital video recorders, retail
checkouts, credit-card transactions, and countless other sources, and
businesses everywhere are privy to a heretofore unimaginable trove of
information about what consumers see and do.
It is difficult to find a product or piece of equipment these days that doesnt
contain coding. Airplanes are equipped with more than a billion lines of code
that generate about ten terabytes of data per engine during every thirty
minutes of operation. To put those numbers in perspective, a flight from
Heathrow Airport in London to John F. Kennedy in New York City would
generate about 650 terabytes of data.
Time stamps are ubiquitous. Think of the auto-dating metadata that gets
captured every time someone takes a picture with their camera or
smartphone, posts to Facebook, or uses a tablet to watch a television show. In
short, it is not hard to assemble a timeline of peoples lives. The average
commuter in London will have his or her picture taken more than 150 times a
day while traveling from home and back from downtown London. Add that
surveillance to the variety of sentiment, temporal, and spatial information
generated during that time frame, and a good deal of big data is there for
businesses to access.
There also is a common misconception that big data is generated only online.
Yes, there are over a billion Google searches and Twitter messages each day,
and Facebook, LinkedIn, MySpace, and Pinterest alone have over a billion
combined users who post text, images, and links. In addition, Web sites track
visitor preferences and habits, as well as internal processes and results. But
big data doesnt stop there. It also includes scientific, medical, and sociological
studies and research; organizational, government, and political documents;
entertainment and arts content in all media; and news of every type.
While big data has the potential to radically improve the way organizations
function, it also has the potential to overwhelm them. Many leaders dont

know where to begin. The key is to develop processes that recognize, mine,
and exploit the information that is relevant to an organizations needs.
Achieving this requires changes in management, processes, and culture. The
choice is stark: Master the data or drown in it.
To make the most of big data requires us to rethink how we think. When
information was stored in analog form, data collection was expensive and
time-consuming. Taking an opinion survey, for example, required face-to-face,
telephone, or mail contact with interviewees. Today, people can respond by
touching a single key, and their responses are recorded and filtered instantly.
With the big-data deluge, there will be some inaccuracy, but it is worth it. A
slightly disorganized sample of 100,000 consumers preferences, for example,
yields better insights than a 100-percent accurate survey of 200 people.
Language translation provides a vivid example. Because language is complex
and idiosyncratic, early online translations were literal dictionary downloads
and overly formal and often contained errors in grammar and tense. Then IBM
found a way to improve the process, using French as its model. Because the
Canadian government conducts its business in both French and English, IBM
used transcripts of that countrys parliamentary proceedings to create software
that translated sentences with far more sophistication and accuracy than a
simple dictionary match. This technique was then applied to other languages
with equally improved results. But there were limits. The syntax was too often
stilted, and there were few colloquialisms. Enter Google. It gathered
translations from across the Internet: business documents, Web sites, song
lyrics, books, even emails that had been translated for one reason or other.
The result was translations that were not only more accurate, but more
colloquial and fluid. The lesson: Vast quantities of messy data yielded a result
superior to the one gleaned from small amounts of orderly data.

What Over Why

There is another key difference with the way information was treated in the
past. Traditionally, the goal was to work backwards from the data to figure out
the why of what the information showed. If, for example, data revealed that
there was more employee absenteeism on Monday than on any other day of
the week, the goal would be to find the reasons behind the behavior. With big
data, the what becomes more important than the why. The primary goal is
to allow organizations to prepare for events, minimize any negative impact, or
seize any opportunity they may present - in this case, change staffing patterns
on Mondays. Oxford scholar Viktor Mayer-Schonberger and Economist data
editor Kenneth Cukier put it this way: Society will need to shed some of its
obsession for causality in exchange for simple correlations: not knowing why
but only what. This overturns centuries of established practices and challenges

our most basic understanding of how to make decisions and comprehend

reality.
The change has profound implications for all organizations. It creates a
practical, proactive, forward-thinking mindset.
Not surprisingly, it is business that is leading the charge. UPS is an example.
The delivery giant wanted to lower the number of truck breakdowns. The
company knew which parts were most likely to give out, so it has placed heat
or vibration sensors on those parts. When the sensors detect a certain
measure, the part is replaced before it fails in the shop, not on the road. The
data doesnt tell UPS why the parts are failing, just that they are. But it does
achieve the goal of minimizing delayed deliveries and idled drivers. With the
immediate mission accomplished, the company can explore the reasons for the
breakdowns.
Vehicle parts are just the beginning. Big data allows businesses to measure
just about everything: sales, product performance and reviews, production
schedules and snafus, customer preferences and habits, stakeholders profiles,
trends, and employee work patterns. Leaders and managers can now
understand their companies in real time with an acuteness that was impossible
even two years ago.
Many startups, of course, embrace big data from day one. For this reason, they
hold important lessons for all organizations.
Consider shoes. Everyone wears them; it is a multi-billion-dollar market. Shoe
stores have been a staple of the retail landscape for hundreds of years. Their
leaders know their inventory, what sells, and what doesnt. They may run
sales and promotions and have a customer loyalty program, but thats about as
far as it goes with knowing and understanding their customers. And for
hundreds of years, it was enough. Then along came Zappos.com, which proved
that consumers would buy shoes online. Zappos tracks not only what
customers buy, but what else they browse, how they respond to promotions,
whether or not they read reviews, and how they navigate the site; it also
establishes a dialogue with customers by encouraging feedback and
suggestions. Using this information, the company categorizes its customers
into cohorts, demographic groups that may share nothing more than a love of
red shoes - and that can be targeted with crimson precision.
The algorithms designed by Zappos data technologists predict exactly how to
target these groups. If a man buys only athletic shoes, the company wont try
and sell him dress shoes but it will certainly alert him to the newest Nikes. If
a woman purchases only flats, it keeps her up to date on the latest styles, but

it leaves out the stilettos.

The beauty of these algorithms is that they self-improve: Every time a
customer makes a purchase (or ignores a promotion) the amount of predictive
data Zappos harvests grows. And why stop at shoes? Zappos has moved into
clothing and accessories. After all, it has a detailed profile of its customers;
why not sell them skirts and shirts and hats and accessories that match their
shoes and lifestyles? The algorithms help Zappos create, in effect, a personal
shopper for each customer.

Data Overrules Instincts

As the acceptance of the big-datas efficacy spreads, leaders have to adjust,
and some of the adjustments may be painful. Traditional thinking about
decision-making and the value of experience must be recalibrated. Leaders
have to be open to having their instincts overruled by data. This can be
difficult for leaders prized for their ability to make decisions that ultimately
rely on gut instinct. This means letting go of some of their authority, which
can be difficult. But to change an organizations decision-making culture,
nothing is more important than starting at the top.
That said, big data is not a substitute for leadership - it is a tool for leaders.
Companies still need vision and inspiration and motivation. What big data
supplies are relevant facts; it is still a leaders job to turn those facts into
actions.
In order to exploit big data effectively, leaders must receive the data most
pertinent to their challenges. The leaders of a large hardware or homeimprovement chain, for example, may ask for a predictive analysis of weather
patterns for the next month - specifically, what are the odds of extreme
storms, such as hurricanes or tornadoes hitting various regions of the country?
The big-data analysts go to work, deliver the science-based prediction, and the
company then ships extra quantities of emergency supplies - plywood,
flashlights, generators, tape, and so on - to the regions most likely to need
them. While this crucial supply-chain information was delivered by the data
scientists, the process was initiated by the leader.
Heres another process where big data is having a profound impact: hiring and
recruiting. In the past, the resume and personal interview were the most
important factors in a hiring decision. Big data is changing that. In the words
of Dan Shapero, LinkedIns vice president of talent solutions and insights,
Recruiting has always been an art, but its becoming a science.
Big data opens up whole new streams of verifiable information about potential

hires that dwarfs those provided by resumes and interviews. But perhaps big
datas greatest human-resources value comes as a predictive tool. By
analyzing information on established high performers, it allows organizations
to determine the attributes that have proven themselves valuable in the
workplace. It can then evaluate potential hires for those characteristics. In
fact, Google has an entire division devoted to people analytics. Thus, an
impressive resume or winning interview is trumped by qualities such as
flexibility, perseverance, social skills, a positive attitude, and emotional
intelligence.
In the days before big data, recruiting was often contracted out to headhunting firms, which would seek out candidates who werent actively looking to
change jobs. With big-data tools, many human-resources departments can
dispense with head hunters and turn to networking and data-aggregating sites
such as LinkedIn and TalentBin. As Jennifer Hasche, an Intuit recruiter, puts
it: With TalentBins search engine, it seems nobody is out of reach. We found
it to be a massive timesaver and critical tool in our discovery of top talent.
The new approach has also benefitted the networking sites: LinkedIns paid
recruiting services accounts for some 60 percent of its annual revenues.
In another sign of the democracy of data, a degree from a prestigious college
carries far less weight these days. Big data promotes meritocracy. Guy
Halfteck, who founded Knack, a Silicon Valley company that uses big data and
games to uncover the qualities of a stellar hire, puts it this way: You might
get into a school because your father got into the school. That is not indicative
and insightful about who you are as a person and your potential.
Many companies these days expect employees to help define their own jobs, to
be self-starters and take initiative. Sometimes straight-A students lack this
trait. They have done what they were asked to do and done it well, but this is
no guarantee they will be innovative thinkers who seek out new challenges.
Googles vice president for people analytics, Prasad Setty, has stated that high
SAT scores and GPAs are unreliable predictors of success at the company and
are no longer used as important hiring criteria. Josh Bersin, founder of
Bersin by Deloitte, a data-driven human-resources consultancy, has this to say
about a conservative insurance company that had a policy of only hiring MBAs
from top schools: They looked at the performance of their best salespeople,
and they found it had nothing to do with where they went to school and
nothing to do with their grades.
In a managerial job or one that requires lots of collaboration, big data has
shown that the key quality is emotional intelligence, the ability to read
peoples unspoken signals and respond appropriately. Knacks Halfteck says,
Whether youre an innovator, a physician, a teacher, a retailer, or a

salesperson, your social abilities, being able to intelligently manage the social
landscape, intelligently respond to other people, read the social situation, and
reason with social savviness - this turns out to differentiate between people
who do better and people who dont do as well.
Interviews simply dont yield a lot of actionable data. A lot of stellar talents
lack social skills and may not make a particularly good impression in an
interview. Steve Jobs is Exhibit A: He would have failed many job interviews.
Data doesnt have those biases. It uncovers the accomplishments and potential
of the Steven Jobses of the world and relays the message to hire this talent.
Sophisticated algorithms can spot hidden potential. By harvesting Twitter
posts, social connections, blog comments, and other digital footprints, a full
professional picture emerges.
Valuable human-resources data also lies in game playing. The way people play
a game reveals a great deal about how they engage, react to stimuli,
collaborate, and respond to stress. Knack has designed games that undercover
behaviors and traits that are markers for high performers. Halfteck labels it
behavioral gig data, saying, We measure everything from creative abilities
to emotional and social intelligence, to how you think and make decisions, how
you learn new information, how curious you are about the world. Knack works
with client companies to design customized games. When the companys top
performers play the games, Knack uses their skills to create a profile for
potential hires.
The human-resources power of big data is being recognized by many of the
largest companies in the world. IBM paid $1.3 billion to acquire Kenexa, a
company that gathers data on and assesses approximately 40 million people a
year. Kenexas chief marketing officer Tim Geisert explains the companys
value: What were bringing to the table is what we call . . . human-insight
analytics. What are the key data points that make people good at what they do
in their jobs? Whats supercharged is the technology that can gather that data
and the platforms which we run that data on, using it for analytics, insights,
and predictability.
According to Meghan Biro, a human-resources consultant and founder of
TalentCulture, Probably the single most important quality in an applicant is
learning agility. This is the ability to dive into a new situation or project and
quickly grasp whats needed, and then . . . get to work. Big data makes finding
these kinds of people infinitely easier. The right filters can help humanresources managers measure skills, talent, and propensity for learning. Its
making the job not only easier but far more productive. What used to take
hours or even days can sometimes be accomplished in minutes. Equally

valuable is big datas ability to winnow the pool, to look for unfavorable
markers that disqualify a candidate, once again saving time, effort, and
money.
It is commonly accepted that good salespeople have outgoing, engaging
personalities. Many do, but big data has shown that charm alone is not
enough. Really great salespeople have something called emotional courage,
which is the ability to persevere in the face of repeated rejection. Big data
allows companies to screen for this trait and not be blinded by an applicants
charisma.
This kind of revelation is exactly where big data proves its worth. Josh Bersins
work with an oil company uncovered a fascinating insight. The company was
having trouble finding engineers and production workers able to thrive in
difficult living conditions, often out in the desert, away from home and family
for long stretches of time. Turnover was astronomical. A data-driven analysis
of those who performed well showed that a multicultural background was a
clear marker for success. These people were comfortable working with a
diverse group of colleagues, and were more willing to adjust to adverse living
and working conditions.
Meghan Biro sums up big datas effect on hiring: We have a whole new
toolkit. Its intriguing to see what big data reveals. I do advise anyone
interested in being hired, even if theyre not actively looking, to keep up an
online presence. Thats how companies find you. All in all, its an exciting time
to be in HR because big data enables great hiring and great hiring enriches
everyones lives and, of course, leads to new levels of performance.

Big Data, Big Possibilities

Big data is changing society and business in profound ways. It is here to stay
and its power will only increase. We have seen that organizations are using it
to increase performance, boost profits, stop crime, and even save lives. Big
data is changing our relationship to knowledge, taking it from a tool of
understanding to a tool of action.

Create, Add, Revamp, Eliminate

To harness the power of big data, follow this framework:
First, create relationships with exponentially expanding networks and the data their
members generate.
Second, add big datas insights to your internal and external reporting system.
Third, revamp your processes, thinking more about what big data can do for you
(Apple has done it with its developer network).

Finally, eliminate your fear of big data because its here to stay.
Lets take each in turn:
1. Create: Identify the sources of big data that are most likely to generate value. This
includes social interactions that enable your organization to see and hear from
customers, employees, partners, and investors about their experiences; provide
feedback on products, services, and strategy; and set the foundation for cocreating new products or services.
2. Add: Include big data in your existing financial and operating reports so your entire
organization - from board to front line employees - can better understand what
customers, employees, and partners are saying, whether to each other or to the
organization. This will support improvements to existing business practices across
the enterprise, including hiring, training, customer service, up- and cross-selling,
marketing, the development of new sales channels, and new product development.
3. Revamp: Reengineer existing business processes and technology investments so
that you can capture the big data generated from todays new sources - including
the interaction between things, not just people.
4. Eliminate: Work to remove existing biases about the importance of and value
created by unstructured data. Encourage all members of the organization accept
and gain expertise in big data so that everyone is working from the complete and
current set of information. It will support good decision-making.

Other Great Reads

Published by New Word City LLC, 2014

www.NewWordCity.com
Barry Libert
All rights reserved. No part of this book may be reproduced, in any form or by
any means, without permission in writing from the publisher.
ISBN 978-1-61230-738-1