Академический Документы
Профессиональный Документы
Культура Документы
1 May 2013
The best way to start the conversation about Big Data is to define it. Its name is perhaps confusing and not quite apt, since it implies that existing data is small, or that we simply have a lot more data. The reality is, the term Big Data is applied to information that cannot be analyzed with traditional tools or processes.
Big Data has three fundamental characteristics: it involves managing a large volume of information, processing the data quickly or in real time, and integrating a large variety of information sources that may be able to draw conclusions from data connections that are not apparent from the start.
A recent study discovered that a large amount of todays business leaders are aware that they do not have access to all of the insights that would help them improve decision-making in their companies. The companies, in turn, are facing increasing challenges in a time in which data is being generated like never before and in which they have the capacity to store this information. This represents a great opportunity for these companies to equip themselves with realtime knowledge that can truly help them understand and adapt to individuals and their needs, and make decisions accordingly. It may seem paradoxical, but while it is possible for todays businesses to access information that can potentially be decisive for their core strategies, their capacity to process, filter and analyze increasing quantities of information is decreasing. The data which could represent a truly golden opportunity just continues to pile up. This is where Big Data comes in as a key player for the business.
1 https://www.ibm.com/developerworks/mydeveloperworks/blogs/SusanVisser/entry/fashbook_understanding_big_data_ analytics_for_enterprise_class_hadoop_and_streaming_data?lang=en
DATA OVERLOAD
AVAILABLE, STORED INFORMATION WORLDWIDE EXABYTES
SOURCE: IDC
2,000 FORECAST 1,750 1,500 1,250 1,000 750 500 250 0 2005 2006 2007 2008 2009 2010 2011
INFORMATION CREATED
AVAILABLE STORAGE
GROWTH FORECASTS
LEADING COMPANY
GEOGRAPHY
RETAILERS
60% potential increase in the operating margin for the retail sector
HEALTH
180,000 Big Data experts will be needed over the next 5 years in the USA
INVESTMENT FINDS
2,470 venture capital fund investments in Big Data companies in the USA in 2011
INTERNET DATA
DATA GENERATION
The world is changing in leaps and bounds. We use more and more technological devices in our daily lives, and thus we are able to capture more things. It has been observed that when we can capture things, we tend to hold on to them. Thanks to technological progress, people and objects are increasingly interconnected 24 hours a day without any type of interruption. This interconnection is rapidly escalating, and the flow of data exchange that it inspires is growing without bounds. The reduction in the size and price of circuits, like those used in smartphones, watches, heart rate monitors, mp3 players, and tablets, etc., contributes to this growth. Thanks to the decreased cost of these circuits, we are now able to endow just about everything with intelligenceeven a floor cleaner like the Roombaand obtain answers from this intelligence in the form of data.
These types of devices are highly reliable, sufficiently enough to have been implemented in security systems for some time now. For example, a freight train has hundreds of sensors that monitor the climate conditions inside the wagon, the status of certain pieces of machinery, or shipments. These processors interpret in real time the data from sensors in parts that are prone to wear, like the bearings, in order to identify the components that are in need of repair before they fail and potentially cause a problem. The rails also have sensors.
This data implies a fundamental change in the way we analyze this data, since it no longer follows a traditional structure and therefore requires more sophisticated technologies and methodologies. The success of an organization will increasingly stem from and depend on its ability to draw conclusions regarding the diverse types of data available to it. Getting ahead of the competition requires, in the majority of cases, identifying a trend, a problem, or an opportunity microseconds before anybody else. Thats why organizations must be able to analyze this information if they want to gain insights and knowledge that will help them with their business. They must start by identifying the opportunities behind Big Data, as this paper seeks to illustrate.
10
11
12
However, other types of companies opt for approximations using cloud-based and open-source tools, like Hadoop, a popular open-source software framework that allows applications to work with large amounts of data and thousands of nodes. Hadoop was inspired by tools used by Google and by non-relational databases
necessary for storing and processing the enormous complexity of all types of data, which in many cases do not follow the logic of ACID (Atomicity, Consistency, Isolation and Durability) guarantees, typical of conventional databases. It seems that solutions of this type will be increasingly adopted in the future, although exciting questions about their implementation and uses remain unanswered.
13
It was precisely with the idea of increasing Big Datas reach that Google introduced BigQuery some time ago, an online service for processing large volumes of information. The service, however, is targeted towards professionals, and therefore it is not free of charge. With BigQuery, Google takes advantage of all its knowledge on processing large volumes of information and making it available to companies that are unable to purchase their own infrastructure, thus
offering them a cloud-based model that provides storage space as well as a data-mining service. Thanks to BigQuery, companies can make their first inroads into processing large volumes of information, although, logically, it may be necessary to hire a specialized service in order to receive more in-depth service or analysis. Even so, Googles initiative seems to be of interest, as it is a way to advertise Big Data around the world.
14
In any case, the utilities and applications that Big Data can provide are already within reach for many users, and in a way that allows them to recognize and understand the massive convergence of data. Any user may consult and use the tools that already exist on the Web. For example, a user may go to Google Maps, write an address, choose the satellite view, and see the traffic in the area that he/she wants to visit in real time, based on information that other users have sent to the network via an Android terminal. Google has also discovered that certain search terms are valid indicators of
the evolution of the flu, and the results are shown on Google Flu Trends.2 Approximate calculations of flu activity can thus be made for certain regions, which could be of use when it comes to taking preventive action. We can find other similar examples to the one just mentioned.
2 http://www.google.org/flutrends/
15
Another facet of Big Data that has a strong potential for further development involves citizen access to public data, which, until now, was only available for analysis by the public administrations. In 2009, the government of the United States was a pioneer by opening the doors to all of its information on the website data.gov. On data.gov, you can access a great deal of information that has been available to US residents for a while now. To date, the site has received more than 100 million visits, and local authorities and institutions have started to release their data to citizens, following President Obamas lead. Cities like San Francisco and New York, and the states of California, Utah and Michigan, among others, have launched their own websites based on the data.gov model. The same is taking place in countries like Canada, Australia and the United Kingdom, and with such wellknown institutions as the World Bank.
Another public-interest use for Big Data was developed by IBM.3 Using Smart Meters, IBM analyzed a neighborhoods power consumption with sensors that provided energy consumption data, with the goal of making that consumption more efficient. Based on this information, the company was able to determine inhabitants energy-usage patterns throughout the day, see how demand varied, and even change some of those patterns by implementing various strategies and client discounts.
3 http://www.ibm.com/smarterplanet/us/en/smart_grid/ideas/index.html
16
17
Even the Leicester Tigers rugby team has started using Big Data to help prevent injuries.4 Thanks to the increasing availability of public data, people have developed hundreds of applications that society can benefit from, for example, applications that allow you to see pollution levels by region, that help travelers find the fastest route to their destinations, and that inform new homeowners about the safety of their neighborhood. Never before has so much valuable, objective information been available to help people make the best decisions possible in their day-to-day lives. As opposed to the way things usually find popularity, Big Data is being propelled by the public sector, as it shows people its value and potential. The time has come for Big Data to expand into the private sector, and for marketing and customerrelations departments to take advantage of the opportunity to increase their profits and productivity, and to be able to adapt their business strategies to the new changes that are to come by using all the information available through Big Data.
4 http://alt1040.com/2012/04/big-data-reduccion-lesiones-rugby
18
The organizations that apply predictive analysis are 2.2 times more likely to beat their opponents
19
20
One of the biggest changes were seeing in the online advertising industry is an increased focus on data and analysis. Marketers are hungry for information about what their audiences do online and how theyre responding to ads. At the same time, its not always easy to navigate with massive amounts of data, so, in order to
be meaningful, that data needs to be combined with insights so marketers understand how to activate on the findings. Lauren Weinberg, VP, Strategic Insights and Research, Yahoo!
21
Large companies are aware of this and are increasingly dedicating departments and resources to data collection and application.
22
74% DEMOGRAPHIC DATA 64% CUSTOMER TRANSACTION DATA 60% USABILITY DATA FROM THE CUSTOMER 35% SOCIAL CONTENT CREATED BY CUSTOMERS AND TARGET 33% SOCIAL NETWORKS AND TIES BETWEEN CUSTOMERS AND TARGET 19% CUSTOMER CELLULAR PHONE/DATA DEVICES
TRADITIONAL DATA
DIGITAL DATA
23
24
Using the same technology with the correct platform and the appropriate tactics, we can achieve more ambitious objectives and provide very valuable information for brands, which can then use this information to enrich their customers experience. All we need are technical and human systems that are able to collect, standardize and mine the information. The implications for customer-service strategies are also significant. Big Data has recently gained relevance because companies are realizing what it can do for them, and that it is a goldmine for finding competitive advantages. When it is applied to the realm of business or marketing, the whole conversation about Big Data revolves around consumer trends, developing new products, and other insights into the market. When McKinsey wrote its report on Big Data5 last year, it identified five different ways in which Big Data can be used to create value, but only one of them mentioned customers, and it did so in order to discuss improvements in consumer segmentation. The Wall Street Journal describes several successful stories from different brands in its blog on Big Data,6 but focuses almost exclusively
on operational issues, process management, and other efficiencyimproving aspects. Efficiency is clearly a goal worth pursuing, but the use of Big Data is much more relevant in the realm of content or customer service. Now that consumers have seen what social media and mass personalization are capable of, they increasingly expect their favorite brands to provide these engagement opportunities. They are not merely passive users waiting to receive a message. Rather, they want to be active participants. Customer experience designers are aware of this. When a customer calls the customer service number, sends an email, or speaks with an employee in a store, they are starting a conversation. At that moment, the brand holds all of the customers attention, even if he or she is annoyed, which means that the brand has been given an important opportunity to define its relationship with its users. The user knows that the brand has gathered information about its customers for its own needs, and he in turn will ask why doesnt the brand do anything usefulfor the customer, not just the brandwith this data.
5 http://www.mckinsey.com/insights/mgi/research/technology_and_innovation/big_data_the_next_frontier_for_innovation 6 http://blogs.wsj.com/cio/category/big-data/
25
Listening to online conversations may help companies provide better services and integrate social channels with customerservice channels, thus hugely improving the user experience. Technically, this can be very difficult to achieve, but Amazon does it particularly well. Amazon has grown quite a lot over the years, but it has always stayed constant as a unique organization. Other organizations, however, have become larger by way of acquisitions, which make data synchronization an extremely technically complex task, with a high demand for resources and investments.
Even so, if the new pattern of relationships between brands and consumers is here to stay, companies must invest in capturing, processing and synchronizing data between channels and platforms, which is something unique to human interactions. If you talk to a friend, for example, and constantly ask him for information that you already have, he would understandably get annoyed. In the era of Big Data, the same rules apply to brands. The ones that follow the rules will win the trust and loyalty of their customers.
26
27
28
But, on the contrary, Seans experience with Amazon was positive and fluid. Amazon surprised Sean by using his data and purchasing-history profile to provide him with a fast and personal repair service, as well as personalized advice, based on his customer history profile. The fact is that Amazon had been collecting information on Sean for years, not just his different addresses and payment information. They created an identity of Sean as a person and they used it to build a two-way relationship with him.
With what CRM is traditionally able to offer, combined with social data, that is processed extremely quickly, and used to obtain massive knowledge of all of the customers as a whole, Big Data becomes truly powerful.
29
30
89%
11%
OF SMARTPHONE USERS HAVE THEIR PHONE AS A CONSTANT COMPANIONS THROUGHOUT THE DAY
DO NOT
31
General Electric
General Electric, in conjunction with the online medical community MedHelp, has launched four applications for the iPhone that track sleep, weight, pregnancy, and state of mind. As the users implement these tools in order to monitor their own development, MedHelp collects all of the data.
32
33
Nike Fuelband
Nike is another brand that charged headfirst into the year with a new product/ service that expands the possibilities of its successful ecosystem Nike+: the Nike FuelBand. This system allows users to track their daily activity and see their progress. Nike+ FuelBand has an LED screen where you can see the information gathered about the activities in your day, sensed and collected via wrist movements. The user sets a goal for how active he/she wants to be during the day and his/her movements are recorded and measured by the bracelet with 20 LED lights, which change from red to green as the user nears his/her goal. Theres a website where all of your NikeFuel points are accumulated, so you can compare your performance based on the time, day, week, month, or year using different types of graphics. You can also compare your data to that of your friends in the Nike+ community. This device can also be synchronized with the iPhone and the data can be viewed using a free application. Although similar devices like Fitbit and Jawbone UP have existed since 2009, Nike waited for the trend to go mainstream, in order to execute a major launch that would position the brand as the leader of its category and as the reference brand for this type of gadget in short, becoming the company that democratized the measurement of sports performance and well-being for all users. Ultimately, Nike has been a true game-changer, offering relevant services to its consumers thanks to data mining. If all of this information is analyzed on a large scale, the opportunities for the brand are infinite. Nike is becoming a company that isnt just focused on products, but on products and services. It used to be that when you bought a product, that was the end of the relationship. Its classic marketing. Great, you bought the product. See you in a year, when the next campaign comes along. That thinking has flipped on its head. Now, the purchase of any Nike product needs to be the beginning of the relationship we have with the consumer. Stefan Olander, VP Digital Sport
34
The system allows users to track their daily exercise and see their progress. As the claim states, Make it count.
35
Trulia
The real estate website Trulia (New York housing sales and rentals), has launched an interactive commute map that allows users to view their route to work in a dynamic format. This is especially useful for those who plan to move to a new neighborhood, since they can easily see on the heat map how long it will take to get to work or to other places. When users specify a starting point, the duration of the trip will immediately be shown in real time on the heat map. Using the slider, users can see the sites they can reach quickly, as well as those that will take longer. Trulia helps its potential customers make better decisions, and positions their site as a more useful space, thus generating traffic and sales. The commute map is a useful tool for communicating a large quantity of information in an easy-to-understand format. It uses the traffic information and the OpenStreetMap data to create a visual image with a range of colors that represent the different travel times.
36
37
The Eatery
The Eatery is an application developed by Massive Help (USA), which lets users take pictures of their food and rate other users food photos based on their perception of whether or not what they see is healthy. Since its launch last year, this platform has acquired a vast quantity of data from hundreds of thousands of users. Massive Health has used the photo ratings to analyze how our friends influence what we eat. If you are obese and you have a partner, there is a 34.5% chance that he or she is also predisposed to obesity. This percentage increases to 57% when its your friends who have weight issues. With this information, Massive Health hopes to help people improve their food habits. Theyve found out that people who eat healthier food tend to stick together, and therefore the application seeks to facilitate contact between people with healthy and not-so-healthy habits in order to promote better attention to food choices.
38
39
Wal-Mart
Walmart gained Big Data experience with its purchase of Kosmix in April of 2011, with which it created WalmartLabs. Kosmixs expertise was in analyzing enormous sequences of data from social networks in order to help companies understand what consumers are saying about products and brands. Wal-Mart is also trying to use social network trends to influence the marketing and inventory decisions on their website and in their stores. Their technology, called Social Genome, uses the aforementioned Hadoop and other open-source tools to capture and analyze in real time the flow of comments made on Facebook, Twitter, and other social networks that reveal what people think about certain products, brands, places, and events. Walmart has even developed its own technology to rapidly analyze the data. WalmartLabs first innovation with this technology was Shopycat, launched in December of 2011. Shopycat is an application that recommends gifts to friends and family members based on your tastes and likes on Facebook. Its objective is to turn insights about the consumer, extracted from social networks, into practical shopping advice. Shopycat is capable of interpreting unstructured data like the feelings behind a Facebook status update, which are difficult for traditional databases to analyze. Shopycat also identifies which items are better gifts than others, using an algorithm that analyzes multiple aspects such as how recently the product was launched, its uniqueness, and the users purchasing behavior on Walmart.com. Walmart is taking an unconventional approach to offering gift recommendations. If the company does not find the best product in line with a recommendation online or in a local store, it will send the user to another retailer who does have that product.
40
41
Privacy
As the relationship between marketing and Big Data evolves, brands need to examine how to obtain information while not only protecting the privacy of their customers or users, but also demonstrating that they are making the effort to do so. In a world where we increasingly capture more and more information, and where information comes from the daily use of all types of devices, we have to be ever more responsible about the use of data. Whats more, consumers and users are also becoming more aware. They are informed about how companies use information and they demand suitable data protection policies that are perhaps not always compatible with maximizing marketing opportunities, even when those opportunities would benefit the users. In this context, the public response is unpredictable and variable. BlackBerry has been severely criticized in public for leaking certain data, and Twitter has been praised for protecting it. Google became the center of attention when The Wall Street Journal revealed that the US government had obtained a secret court order to force Google and the Internet service provider Sonic.net to give up all of the email account information of the famous hacker and WikiLeaks volunteer, Jacob
42
Appelbaum, who had not been accused of a single crime. The Wall Street Journal disclosed how the ISP secretly fought to avoid providing the information until it was forced to do so. Google, in turn, did not comment on the WSJ exclusive, thus creating discontent amongst online users. These types of cases generate a great deal of controversy. On the other hand, we often lose sight of the idea that certain data is personal and must be protected. For example, the Ritz-Carlton chain has taken big steps forward in the hotel industry, improving its hospitality by collecting a lot of data from its customers, with the sole goal of improving customer service.
For now, this seems valid and no one has complained. That said, it can also be counterproductive for a service to become too good as a result of data analysis: the customer who notices how proposals or content are always personalized may feel watched or frightened about the companys data-gathering methods. Balance appears to lie in a combination of strict data-protection policies that allow information to be used to improve services, but are always transparent with regard to what information is being used and why.
43
44
45
Netflix
Netflix, a company that streams television series and movies online, recently purchased the license for a television series, surpassing the bid proposed by the cable TV channels HBO and AMC, in order to guarantee their rights to the series House of Cards. This is the first time that Netflix has invested in original content. Netflix, since its founding, has distributed television content using a subscription model (physical shipment of DVDs through the mail), and now has broadened its business to provide on-demand video streaming. The content is transmitted online to consoles like the Xbox 360, Nintendo Wii, the PS3, and other devices like Blu-ray players and Smart TVs connected to the Internet, in addition to smartphones, tablets and computers. The series the company purchased is a remake of a BBC political thriller. It will be directed by David Fincher and will star Kevin Spacey. What Netflix did was collect large quantities of data from all of its subscribers in order to determine if they would want to watch this combination of political thriller, director, and actors. The answer was yes. And not just that, but the same data that helped Netflix decide which series to purchase will now help the company promote it effectively among their subscribers through their recommendation system, which suggests 75% of what users end up watching, according to the company. To understand the context, it helps to keep in mind that in the month of June, Netflix streamed more than one billion hours of online video to its subscribers. Well-managed data collected on its viewers can help the company find a new series in the future or movies that will be in line with what Netflix customers want to watch.
46
47
48
49
6 Key Points
BIG DATA IS ALSO FOR MARKETING:
The term Big Data refers to infrastructures and systems so broad and powerful that they can seem unrelated to marketing. But Big Data in fact represents a real opportunity to develop strategies, campaigns, customer experience models, and CRM based on access to and use of never-before-seen levels of data, even when it doesnt quite reach the volume truly required.
50
51
An emerging issue
Big Data is starting to enter inaccessible realms. It is always possible to collect more and more pieces of data and ask ourselves ever more complex questions. The European Organization for Nuclear Researchs Large Hadron Collider atom smasher generates so much data that most of it is ignored and deleted, in the confidence that nothing of importance is being discardedunlike, for example, in the healthcare world, where clinical histories, or all of the medical images, such as X-rays and MRIs, could be important. There will always be a doctor who wants to cross-reference data from, for example, all of the X-rays of tumor patients still alive after five years, who have families and no alcohol drinking in their background. Or perhaps we might want to analyze power-consumption data from all power meters to the minute, in order to make appropriate consumption decisions. Why not have meters in every outlet and in every appliance to customize electricity charges as much as possible? Or perhaps someone might want to collect all of the tweets that mention a specific subject and correlate them with news items; or follow the movement of every vehicle on the road; or study the influence of rumors propagated on social media about stock exchanges and financial products, or about recently premiered movies or new products. And what about a system that links a buyers personal data from his NFC-enabled payment device (NFC is soon to be implemented in cellular phones) with every item purchased in the supermarket, through the NFC
52
target incorporated into each product unit? This will soon revolutionize the way we pay for our groceries. The list of questions that industries, sectors and companies can ask themselves is never ending. So is the list of answers, although the majority of them start from a shared premise: concern for the consumer and a push to unveil all the hidden potential of this knowledge. In order to apply marketing strategies based on Big Data principles, first we must invest in infrastructure, and systems, and resources with which to analyze all the data in the spirit of a search that does not rule out deep connections between data and events that on the surface seem completely unrelated. And of course, we must have the will and the resources to
activate that knowledge in specific strategies and actions, whether it be the launch of a new product or the creation of a cell-phone application that distributes new brand content in a new or more effective way. The reward, in the form of value added for the consumers and growth and loyalty for the brands, is waiting for you. It is Big Data.
53
Sources
El Blog de Enrique Dans ZDNet ALT1040 Robert Kirkpatrick: How The United Nations ls Using Social Data To Spot Disasters The Wall Street Journal TED talk: Kevin Slavin: How algorithms shape our world
Public Technology
Fast Company
TheNextWeb
VentureBeat
The Guardian
lnformation Management
Forbes
Business lnsider
54
55
WRITTEN BY JUAN MANUEL RAMREZ DANIEL CAMPRUB EDITED BY GRACE CHANG DESIGNED BY KATHLEEN HANNA
56