Analytics for CRM

Database, Data warehousing, OLAP and Data

Mining/ Text Mining/ Big data
 LEARNING OBJECTIVES

• How does Data Analytics work

• How do I use these technologies to improve relations with customers
• What are Customer Performance Indicators
• Case: Big Basket

2
 The Question is
Can you do it for all Customers?

3
 Fatal Mistakes in Marketing
4

 “all customers are equal”

 “customer at any cost”
 “selling products at cost or below”
 “handing free goods and services to new customers who might not
return”
 Fatal Mistakes in Marketing
5

 More focus on website traffic than profitability………

 For home delivery service the average delivery costs often exceed the cost of
products being delivered
 Per delivery losses force these companies to expand product offerings and put
purchase minimums on their deliveries to protect their costs
 What about the loyal customer who spends on an average Rs100 per week and
wants that Dove bar without surcharge?
 10 percent of a company’s customers account for 90 percent of the profits
 WHICH 10 PERCENT?
 Such companies have been investing more in basic infrastructures-warehousing
and distribution and less in analytical capabilities!!
 Fatal Mistakes in Marketing
6

 Without a clear view of the customers a company wants to attract

and keep, the best logistics and best procurement in the world won’t
matter.
 What’s best for You?
7

 Multi channel CRM: keeping the balance

 Web is more cost effective. But what should be the combination of channels will
come from analytics
 Operation CRM begins with capturing the customer
8
database
 • Make recommendations for products that a customer is most
OPERATIONAL likely to respond positively which are based on the customer
CRM profile
• Make split second advertising decisions for online customers to
avoid abandoned shopping carts

9
 Operational CRM
• Front office CRM
• Involves touch points where direct
customer contact occurs
• The focus is on smoothly running
the operations of the organization
• Where you look at
– individual transactions,
– one sale,
– one complaint,
– one prospect.

10
WHAT ARE THE DATA MANAGEMENT TECHNOLOGIES AND
TECHNIQUES THAT SUPPORT THIS?

11
Database
• Store data generated by business apps, sensors, operations and
transaction processing systems (TPS) For customers/ customer facing
employees.
• Keeps track of
– each transaction
– current data (one month or a few months)
• The purpose of each of these transactions is to help in the day to day
operations of the company.

12
 • The operational CRM system is used all the time and by
several people at the same time. But each time it is used,
OPERATIONAL not too much data is accessed from the data base.
CRM – A sales man may use it to check the preferences of one customer.
– Check the status of one order
– Check the status of AMC of a customer
• These transactions do not hog computing resources –the
database, hard disk, CPU, RAM etc.

13
But when a query for analytical CRM is run, things are
different.
– While an operational query requires access to one record (or a
few records), in the database, analytical query needs to access
the entire database.
– This slows down the system as the system works on a time
sharing basis.

14
• An analytical query may be
– Which categories of prospects responded to a mailer in the past.
• Therefore, there are two sets of databases for operational and analytical
CRM (a database and a data warehouse).

15
16

 Customers keep combining channels & expect the company to know the past
history of interactions, purchases whenever they call.
 In marketing, company wants to target more effective campaigns by tracking past
patterns of purchase & behavior
 Information about specific calls is loaded from the call center system into a
database system used by an analyst in the customer service department to
evaluate trouble ticket resolutions
 Unhappy scenarios-Poor Decisions on Customers
17

 Marketing approaches existing

customers and some of them are
unhappy 
 CSR responds to a complaint
without recognizing that it’s a
high value customer-failure to
respond correctly means you
loose a high value customer
 Multiple overlapping tools within
a single company involves
duplicate effort in technology
acquisition, systems integration,
installation, deployment and
18

The lack of a single interface

for customer data forces the
CSR to access multiple
systems to aggregate info
about the customer-while the
customer waits on the line
19

 Different applications have widely different means of tracking and reporting on

customers, leaving companies with islands of unrelated data. The result is a
fragmented customer view that impairs the company’s ability to determine key
metrics like overall customer profitability and lifetime value
-Forrester Research
 Integratedness is important
20

 Of past interactions,
 As also
 Geography/ regionality,
Psychographics (personality/
lifestyle/ hobbies)
 Preferred sales channel,
Demographics (age, race,
sex),
 Number of products,
 Lifetime value,
 Life stage,
 Privacy preferences, Billing systems, Order and provisioning systems
 etc ERP system, POS system, Marketing
Call center
21 The Case for Integrated Database
 What more can you Analyze?

• Buying attitudes of customers

• Customer satisfaction/ retention/increased profitability

22
23 CRM and Datawarehouse
• From Database to Data Warehouse
– PROCESS OF BRINGING TOGETHER ALL THE HISTORICAL DATA OF AN
ORGANIZATION FROM MULTIPLE SOURCES AND STORING IT IN ONE PLACE.
– A collection of data that supports management decision making/ Analytical
CRM
– Historical data; updated from database
– Summaries of past data
– Do away with inconsistencies
– Updated from the database at a pre-determined frequency (depending on
business and level of activity- Airtel, Nestle, Amazon, BPCL etc)
– DW are composed of snapshots taken at various times
– Non-volatile: no deletion of data in the DW

24
 Data Warehousing

• An OLTP application (ERP/Operational CRM) would access a database which

typically has current data.
• An analytical application would access a data warehouse which has large
quantities of historical data
• Such a DW is unified across
• Locations
• Product lines
• Touch points
• Multiple applications

25
 Data Warehousing
– A data warehouse organizes the data in multiple ways-by
subject, functional area, vendor, customers and product
– Concept of granularity in DW:
• Therefore, while the operational CRM would store every
transaction in detail, the data warehouse which is a part of
analytical system would only store summaries.
• This makes the DW smaller and cuts down the time for
answering queries. But it also restricts the queries you can ask.*
– Companies also use two levels of storage
• Summary data on expensive hard disk drives
• Last level of historical detail on cheaper media such as
magnetic tape.

26
 Data Warehousing
– IT scrambles to provide enterprise data to CRM application
without understanding which data will support the CRM
strategy
– The business has to provide the inputs!
– If DW is built using disparate CRM systems there is more
work/ expense than if CRM was built around the DW the
first time
– CRM is not mandatory for a DW; but
– DW is mandatory for CRM!!

27
Phases in Building a Data Warehouse: ETL and CDC
• ETL refers to three procedures – Extract, Transform, and
Load – used in moving data from databases to a data
warehouse.
– Extraction Phase
• Builders create the files from transactional db and save on server
– Cleansing/ Transform Phase
• Data is made consistent, standard formats
– Loading Phase
• Builders transfer files to data warehouse database
• CDC, the acronym for Change Data Capture, refers to
processes which capture the changes made at data sources
and then apply those changes throughout enterprise data
stores to keep data synchronized.
• CDC minimizes the resources required for ETL processes by
only dealing with data changes.

28
Real time DWH and Analytics: Active Data warehouse (ADW)
• An ADW provides real-time data warehousing and analytics, not for executive strategic
decision making, but rather to support operations. Made possible by massive increase
in computing power, processing speed and memory
– Interacting with a customer to provide superior customer service,
– responding to business events in near real time,
– or sharing up-to-date status data among merchants, vendors, customers, and associates.
• Example: Banks calculate a profitability score to determine the level of service to
provide a customer. Low score customers are pushed to IVRS, high score to agents
• Trivago calculates the best deals for you on a daily basis based on your search history,
destinations of interest etc. if you search for flights and hotels in London, making a
personalized offer based on the prevailing rates across travel sites
• Monitoring and regulating the temperature and climate conditions of perishable foods
as they are transported from farm to supermarket, resulting in better quality and yield
(less spoilage.)

29
Phases in Building a Data Warehouse (Cont.)

30
 Data Mart
Data marts are lower-cost, scaled-down versions that can be implemented in a much
shorter time, for example, in less than 90 days.
Data marts serve a specific department or function, such as finance, marketing, or
operations.
Since they store smaller amounts of data, they are faster, and easier to use and
navigate.

31
Uses of Data Warehousing

32
33 Analytical CRM
 Operational & Analytical CRM
• Analytical CRM
– Back office CRM/ aggregate analysis/ strategic CRM
– Understand and analyze the activities that occurred
in front office
– When the management tries and analyses large
chunks of historical data, and comes to the
conclusion as to
• Which customers are likely to stick with the company
• Which customers are likely to leave
• Which ones should leave
• Which product has maximum complaints
• Which customers grumble but will stick by you
– Involves Technology to analyze the data and
suggest refinements in front office/ business
actions for better customer loyalty/ reduce the
gaps/ missing links in interaction. 34
 Analytical CRM
• Senior Management
• Take decisions which impact several customers, not just one.
• Decisions which impact the business as a whole
– Expansion (location, products, real estate )
– Marketing campaign, pricing schemes etc
• Such decisions are based on the preferences of a large number of customers. This
means historical data.
• Which customers..??? Those who bring in money
• Analytical CRM is analysis of large amounts of data/ facts collected as part of
operational CRM/ ERP etc.

35
Analytical CRM

36
 Some Examples
1. Reward customers with personalized discounts and perks for using lower cost
channels
2. Proactively offer products and services that fit a customers needs based on his
past purchases
3. Adjust customers marketing expense based on lifetime value scores
4. Analyze combinations of touch-points across channels to predict a customers
next likely purchase
5. Tailor commissions and incentive programs for sales partners
6. Prevent a customer from churning by offering incentives based on individual
preferences
7. Provide customers in the highest value tier with personal respresentatives who
understand their history and preferences
37
 Customer Relationship Management Systems

ANALYTICAL CRM DATA WAREHOUSE

FIGURE 9-10

Analytical CRM uses a customer data warehouse and tools to analyze customer data collected from the firm’s customer touch points and from
other sources.
39 Major Types of Data Analysis
OLAP
Data Mining
Click Stream Analysis
Personalization and Collaborative Filtering
40
Need for Data Quality: Data Errors Increase Costs Downstream
• Example: Although the average cost of processing an insurance claim may be only
Rs100, the average downstream costs due to data errors is Rs 3000.
• This increase includes costs from
– manually handling exceptions,
– from customer support calls initiated due to errors in claims,
– and from the reissuing of corrected documents originally processed incorrectly.
• This does not include the soft costs like lost revenues due to customer dissatisfaction.
• Sometimes the soft costs might exceed hard costs…!!
• Errors can be difficult, time-consuming, and expensive to correct, and the impacts of
errors can be unpredictable or serious.
• Poor data degrade the value of analytics.
• Data quality can be accomplished through cleaning up new data by
removing redundancies, filling in the blanks and missing fields and
organizing data into consistent formats.
41
On-line Analytical Processing (OLAP)
and Data Cubes
On-line Analytical Processing (OLAP)
 OLAP software enables multidimensional
views and analysis of business data.
 Designed for live and ad-hoc data access
and analysis.
 At the core of the OLAP concept, is an OLAP
Cube. The OLAP cube is a data structure
optimized for very quick data analysis.
 Pick up 2-, 3- or n- factors from the DW and
convert it into a matrix. For example, customer,
time, geography, branch etc. Such a matrix is
called a data cube.
Multidimensional View of Database
(Data Cube)

43
 OLAP and Data Cube

 Data cube stores facts and you can view

these facts through multiple dimensions.
 All the data is picked from DW.
 Facts are stored in fact tables (data that
is business numbers), and have a foreign
key relationship with a number
of dimension tables (relatively constant
data)
 Example : Location Dimension ,
Product Dimension. ... Example :
Sales Fact, Order Fact.
 Each Dimension represents an entity
(with attributes)

44
OLAP Functionalities

The various functionalities

of OLAP are discussed
below:

 Drill down and drill up

Examples of Drill Down and Drill Up
 Slice and Dice
 Other functionality

Example of Slicing and Dicing Data

45
 Roll-Up

In this aggregation process, data is

location hierarchy moves up from
Get me the average city to the country.
balance for all customers
in US

46
 Drill-down

Focused on detailed data

Moving down the concept hierarchy

Increasing a dimension

Quarter Q1 is drilled down to months

January, February, and March.
Corresponding sales are also registers.
In this example, dimension months are
added.

47
 Slice

Here, one dimension is

selected, and a new sub-cube
is created.

Dimension Time is Sliced with

Q1 as the filter.
A new cube is created
altogether.

48
 Dice
This operation is similar to a slice. The
difference in dice is you select 2 or more
dimensions that result in the creation of a sub-
cube.

49
 Advantage-OLAP (On-line Analytical Processing)
50

 Typically OLAP is used for marketing, budgeting, forecasting and similar

applications.
 Fast analysis of shared multi-dimensional databases such as Sales& marketing
analysis
Financial reporting & consolidation, Budgeting & planning, Product profitability and
pricing analysis
 OLAP provides the building blocks for business modeling tools, Data mining tools,
performance reporting tools.
 For example, it can be used to find out about the sales of this year in Dabur
compared to last year? What is the prediction on the sales in the next quarter?
What can be said about the trend by looking at the percentage change?
 Online Analytical Processing-Disadvantage
51

 OLAP requires organizing data into a star or snowflake schema.

These schemas are complicated to implement and administer
 You cannot have large number of dimensions in a single OLAP cube

 Transactional data cannot be accessed with OLAP system.

 Any modification in an OLAP cube needs a full update of the cube.

This is a time-consuming process

Examples of OLAP tools: Dundas BI, Sisense, IBM Cognos Analytics,
InetSoft, SAP Business Intelligence, Halo.
Beyond OLAP
 Which customers are likely to leave-predict customer
churn

May be to run a focused marketing campaign for these
customers

53
 See correlations/ patterns between customers who have churned and may be
average balance during the month, number of transactions during the month
etc
 What you need is the combination of factors (known and unknown) which
affected the customer churn…..PATTERN
 With OLAP, you have to personally look for patterns..
 what we need is a tool that can suggest patterns…???
 So that we can apply this relationship to other customers……….

54
Data Mining
 E R N S
PATT
OWN
UNKN

56
 Data Mining
57

 Helps data analysts look for information they don’t yet know to look
for
 Software generates multiple options through complex correlations
 Often involving no hypothesis
 Customer’s next likely purchase
 Optimal store layout
 Most favorable release date for a movie in preproduction
 Directed and undirected data mining (where we do not have a goal as such)
Data Mining
 Discover knowledge that you did not know existed in the
databases.
 It combines many methods from artificial intelligence, statistics
and database management.
 While OLAP relies on data that has been summarized according to
particular dimensions
 Data mining refers to the analysis of large quantities of data stored in data
warehouses and to extract various trends/ rules or patterns from
detailed data.
 If OLAP would analyze the customer segments likely to churn, data
mining would examine individual customers touching upon each of the
millions of records in the database.

58
 OLAP vs. Data Mining
59

 While OLAP requires the analyst to have a query or hypothesis in mind

 Data mining can generate information to show unknown patterns and relationships
without the analyst knowing about them

 For example, During Covid-19 time what has been the purchase of home office
workers.
 With OLAP, an analyst would have to guess the products (printer, toner, power supplies
etc) and then run the query to identify customers
 Data mining can identify clusters of customers who buy similar products and can
recognize out of category purchases such as coffee, beer and waste baskets
 OLAP vs. Data Mining
60

OLAP and data mining are used to solve different kinds of analytic problems:

– OLAP provides summary data and generates rich calculations. For example, OLAP answers
questions like
"How do sales of mutual funds in North India for this quarter compare with sales a year ago?
What can we predict for sales next quarter? What is the trend as measured by percent
change?”

– Data mining discovers hidden patterns in data. Data mining operates at a detail level instead of
a summary level. Data mining answers questions like
"Who is likely to buy a mutual fund in the next six months, and what are the
characteristics of these likely buyers?"
61
OLAP vs. Data Mining
OLAP and data mining can complement each other.

For example, OLAP might pinpoint problems with sales of mutual funds in
Haryana State.
Data mining could then be used to gain insight about the behaviour of individual
customers in Haryana State.
Finally, after data mining predicts something like a 5% increase in sales, OLAP
can be used to track the net income.
• Or, Data Mining might be used to identify the most important attributes
concerning sales of mutual funds, and those attributes could be used to design
the data model in OLAP.
 BI Technologies
62

 Both data mining and OLAP are two of the common Business Intelligence(BI)
technologies.
 Business intelligence refers to computer-based methods for identifying and
extracting useful information from business data.
 Data Mining and Online Analysis (Cont.)
63
 Important!
64

 For data mining, the use of correct algorithms and their application to business
problems is important
 Data Mining throws up patterns
 The User / Analyst decides the “interestingness” / utility
 Business people generally don’t mine the data themselves-the analyst does it
for them and represents it graphically or as part of a visualization to be used
to manage relationships with customers
 Analytical CRM means understanding how the customer interacts and using this to
refine business processes to provide them with a better experience.
HOW DATA MINING WORKS?
TECHNIQUES
65
 Techniques to filter out the prospect data base
 Clustering-group of points with similar characteristics
 Probability of a new prospect falling into a specific cluster
 overlapping clusters

Interest
Income

66
Techniques to filter out the prospect data base
 Decision Trees (inverted)-at every node we have a question we ask of data
 When there are no further questions, the node becomes a leaf
 In real life, the branches are not clear cut and we work with probabilities
 Training set/ data set
 This model is then used to predict the future

67
68
TYPES OF DATA ANALYSIS

69
 Data Mining and Predictive Modeling
70

 Using past/ historical data to find patterns and build a model

to predict future behavior
 The next product a customer is likely to purchase based on historical
purchases by that customer and other customers who have purchased the
same product
 Most CRM vendors have predictive modeling into its toolset to provide lists of
customers most likely to respond to a given marketing campaign
 Training set/ test set
 Data Mining and Sequential Analysis
71

 Sequential Analysis identifies combinations of activities that happen

in a particular order
 Are customers following any particular order
 Is there a pattern in order cancellation or in slowdown of purchases
 Data Mining and Association Analysis
72

 Association Analysis detects groups of similar items or events

 Detects items or events that occur together
 Applied to market basket analysis to help understand products purchased
together
 Butter and bread
 Help a company to decide which products to advertise, or discount and
which customers to target for which product
 Data Mining and Click stream Analysis
73

 Click stream Analysis detects web visitors footprint around the site
 How the user arrived at the site
 How long he stayed
 What he did during the visit
 When he returned
 Click stream data is stored either as part of company’s data warehouse or in a dedicated click
stream data store called data web house.
 Companies want their websites to be as sticky as possible
 Analyze why a customer left prematurely
 Determine the value of abandoned shopping cart
 At what point during the shopping trip, the cart was actually abandoned (when the
customer saw the charges/ when he saw a survey!)
 Customers Click stream Analysis
74

 Click stream Journey of a customer can reveal interesting patterns

 Help improve the design of a website
 Look for product affinities
 Suggest cross sell and upsell strategies
 Enhance understanding of customer when combined with demographic,
psychographic and past behavior data.
 Which can be used to personalize customer communication!!
 Customers Click stream Analysis
75

 What if the analysis reveals

 The customer purchases only on sale, low margin products
 No cross-selling, no upselling , no loyalty
 DO NOTHING!
 Criticism of clickstream analysis…..are companies penalizing the loyal customers!
 Personalization and Collaborative Filtering
76

 Accurate product recommendations in real time

 Targeted promotions in real time
 Maintaining customer loyalty and driving purchases higher
 Personalization can involve customization of website, look, feel, features
 Personalization can involve localization of website, look, feel, features
 Two types
 Rules based Personalization

 Adaptive Personalization/ collaborative filtering: learn as it goes

77 Adaptive Personalization Case Studies
Bigbasket: Enhancing the Customer
Experience through Analytics
 Recommender Systems are of three types
79

1. Content based recommendations: similar to items previously purchased by a

customer, historical purchase data e.g. holiday destinations recommendation
2. Collaborative filtering: based on similar users/ profiles e.g. books/ movies
3. Knowledge based: based on demographic, psychographic and other data of
users and their relationship

 For Big Basket, Content based and Knowledge based

 For Amazon/ Flipkart, Collaborative filtering and Knowledge based
 What is the difference in recommender system requirements between
big basket and other e-commerce companies such as Amazon and
Flipkart
80

 Groceries
 Small value, daily need items
 High volume-up to 80 in one order
 Mostly through mobile
 Repeat purchases: vegetable, bakery, dairy products
 Smart Basket: Individual Customer
 Did you forget: Recommender based on the purchase history of an individual customer
(not of other customers having similar profiles)
 Amazon and Flipkart:
 non repeat purchases
 Would look at customers with similar interests, profiles, customer ratings
 Customer forgetfulness is not an issue here!!
 Predictive Analytics at Amazon
81

 Amazon.com Inc., the world’s largest e-commerce company uses high-end

analytics and algorithms in a number of areas such as
 recommending relevant products to users,
 showing users relevant search results,
 displaying ads to users that they are very likely to click on,
 predicting future demand for products,
 detecting spam reviews and detecting fraudulent orders
 Amazon earns at least 35% of its revenues through product recommendations
 Use of Association Rules at Amazon
82

 Amazon also leverages data about products that are purchased together to show
users “people who bought product X also bought product Y.
 The product search tries and shows users the most relevant search results
 Online advertising shows users ads they are most likely to click, based on past
click history.
 Click stream Analysis-Dynamic Pricing at Amazon
83

 Differentiate consumers and their price sensitivity

 Compare a shoppers desire for the product with perceived ability to
pay for the product
 Giving Amazon the capability to charge differently from different
customers!!
 Prices appear higher for regular customers??
 Consumer Privacy??
 Case Questions
84

 What are data challenges in developing product recommender

systems and how to handle them
 What are data challenges in developing product recommender
systems and how to handle them
85

 Data Quality: similar SKU’s placed in different categories such as Noodles are
tagged in both Noodle and Ready Spice Masala categories/ proper categorization
is a must.
 Customers who were new or had very few transactions with Big Basket may not
yield good results
 For Analytics purpose, the data in E-R table needs to be brought together in a
fact-dimension table
 Member: Member id
 Order: order number
 SKU
 Created on
 Challenges for Big Basket
86

 Separate model for every individual customer based on repeat purchase behavior
to suggest a smart basket
 For N customers, N models are to be built
 This demands huge computational power
 Every time the customer places a new order, the model needs to be rebuilt and
updated as the preferences would have changed
 Before making Did you forget recommendation, it is important to check if the
items are available in the inventory (in real time)
 The model predictions need to be integrated with inventory systems in real time before
making recommendation
 Big data solutions are used to build such models
 Big Data
87

 Big data is an all-encompassing term for any collection of data sets so large and
complex that it becomes difficult to process them using traditional data processing
applications.
 The challenges include analysis, capture, curation, search, sharing, storage,
transfer, visualization, and privacy violations.
 Big data is difficult to work with using most relational database management
systems and desktop statistics and visualization packages, requiring instead
"massively parallel software running on tens, hundreds, or even thousands of
servers
 Variety and Volume
88

 The goal of big data initiatives is the ability to analyze

 diverse data sources: transactional, social, mobile, cloud, web, sensor data
 new data types-voice or text or log files or images or video
 A retail bank, for example, is getting a handle on its multi-channel customer interactions
for the first time by analyzing log files.
 A hotel firm is analyzing customer lines with video analytics.
 A health insurer is able to better predict customer dissatisfaction by analyzing speech-to-
text data from call center recordings.
 large data sets.
 Companies can have a much more complete picture of their customers and
operations by combining unstructured and structured data.
 Big Data Characteristics
89

 Volume: Transaction-based data stored through the years. Unstructured data streaming in from
social media. Increasing amounts of sensor and machine-to-machine data being collected.
 Variety: Data today comes in all types of formats. Structured, numeric data in traditional
databases. Information created from line-of-business applications. Unstructured text documents,
email, video, audio, stock ticker data and financial transactions.
Velocity: Data is streaming in at unprecedented speed and must be dealt with in a timely manner. 
 Veracity: data flows can be highly inconsistent with periodic peaks.
90
Hadoop and Map Reduce
• Hadoop implements MapReduce in
two stages:
• Map stage: MapReduce breaks up the
huge dataset into smaller subsets;
then distributes the subsets among
multiple servers where they are
partially processed.
• Reduce stage: The partial results from
the map stage are then recombined
and made available for analytic tools

91
Big Data and Data Warehouse Coexistence
 Big Data
93

 Retail Store
 POS data: We already know what the customer bought
 RFID Tag attached to shopping cart
 Now we know where the customer stopped – and possibly could have bought – but
didn’t
 Using the video captured by the Security Camera
94
 IoT + Big data is changing how we manage customer
95
relationships
 With Big data and IoT we can go beyond the transaction to every
little detail of customers actual experience
 When customers enter the store
 How long they are there
 What products they looked at
 How long
 This vivid view of end to end experiences is changing the way
people think about, measure and manage customer relationships
 Customer Performance Indicators
• Outcome that customers desires
• Measurable in increments that customer value
• Example: Insurance vehicle: fast quote

96
Thank You