Bill's Presentation

Predictive Analytics and
Machine Learning:
An Overview
Bill Haffey
April 27, 2012
Business Analytics software

© 2010 IBM Corporation
Statistical Analysis and Machine Learning
Statistical Analysis
–Confirm Hypotheses
–Data Requirements User
–More Assumptions
Driven
–Design importance
–General Population Predictions
Machine Learning
–Generate Hypotheses
–Exploratory
–Less Data Prep Data
–Fewer Assumptions Driven
–Individual Predictions
–Results Oriented

Statistics – Use Case Examples

Used often in experimental design, clinical trials and survey research
with complex sampling designs
– N.O.R.C. and Gallup use extensive inferential statistics
accurately representing survey data on how people think and feel
about the world today.
– NIH uses inferential statistics to analyze clinical data to reveal
significant differences in treatments and interventions.
– CDC – extensive epidemiological studies require inferential
statistics
Used to create data when you don’t have it:
– The data is ‘expensive’
– Sample size
– What types?
– Sample, infer

© 2009 SPSS Inc. 3
In a nutshell…
Machine Learning works by…
–Clearly defining business goals

–Data exploration:
• Discovery of patterns – hypothesis generation
• ‘Weighting’ of important inputs
• (With some algorithms) dismissal of non-influential factors
–Training/Refining/Validation of Models
–Reliance on domain/data expertise, rather than analytical skills
–Model deployment
• Model export in common formats (.xml, .pmml)
• Automated update

Machine Learning
Three classes of algorithms Group cases that
Cluster exhibit similar
Supervised vs. characteristics.
“Differences”
unsupervised
Complementary
What events
occur together?
Given a series of
actions; what
action is likely to Data
occur next?
Mining Predict
“Relationships”
Associate
“Patterns”
Predict who is likely
to exhibit specific
behavior in the
future.
© 2007 SPSS Inc. 5 © 2010 IBM Corporation

Supervised Learning: Profile and Predict
Build a predictive profile of the Credit ranking (1=default)
Cat. %
Bad 52.01 168
Good 47.99 155
n
Total (100.00) 323
historical outcome using a Weekly pay
Cat. %
Bad 86.67 143
n
Paid Weekly/Monthly
P-value=0.0000, Chi-square=179.6665, df=1
Monthly salary
Cat. %
Bad 15.82 25
n
collection of potential input fields.

Good 13.33 22 Good 84.18 133
Total (51.08) 165 Total (48.92) 158
Age Categorical Age Categorical

P-value=0.0000, Chi-square=30.1113, df=1 P-value=0.0000, Chi-square=58.7255, df=1
Young (< 25);Middle (25-35) Old ( > 35) Young (< 25) Middle (25-35);Old ( > 35)
Cat. % n Cat. % n Cat. % n Cat. % n

Bad 90.51 143 Bad 0.00 0 Bad 48.98 24 Bad 0.92 1
Good 9.49 15 Good 100.00 7 Good 51.02 25 Good 99.08 108
Total (48.92) 158 Total (2.17) 7 Total (15.17) 49 Total (33.75) 109
We will “Supervise” the learning

Social Class
P-value=0.0016, Chi-square=12.0388, df=1
Management;Clerical Professional
Cat. % n Cat. % n
Bad 0.00 0 Bad 58.54 24
Good 100.00 8 Good 41.46 17
process as the algorithm attempts

Total (2.48) 8 Total (12.69) 41
to model the outcomes using

provided inputs
Explores all combinations,
interactions and contingencies.
Use this profile to understand and
predict future cases.
6
© 2009 SPSS Inc. © 2010 IBM Corporation
Profile and Predict

Neural Networks
–A technique for predicting outcomes based on inputs,
where the inputs are weighted on hidden layers
–Behaves similar to the neurons in your brain
–‘Back-propagation’ to adjust weights based on hits/misses on
training data
–Requires minimal statistical or mathematical knowledge
7 SPSS Inc.
© 2009 © 2010 IBM Corporation
Neural Network Anatomy
8 © 2010 IBM Corporation

Neural Network Output

Neural Network Summary
Excellent for modeling complex relationships and predicting outcomes

–Can handle nonlinearity and interactions with ease
Good for solving many different problem sets (categorical, binary, scale
predictors and outcomes)
Very poor (Black Box) at describing the relationships among predictors
and outcomes

Profile and Predict
Decision Trees and Rule Induction

–Classification systems that predict or classify
–Technique that shows the ‘reasoning’
– contrast with Neural Network
–Builds sets of easy to understand ‘If – Then’ Rules
–Eliminates factors that are unimportant

Start Here: Claim Amount

Decision Tree Output

Decision Trees
Excellent at uncovering and modeling complex relationships

Very accurate on even small data sets to inform decision
making.
Can handle nonlinear relationships with complex interactions.
Very easy to understand and describe to others.
Time to insight in minutes.

What is Unsupervised Learning?
A Machine Learning technique

useful when we do not know the
output or outputs
Can be thought of as finding
‘useful’ patterns above and
beyond noise…or “fishing” for
information

Cluster and Associate
Clustering
– An exploratory data analysis technique
– Reveals natural groups within a data set
– No prior knowledge about groups or
characteristics
– ‘Large’ groups interesting, but so are ‘small’
groups
– Not always an end in itself
Associations
– Finds things that occur together – ex: events in
a crime incident
– Associations can exist between any of the
attributes: (no single outcome like Decision
Trees)
Sequential Associations
– Discovers association rules in time-oriented
data
– Find the sequence or order of the events
15 SPSS Inc.
© 2009 © 2010 IBM Corporation
What kinds of things can you do with Machine Learning in the

Public Sector/Federal?
Manage Thwart Detect Fraud Clean up the

Human ‘Insider Streets
Capitol Threat’

IBM SPSS Text Mining
Human Capital Management

Case Study: U.S. Army Reserve - OCAR

Challenge – Reduce and determine reasons for reserve attrition
• Reserve soldiers have careers and responsibilities outside of the U.S. Army,
making high attrition rates an ongoing challenge.
• Need to determine the characteristics that lead to attrition and the types and
levels of incentives that can aid in retaining a soldier
Solution – IBM SPSS Modeler
• SPSS Modeler used to classify soldiers at risk of attrition, including the

analysis of military occupational skills (MOS) in classifying attrition
• SPSS Modeler to create models for incentive planning.
Benefits
• Predicted attrition using demographic data for army reservists.
• Created a predictive model to analyze why reservists leave and used this
model for scoring the possibility for attrition of candidates on a weekly basis.
• Modeled the soldier incentive types and levels that would minimize cost
and attrition
Retention
Modeling Current
Employees
(Education, job
Likelihood of Success history, experience,
Process If Experience = Info Systems

And Education = Undergrad
And Years Working <= 5
demographics)
Current
And Communication Skills > 7 Data
Then Success = Medium(35, 0.78)
Likelihood to Separate Payroll

If Education= Post Graduate Identify characteristics of (Comp plans,
And Years Working >= 7 employee success and salary)
And used “travel” (sentiment NEGATIVE) attrition / (dis)satisfaction
And Commute >= 30mins
Then Leave = YES (94, 0.927)
Survey Data
(Attitudes, non work
Retention Incentives related factors)
1. Salary Increase , prob 0.23
2. Not applicable
3. Flexible Schedule, prob 0.87
4. PerformanceAward, prob 0.36 Managers reports on Data
5. Benefits, prob 0.54 employee satisfaction Collection
… and performance
© SPSS 2009 © 2010 IBM Corporation

Retention
Modeling Current
Employees
(Education, job

demographics)
And Communication Skills > 7


Predictive Modeling
Survey Data
2. Not applicable
4. PerformanceAward, prob 0.36 Managers reports on
5. Benefits, prob 0.54 employee satisfaction
… and performance

Retention
Modeling Current
Employees
(Education, job

demographics)
And Communication Skills > 7


Decision Optimization
Survey Data
2. Not applicable
4. PerformanceAward, prob 0.36 Managers reports on
5. Benefits, prob 0.54 employee satisfaction
… and performance

Medicaid Fraud:
Trust Solutions CMS Case Study

Case Study – Anita Nurse
Apriori results showed Anita as being one of the

patients billed for services from various HHAs
– Therapy at Home HHA
– Friendly Therapy & Fun HHA
– Comfortable Quarters HHA
– Rehabilitation & Recreation HHA
The output does not show the frequency by which
Anita moved back-and-forth

Case Study – Anita Nurse

Further data analysis
revealed that Anita
actually moved
among these 4
providers in an 18
Therapy At Home Friendly Therapy & Fun month period!
Given the nature of
HHA, TS clinicians
and investigators felt
that this was unusual
Shows Anita moved
among these
providers but not the
order
Comfortable Quarters Rehabilitation & Recreation

Sequencing Algorithm
Specific type of association technique where it is not only important

that a relationship exists, but the order of ‘events’ is of interest
Sequencing rules are used to answer questions like:
–Does the pattern of a purchase predict the future purchase of
another item?
–Do people buy health club memberships after visiting a doctor?
–Does a patient visit provider B after visiting provider A?

Case Study – Ben Feelinsick
Apriori results had shown that Ben had been billed for services by 2
different providers:
–Healing Nurses HHA
–Therapy at Home HHA
Initially, this might not be of interest, or not as much as Anita…
Examining the data further using sequencing, a suspicious pattern
emerges!

Case Study – Ben Feelinsick

2/14/2000 2/19/2000 In only a 4 month
Dx 340 (MS) Dx 340 (MS)
span, Ben moved
between these 2
providers a total of 7
Therapy at Home 2/28/2000 times
Dx 5990 (Ur Trct Inf) 3/12/2000
Dx 5990 (Ur Trct Inf)
How does this
pattern benefit the
patient?
3/19/2000
3/13/2000
It is suspicious that
Dx 340 (MS) the patient moves
so much during this
short time
3/27/2000
Dx 340 (MS)
4/21/2000
Healing Nurses

January 12, 2012
Law Enforcement

Law Enforcement
Problem: Spiraling crime rates, limited officer resources -- better
deployment decisions required
Solve: (In addition to incident data) weather, city events,
holiday/payday cycles, etc – better picture of criminal incidents,
more accurate prediction, more effective deployment

Model mechanics – law enforcement scenario
Night Day
NPD PD/PD+1 Clear Rain
N3D 3D


Insider Threat Detection and Analysis

What is an insider threat?
A current or former employee, contractor, or business

partner who:
has or had authorized access to an organization’s network,
system, or data
and
intentionally exceeded or misused that access in a manner that
negatively affected the confidentiality, integrity, or availability of
the organization’s information or information systems
Source: U.S. CERT

Insider Threat Analysis – Use Case
Common Data Environment: Using Machine Learning:

Merge and exploit data from all sources using all
Audit data – network and server logs, files
relevant data attributes
accessed, emails and content, employee
Model normality to identify anomalous behavior
demographics
Trend/ Predict which employee is not behaving
Large volumes like peers
Disparate sources
Different data formats - structured
and unstructured

What is Normal?
Baseline Activity
Including resource usage, work hours,
document type…
Change in Cluster
Used to baseline activity of employees Membership
against:
Their own past history
The past history of their peers (job
title, department, project)
Used for both Reactive and Proactive

Analysis
Spikes in Activity
Reversals in Trends

Reactive Analysis
A K-Nearest Neighbor algorithm is

used to easily identify employees
whose behavior closely matches that
of the person being audited.
…other
Segmentation
algorithms and
Association
algorithms are
also used to
group people
based on
behavior patterns

Proactive Analysis
Analysis
Analysis of
of documents
documents
accessed
accessed by by
employees
employees andand how
how
closely
closely each
each person
person isis
associated
associated toto certain
certain
topics of interest
topics of interest
Most
Most of of the
the work
work done
done within
within
proactive
proactive analysis
analysis is is used
used toto
contribute
contribute to to an
an individual’s
individual’s riskrisk score
score
or
or to
to create
create aa model
model to to classify
classify the
the
likely
likely risk
risk for
for that
that individual.
individual.
Thanks

Bill&#39;s Presentation

Загружено:

Сведения о документе

Авторское право

Доступные форматы

Поделиться этим документом

Поделиться или встроить документ

Параметры публикации

Этот документ был вам полезен?

Это неприемлемый материал?

Авторское право:

Доступные форматы

Bill&#39;s Presentation

Загружено:

Авторское право:

Доступные форматы

Predictive Analytics and

Business Analytics software

Statistical Analysis and Machine Learning

© 2010 IBM Corporation

Statistics – Use Case Examples

© 2010 IBM Corporation

–Clearly defining business goals

© 2010 IBM Corporation

© 2007 SPSS Inc. 5 © 2010 IBM Corporation

Supervised Learning: Profile and Predict

Build a predictive profile of the Credit ranking (1=default)

Total (100.00) 323

historical outcome using a Weekly pay

collection of potential input fields.

Age Categorical Age Categorical

Cat. % n Cat. % n Cat. % n Cat. % n

We will “Supervise” the learning

process as the algorithm attempts

to model the outcomes using

Profile and Predict

Neural Network Anatomy

8 © 2010 IBM Corporation

Neural Network Output

9 © 2010 IBM Corporation

Neural Network Summary

Excellent for modeling complex relationships and predicting outcomes

10 © 2010 IBM Corporation

Profile and Predict

Decision Trees and Rule Induction

11 © 2010 IBM Corporation

Start Here: Claim Amount

12 © 2010 IBM Corporation

Excellent at uncovering and modeling complex relationships

13 © 2010 IBM Corporation

What is Unsupervised Learning?

A Machine Learning technique

© 2010 IBM Corporation

Cluster and Associate

What kinds of things can you do with Machine Learning in the

Manage Thwart Detect Fraud Clean up the

© 2010 IBM Corporation

Human Capital Management

Business Analytics software

Case Study: U.S. Army Reserve - OCAR

Solution – IBM SPSS Modeler

• SPSS Modeler used to classify soldiers at risk of attrition, including the

Process If Experience = Info Systems

Likelihood to Separate Payroll

© SPSS 2009 © 2010 IBM Corporation

Process If Experience = Info Systems

And Communication Skills > 7

Likelihood to Separate Payroll

© SPSS 2009 © 2010 IBM Corporation

Process If Experience = Info Systems

And Communication Skills > 7

Likelihood to Separate Payroll

© SPSS 2009 © 2010 IBM Corporation

Business Analytics software

Case Study – Anita Nurse

Apriori results showed Anita as being one of the

© 2010 IBM Corporation

Case Study – Anita Nurse

Comfortable Quarters Rehabilitation & Recreation

Specific type of association technique where it is not only important

Bill's Presentation

Bill's Presentation