AIN104 – Machine Learning

in Enterprise Applications

PUBLIC
Speakers 2018

SAP TechEd Las Vegas Erik Marcadé

October 2–5, 2018

SAP TechEd Barcelona Erik Marcadé

October 23–25, 2018

© 2018 SAP SE or an SAP affiliate company. All rights reserved. ǀ PUBLIC 2

 Take the session survey.
We want to hear from you!

Be sure to complete the session evaluation

for this session AIN104
on the SAP TechEd mobile app.

Download the app from

iPhone App Store or Google Play.

© 2018 SAP SE or an SAP affiliate company. All rights reserved. ǀ PUBLIC 3

Disclaimer

The information in this presentation is confidential and proprietary to SAP and may not be disclosed without the permission of SAP.
Except for your obligation to protect confidential information, this presentation is not subject to your license agreement or any other service
or subscription agreement with SAP. SAP has no obligation to pursue any course of business outlined in this presentation or any related
document, or to develop or release any functionality mentioned therein.
This presentation, or any related document and SAP's strategy and possible future developments, products and or platforms directions and
functionality are all subject to change and may be changed by SAP at any time for any reason without notice. The information in this
presentation is not a commitment, promise or legal obligation to deliver any material, code or functionality. This presentation is provided
without a warranty of any kind, either express or implied, including but not limited to, the implied warranties of merchantability, fitness for a
particular purpose, or non-infringement. This presentation is for informational purposes and may not be incorporated into a contract. SAP
assumes no responsibility for errors or omissions in this presentation, except if such damages were caused by SAP’s intentional or gross
negligence.
All forward-looking statements are subject to various risks and uncertainties that could cause actual results to differ materially from
expectations. Readers are cautioned not to place undue reliance on these forward-looking statements, which speak only as of their dates,
and they should not be relied upon in making purchasing decisions.

© 2018 SAP SE or an SAP affiliate company. All rights reserved. ǀ PUBLIC 4

Agenda

Memory: Slides from 2016/2017

Intelligent Enterprise in 2018

What is needed to go further

© 2018 SAP SE or an SAP affiliate company. All rights reserved. ǀ PUBLIC 5

Slides from 2016/2017
From Human decisions systems to machine decisions
Productivity
Autonomous Machines propose/
Chasm!!! Systems make decision
Humans make decision
Data is de-
Data is aggregated Prescriptive
normalized,
Smart/Augmented flattened, fine grain
for visualization Analytics
Analytics
Predictive
Time: Instantaneous Analytics/ Human Machine Time: Train and
Machine Interface Apply!
UI integration at best Learning
Process integration
at least
Self-Service BI
Ad Hoc Agile Huge data
Visualization Optimized actions
Reports & volume generated
Standard OLAP What will happen?
Cleaned Reports
Raw
Data Why did it happen?
Data
What happened?

Maturity of Analytics Capabilities

© 2018 SAP SE or an SAP affiliate company. All rights reserved. ǀ PUBLIC 7

Analytics landscape and terminology: In the world…

Augmented/ Cognitive / Artificial

Analytics Advanced Analytics
Smart Analytics Intelligence
Predictive /
Diagnostic / Semi- Knowledge Representation and
Descriptive Assistance Machine Prescriptive
Discovery Structured Control
Learning

Powered by Math for

Powered Powered by Math for Processes
Powered by Interacting with Humans
by Math (Machines make operational/
Humans for Humans or for Autonomous
for Humans tactical decisions)
Systems

© 2018 SAP SE or an SAP affiliate company. All rights reserved. ǀ PUBLIC 8

Analytics landscape and terminology: In the world…

Augmented/ Cognitive / Artificial

Analytics Advanced Analytics
Smart Analytics Intelligence
Predictive /
Diagnostic / Semi- Knowledge Representation and
Descriptive Assistance Machine Prescriptive
Discovery Structured Control
Learning

Reporting Navigation Key Influencers Classification Text Analytics Recommendations Speech To Text Image/Video
Dashboard OLAP Hidden structure (rules) Regression Entity Extraction Scheduling Text To Speech Classification

Agile Outliers/ Time Series Search Model Natural Language Face Detection
Visualization Anomalies/ Management Processing Authentication
Clustering Taxonomy/Sort
Spatial Point of Interest Rule Systems Natural Language Visual
Associations Link Analytics
Visualization Segments/ Groups/ Optimization Understanding Recognition
Bands Sequences Knowledge Graph
BPM Automation Natural Language Reasoning
Model-based Spatial Analytics Generation Process Control
What-if analysis Signal Processing Conversational Gesture Control
Translation Robotics
Sentiment Common Sense
Analysis
Emotion Analysis

Powered by Math Powered by Math for

Powered by Humans for Powered by Math for Processes (Machines
Interacting with Humans or
Humans for Humans make operational/tactical decisions)
Autonomous Systems
© 2018 SAP SE or an SAP affiliate company. All rights reserved. ǀ PUBLIC 9
Intelligent Enterprise in 2018
The vision of the intelligent enterprise

Network & Cloud

Spend Management Platform

People
Engagement
Intelligent
Technologies Digital
Digital Core Platform

Intelligent Suite AI
Machine Learning
Analytics
IoT
Manufacturing
& Supply Chain

Customer
Experience Data
Management

© 2018 SAP SE or an SAP affiliate company. All rights reserved. ǀ PUBLIC 11

 Powered by PA
Use Case: SAP Sales Cloud
Opportunity Scoring
Key Message
Improve win rates and achieve quota targets by focusing on deals with high propensity of closing. Accelerate sales execution by aligning your
sales team’s efforts on highly scored deals with by understating the key influencer signals for each deal in each stage

Unique Value of Cloud for Sales

• Sales Acceleration
• Predictable sales pipeline and revenue
forecast
• Better prioritization of opportunities.
• Planning for high risk opportunities
• Insight into why deals are high/low risk.
• Powered by SAP Predictive Analytics

© 2018 SAP SE or an SAP affiliate company. All rights reserved. ǀ PUBLIC 12

 Opportunity Scoring: What are the Feature Buckets?
75+ features derived from feature engineering using SAP & non-SAP datasets

Sales Activity Marketing Finance Service

• Campaigns • Risk score
• Industry • Engagement score • Net Promoter Score
associated
• Average annual (NPS)
• Source • Engagement trend
• Lead score revenue spend
• Escalation score
• Forecast Category • Follow up activities
• Buying propensity • Estimated
• Open Tickets
• Age • Last update score commission
• Product Usage
• Sales Area • Influencer score • Cost of Marketing • Cost of sales (CAC)
(CAC) • Returns
• Account win rate • .. • Discount received
• Marketing • Warranty remaining
• Deal size • Order consistency
conversion win
• …
• Pushes to Close rate • Average order size
date
• … • …
• Pushes to Revenue
• ..

© 2018 SAP SE or an SAP affiliate company. All rights reserved. ǀ PUBLIC 13

Summary of ML at C/4HANA Sales & Service (2018 Q1: kudos to C/4 team)

Number of Production Number of Live Customers

Number of Customers Deployments

65 33 9

Number of Scoring Average Accuracy Number of Customer

Predictions/Day during Training References

720k 70% 2

© 2018 SAP SE or an SAP affiliate company. All rights reserved. ǀ PUBLIC 14

SAP ML References – Bentley Systems
“ I have been overwhelmingly and pleasantly surprised with the SAP C4C ML Pilot by
engaging with us at the right time and bringing in best practices from other pilot customers
that we can learn from to make the product better than we can have imagined.

What we are seeing coming out of SAP is leading edge. It is pushing the envelope, coming out
quick, getting there much faster than we could get there ourselves.

Working with SAP has really cemented our approach on how we deploy ML without having to
turn our architecture around”

David McKenney | VP of Sales | Bentley Systems

© 2018 SAP SE or an SAP affiliate company. All rights reserved. ǀ PUBLIC 15

Key Metrics for Opportunity Scoring
High correlation between Process discipline, Quality of data & Model performance
Model Performance Metrics Data Check Criteria Process Metrics
Data Volume
Customer Data Data Volume Average Active Win Rate by Average
KI KR w. Updates Sales Average # Sales
Volume (last 12 Updates Opportunities Count, Days to
(Accuracy) (Confidence) (Training Velocity Days to Win
(overall) months) /Opportunity (>2 Updates) Amount Lose Phases
Population)

1 85% 99.36% 270K 50K 42K 13.6 99.7% 19%, 19% 61 159 4

2 68.5% 99.35% 574K 103K 87K 4.8 82% 52%, 52% 105 174 4

3 66% 99% 373K 63K 60K 5.4 81% 86%, 92% 64 138 3

4 63% 99% 41K 13K 10K 5.0 84% 62%, 39% 60 91 4

5 79.4% 98% 13K 5.6K 4.8K 3.1 49% $5.99 M 27%, 25% 115 135 6

6 Aug 3
73% 98% 300K Aug 17
21K Aug
20K17 5.5 Aug 24
93% Aug 24 65%, 58% 118 Sep 4
618 4
Aug 10
7 67% 97% 70K 70K 10K 5.0 84% 63%, 39% 108 225 5

8 55.25% 95% 6K 2.2K 1.8K 5.8 92% 52%, 60% 133 162 1

9 67% 96% 18K 6.5K 3.8K 2.7 86% 96% 118 618 3

10 60% 94% 30K 16K 16K 1.6 17% 52%, 47% 84 155 6

© 2018 SAP SE or an SAP affiliate company. All rights reserved. ǀ PUBLIC 16

 Key Lessons Learnt (kudos to C/4 team)

• Machine Learning (ML) cannot be a “backbox”.

Explainable AI is key to build trust and make transparent the reasoning behind the prediction

• Clean Data beats More Data & More Data beats Better Algorithms.
Focus more on data quality and process maturity then type of algorithm used

• Model is never perfect but it can be useful.

Optimize on business goal and trying to minimize loss from model

• Only a small fraction of ML production systems is ML code.

Vast majority is for supporting ML and Data infrastructure

• ML in Operations is a way to measure the ‘maturity’ of business process.

Data collected through humans is biased

© 2018 SAP SE or an SAP affiliate company. All rights reserved. ǀ PUBLIC 17

End to end predictive workflows in Applications

Predictive Analytics integration in

applications and business processes

Smart Assist providing intelligence to BI &

Planning workflows

Smart Predict to enable business users to

build custom models*
*Back injection of predictive models end 2018

© 2018 SAP SE or an SAP affiliate company. All rights reserved. ǀ PUBLIC 18

Steps to understand/buy-in and customize

Application
Browse Predictive
Question on
Model repository Expose Insights from
Opportunity Publish the new version
from SAC & Select Model (Key Influencers,
Scoring in (template) to C/4
proper etc..)
C/4
Scenario/model

Analytics Tool
Do the Magic ???
Data Access from SAC Remove one variable,
to build stories Use a template for Build another version of
(eventually use Smart cluster + model, use the model
Discovery) segmented modeling,
etc…

© 2018 SAP SE or an SAP affiliate company. All rights reserved. ǀ PUBLIC 19

SAP Leonardo embedded in S/4HANA Live!
Machine Learning and Predictive Analytics

SAP S/4HANA Contract Consumption

SAP Leonardo
Sourcing and
Procurement
+ SAP Predictive Analytics = Proactively renew contracts by predicting their expiration
and consumption

SAP S/4HANA SAP Leonardo Stock in Transit

Material Management + SAP Predictive Analytics = Mitigate production risks by forecasting stock in transit delays

SAP S/4HANA SAP Leonardo Sales Quotation Conversion

Sales Analytics + SAP Predictive Analytics = Improve Sales Forecasts

SAP S/4HANA SAP Leonardo Payment Block – Cash Discount at Risk

Procure-to-Pay + SAP Predictive Analytics = Avoid loss of cash discounts

SAP S/4HANA SAP Leonardo Smart Alerts for Profit & Loss Analysis
Finance + SAP Predictive Analytics = Get informed of hidden unusual costs and revenue
opportunities

SAP S/4HANA SAP Leonardo Project Cost Forecast

Idea + SAP Predictive Analytics = Mitigate project budget overruns

© 2018 SAP SE or an SAP affiliate company. All rights reserved. ǀ PUBLIC 20

SAP Leonardo intelligent applications for Live
Machine Learning and Predictive

SAP S/4HANA® + SAP® Leonardo = ML Enabled Business Scenarios

SAP Business Integrity Screening

SAP S/4HANA SAP Leonardo
Finance + SAP Predictive Analytics = Increase accuracy of fraud alerts
with predictive analytics

SAP S/4HANA SAP Leonardo SAP Tax Compliance

Finance + SAP Predictive Analytics = Smart Automation on Compliance Issue Processing

SAP C/4HANA SAP Leonardo SAP Opportunity Scoring

Cloud 4 Customer + SAP Predictive Analytics = Increase win rates with Predictable sales pipeline
and revenue forecast

© 2018 SAP SE or an SAP affiliate company. All rights reserved. ǀ PUBLIC 21

SAP Leonardo embedded in S/4HANA 1808-1811
Machine Learning and Predictive Analytics

SAP S/4HANA SAP Leonardo Dynamic buffer level adjustment

Material Management + SAP Predictive Analytics = Optimizes inventory buffers based on replenishment leads time

SAP S/4HANA SAP Leonardo

Sales + SAP Predictive Analytics = Sales Forecast Prediction
Forecast sales based on predictive analytics

SAP S/4HANA SAP Leonardo Delivery Performance

Sales + SAP Predictive Analytics = Forecast sales based on predictive analytics

SAP S/4HANA
Sourcing & Procurement + SAP Leonardo
SAP Predictive Analytics
= Defect Code recognition
Propose defect code when free text entered

SAP S/4HANA
Material Management + SAP Leonardo
SAP Predictive Analytics
= Supplier Delivery Delay
Forecast sales based on predictive analytics

© 2018 SAP SE or an SAP affiliate company. All rights reserved. ǀ PUBLIC 22

What is needed to go further
The Business Question Life Cycle
Design Intelligent
Business
Operations
Business Business Scenario
Question Question

Analytics Tool Application

Insights

ok
Apps Processes
Smart nok
Assist
Analytics User Analytics User Application DevOps.
Developer.
ML API

Model Operationalization
ML API
ok
Smart
Predict

nok

ML API

ML API
Notebook Orchestration

Pipeline Landscapes Data Science Ops

Production
Data
Scientist
Data Science Design Tools Training
Cloud Ops

Business LCM & Operations Tools

Question
© 2018 SAP SE or an SAP affiliate company. All rights reserved. ǀ PUBLIC 24
Data Science: NOT a linear process

Business
Understanding Activate
Version

Data Deviation
Evaluation Deployment
Understanding Control

Scenario Model
LCM LCM

Train /
Data Evaluate
Modelling Retrain
Preparation

© 2018 SAP SE or an SAP affiliate company. All rights reserved. ǀ PUBLIC 25

ML API – Focus on LCM and Operations

Operationalization Front-End Intelligent Application

Data Science Business User
Ops

ML API (LCM and Ops)

PAi Current Scope

Repository LCM Operation

User Rights Licensing
Management Management Governance
Management
and Monitoring and Monitoring

• Catalog • Tasks • User rights, • Usage Tracking, • Subscription,

• Scenarios • Train • Profile Management • Billing, • Unsubscription,
• Model • Control • Audit,
• Model Versions • Retrain • Approval,
• Apply • Compliancy

© 2018 SAP SE or an SAP affiliate company. All rights reserved. ǀ PUBLIC 26

SAP TechEd online / SAP Community

Access replays of:

 Keynotes
 SAP TechEd live interviews
 Selected lecture sessions

http://sapteched.com/online

Continue your SAP TechEd discussion after

the event within SAP Community:
 Read and reply to blog posts
 Ask your questions
 Join conversations

sap.com/community
See all SAP TechEd blog posts

© 2018 SAP SE or an SAP affiliate company. All rights reserved. ǀ PUBLIC 27

Further information

Related SAP TechEd learning journeys

AIN5 – How to unleash your data’s potential with Predictive Analytics

Related SAP TechEd sessions

AIN104 – Machine Learning in Enterprise Applications
AIN214 – Making SAP Software Applications Smarter by Embedding Machine Learning
AIN215 – Introducing Predictive Capabilities in SAP Analytics Cloud
AIN366 – Create Smart Dashbaords with Predictive Modeling in SAP Analytics Cloud
AIN368 – Support the Intelligent Enterprise with SAP S/4HANA and Machine Learning
AIN866 – SAP Predictive Analytics and the Intelligent Enterprise: Roadmap
AIN867 – SAP Analytics Cloud – Smart Predict: Roadmap

Public SAP Web sites

SAP Community: https://www.sap.com/uk/community/topic/predictive-analytics.html
SAP products: https://www.sap.com/uk/products/predictive-analytics.html

SAP training and certification opportunities

www.sap.com/education

© 2018 SAP SE or an SAP affiliate company. All rights reserved. ǀ PUBLIC 28

 Thanks for attending this session.

Feedback Contact for further topic inquiries

Please complete your session Erik Marcade

VP Advanced Analytics
evaluation for AIN104. Erik.Marcade@sap.com

© 2018 SAP SE or an SAP affiliate company. All rights reserved. ǀ PUBLIC 29

Follow us

www.sap.com/contactsap

© 2018 SAP SE or an SAP affiliate company. All rights reserved.

No part of this publication may be reproduced or transmitted in any form or for any purpose without the express permission of
SAP SE or an SAP affiliate company.
The information contained herein may be changed without prior notice. Some software products marketed by SAP SE and its
distributors contain proprietary software components of other software vendors. National product specifications may vary.
These materials are provided by SAP SE or an SAP affiliate company for informational purposes only, without representation or
warranty of any kind, and SAP or its affiliated companies shall not be liable for errors or omissions with respect to the materials.
The only warranties for SAP or SAP affiliate company products and services are those that are set forth in the express warranty
statements accompanying such products and services, if any. Nothing herein should be construed as constituting an additional
warranty.
In particular, SAP SE or its affiliated companies have no obligation to pursue any course of business outlined in this document or
any related presentation, or to develop or release any functionality mentioned therein. This document, or any related presentation,
and SAP SE’s or its affiliated companies’ strategy and possible future developments, products, and/or platforms, directions, and
functionality are all subject to change and may be changed by SAP SE or its affiliated companies at any time for any reason
without notice. The information in this document is not a commitment, promise, or legal obligation to deliver any material, code, or
functionality. All forward-looking statements are subject to various risks and uncertainties that could cause actual results to differ
materially from expectations. Readers are cautioned not to place undue reliance on these forward-looking statements, and they
should not be relied upon in making purchasing decisions.
SAP and other SAP products and services mentioned herein as well as their respective logos are trademarks or registered
trademarks of SAP SE (or an SAP affiliate company) in Germany and other countries. All other product and service names
mentioned are the trademarks of their respective companies.
See www.sap.com/copyright for additional trademark information and notices.