You are on page 1of 36

Psychometric testing in credit risk

A digital readiness primer and case study

@eflglobal //
EFL Intro

Where EFL works best

Digital readiness primer

Agenda HCD: Designing for the feature

Equity Bank case study

Traditional credit scoring is blind to billions of people
3 billion are unscorable, another 3 billion will enter the middle class in the next 20 years

Covered by
credit Not covered
•  Billions of good applicants rejected
covered, •  Enormous missed opportunity


3 Copyright © 2017. All Rights Reserved.

We are entering a new era of lending
And credit scoring needs to adapt to serve more people with end to end digital processes

Loan officers Loan officers Loan officers Digital approvals

with deep + + without loan officers
relationships Traditional Traditional and +
credit scoring alternative scoring Traditional and
alternative scoring

1990’s 2000’s 2010’s Today

Predictive power and availability vary by data source
Psychometric is only data source with 100% availability – everyone has a personality

More Available
(Hit rate on sample of low-income applicants in emerging markets)



Less Available

0% 5% 10% 15% 20% 25% 30% 35% 40% 45%
Less Predictive More Predictive

Predictive Power
(Out of sample gini coefficient with comparable model & sample)

6 Copyright © 2017. All Rights Reserved.

EFL is the proven universal score for today’s lenders
Empowering digital and in-person approvals across $1.5B+ and counting in disbursed loans

Alternative scoring leader Universal coverage Third party vetted

•  4 years R&D at Harvard on •  Score anyone, anywhere •  Published 3rd party

psychometric, demographic, with our psychometric and validations by World Bank
social & CDR scoring behavioral science based and others Link
•  6 years implementing •  Won multiple awards
scores with lenders across •  No client history or 3rd including G-20 SME
Africa, Latin America & Asia party database needed Finance Challenge, African
Business Award for
Innovation, and Mahindra
Finance Challenge

7 Copyright © 2017. All Rights Reserved.

EFL assesses risk with a short online loan application
How we score anyone The character traits our
assessment measures
EFL Algorithms
Analyze the answers and
Unscorable Applicant determine risk level Score Generation
Takes EFL assessment on Create 3-digit EFL Score and
the web, mobile or SMS
2 Score Confidence rating Attitude

1 3 Control

Entrepreneurial potential

Interaction metadata

Continuous Learning Social Behavior
EFL receives monthly loan
repayment used to constantly
customize and improve

8 Copyright © 2017. All Rights Reserved.

EFL Intro

Where EFL works best

Digital readiness primer

Agenda HCD: Designing for the feature

Equity Bank case study

EFL works best when there’s a clear need for an
alternative credit score, and the FI is challenged by:

1. 2. 3.

Low Acceptance Rates High Levels Costly & Lengthy Loan

(due to lack of data) of Default Application Processes

High rejection rates due Default rates over target; Loan application process
to lack of data – too hesitant to dramatically takes time and is resource-
many grey zone or thin decrease acceptance intensive for lender
file clients rate

10 Copyright © 2017. All Rights Reserved.

Use the EFL score to…
Our partners typically use our behavioral psychometric score in one of three ways:

1. 2. 3.

Increase Your Decrease Your Increase Your

Acceptance Rate Default Rate Operational Efficiency;
Lower Application Cost

Approve more people, Decrease and control Fast track: Cut pre-filters
without taking on more your default rate, with for high-scoring
risk limited volume impact applications

11 Copyright © 2017. All Rights Reserved.

‘Institutional Readiness’ plays a large role in success
We’ve noticed certain traits within organizations that allow the EFL digital credit assessment to thrive

High Volume,
Credit Score Ready Driven to Push Innovation Mindset

Ability to receive a 3-digit, Strong desire across Willingness to try new solutions
numerical, credit score and use it in management & teams to and consider a departure from
their existing credit decision making implement and scale traditional processes.
process. solution quickly.

Receiving thousands of credit Motivated team, with Willingness to take calculated

applications each month. willingness and motivation to risks and move in digital direction.
change existing process
Extra Plus? Ability to combine EFL
score with other credit scores via
scorecard or ensemble model

12 Copyright © 2017. All Rights Reserved.

EFL Intro

Where EFL works best

Digital readiness primer

Agenda HCD: Designing for the feature

Equity Bank case study

Financial institutions are going digital globally
Digital technologies can cut the cost of providing financial services by 80 to 90% and improve scalability

Annual cost to serve one customer in emerging economies,

2014 ($)

Accounts Cash-in, cash-out Transactions Total Traditional

bank branch


Cost savings
due to digitization

Source: McKinsey Global Payments Map; “Fighting poverty, profitably: Transforming the economics of payments to build sustainable, inclusive financial
systems,” Bill and Melinda Gates Foundation, September 2013; McKinsey Global Institute analysis

14 Copyright © 2017. All Rights Reserved.

Going digital? It’s more than building an app
What does it mean for a bank to be “digital”?

Being digital means being scalable. If you have to add staff to increase lending, you aren’t digital.

Acquisition Affordability Risk KYC / ID Servicing

Online acquisition is Web scrappers are Multiple companies, Proving ID online is Few companies are
possible, though emerging to support including EFL are an emerging field, focused on are looking
usually pricey. digital affordability, proving that risk however, in EM to outsource
however, limited profiles can be built limited databases collections.
coverage and via digital coverage is a factor.
inconsistent data interactions.
remain an issue

15 Copyright © 2017. All Rights Reserved.

But there are challenges to going digital
Moving the credit application process from in person with a loan officer to online comes with its own risks

Increased Risk
Digital applications can come with increased
risk and fraud. Effective credit scoring is key EFL is your digital
to keeping a healthy portfolio.
loan officer.

Efficiently assess risk

Lose Personal Touch on anyone, anywhere,
No human interaction can dampen the bank- online. Build trust with
to-customer relationship. Tools are needed applicants.
to increase human designed interaction.

16 Copyright © 2017. All Rights Reserved.

Going digital is a process
We work with partners at each stage

Fully digital process;

pushing for scale
No existing digital
processes Integrate EFL into a
digital application:

Use EFL as a stand alone •  Control risk while
tool, focused on: Hybrid model – maintaining
•  Customer Acquisition some parts digitized efficiency.
•  Reject Recovery •  Increase
Add EFL to partially digital acceptance rates.
process flow, see results in:
•  Operational efficiency


17 Copyright © 2017. All Rights Reserved.

EFL Intro

Where EFL works best

Digital readiness primer

Agenda HCD: Designing for the feature

Equity Bank case study

Product Design: Understand Your “Bones”
What framework are you working within?

•  SMS = a necessarily text-heavy product

•  Question length limited to 160 chars –containing both question and answer key
•  Need to battle potential mistrust of the product – concerns of SMS scams
•  Open answer responses – need to validate and send appropriate responses

Understand your customers:

•  Design for the mass-market, and try to incorporate the low-common denominator as
much as possible. People will struggle with:

Literacy Understanding Technology Connectivity

19 Copyright © 2017. All Rights Reserved.

Know What You NEED
Why does someone
•  Relatively easy to understand (both
the question and desired answer
Intelligence General Behavior

•  Predictive even within question

number and character limits Honesty Stability

•  Forgiving if users make mistakes

and need to correct answers
Fraud Attitudes
•  Trust building – Reliable product
that is connected to bank and
reward believed
Product design PROCESS

3. Data analysis of
flows and field 1. Field observation
interviews: ensure & interviews: on
which key metrics both question
are improving and framing and content
which aren’t

2. Experimental design:
validate/quantify hypotheses
from field via more real-life
channel using flow split-testing
and within-flow experiments

Watch! How and when are people

interacting with their phones?

Get to know your competition – what mobile services are people

using? What do they like / dislike about them?

Build: Fast and easy prototypes

Paper print outs of phones with text questions

“Hey! Can I ask your opinion?”

Talk with people- get them to use your product, explain as little as

“What do you think it means?”

22 Copyright © 2017. All Rights Reserved.

2 Create the RIGHT data
Test many versions of your product, over wide audiences, through various

Parallel Flows
Test two similar products at the same time, with different
versions of the same content you need to test – isolates
the “treatment effects”
In Person

Single Flows

Intersperse new iterations of product parts within your

current product iteration to see if behavior changes –
controls for individual differences

Distance Testing

Output: Well-structured quantitative and qualitative data to analyze

23 Copyright © 2017. All Rights Reserved.

3 Explore Your DATA
Define metrics for what makes the product “good” – explore the data to see what's
•  Answer distribution (extreme values vs. well-distributed
•  Answer timing (confusing questions take longer)
•  Drop off rates (do certain questions cause people to disengage?)
•  % of unusable answers
…Do we see an improvement in performance
metrics as a result of a new iteration?

Percent of responses with extreme

answers or unusable answers

10 Point Scale
Reply with a NUMBER ONLY on a scale from 0 - 35%

10 (0=never 5=sometimes 10=always). Do you 30%

show up late or miss appointments? 25%



Percentages 10%

What PERCENT (%) of the time do you show up 5%

late or miss appointments? 0%

10-point scale Percent scale

24 Copyright © 2017. All Rights Reserved.

Final SMS Content: Three Targeted Core Traits
Objective: Identify a few core traits that are most relevant to the population. Focus a majority of
psychometric content around these to max predictive power.

Understanding the
user’s concept of Understanding the
honesty and fairness user’s sense of
in the context of her personal responsibility
financial life for and control over
her life
Locus of
Integrity Control
% of responses with
no valid answer

mindset Understanding the
user’s perspective on
opportunities and
Locus of Confidence, motivations to take risks
Control, Optimism, to improve her life
Integrity, GSE

25 Copyright © 2017. All Rights Reserved.

Final SMS Product

Maximizing predictive power Reply with a

1)  3 core traits focused on
you found 100,000
2)  Granular answer potential (%s) KSH today, how
much would you
… through ensuring understandability spend on leisure?
1)  Answer in percentages
2)  Non-hypotheticals – concrete questions / “mini stories”
3)  Data quality check – smartphone? Help on survey?
… and building trust
1)  Human approach – make people feel comfortable
2)  Start with easy, but non-invasive questions (back load more intrusive
demographic questions)
3)  Trust this product – start with the person’s name

26 Copyright © 2017. All Rights Reserved.

EFL Intro

Where EFL works best

Digital readiness primer

Agenda HCD: Designing for the feature

Equity Bank case study

Overview - Scoring on the feature phone
Equity extends credit through chat, proves ability to boost acceptance rates

How does it work?

The Challenge
High volume, feature phone mobile lending comes with Tine, an Equity
significant risk. Equity needed an innovative solution to accept customer, applies
more of their customers without taking on additional risk. for a loan via her
Equitel SIM card.

The Engagement
EFL designed a new product – taking the learnings of our
traditional psychometric content, and transforming them into a 22
She’s new to the
SMS question assessment. Instead of being flat-out rejected, low bank, so she
data customers could take the assessment remotely on their receives the first
feature phones, and gain an opportunity to be approved.
EFL SMS asking
her to start the test.
The Results
Collecting 24k completed SMS tests, EFL built a custom SMS
model, and saw significant predictive power in the data Tine’s survey is
(psychometric & meta). The strong model results indicate analyzed, and her
potential SMS uses across markets / channels. credit score sent back
to Equity in real time.
Tine is disbursed!

28 Copyright © 2017 EFL. All Rights Reserved.

Designing the SMS Pilot – Process & Criteria
Equity & EFL partnered in 2013 to develop a custom
feature phone credit scoring tool

1 Design SMS

2 Build & Integrate

SMS Platform

1. The SMS channel Test

3 Run Pilot;
Gather Results
will facilitate a high
conversion rate.
Rate (%s in

4 Build Custom
2. SMS psychometric
5 Out-of-Sample Predictive
testing will enable Equity Psychometric
Model Power of SMS
Validation to better segment and Assessment Model
predict risk

29 Copyright © 2017. All Rights Reserved.

Assess willingness to repay through a conversation
Through EFL’s chatbot platform, we converse with your clients, one SMS at a
time. Through an automated conversation, we ask our traditional psychometric
questions – straight to their phone. Predictive, real-time, credit scores available
in minutes.
What percent (%)
of businesses
Derives predictive power from: cheat their
1.  Psychometric answers

2.  Timing data SMS – USSD

3.  Free-text input data ~20 psychometric questions
4.  Self-reported demographic data 4 test sections
Any phone, any location
Only need to score once

30 Copyright © 2017. All Rights Reserved.

Seamlessly assess an applicant’s character


Benson, a new Instead, he immidietely Benson engages, Credit Score

customer, applies for receives an inia9on conversa9on style, in the returned to lender
loan. Normally, he SMS masked from the SMS test.
would be rejected. lender

Behavioral – Text Analysis; Timer Analysis Psychometrics

What time of day are they responding?
What are response answer patterns?
Do they understand the question?
How is their spelling? Grammar?
What words or phrases do they use?
© 2016, EFL Global. All rights reserved. 31
EFL SMS test seamlessly fits in with existing UX mobile
lending product flow
Before rejecting low-data and new applicants, Equity can digitally score and asses risk on borrowers.


Benson, a customer Immidietely Benson engages,conversa9on Credit Score

of Equity Bank, receives inia9on style, in the SMS test. -
applies for loan via SMS from - Returned in real-
Equitel SIM SDK/ dedicated Equity Answers test in 4 secPons. SMS Pme through API to
USSD menu short code reminders sent to keep Benson Equity’s CU
- - focused.
Benson replies to
the SMS (zero- Total test )me <15 minutes,
rated) and begins taken at once or over hours/
the test days

32 Copyright © 2017. All Rights Reserved.

Results: Predictive Model; Strong Usability

Customized Model Out of

High Sample
With 24k tests completed, EFL built a Default
customized model for SMS – achieving a 63.7
predictive power of 63.7% AUC

Default Rate
Default Rate
Usability validated through low drop off
Average completion rate of 78%, with an
average test time of 33 minutes.
Low EFL Score Quartiles
80% of drop-off occurred within first 4 questions. Default
1 Worst 12 23 34 54Best
56% of tests were completed within 15 minutes. Worst Scoring Best Scoring

bad45_m3 bad30_m3 bad60_m3

33 Copyright © 2017. All Rights Reserved.

Ability to increase acceptance rates by 20% when
combined with internal behavioral score
Cumulated Default Rates

Default Rate
EFL & Equity’s Internal
Behavioral Score
Combining the EFL score with an
internal behavioral (transactional) score:
boost acceptance rate by 20% High 1.2x 1x Low
without taking on additional risk Acceptance Rate
Ensemble Experian
Behavioral EFL

34 Copyright © 2017. All Rights Reserved.

EFL finds different bads, and different goods, than
traditional credit bureau scores
Cumulated Default Rates

Default Rate
EFL & Traditional Bureau
Combining the EFL score with a
traditional bureau score:
boost acceptance rate by 53% Default

without taking on additional risk

High Low
Acceptance Rate
53% Acceptance

Ensemble Bureu
Bureau EFL

35 Copyright © 2017. All Rights Reserved.

EFL Intro

Where EFL works best

Digital readiness primer

Agenda HCD: Designing for the feature

Equity Bank case study