Speaker Name & Country :

Roger Austin (UK), Kieran Barnes (UK)

On behalf of UK IFoA
Solvency & Capital Management Working Party

Topic:

Delivering Appropriate Expert Judgement

Agenda

Introduction and background

Framework
Process using a worked example
Validation
Recap of process and conclusions
Opportunity for discussion
Paper: https://www.actuaries.org.uk/documents/expertjudgement

11 Nov 2016

Expert Judgement

Introduction and background

11 Nov 2016

Expert Judgement

Introduction
What is an expert?

Expert - you will know one when you see one.

Wikipedia definition: An expert:

is someone widely recognized as a reliable source of technique or skill

whose faculty for judging or deciding rightly, justly, or wisely is accorded
authority and status by their peers or the public in a specific welldistinguished domain
...can be believed, by virtue of credential, training, education, profession,
publication or experience, to have special knowledge of a subject beyond
that of the average person, sufficient that others may officially (and legally)
rely upon the individual's opinion.

11 Nov 2016

Expert Judgement

Introduction
So what is expert judgement?

Not something new (actuaries have been doing this since the profession started).
Important in other professions too (medical, legal, etc).
I will confine my comments to insurers balance sheet.
Inherent in wide range of areas e.g. choice of methodology, choice of datasets,
how to deal with insufficient / unreliable data, etc.
Relevant for various items of the balance sheet (not just capital).
Relevant for all insurers (not just those using an internal model).

An increasing area of focus for regulators, particularly in the EU with

Solvency II Directive.
Challenging area for many insurers: one of the drivers for our paper.
Approach needs to be proportionate.
11 Nov 2016

Expert Judgement

Introduction
What is a model?
Simplification of reality
so judgement is inherent in all models.
Some judgements have small impact; others have significant
impact.
When does a judgement become an expert judgement?

Should be defined in firms system of governance.

Materiality will be an important factor
as will proportionality.

11 Nov 2016

Expert Judgement

Expert judgement versus judgement

Judgement

Expert Judgement
mortality improvement

data manipulation

mortality risk factors

11 Nov 2016

Expert Judgement

Implications and scope

So what is the consequence of something being considered
expert judgement rather than judgement?
Approach to forming
judgement

How it is documented

Key categories of expert

judgement

Methodology

Additional rigour

Assumptions (Inc. parameters)

Approximations

How it is validated

How it is monitored

Are expert judgement and data mutually exclusive?

11 Nov 2016

Expert Judgement

Formation of judgement
Sources of information

Expert views

Decision-makers

Good process is essential, and needs to be tailored and

proportionate in line with materiality.
Sometimes the experts may also be the decision-makers.
Modify process to fit context / complexity.
11 Nov 2016

Expert Judgement

Framework

11 Nov 2016

Expert Judgement

10

Framework: Overview

Expert judgement policy.

Governance structure.
Strong process.
Documentation.
Appropriate validation.
Must be proportionate.

11 Nov 2016

Expert Judgement

11

Framework: Expert judgement policy

The expert judgement policy could cover aspects such as:

What is meant by expert judgement.

When the policy applies and limitations.
Interaction with any materiality, proportionality and validation policies.
Requirements of the management board in relation to expert judgement.
Requirements of executive and operational owners.
Documentation requirements.
Reporting requirements, including escalation.
Requirements on the expert, including defining when an external expert needs to be sought.
Required review (both internal and external).
Required frequency of refresh and review.

Different approaches possible

but important to be somewhere within policy framework.
11 Nov 2016

Expert Judgement

12

Process

11 Nov 2016

Expert Judgement

13

Process overview
1. Preliminary
assessment of
judgement

5. On-going
monitoring

4. Decision
making

11 Nov 2016

2. Defining
the problem

3. Elicitation
of expertise

Expert Judgement

14

Some useful concepts

Plausible range

1st quartile

Plausible range

Central estimate

Output metric (lower)

3rd quartile

Output metric (upper)

Impact range

Uncertainty total impact

(Impact range) = Uncertainty total impact

Regions of expert judgement

11 Nov 2016

Expert Judgement

15

An example
Situation:

Regional Group but without experience in the target market

New product with a return of premium guarantee after 10 years
Needs to set the lapse assumption.

11 Nov 2016

Expert Judgement

16

Preliminary assessment of judgement

Identify judgement

Lapse assumption

Assess whether in the

scope of the EJ process

Key risk so inside expert judgement process

11 Nov 2016

Expert Judgement

17

Defining the problem

Defining the problem

Lapse assumption

Terminology

Defined as the proportion of the initial population which will still be in-force after ten
years for a medium net worth policyholder age 40-50 with a family

Articulate what the EJ relates

to and why it is needed

Area of judgement: Assumptions

Metrics of interest: IFRS profit, Embedded Value profit, statutory balance sheet, internal
economic capital forecasts

11 Nov 2016

Expert Judgement

18

Defining the problem

High level understanding of
the firms exposure

Higher lapses early in the contract reduce profitability

Higher lapses late in the contract improve profitability
Higher lapses following a fall in the unit fund improve profitability

Areas where judgement may

need to be broken down

Pricing teams may require more granular assumptions than the financial reporting teams

Trigger of expert judgement

New product launch

11 Nov 2016

Expert Judgement

19

Defining the problem

Previous work and drivers to
change

Prepare an initial estimate of

the plausible range

11 Nov 2016

New assumption so no previous judgements to review

Three estimates
1. Unit linked contract with no guarantee in country X
Result = 50% remain in force.
2. Industry data on guarantee product in country Y
Result = 65% remain in force taking average across fund growth scenarios
3. Guesstimate lapses reduce to 0% p.a. when guarantee is more than 120% of unit
fund
Result = 80% remain in force taking average across fund growth scenarios

Expert Judgement

20

Defining the problem

Assumed sample policy with an initial investment of $10,000
Scenario

Impact of plausible range

Assess the potential for

reducing the plausible range

11 Nov 2016

Method 1
Method 2 (central)
Method 3

Remaining inforce
50%
65%
80%

EEV (3% fund

growth) $

800
1100

1200

Difference to central
estimate $
-300
0
100

Difference to central
estimate %
-27
0

Further approaches could be used to reduce the plausible range:

- Further research on generally accepted actuarial techniques and industry data (30
days)
- Research on reasons for differences between lapse rates for gteed products in
Countries A, B and C and extrapolation to Country X (55 days)
- Consultants experience (100 days)

Expert Judgement

21

Defining the problem

Assess appetite for reducing
the plausible range

Balance between
A. Desire to reduce the plausible range; and
B. Calendar time (time to market), staff time and cost
Board decides to spend up to $150k and 80 person days

Prepare an overview of the

need for expert judgement

Key assumption
Board is comfortable with a 3 month timescale to conduct further analysis

Identify the personnel involved

and their roles

The internal actuary is to explore methods of improving the accuracy of the central
estimate and reducing the plausible range
The external consultant is to provide
The marketing team is to provide an expert opinion on

Set out brief for experts. Clarify and finalise the brief.
11 Nov 2016

Expert Judgement

22

Elicitation of expertise
Decide on approach

Elicit and consolidate

11 Nov 2016

I. In writing
II. Individual interview
III. Group interview no decision makers
IV. Group interview with decision makers
Other approaches viable
Elicitation manager:
Consolidates information
Highlights key areas of agreement / disagreement
Expert A: 40% remaining in-force
Expert B: 45% remaining in-force
Expert C: 50% remaining in-force

Expert Judgement

23

Decision making
Scrutiny and challenge

Further challenge by the decision makers.

Takes account of consistency with other judgements

Decision making

Need to avoid bias.

Clearly documented thought processes.
Capture in an expert judgement register.
Communication back to experts.

11 Nov 2016

Expert Judgement

24

On-going monitoring
Review

Review in 1 year

For example:
Triggers for non-scheduled
review

Experience on other products.

Significant increase in business volumes.
Formal guidance from the regulator.
Expert judgement register could facilitate.

11 Nov 2016

Expert Judgement

25

Validation

11 Nov 2016

Expert Judgement

26

Role of validation & validation process

Validation: required by Solvency II but wider applicability.
Judgement is hard to validate, but it can be done.
Key features of our proposed process aid validation:
Logical structure.
Clearly set out thought processes.

Validation tools can be used for expert judgement.

Consistency.
Use of industry benchmarking
but take care with potential systemic risks.

11 Nov 2016

Expert Judgement

27

Recap of process and conclusions

11 Nov 2016

Expert Judgement

28

Process summary
Proportionality
Elicitation of Expertise

Define the Problem

Brief for Expert

Decision Making Process

Final Decision

Documentation
Expert Judgement Register

Challenge
2nd Line Review

Higher Governance
Regulatory Oversight

Ongoing Monitoring

11 Nov 2016

Expert Judgement

29

Conclusions
Expert judgement is inherent in models: embrace it.
Recent regulatory changes have emphasised the need to have
transparent, evidence-based judgements.
Need a strong framework to ensure it is easy to manage:
Expert judgement policy.
Sensible governance structure.
Robust but proportionate process tailored to firms needs.
Backed up by good documentation, validation and monitoring.

11 Nov 2016

Expert Judgement

30

Questions

Comments

The views expressed in this presentation are those of the presenters rather than necessarily
being the views of the IFoA or the Bank of England.
Paper: https://www.actuaries.org.uk/documents/expert-judgement.
Contact details: roger.austin@aprllp.com / kieran.barnes@bankofengland.co.uk

11 Nov 2016

Expert Judgement

31