A Practitioners Guide to Using ACEs to Build Hope

and Resilience

SenseMaker:
Evaluating Complex
Adaptive Systems
Keil Eggers
May 9. 2016

On Big Words
Will be using new vocabulary throughout
the presentation
If something is confusing, please stop and
ask
Learning this for the first time together!

2 | May 2016

Organizing a Party
This Lunch and
Learn is a
celebration
Blindfolded and
swinging
Lets break this
bad boy open.

3 | May 2016

We All Know the Story

Why are we doing
this?
Why cant we just
get back to the
work?
Reporting time vs.
Time spent on
programming.
4 | May 2016

What is Evaluation?
Why do we do it?
What outputs are we expecting from an
evaluation?
In what contexts does CPPR help with
evaluation and what do partners expect?

5 | May 2016

Where We Are
Drive to anticipate the future.
Fail-safe programs to achieve set
outcomes
Robustness: prediction and prevention of
negative outcomes
Hypothesis-based
6 | May 2016

What are those?!?

Logic Models

Resources
(Inputs)
People:
Trained

Lemonade for
Life Trainers
(TtT model)
Experienced

MIECHV
professionals

Knowledge:
Adverse

Childhood
Experiences
research
Hope &

resilience
research

7 | May 2016

Program
Components
(Activities)
Lemonade for Life:
Prerequisite

online ACEs
module
1 day training

Participant

resource
materials
(Handbook &
handouts)
Coaching

Lemonade for Life

Theory of Change
Outputs

Number of
professionals
trained in
Lemonade for
Life
Number of
families engaged
in meaningful
ACEs discussion

Intermediate
Outcomes

Professionals
have
hope/mindset
Increased family
engagement in
services
Improved
programmatic
outcomes

Long-Term
Outcomes

Strong & resilient

families
Fewer children
impacted by
ACEs
Hopeful
communities

Order vs. Complexity

Conformity carries risk
Order is not usability or adaptability.
Complexity values action
Good enough action
Learning and SenseMaking can be accurate
enough to move on

Irony of Bureaucratic control

The VISION
Anticipatory awareness
Understand the evolutionary potential of
the present

Safe-to-Fail experiments
amplify success
dampen failure

Resiliency to recover from surprises

9 | May 2016

SenseMaking
Understanding actions and their
consequences that emerge from efforts to
create order
Making sense from knowledge of the past
Generate future action

Continuous and evolving, not rigid like

annual planning to meet targets
10 | May
2016

Cynefin (Kun-evin)
The place of your multiple belongings,
religious, tribal, spiritual, geographical,
historical, etc that profoundly influence
what you are (individually and collectively),
but of which you are only ever partially
aware

11 | May
2016

What is SenseMaker?
Software developed by Dave Snowden
and Cognitive Edge
Based on complexity theory, ecology, and
distributed ethnography
Creates human sensor network through
capturing self-identified narrative
fragments

Features
Distributed cognition
Fine-grained material
Disintermediation- original material accessed
directly, not mediated by analysts to decision makers
Evidence-based hard and soft data for actionable
results
Identifies weak signals of hidden opportunities
Semi-constrained signification to not limit to
hypothesis
Mix of soft and hard data to make sense of complex
realities, to distill patterns and to respond in a timely
manner
13 | May
2016

The SenseMaker
Survey

How to Use SenseMaker

Ongoing Capture
Apple/Android mobile Application or paper
surveys entered on backend
4-5 minutes
Downloaded to central server for instant
analysis
15 | May
2016

Analysis
Clusters- what is important to look at
based on patterns
Vector analysis
Movement of stories over time

Fitness Mapping

17 | May
2016

Using the Fitness Map

Three filters on the triangle demonstrate
strength or weakness of the labels
Form the axes on the fitness map.
Visualization and statistical instruments allow
patterns to be detected in the metadata, and
once a pattern is detected, the ability to go to
the supporting narrative immediately and
without interpretation, enables more effective
decision making with respect to that pattern
(e.g., encourage or dampen)
18 | May
2016

Human Sensor Network

Micro-narratives
Mass capture (100 to e.g. 3000 stories)
show diversity of perspectives
Create an accurate description of reality
for program decision-making

Features of SenseMaker
Self-signification
Generates quantitative (visual) data
Patterns & trends against topics of
interest
Analysis, interpretation and use for
action/interventions
Mix of soft and hard data to make sense
of complex realities, to distill patterns and
to respond in a timely manner
20 | May
2016

Outputs
Fine-grained data
See clusters of stories, and read
individual narrative data points
Original material directly accessed by
decision makers
Explorer software for easy analytics
Tableau output- visual representation of
quantitative data

Self-Signification
The process of tagging narrative
Old Approach: Hierarchical taxonomy,
marking key words and assigning the
material to a category
Heavy work for analysis
But semantically, very difficult due to
cognitive science and the nature of
language
22 | May
2016

Why Micro-Narratives?
Not a constructed story to prove a point
Large amounts of fragmented narratives
Unstructured, anecdotal way of seeing
world
Tell specific anecdotes, self-signify what it
means.
You cant ask generally Does this work?
because sometimes it does, sometimes it
doesnt.
Continuous free capture of narrative with
instant feedback.
23 | May
2016

Surveys can be limiting

Importance of wide-net story capture
Multiple identities of the person
Gaming the system
The best we can do is learn, act, and
improve. Through continuous real-time
input.

24 | May
2016

SenseMaker and CPPRs

Values
The Center for Public Partnerships and
Research (CPPR) assists partners with
addressing complex social issues through
targeted research and evaluation as well
as comprehensive project management
and systems development.
Mission: To optimize the well-being of atrisk children, youth, and families by
generating responsive solutions that
improve practice, inform policy, and
advance knowledge.
25 | May
2016

The Partnership with

Cynefin Centre
The Cynefin Centre for Applied
Complexity at the University of Bangor in
Whales
Dave Snowden, founder of Cognitive Edge
Leader in complexity science and Knowledge
Management field

International Research Consortium

Programmatic research that we can
participate in
Additional partnerships possible- private
sector, governments, international grant

How do we go from
where we are to
where we want to
be?
27 | May
2016

What to do with the

data?
Share back with those telling stories
Responsive TA
Flexibility and Creativity

Thinking Big
Could change landscape of social service
reporting
LfL as a first experiment leading to wider
applications
Develop expertise in innovative narrative
evaluation

What story are we trying

to tell and who is telling
it?

Questions and
Discussion