Improving Intelligence Analysis

This book on intelligence analysis written by intelligence expert Dr Stephen

Marrin argues that scholarship can play a valuable role in improving intelligence
analysis. It comprises new essays and some previously published material in
revised and expanded form.
Scholars who write about intelligence analysis build knowledge that is useful
for intelligence practitioners, but the knowledge only becomes useful if the gap
between them is bridged. This book is intended to help bridge the gap by provid-
ing a guided roadmap through the scholarship on mechanisms and methods for
improving intelligence analysis processes and products. A wide variety of poten-
tially useful ideas are addressed in this volume, including the nature of intelli-
gence analysis as an art and science and mechanisms to improve both, the
creation and operation of analytic teams, the development of training and educa-
tion programs, the exploitation of best practices from other fields such as medi-
cine, and the creation and promulgation of formal professional practices. It is
from ideas for improvement such as these that scholarship can have its greatest
impact on the practice of intelligence analysis.
The book will be of great interest to students of intelligence studies, strategic
studies, US national security, US foreign policy, security studies and political
science in general, as well as to intelligence analysts and managers.

Stephen Marrin is a Lecturer in the Centre for Intelligence and Security Studies
at Brunel University in London. He has a PhD in Foreign Affairs from the Uni-
versity of Virginia and is a former Intelligence Analyst in the CIA.
Studies in Intelligence Series
General Editors: Richard J. Aldrich
and
Christopher Andrew
British Military Intelligence in the
Palestine Campaign, 1914–1918
Yigal Sheffy
British Military Intelligence in the
Crimean War, 1854–1856
Stephen M. Harris
Allied and Axis Signals Intelligence
in World War II
Edited by David Alvarez
Knowing Your Friends
Intelligence Inside Alliances and
Coalitions from 1914 to the Cold War
Edited by Martin S. Alexander
Eternal Vigilance
50 years of the CIA
Edited by Rhodri Jeffreys-­Jones and
Christopher Andrew
Nothing Sacred
Nazi Espionage against the Vatican,
1939–1945
David Alvarez and Revd.
Robert A. Graham
Intelligence Investigations
How Ultra Changed History
Ralph Bennett
Intelligence Analysis and Assessment
Edited by David A. Charters,
Stuart Farson and Glenn P. Hastedt
Tet 1968
Understanding the Surprise
Ronnie E. Ford
Intelligence and Imperial Defence
British Intelligence and the Defence of
the Indian Empire 1904–1924
Richard J. Popplewell
Espionage
Past, Present, Future?
Edited by Wesley K. Wark
The Australian Security Intelligence
Organization
An Unofficial History
Frank Cain
Policing Politics
Security Intelligence and the Liberal
Democratic State
Peter Gill
From Information to Intrigue
Studies in Secret Service based on the
Swedish Experience, 1939–1945
C. G. McKay
Dieppe Revisited
A Documentary Investigation
John P. Campbell
More Instructions from the Centre
Christopher and Oleg Gordievsky
Controlling Intelligence
Edited by Glenn P. Hastedt
Spy Fiction, Spy Films, and Real
Intelligence
Edited by Wesley K. Wark
Security and Intelligence in a
Changing World
New Perspectives for the 1990s
Edited by A. Stuart Farson,
David Stafford and Wesley K. Wark
A Don at War
Sir David Hunt K.C.M.G., O.B.E.
(reprint)
Intelligence and Military Operations
Edited by Michael I. Handel
Leaders and Intelligence
Edited by Michael I. Handel
War, Strategy and Intelligence
Michael I. Handel
Strategic and Operational Deception
in the Second World War
Edited by Michael I. Handel
Codebreaker in the Far East
Alan Stripp
Intelligence for Peace
Edited by Hesi Carmel
Intelligence Services in the
Information Age
Michael Herman
Espionage and the Roots of the Cold
War
The Conspiratorial Heritage
David McKnight
Swedish Signal Intelligence
1900–1945
C.G. McKay and Bengt Beckman
The Norwegian Intelligence Service
1945–1970
Olav Riste
Secret Intelligence in the Twentieth
Century
Edited by Heike Bungert,
Jan G. Heitmann and Michael Wala
The CIA, the British Left and the
Cold War
Calling the Tune?
Hugh Wilford
Our Man in Yugoslavia
The Story of a Secret Service Operative
Sebastian Ritchie
Understanding Intelligence in the
Twenty-­First Century
Journeys in Shadows
Len Scott and Peter Jackson
MI6 and the Machinery of Spying
Philip H. J. Davies
Twenty-­First Century Intelligence
Edited by Wesley K. Wark
Intelligence and Strategy
Selected Essays
John Robert Ferris
The US Government, Citizen
Groups and the Cold War
The State–Private Network
Edited by Helen Laville and
Hugh Wilford
Peacekeeping Intelligence
New Players, Extended Boundaries
Edited by David Carment and
Martin Rudner
Special Operations Executive
A New Instrument of War
Edited by Mark Seaman
Mussolini’s Propaganda Abroad
Subversion in the Mediterranean and
the Middle East, 1935–1940
Manuela A. Williams
The Politics and Strategy of
Clandestine War
Special Operations Executive,
1940–1946
Edited by Neville Wylie
Britain’s Secret War against Japan,
1937–1945
Douglas Ford
US Covert Operations and Cold
War Strategy
Truman, Secret Warfare and the CIA,
1945–53
Sarah-­Jane Corke
Stasi
Shield and Sword of the Party
John C. Schmeidel
Military Intelligence and the Arab
Revolt
The First Modern Intelligence War
Polly A. Mohs
Exploring Intelligence Archives
Enquiries into the Secret State
Edited by R. Gerald Hughes,
Peter Jackson and Len Scott
US National Security, Intelligence
and Democracy
The Church Committee and the War
on Terror
Edited by Russell A. Miller
Intelligence Theory
Key Questions and Debates
Edited by Peter Gill, Stephen Marrin
and Mark Phythian
East German Foreign Intelligence
Myth, Reality and Controversy
Edited by Thomas Wegener Friis,
Kristie Macrakis and
Helmut Müller-Enbergs
Intelligence Cooperation and the
War on Terror
Anglo-­American Security Relations
after 9/11
Adam D.M. Svendsen
A History of the Egyptian
Intelligence Service
A History of the Mukhabarat,
1910–2009
Owen L. Sirrs
The South African Intelligence
Services
From Apartheid to Democracy,
1948–2005
Kevin A. O’Brien
International Intelligence
Cooperation and Accountability
Edited by Hans Born, Ian Leigh and
Aidan Wills
Improving Intelligence Analysis
Bridging the gap between scholarship
and practice
Stephen Marrin
Improving Intelligence
Analysis
Bridging the gap between scholarship and
practice

Stephen Marrin
First published 2011
by Routledge
2 Park Square, Milton Park, Abingdon, Oxon OX14 4RN
Simultaneously published in the USA and Canada
by Routledge
711 Third Avenue, New York, NY 10017
Routledge is an imprint of the Taylor & Francis Group, an informa business
© 2011 Stephen Marrin
The right of Stephen Marrin to be identified as author of this work has
been asserted by him in accordance with sections 77 and 78 of the
Copyright, Designs and Patents Act 1988.
All rights reserved. No part of this book may be reprinted or reproduced or
utilized in any form or by any electronic, mechanical, or other means, now
known or hereafter invented, including photocopying and recording, or in
any information storage or retrieval system, without permission in writing
from the publishers.
Trademark notice: Product or corporate names may be trademarks or
registered trademarks, and are used only for identification and explanation
without intent to infringe.
British Library Cataloguing in Publication Data
A catalogue record for this book is available from the British Library
Library of Congress Cataloging-­in-Publication Data
A catalog record has been requested for this book
ISBN: 978-0-415-78068-1 (hbk)
ISBN: 978-0-203-81020-0 (ebk)
Typeset in Times
by Wearset Ltd, Boldon, Tyne and Wear
Contents

Preface viii

  1 Bridging the gap between scholarship and practice 1

  2 Describing intelligence analysis 9

  3 Improving the science of intelligence analysis 21

  4 Improving the art of intelligence analysis 37

  5 Improving intelligence analysis with analytic teams 53

  6 Improving intelligence analysis through training and education 77

  7 Using analogies to improve intelligence analysis 99

  8 Improving intelligence analysis as a profession 127

  9 The importance of scholarship to practice 147

Notes 151
Select bibliography 174
Index 178
Preface

Academic scholars and intelligence analysts are members of the same family;
perhaps distant cousins. The production of academic scholarship and the produc-
tion of finished intelligence analysis is similar, relying on the individual’s ability
to acquire information (research), evaluate that information (think), and com-
municate findings (write and speak). Yet at the same time there is a significant
difference between scholars and analysts.
I know about both the similarities and differences firsthand. In 1996, after
graduating from Colgate University with a bachelor’s degree in political science,
I began working at CIA as an analyst. I was hired based on my demonstrated
academic strengths, but at CIA discovered subtle but important differences
between the two fields. Academia exists to develop knowledge, and places a lot
of emphasis on underlying causal forces and their effects. The same cannot be
said about intelligence organizations, which are in the business of creating useful
knowledge. While they need to have some understanding of underlying causal
forces, or drivers, they are more interested in communicating the implications or
significance of events than devoting much time or energy to building causal
models. That, after all, is the role of academicians.
Some practitioners are able to find positions where they can productively
employ the skillset of the academician in pursuit of organizational goals, but I
was more fascinated by studying the practitioners themselves. As an undergradu-
ate at Colgate University, I wrote a senior thesis on the causes of intelligence
failure and an honors thesis on the future roles and missions of the US intelli-
gence community. Then later, while working at CIA, I also became fascinated
by the analytic process itself, including methods and practices. So I left CIA to
pursue graduate and doctoral studies at the University of Virginia, acquiring a
MA degree from the Department of Politics in 2002 with a thesis on CIA’s train-
ing program for new analysts, and a PhD in 2009 with a dissertation evaluating
the intersection of analysis and decision.
As a student of intelligence analysis for almost 20 years now, I have looked
at, studied, and written about intelligence analysis from many different perspec-
tives. This book pulls together much of the thinking and writing I’ve done over
the past ten years on the methods and processes of intelligence analysis and how
they can be improved.
 Preface   ix
More than half the content of this book is original, with the other half relying
on earlier writings that have been revised and updated. Specifically, Chapters 1,
2, and 9 have not been published elsewhere. Most of Chapters 3 and 4 on the art
and science of intelligence analysis are original, but a small subset of material
was drawn from “Adding Value to the Intelligence Product” in Handbook of
Intelligence Studies, edited by Loch Johnson (Routledge, 2006), and
“Intelligence Analysis: Structured Methods or Intuition?” as published in the
American Intelligence Journal, Vol. 25, No. 1, (Summer 2007). Most of Chapter
5 on the value of analytic teams is original, but a section of it was previously
published as a chapter titled “Improving CIA Analysis by Overcoming Institu-
tional Obstacles” in Bringing Intelligence About: Practitioners Reflect on Best
Practices, edited by Russell Swenson (Joint Military Intelligence College’s
Center for Strategic Intelligence Research, May 2003). Additionally, sections of
Chapters 6, 7 and 8 were drawn from articles published in the International
Journal of Intelligence and Counterintelligence; specifically “Training and
Educating US Intelligence Analysts,” Vol. 22, No. 1. (Winter 2008–2009);
“CIA’s Kent School: Improving Training for New Analysts,” Vol. 16, No. 4
(Winter 2003/2004); “Improving Intelligence Analysis by Looking to the
Medical Profession,” Vol. 18, No. 4. (2005); and “Modeling an Intelligence
Analysis Profession on Medicine,” Vol. 19, No. 4. (Winter 2006–2007).
I would like to thank each of the publishers for permission to reprint portions
of these previously published articles. In addition, I would also like to thank
Mercyhurst College for a research grant providing additional time to research
and write the book, as well as the able research assistance of Aleksandra Bielska
whose diligent work was much appreciated.
Finally, as a requirement related to my prior employment at CIA, this book
has been reviewed by the CIA to prevent the disclosure of classified information.
Nothing in the contents should be construed as asserting or implying US
Government authentication of information or Agency endorsement of the
author’s views.
1 Bridging the gap between
scholarship and practice

Improving intelligence analysis requires bridging the gap between scholarship

and practice. Compared to the more established academic disciplines of political
science and international relations, the intelligence studies scholarship is
generally quite relevant to practice. Yet a substantial gap exists nonetheless.
Even though there are many intelligence analysts, very few of them are aware of
the various writings on intelligence analysis which could help them improve
their own processes and products. If the gap between scholarship and practice is
to be bridged, practitioners would be able to access and exploit the literature in
order to acquire new ways to think about, frame, conceptualize, and improve the
analytic process and the resulting product.

What knowledge do intelligence analysts need?

Intelligence analysis involves the interpretation of information about the adver-
sary or environment for purposes of assisting decisionmaking. There are many
different kinds of intelligence analysts in many different disciplines, from civil-
ian national security to military to law enforcement to business. As a result,
intelligence analysis as a professional discipline has practitioners all across the
world, some of whom join professional intelligence analysis-­related associations
and explore the nature of the discipline in conferences and workshops, contrib-
ute to the growing dedicated literature on intelligence analysis, and even take
college and university courses dedicated exclusively to exploring the nature and
practice of intelligence analysis.
In order to successfully achieve their purpose, intelligence analysts need to
possess both subject matter knowledge related to their specific analytic focus—
the kind of knowledge necessary to describe, explain, evaluate, and forecast the
actions of the adversary or the environment—as well as process knowledge
related to exactly how to do the work of analysis. To acquire subject matter
knowledge useful for improving intelligence analysis, one might look to area
studies, comparative politics, international relations, and other subject matter
disciplines. But if one wants to acquire knowledge on the processes, concepts,
and context for understanding and improving intelligence analysis, one would
look to the intelligence studies literature.
2   Bridging the scholarship/practice gap
Intelligence studies as a field of knowledge is about intelligence itself, and the
portion of it that overlaps analysis specializes in building up the kind of process
knowledge that intelligence analysts require to do their jobs successfully. This
knowledge includes how intelligence is collected, analyzed, processed, and dis-
tributed, all within group, organizational, cultural, and national contexts.
This knowledge can then be used by practitioners to improve their perform-
ance. Doing this well requires being self-­conscious about what the analytic
process involves: the management, organization, and processes related to the
performance of intelligence analysis. It also requires the creation, interpretation,
and exploitation of knowledge about intelligence analysis. While practitioners
can and do develop this kind of knowledge,1 they do so primarily on an ad hoc,
opportunistic basis and do not have the same kind of infrastructure for growing
knowledge about process that exists in academia. As a result, the emphasis on
process knowledge related to intelligence analysis should lead one automatically
to the world of scholarly literatures.

Scholarship can provide that knowledge

Despite the significant potential benefit that scholarship on intelligence analysis
can provide the practitioner, it has been portrayed as scarce and therefore diffi-
cult to find and exploit. Roger George and James Bruce, two former CIA officers
and the editors of a recent book on intelligence analysis, report that “as of 2007,
the body of scholarly writing on intelligence analysis remains . . . surprisingly
thin.”2 In addition, Amy Zegart, a professor at the University of California at Los
Angeles, has argued that intelligence studies has been ignored by academia,
citing its failure to teach courses and publish works on intelligence.3 But is this
really the case? Is the literature as scarce as these experts portray?
Fortunately, the state of the intelligence studies literature in general and the
intelligence analysis literature specifically is not as bleak as these authors
describe. When one surveys the extensive intelligence studies literature in all its
variety over the 60 plus years of its existence, the literature can appear to be
quite large indeed.4 Through books, journal articles, conference papers, and the
like, it is difficult for any scholar to master the large literature on intelligence
analysis, which is growing larger every day. This ever-­expanding literature
makes up the body of knowledge in the field, and is the knowledge base that
practitioners can exploit to answer the questions necessary to improve analytic
practice.
This literature has significant potential value for practice because, unlike
other kinds of scholarship, the intelligence studies scholarship is much closer to
practice than it is to theory. In other words, the literature itself is generally
applied in nature. The questions asked generally address real-­world, practitioner-
­oriented problems and the answers generally provide real-­world, practitioner-­
oriented solutions.
Levels of abstraction employed in the intelligence literature are minimal, and
even what little intelligence and intelligence analysis theory exists is primarily
 Bridging the scholarship/practice gap   3
derivative of an effort on the part of intelligence practitioners to understand and
explain the intelligence function. Specifically, “in 2005, the Office of the Direc-
tor of National Intelligence (ODNI) in partnership with the RAND Corporation
convened a one-­day workshop . . . to discuss how theories underlie our intelli-
gence work and might lead to a better understanding of intelligence.”5 A byprod-
uct of this workshop included four subsequent panels at International Studies
Association conferences all addressing intelligence theory,6 as well as the book
Intelligence Theory: Key Questions and Debates.7 In other words, intelligence
theory has not been developed separate from practice in an academic ivory
tower, but rather hand-­in-hand with practice in a form of theory/practice
partnership.
As for how all of this relates to intelligence analysis, this means that embed-
ded in the scholarship are answers to questions that relate to the kinds of process
knowledge that would be most effective in improving intelligence analysis. An
example might be the effort to develop a general theory of intelligence, and
explain how it could help shape a more specific theory of intelligence analysis.8
The scholarly literature on the methods and processes of intelligence analysis
begins in the 1940s with George Pettee, Sherman Kent, and Willmoore Kendall,
is continued in the 1950s by Roger Hilsman and Washington Platt, is deepened
in the 1960s by Klaus Knorr and Sherman Kent, then in the 1970s is extended
directly into analytic methods by Richards Heuer and others. By the 1980s, the
scholarly literature on intelligence analysis had become well-­established, and
even though coverage was relatively spotty the kinds of questions asked were
increasing in variety. In the 1990s and into the 2000s, particularly due to increas-
ing openness after the end of the Cold War, the literature on intelligence analysis
exploded in terms of variety and quality of contributions to scholarship.
A careful reading of the accumulated literature on intelligence analysis would
provide a practitioner with a substantial amount of useful knowledge, including
criteria and metrics for evaluating analytic quality, best practices in terms of
employing analytic methods to support inferences and judgments, guidance for
how to develop core competencies necessary for the production of quality intel-
ligence analysis, an understanding of the significance of organizational structures
and processes in the development and aggregation of different kinds of analytic
expertise in a team or unit context, and so on.
Unfortunately, despite the potential benefit for the practitioner inherent in this
literature, a significant gap between scholarship and practice prevents the poten-
tial value of the former for the latter from being realized.

Gap between scholarship and practice

Intelligence analysts should be able to benefit from the scholarship on intelligence
analysis that is produced, but many—perhaps most—do not. While both scholars
and practitioners evaluate information, there is a significant functional differentia-
tion that separates the two fields, as well as a resulting personality differentiation
of the people who choose to go into them. Scholars are contemplative and con-
4   Bridging the scholarship/practice gap
ceptual because their primary mission is to understand and explain: to maximize
the growth of knowledge. Relative to scholars, intelligence practitioners, even
intelligence analysts, are focused on getting the job done.
As a result of the differences between the worlds of academia and practice,
intelligence analysts have a tendency to roll their eyes at the word “literature” as
the touchstone of the irrelevant academic. In some cases they have a point.
Sometimes theory is overly removed from the day-­to-day issues of the practi-
tioner, and while it can help in the process of understanding and explaining big
issues it may not have much relevance for the working level practitioner
immersed in the highly detailed minutiae involved in interpreting raw
intelligence.
In addition, the culture of the intelligence field is infused with a distinct ori-
entation that the best way to learn is not by reading but by doing; that somehow
academia and its emphasis on developing, documenting, and disseminating
knowledge is a lesser form of learning and that the only real knowledge worth
having is that gained by experience. But what is experience other than a trial and
error process of learning from your own mistakes? In order to gain a lot of
experience, you have to make a lot of mistakes. On its face, that seems to be a
pretty ineffective way to learn.
Another way to learn is from the experiences and mistakes of others. And
where do you find those lessons? By reading what others have written, in the
literature. While some literatures—such as parts of the international relations lit-
eratures–are deeply theoretical and derivative of an ivory tower completely sepa-
rated from the average intelligence practitioner, others have direct applicability
to practitioners and the intelligence studies literature is one of those.
In the intelligence field, much of the literature consists of lessons derived
from the experiences of practitioners who then became academicians, or acade-
micians who became practitioners, and do what academicians are good at: devel-
oping, documenting, and disseminating knowledge consisting of their
understandings and explanations for intelligence practice and its improvement.
Their goal is not to create abstractions for the purposes of navel-­gazing, but
rather to conceptualize the function of intelligence in such a way as to make its
practice easier to manage and improve. The literature thus provides the mechan-
ism for knowledge transmission to change from verbal lore to a more structured
process of formalized education that enables current and future practitioners to
learn from the mistakes of those who came before them.9
For the practitioner, the literature on intelligence analysis should be viewed
as a knowledge repository, where they can go to find out what people have
learned about intelligence analysis thus far. Yet the pernicious effects of func-
tional differentiation—even when complementary and geared to achieve the
same goals—frequently leads to competition, rivalry, and “us versus them”
thinking that progresses to a form of stereotyping.10
When scholars and practitioners get together, sometimes there is collaboration
and sometimes there is competition. In terms of the latter, on multiple occasions
it has been suggested that those scholars who do not have previous experience as
 Bridging the scholarship/practice gap   5
intelligence professionals should not teach intelligence studies because they
bring very little useful knowledge to the table, with the obvious implication that
experience in the field of intelligence trumps knowledge about intelligence.
To use one of these discussions as an illustration, in February 2008 a contribu-
tor to the electronic mailing list of the International Association for Intelligence
Education (IAFIE) suggested that practical experience in intelligence was the key
criterion for being considered an intelligence expert and that all others who did not
have practical experience in the field—to include scholars since “studying and cri-
tiquing alone do not count”—were the equivalent to “back seat drivers.”11 In
response, Geneva College professor Thomas Copeland suggested that such a defi-
nition was overly limiting, and that while experience in the practice of intelligence
is “vital,” he believes that “at least a few academics have had some useful things to
say” about the practice of intelligence.12 He also pointed out that if scholars cannot
be considered experts on intelligence issues, then intelligence analysts cannot be
considered experts on international issues or situations that they have not experi-
enced first hand, such as terrorism or drug trafficking.13
Interestingly, Copeland’s defense of the scholar in the face of practitioner
preference for “real world” experience addresses exactly the same challenges
that intelligence analysts have faced vis-­à-vis policymakers’ questions about
their value. For much of their history, intelligence analysts have been perceived
by policy officials as irrelevant ivory tower academics with book knowledge but
no real sense of how policy was made and no real world experience. As
described by Roger Hilsman in the early 1950s, these policy officials “distrust”
the intelligence analyst, seeing him or her

as a long-­haired academic, poring over musty books in dusty libraries far

from the realities of practical life. On the other hand, they think that
“practical” experience is the true—in fact, the only—path to knowledge and
judgment.14

If the value of knowledge produced by intelligence analysts has value to the

policymaker, then the value of knowledge produced by the scholar may have the
same kind of value for the intelligence analyst. The failure of practitioners to
listen to the knowledge available to them regardless of source may prevent them
from performing at peak levels of efficiency and effectiveness.
But how many practitioners are aware of the accumulated knowledge and
wisdom available to them in the scholarship? The answer appears to be “not
many” because of the presence of the gap that exists between scholars and prac-
titioners and the general failure on the part of both to bridge that gap.

Negative consequences resulting from the gap

The following illustrations of the negative consequences that can result from the
gap between scholarship and practice are drawn from personal experiences as an
analyst at CIA. Before arriving at CIA, I studied the causes of intelligence failure
6   Bridging the scholarship/practice gap
and read the writings of scholars like Robert Jervis, Richard Betts, Michael
Handel, Art Hulnick, and Loch Johnson. Then I went to work at CIA as an
analyst, and quickly discovered that the best practices described in the scholar-
ship were not being followed by the practitioners.
For example, as a junior analyst I was asked to explain why a particular situ-
ation was occurring overseas. Because there was no actual evidence about that
particular situation, I started the paper with the phrase “We do not know why”
the situation is occurring, and then proceeded to supply three plausible explana-
tions given the intelligence available. I was subsequently told by my manager
that the first line had to be deleted, because CIA analysts never admit when they
do not know something. Yet the literature on intelligence analysis that I was
aware of clearly indicated that decisionmakers had an interest in knowing not
only what analysts know, but also what they do not know. This is the core idea
behind the Colin Powell-­derived statement “Tell ’em what you know; tell ’em
what you don’t know; and tell ’em what you think,”15 but is also embedded in
longtime CIA analytic methodologist Jack Davis’ writings and interviews from
the 1990s. If the manager had known that the scholarship indicated it is import-
ant to keep decisionmakers informed of gaps in knowledge or information, then
he might not have requested that the opening sentence be deleted.
In addition, that same manager then asked me to look at the three explana-
tions I provided, and choose the one that had the strongest support behind it and
say that “we (as in CIA) believe” that this one explanation was the reason for
what was happening overseas. Unfortunately, we really had no reason to believe
that explanation was any better or more accurate than the others; it was just the
one explanation with the most solid (or perhaps more accurately, least flimsy)
evidentiary base. If the manager had been aware of the scholarship on intelli-
gence analysis, he might have realized that multiple interpretations from a set of
data are possible, and that decisionmakers frequently want to know alternative
interpretations as well as the analyst’s judgment or “best guess.” Unfortunately,
the analytic office that I was in at that time had a strong ethic of putting forth a
single analytic line rather than multiple interpretations, thus demonstrating a gap
between scholarship and practice that had significant negative consequences on
the quality of the resulting analytic product.
As a final illustration, sometimes analytic processes not informed by scholar-
ship can lead to suboptimal outcomes from the policymaker’s perspective as
well. There was at one point an analytic dispute over the CIA’s analysis regard-
ing a particular foreign political situation. However, because the analytic office
promoted only one single analytic line, one side in the dispute was chosen as the
CIA’s official position, and the other side in the dispute was not. Thus far there
is nothing unusual about this case; these kinds of analytic disputes occur all the
time. What makes this case different, however, is that the analyst who lost the
fight was a subject matter expert who had greater subject matter knowledge of
the country in question as well as greater foreign language ability than the ana-
lysts who won the fight. It was precisely because of this additional expertise that
the losing analyst had come to a new position which was considered an outlier in
 Bridging the scholarship/practice gap   7
the office, but his interpretation did not win in the bureaucratic contest to be the
CIA’s view on that issue. Then, six months later, this outlier judgment proved to
be correct when the event forecast by the loser took place, surprising decision-
makers who then, unhappy at being surprised, described the event as an intelli-
gence failure; a term subsequently used on the front pages of major newspapers
across the country.
Here is the problem: the literature indicates that a significant value of intelli-
gence analysis is not just “making the call” or even providing multiple possible
interpretations for it should they exist, but also explanations for why those inter-
pretations differed. This is the value of dissent; the clash of opinions that fre-
quently leads to insight once the reason for the different opinions becomes clear.
In the end, even though a CIA analyst had produced an accurate estimate of what
was going to happen overseas, senior decisionmakers were unhappy with the
result because the CIA office in question failed to follow best practices described
in the literature and instead promoted only a single analytic line.
Because of these incidents, and a series of others related to apparent disregard
for analytic expertise, disinterest in trying to identify ground truth rather than
just report what was in the raw intelligence, and so on, I became a disillusioned
analyst in relatively short order. This disillusionment occurred not because I did
not see value in the enterprise, but rather because I believed the function of intel-
ligence analysis was too important to allow these kinds of flawed practices to be
embedded within analytic processes. As someone who arrived at CIA aware of
the literature and its efforts to document best practices from the past, it was frus-
trating to witness intelligence failures because those best practices were not
implemented, and in-­depth subject matter expertise was ignored. The following
chapters, therefore, rely on and draw from the intelligence literature to identify
analytic best practices, using as a foundation the observations and insights of
many other intelligence practitioners and scholars.
The core premise of this book is that intelligence scholarship can provide
knowledge and insight useful for the analytic practitioner; so useful, in fact, that it
will help improve the quality of the resulting intelligence analysis. The scholar’s
knowledge and insight comes from familiarity with the literature, making up the
accumulated body of knowledge that exists about intelligence analysis; and the
scholar’s ability to mine that body of knowledge, finding patterns and insight by
combining its contributions in different ways, and by organizing and reorgan­
izing the content embedded within it using different kinds of prisms or frames of
reference in doing so.
Each of the following chapters provides an answer to a particular question or
set of questions that has relevance to the intelligence analysis practitioner in
terms of improving performance over time:

• To provide the foundation for subsequent discussions, what is it that intelli-

gence analysts do exactly?
• Is intelligence analysis a science or an art? If it is a science, what methods
are the best to use in analyzing intelligence and how can they be improved?
8   Bridging the scholarship/practice gap
If intelligence analysis is an art, what does that mean and how exactly can
the art of analysis be improved?
• How might the use of analytic teams integrate different kinds of expertise as
a way to improve the quality of the analytic product?
• To what extent can training and education be used to improve intelligence
analysis?
• Do better practices exist in other fields that can be used to improve intelli-
gence analysis and, if so, what are they?
• Can intelligence analysis be improved systemically across the board through
a professionalization process, and, if so, how?

The answers to all of these questions have potential relevance for the practice
and improvement of intelligence analysis, but the answers derived here are not
drawn from experience alone but rather experience combined with knowledge as
derived from the intelligence studies literature. In other words, the goal here is to
see what we can learn from others, including those who came before us.
2 Describing intelligence analysis

Before we can work to improve intelligence analysis, we first need to establish

what it involves and how well it has been practiced. Unfortunately, while there
are a lot of general descriptions of the analytic process, there is very little
detailed scholarship that describes exactly who analysts are and exactly what
they do on a daily basis.
There are many different kinds of intelligence analysts who do different kinds
of analysis. Some kinds of analysts can be differentiated based on source of
information evaluated: single-­source analysts evaluating signals intelligence
(SIGINT), imagery intelligence (IMINT), human intelligence (HUMINT), and
open source intelligence (OSINT). Then there are all-­source analysts who assess
information from all sources at the same time. In addition, the substantive
subject matter that the analyst focuses on can be different. Some are responsible
for a defined geographic area (for example, individual countries or entire
regions), while others are responsible for particular kinds of threats or groups.
Another difference could be the prism used to evaluate the threat. For example,
the standard differentiation between different kinds of power–politics, eco-
nomics, military, and leadership—can provide a framework for analysts to break
down (or actually analyze) a country or group before then evaluating its
significance.
Finally, analysts can also be differentiated based on the kinds of problems
they address. On one end of the scale there are tactical intelligence analysts pro-
ducing current intelligence, who work primarily with descriptive data that
address the who, the what, the where, the when and sometimes the how. On the
other end of the scale there are strategic intelligence analysts who frequently
address motivation or causation, or the why. The tactical analyst requires a
facility with tangible data, pattern recognition, and frequently the need to “make
the call,” whereas the strategic analyst requires greater facility with concepts,
abstractions, uncertainty, and ambiguity. The strategic analyst requires an
ability to critically evaluate a situation, assess it for significance, match the
assessment of significance against either decisionmaker interest or national
interest, reframe or re-­conceptualize the situation as needed, and construct an
argument about that significance using whatever information, including raw
intelligence, is available.
10   Describing intelligence analysis
Unfortunately the broader community of practitioners and scholars has not
effectively distinguished one kind of analysis from another. To illustrate this
point, there is a saying that Native American Inuit (Eskimos) have many differ-
ent words for snow, perhaps because the different kinds of snow really matter to
them.1 The difference between dry hard snow and wet heavy snow may affect
their ability to travel, or build igloos, or hunt, and that can affect their daily lives.
But we do not have nearly as many words for snow; instead we have only a
handful. Snow, sleet, hail, slush. Other than that, we do not have any more words
for snow, probably because the different kinds of snow do not matter as much to
us. We get in our cars and drive through it, or shovel it out of our way. But we
do not live in it in the same way the Inuit do, and because of that fail to take note
of the differences in our communications.
Here is where the analogy has relevance: there are different kinds of intelli-
gence analysis, but we do not yet have a vocabulary to distinguish one kind from
another. Previously, being able to distinguish one kind of analysis from another
did not matter. All different kinds of intelligence analysis involved assessing
information and communicating that assessment to a decisionmaker, and the
analyst learned how to do it primarily through on-­the-job training. But as the dif-
ferent kinds of intelligence analysts proliferate among many different organiza-
tions, substantive domains and countries at the same time that the field is
developing more sophisticated analytical tools and training programs, we are
now at a point where failure to differentiate between different kinds of analysts
will lead to inefficiencies in terms of analytic processes and analytic training. In
order to maximize effectiveness, some effort must be devoted to studying and
evaluating the analytic competencies and skillsets that lie at the core of all intel-
ligence analysis, as well as the differences between analysts and analysis derived
from the varied tasks they are asked to accomplish.
In terms of core similarities, most intelligence analysts share a certain set of
characteristics.2 The basic job of all analysts involves research, reading, think-
ing, writing, and briefing. Analysts are first and foremost thinkers and evalua-
tors, who communicate their findings in both written reports and verbal
communications. In order to have sufficient content to think about, they also
should be proficient researchers, equipped with a variety of analytic tools appro-
priate to the kind of analysis that they do. They should also be proficient ana-
lysts. According to Ronald Garst and Max Gross, both former administrators for
the then-­Joint Military Intelligence College, analysis

is the ability to see patterns. In a world of information overload, the mark of

the capable analyst is the ability to separate the wheat from the chaff, the
important from the less important or even the trivial, and to conceptualize a
degree of order out of apparent chaos.3

Garst and Gross go on to say that “armed with knowledge of the area or topic, and
in possession of a mass of information, the analyst must make sense out of informa-
tion that is often fragmentary, ambiguous, contradictory, and subject to deception.”4