FEATURE

Big data: an information

security context
Conrad Constantine, AlienVault

Conrad
Constantine

It looks like big data is here to stay. When it first emerged as the next big
thing a few years ago, it didnt take too long for the information security
industry to realise it had applications within the field and quickly it was being
pitched as yet another silver bullet solution.
We love parroting the line that silver
bullets dont exist in infosecurity and
yet every time something new and
shiny comes along, excitement trumps
reason every time. The information
security field has always suffered from
a very special form of hubris that
feeling that somehow our problems
are so unique to us that no other
field could possibly have encountered
anything of the scope or scale of
intrinsic complexity and innumerate
factors determining the outcome of
any action. Yet here we are, welcoming
in a new age of mathematically-driven
analysis of our data.
And theres the rub. Information
security people, by and large, are not
good at mathematics, data modelling or
programming. Infosecurity has become
the new hotness for people looking to
go into university for something that
will get them a guaranteed career with
lots of money. The hand-wringing
among old hands over the transition
of our field from craft to trade can fill
volumes. Infosecurity rookies come
fresh from university with a smattering
of familiarity with core concepts and
skills, into a field that demands mastery
of them all.

We put people fresh out of

a two-year technical security
degree into front-line defence
positions for the worlds largest
corporations and wonder why
the news is full of stories of
major breaches
A decade ago our problem was the
lack of skilled penetration testers, a
18

Network Security

problem we no longer face breaking

into systems has become a rather
deterministic skill that takes a minute
to learn, a lifetime to master and yet
the defensive side of thing presents an
obliquely different learning curve.

Time on the streets

A skilled police detective will point to
their time on the streets, learning all
the things that only direct experience
with the public and the criminal mind
can teach. No matter how extensive
the courses at the academy are, they
can only present information, not the
understanding and empathic ability to
read between the lines that experience
brings. As any good police drama will
emphasise acceptance to the homicide
division only comes after an officer
has worked in every other area of the
departments operations beforehand.
And yet, every day we put people
fresh out of a two-year technical
security degree into front-line defence
positions for the worlds largest
corporations and wonder why the news
is full of stories of major breaches that
went unnoticed by these security teams
for months. You cant protect what you
dont understand after all, and with the
massive influx of academy rookies into
the field, should we be so surprised
when its so difficult to find those
people with the 10,000 hours widely
held to be required for mastery?
In a field like network defence, where
the attacker only has to be correct
once, but a defender must be correct
every time, mastery is an unfortunate
prerequisite to effectiveness.

Big queries
But lets bring this back around to
big data an easily digestible name
for the emergence of commodity
software designed to allow synchronous
N-dimensional analytics quite
the mouthful to anyone without a
background specialising in the data
sciences. Data has always been big: an
intrinsic side-effect of Moores law can
be expressed as utilisation will always
expand to fill capacity. No, the real
nature of big data is big queries the
ability to ask questions of our data that
have been computationally unfeasible
before.
Ask anyone working frontline security
operations and analysis weve had big
data for years terabytes of logs we need
to sift through to find that single log
entry that delivers the smoking gun to
us. And well regale you with stories of
waiting hours, days even, for that search
to return results. If big data were nothing
more than a leap beyond isometric
increases in the speed of querying our
vast repositories of data in accordance to
their volume, the average security analyst
would be quite happy with that.

The convergence of data

science with security analytics
was not an overnight event,
more so because it was not
a creation of the information
security world
And yet, big data becomes the next
big thing a critical re-evaluation and
re-tooling of our analytical abilities.
This is not about being able to query
more data but being able to query
all data; beyond being able to grep
through log data faster, is the ability to
distil everything we have ever recorded

January 2014

FEATURES/NEWS
technical skills that are only hastily
covered in the current educational tracks
for infosecurity.
If security big data is going to do
more than keep buzzword-pace with
the rest of the technology world, it will
inevitably draw upon prior expertise
from other fields. True, they will have
to acquire some of the experience and
domain knowledge of the security field
a task that may be far less challenging
to people with a background in data
science than for our current crop of
security graduates to replicate in reverse.

If this is our new normal, the core

technology that drives all workflow
and action how are we going to
address that in education, training
and certification? Information security
expertise requires experience and
competence across a wide variety of
information technology domains, yet
how will we address the incursion of
a skill so few of us are qualified with
beyond cursory familiarity, only to find
ourselves exclaiming: Help, a data
scientist took my security job!?

Information security
expertise requires experience
and competence across a
wide variety of information
technology domains

Big data can achieve nothing by itself,

it is merely an engine to enable the
asking of better questions questions
that arise only through experience
with real world data. To express those
questions programmatically from big
data systems requires a certain set of

The hubris of the infosecurity field

to believe it deals with entirely unique
and unsolvable problems may finally
see new light as other domains of
expertise come to accept that security
is everybodys problem. Information
security has matured after two
decades of relevance we should expect
nothing less but are we following suit
with it? Big data was not our creation,
and there exists far more talent for
asking the right questions from data,
outside of our field.

For Conrad Constantine, research

team engineer at AlienVault, an early
background in searching for forbidden
knowledge, pushing computing hardware
to its limits and a nose for the truth,
made for a perfect storm toward a career
in incident response, where, for over a
decade and a half, he has been on the
front lines of defence work in telecom,
medical and media corporations, not
least of which being at ground zero for
the 2011 RSA Breach. A firm believer
that incident response must become
an accessible and effective discipline
available to all, he works on bringing
the mysteries of open source intelligence
generation and defensive agility to those
willing to take the leap from fear to
action.

...Continued from page 3

Many people in the security industry
remain unconvinced by RSAs denials.
These include Mikko Hyppnen, chief
research officer at F-Secure, who recently
cancelled his planned presentation at this
years RSA conference. He was due to
give a talk on Governments as Malware
Authors.
Now several other researchers and
speakers have followed his lead. They
include: Jeffrey Carr, chief executive of
Taia Global; Josh Thomas of Atredis

Partners; Chris Palmer, a software

security engineer at Google; Adam
Langley, a Google cryptographer;
Chris Soghoian, principal technologist with the ACLUs Speech, Privacy
and Technology Project; Alex Fowler,
Mozillas global privacy and public
policy leader; and Marcia Hofmann, a
digital rights lawyer at the
There has been an attempt to remove
an NSA employee from an influential cryptographic standards body.
The Crypto Forum Research Group

(CFRG) is part of the Internet Research

Task Force (IRTF) and is co-chaired by
Kevin Igoe, who works for the NSA.
Some members of the group wanted
him to step down following his part in
the adoption of a weakened version of
the Dragonfly key exchange protocol.
This followed the revelation that the
NSA has been active in trying to promote flawed technologies in order that
it could develop backdoors in widely
accepted protocols and products.
Continued on page 20...

from our information systems, into

information pictures that no single
human mind could perceive from the
uninstalled source material.
And here is where the two observations
collide. The convergence of data science
with security analytics was not an
overnight event, more so because it
was not a creation of the information
security world to begin with. The path
of convergence first came with an
overlapping field fraud detection and
investigation where data analytics has
been a key driver for many years now
in identifying what constitutes normal
and abnormal patterns of activity. For
anyone who has ever found their debit
card locked out after a transaction they
consider normal, well theres the data
analytics in action, running into an edge
case. These algorithms are refined over
time, iteration by iteration, and their
designers learn to ask ever more elegant
questions about their datasets.

Better questions

January 2014

About the author

Network Security

19