application of scaling performance
An
Introduction
to NoSQL
Data
Management
for Big Data
by David Loshin |
September 4, 2013
3:24 pm | 1
Comments
Editors note: This big data storage
article is part of a series
examining issues
related to evaluating
and implementing big
data analytics in
business.
As with any emerging
yet complex application
development
framework, successful
implementation relies
on an ecosystem of
different components
that can be combined
to address the
development of the
appropriate solution.
For big data, that
ecosystem revolves
around some key
architectural artifacts,
including scalable
storage, parallel
computing, and data
management
paradigms that are
united through an
development platform.
And while it is
on a big data appliance.
However, many
important to review the
ways these different
algorithms that expect
to take advantage of a
components are
blended together, from
high-performance,
elastic, distributed data
the perspective of
application
environment are not
suited to consume data
development, the in traditional RDBMS
interdependence systems. That means
between analytic developers must
algorithms and the consider different
underlying data methods for data
management management.
framework warrants a
more in-depth review of These analytic
algorithms can employ
paradigms. one of an array of
alternative means for
Related Stories data management that
The evolution of the are typically bundled
enterprise data warehouse
under the term NoSQL
starring Hadoop.
databases. The term
Read the story
How Pig, Hive and NoSQL conveys two
Zookeeper build apps on different concepts. The
Hadoop and MapReduce.
first suggests a data
Read the story
management
Sentiment analysis tool
designed to predict framework that is not a
veterans suicide risk. SQL-compliant one. The
Read the story second more generally
Why more data and simple
acknowledged (that is,
algorithms beat complex
more frequently
analytical models.
Read the story presented) meaning is
The reason is that the term stands for
straightforward: when Not only SQL,
applications already suggesting
depend on a traditional environments that
relational database combine traditional SQL
(RBDMS) model and/or (or SQL-like query
a data warehousing languages) with
approach to data alternative means of
management, it may be querying and access.
sufficient to port the Schema-less
RDBMS tools as a way Models: Increasing
Flexibility for Data intended to allow A key-value store is a
Manipulation different models to be schema-less NoSQL
NoSQL data systems adapted to specific model in which data
provide a more relaxed types of analyses. For objects are associated
approach to data example, some are with distinct character
modeling often referred implemented as key- strings called keys,
to as schema-less value stores, which similar to the data
modeling, in which the nicely align to big data structure known as
semantics of the data programming models a hash table. Many of
are embedded within a like MapReduce. the NoSQL architectures
flexible connection Although the relaxed rely on variations on the
topology and a approach to modeling key-value theme, in
corresponding storage and management paves that unique keys are
model. This provides the way for employed to both
greater flexibility for performance identify entities and to
managing large data improvements for locate attribute
sets while analytical applications, information about those
simultaneously it does not enforce entities. This pervasive
reducing the adherence to strictly- use of unique keys
dependence on the defined structures, and lends a degree of
more formal database the models themselves credibility to this basic
structure imposed by do not necessarily approach to a schema-
the relational database impose any validity less model.
systems. rules. This potentially As an example,
The flexible model introduces risks consider the data
enables automatic associated with subset represented in
distribution of data and ungoverned data Table 1, where
elasticity with respect management activities the key is the name of
to the use of such as inadvertent the automobile make,
computing, storage, inconsistent data while the value is a list
and network bandwidth replication, of names of models
in ways that dont force reinterpretation of associated with that
specific binding of data semantics, and automobile make.
to be persistently currency and timeliness Table 1: Example data represented in key-value
stored in particular issues. This article
physical locations. discusses four different Key Value
NoSQL databases also NoSQL approaches:

provide for integrated
data caching that helps Key-value stores BMW {1-Series, 3-Series

reduce data access Document stores GT, 7-Series, X3,

latency and speed Tabular stores Buick {Enclave, LaCrosse

Object stores
performance. Cadillac {CTS, DTS, Escala
Escalade EXT, SRX,
The loosening of the
Key-Value Stores
relational structure is
 consider the (such as the number of
representations of times specific phrases
The key-value store
those values and how occur within massive
does not impose any
they are to be linked to numbers of documents)
constraints about data
the key. and for producing those
typing or data
results for reports, the
structure. It is the
Key-value stores are model does pose some
responsibility of the
essentially long, thin potential drawbacks.
consuming business
tables, and can be One weakness is that
applications to interpret
indexed by key value to the model will not
the semantics of the
speed data queries (in inherently provide any
data organization.
that there are not many kind of traditional
columns associated database capabilities
The core operations
with each row). The (such as atomicity of
performed on a key-
tables rows can be transactions, or
value store include:
sorted by the key value consistency when
Get(key), which to simplify finding the multiple transactions
returns the value key during a query. A are executed
associated with the query essentially simultaneously). Those
provided key. comprises two steps: capabilities must be
Put(key, value),
the first step is to provided by the
which associates
the value with calculate the unique application itself.
the key. key, and the second is
Multi-get(key1, to use that key as an Another potential
key2, .., keyN), index into the table. weakness: as the
which returns the Because of the need to volume of data
list of values
calculate the key to increases, maintaining
associated with the
list of keys. access any information unique values as keys
Delete(key), about the entity, it is may become more
which removes the difficult to expect to difficult; addressing this
entry for execute general SQL- issue requires the
the key from the style queries such as introduction of some
data store.
what are the most complexity in
popular models of cars generating character
based on sales? These strings that will remain
When using a key-value
kinds of questions unique among an
store, ensuring that the
would typically be extremely large set of
values can be accessed
answered using code, keys. For example, a
means that the key
as opposed to a query global company may
must be unique. To
engine. attempt to manage
associate multiple
data associated with
values with a single key
While key-value pairs millions of customers,
(such as the list of car
are very useful for both many of whom sharing
models in the example
storing the results of the same or similar
in Table 1), the
analytical algorithms names. Duplication in
developer must
the set of names will
mean that the name
itself will be insufficient
when used to
differentiate different
entities. The upshot is
that additional data
attributes will need to
be added to the
composed character
string to be used to
generate a unique key.
Document Stores
A document store is
similar to a key-value
store in that stored
objects are associated
(and therefore accessed
via) character-string
keys. The difference is
that the values being
stored, which are
referred to as
documents, provide
some structure and
encoding of the
managed data. There
are different common,
standard encodings,
including XML
(Extensible Markup
Language), JSON (Java
Script Object Notation),
BSON (which is a binary
encoding of JSON
objects). Aside from
these standard
approaches to
packaging data, other
means of linearizing the
data values associated
with a data record or
object for the purposes
of storage may be with stored content,
employed. which essentially
Figure 1 shows provides a way to query
examples of data the data based on the
values collected as a contents. For example,
document using the example in
representing the names Figure 1, one could
of specific retail stores. search for all
Note that while the documents in which
three examples all MallLocation is
represent locations, the Wheaton Mall that
representative models would deliver a result
are different. The set containing all
document documents associated
representation embeds with any Retail Store
the structure of the that is in that particular
model, allowing the shopping mall.
meanings of the
document values to be Tabular Stores
inferred by the Tabular, or table-based
application. stores are largely
descended from
Googles original
BigTable design to
manage structured
data. Hadoops HBase
model is an example of
a NoSQL data
management system
that evolved from
BigTable. (For
background on BigTable
design, see this
paper via Googles
research website. )
The tabular model
allows sparse data to be
stored in a three-
Figure 1: Example of
dimensional table that
document store.
is indexed by a row key
One key distinction
(that is used in a
between a key-value
fashion that is similar to
store and a document
the key-value and
store is that the latter
document stores), a
embeds attribute
column key that
metadata associated
indicates the specific
attribute for which a
data value is stored,
and a timestamp that
may refer to the time at
which the rows column
value was stored.
As an example, various
attributes of a Web
page can be associated
with the Web pages
URL: the HTML content
of the page, URLs of
other web pages that
link to this Web page,
the author of the
content. Columns in a
BigTable model are
grouped together as
families and the Analytics
timestamps enable
management of
multiple versions of an
object. The timestamp
can be used to maintain
historyeach time the
content changes, new
column affiliations can
be created with the
timestamp of the when
the content was
downloaded.
More Articles in This
Series
Market and Business
Drivers for Big Data
Analytics
To best understand what
big data can mean to technology that promises
your organization, start by
understanding the
conditions that has led to
its growing acceptance. In
this article, the first in a
series, David Loshin
explains the economic
drivers that make new
analytics applications worth
evaluating given todays
exploding data volumes,
and the technology
innovations that make such
systems more accessible to
more companies.
Business Problems Suited
to Big Data Analytics
Enterprises need clear
processes for determining
the value proposition of a
big data analytics project.
In this article, David Loshin
examines the applications
that make sense for these
projects and the criteria
that enterprises should use
to weigh the costs and
benefits of such a strategic
investment.
Achieving Organizational
Alignment for Big Data
on an ecosystem of
Numerous aspects of big
data analytics hold appeal,
and while individuals within
an organization can test
drive them, these new
technologies need to win
adoption in a broader
enterprise setting.
Managers need to answer:
What is the process for
piloting technologies to
determine their feasibility
and business value? And:
What must happen to bring
big data analytics into
organizations system
development lifecycle?
Developing a Strategy for
Integrating Big Data
Analytics into the
Enterprise
As with any innovative
involved and how
business value, there is a
rush to embrace big data
analytics as a key source of
business value. This article
explains how to consider
the challenges and issues
involved in bringing big
data analytics into
production.
Data Governance for Big
Data Analytics:
Considerations for Data
Policies and Processes
With emerging big data use
cases, datasets created for
one purpose can be used
for an entirely different
purposea dynamic that
challenges traditional
approaches to data
governance. This article
explores ways to manage
this conflict and build new
governance policies.
Considerations for Storage,
Appliances and NoSQL
Systems for Big Data
Analytics Management
Big data management and
analytics applications rely
components that can be
combined in a variety of
ways to address application
requirements. This article
examines three aspects of
this ecosystem and
associated technologies:
storage, appliances, and
data management.
An Introduction to Big Data
Application Development
and MapReduce
For any target big data
platform, you must have an
application development
framework that supports a
system development
lifecycle and provides a
means for loading and
executing the developed
application. This article
discusses the principles
programmers use the
MapReduce and ECL
frameworks to analyze big
datasets.
Understanding the Big Data
Stack: Hadoops Distributed
File System
Hadoop is a collection of
open source projects,
combined to enable a
software-based big data
appliance. This article
introduces a core aspect of
Hadoops utilities, the
Hadoop Distributed File
System.
How Pig, Hive and
Zookeeper Build Apps on
Hadoop and MapReduce
This article examines the
prototypical big data
platform using Hadoop, and
how Pig, Hive, HBase,
Zookeeper and Mahout
address these pieces of the
puzzle.
Object Data Stores
Object data stores are
essentially a hybrid
approach to data
storage and
management; in some
ways, object data stores
and object databases
seem to bridge the
worlds of schema-less
data management and
the traditional relational
models. On the one
hand, approaches to
object databases can
be similar to document
stores except that while
the document stores
explicitly serialize the
object so the data
values are stored as
strings, object
databases maintain the
object structures. That
is because they are
bound to object-
oriented programming
languages such as C+
+, Objective-C, Java,
and Smalltalk.
As opposed to some of
the other NoSQL
models , object
database management
systems are more likely
to provide traditional
ACID compliance (that
is Atomicity,
Consistency, Isolation,
and Durability)
characteristics that are
bound to database
reliability. Yet this is one
of the few similarities to
a traditional relational
database, and it is
important to remember
that object databases
are not relational
databases and are not
queried using SQL.
Considerations for
Implementing NoSQL
The decision to use a
NoSQL data store
instead of a relational
model must be aligned
with business users
expectations. The key
question: How will the
performance of a
NoSQL data store
compare to their
experiences using
relational models?
As should be apparent,
many NoSQL data
management
environments are
engineered for two key
criteria:
Fast accessibility,
whether that
means inserting
data into the model
or pulling it out via
some query or
access method, and
Scalability for
volume, so as to
support the
accumulation and
management of
massive amounts
of data.
Both of these criteria
are addressed through
distribution and
parallelization, and the
NoSQL styles described
above are amenable to
extensibility, scalability,
and distribution.
Moreover, these
characteristic features
dovetail with
programming models
like MapReduce that
effectively manage the
creation and running of
multiple parallel
execution threads. The
key is leveraging data
distribution.
Fortunately, distributing
a tabular data store or a
key-value store allows
many queries and data
accesses to be
performed
simultaneously,
especially when the
hashing of the keys
maps to different data
storage nodes. NoSQL
methods are designed
for high performance trying out the different not completely replace
computing for reporting approaches, and it may a relational database
and analysis, and smart make sense to develop management system.
data allocation a simple pilot project Choosing to use NoSQL
strategies will enable model that can be is not necessarily an
linear performance deployed in different easy decision. One
scalability in relation to ways to explore the must weigh the
data volume. similarities and business requirements
differences in terms of as well as the skills
There are many new ease-of-use, space needed to transition
companies who have performance, and from a traditional
embraced the different execution speed. approach to a NoSQL
NoSQL models and are approach before
bringing their Yet while the committing to the
customized versions to performance behaviors technology.
market. If you are for NoSQL data
interested in NoSQL, management systems
there is little risk in are appealing, they will