Nuevas Capacidades SQL Server 2014

lvaro Olivares
Consultor Microsoft Consulting Services

2014

2014

2014

2012

2012

Table Constructs
Benefits

High performance data operations Main-Memory Optimized

Optimized for in-memory data Indexes (hash and range) exist only in memory No buffer pool Stream-based storage for durability

Fixed schema no ALTER TABLE, must drop/recreate/reload No LOB datatypes; row size limited 8060 No constraints support (PK only) No Identity or Calculated columns, CLR etc

In-Memory OLTP Tech Pillars

Data and table size considerations

Size of tables = (row size * # of rows) Size of hash index = (bucket_count * 8-bytes) Max size SCHEMA_AND_DATA = 512 GB

IO for Durability
SCHEMA_ONLY vs. SCHEMA_AND_DATA Memory Optimized Filegroup Data and Delta files Transaction Log Database Recovery

Drivers

Hardware trends
Steadily declining memory price, NVRAM

CREATE TABLE [Customer]( [CustomerID] INT NOT NULL PRIMARY KEY NONCLUSTERED HASH WITH (BUCKET_COUNT = 1000000), [Name] NVARCHAR(250) NOT NULL Hash Index INDEX [IName] HASH WITH (BUCKET_COUNT = 1000000), [CustomerSince] DATETIME NULL Secondary Indexes ) are specified inline WITH (MEMORY_OPTIMIZED = ON, DURABILITY = SCHEMA_AND_DATA);

This table is memory optimized

This table is durable

CREATE TABLE DDL Table code generated Compiler invoked Table DLL produced Table DLL loaded

Benefits

Frictionless scaleup

Impact of no locks or latches

Write-Write Conflict: Design Application for condition with try.catch Applications dependent on locking: May not be a good candidate Can take an application lock

In-Memory OLTP Tech Pillars

High Concurrency
Multi-version optimistic concurrency control with full ACID support Core engine uses lockfree algorithms No lock manager, latches or spinlocks

Multiple versions of records

Increases the memory needed by memory optimized tables Garbage Collection used to reclaim old versions

Drivers

Hardware Trends
Many-core processors

Transaction scope
Support for Isolation Levels: Snapshot, Repeatable Read, Serializable Commit time validation again have retry logic to deal with failure
11

Multi-version

Optimistic

Time
1 2 3

Transaction T1 (SNAPSHOT)
BEGIN

Transaction T2 (SNAPSHOT)
BEGIN UPDATE t SET c1=bla WHERE c2=123

UPDATE t SET c1=bla WHERE c2=123 (write conflict)

First writer wins

Native compiled stored procedures

Benefits

Non-native compilation
No different than T-SQL calls in SQL Server today Easier app migration as can still access Memory Optimized (MO) tables All Objects. Access for transactions across MO and b-tree tables T-SQL surface area (limit on MO interaction) Statistics updates can be utilized to modify plan at runtime Ad-hoc query patterns 14

Efficient, business-logic processing

Performance

High. Significantly less instructions to go through Application changes development overhead Can only interact with Memory Optimized tables Limited. Statistics utilized at CREATE -> Compile time Limited (e.g., no ALTER procedure, compile-time isolation level)

Migration strategy
In-Memory OLTP Tech Pillars

T-SQL compiled to machine code

T-SQL compiled to machine code via C code generator and VC Invoking a procedure is just a DLL entry-point Aggressive optimizations @ compile-time

Access to objects

Support for T-SQL constructs Optimization/stats and query plan Flexibility

Drivers

Hardware trends
Stalling CPU clock rate

 Interpreted T-SQL access Access both memory- and diskbased tables Less performant Virtually full T-SQL surface area When to use Ad hoc queries Reporting-style queries Speeding up app migration

Natively compiled stored

procedures

Access only memory optimized tables Maximum performance Limited T-SQL surface area

When to use OLTP-style operations Optimize performance critical business logic

Client App

No improvements in communication stack, parameter passing, result set generation 10-30x more efficient

TDS Handler and Session Management In-Memory OLTP Compiler Natively Compiled SPs and Schema Query Interop Buffer Pool for Tables & Indexes SQL Server.exe Parser, Catalog, Algebrizer, Optimizer Proc/Plan cache for ad-hoc TSQL and SPs

Key Existing SQL Component In-Memory OLTP Component Generated .dll

Interpreter for TSQL, query plans, expressions Access Methods

Reduced log bandwidth & contention. Log latency remains

Checkpoints are background sequential IO

In-Memory OLTP Engine for Memory_optimized Tables & Indexes

Memory-optimized Table Filegroup

Transaction Log

Data Filegroup

SQL Server engine

New high-performance, memory-optimized online transaction processing (OLTP) engine integrated into SQL Server and architected for modern hardware trends

Memory-optimized table file group

Transaction log

Data file group

 Whats being delivered:

Main benefits

Data Stored Row-Wise: Heaps, b-Trees, Key-Value

By using SQL Server 2012 In-Memory ColumnStore, we were able to extract about 100 million records in 2 or 3 seconds versus the 30 minutes required previously.
- Atsuo Nakajima Asst Director, Bank of Nagoya

In-memory In the Data Warehouse

In-Memory ColumnStore Both memory and disk Built-in to core RDBMS engine

Customer Benefits:

C C C 1 2 3
Columnstore Index Representation

10-100x faster
Reduced design effort Work on customers existing hardware

C C C6 4 5

Easy upgrade; Easy deployment

21

Traditional Storage Models

Data stored row-wise: Heaps, b-trees, key-value

Relational, dimensional, map reduce

In-Memory ColumnStore Storage Model

Data stored Column-wise

Each page stores data from a single

column Highly compressed

More data fits in memory

C1

C2

C3

C4

C5

C6

Each column can be accessed

independently

Fetch only columns needed Can dramatically decrease I/O

Row Groups & Segments

A segment contains values for one

Segments
C1 C2 C3 C4 C5 C6

column for a set of rows Segments for the same set of rows comprise a row group Segments are compressed Each segment stored in a separate LOB Segment is unit of transfer between disk and memory

Row group

Processing an Example

Row Groups

Segments

Compress Each Segment*

Some Compress More than Others

*Encoding and reordering not shown

Segment Elimination

Segment Elimination

Batch Mode
Improving CPU Utilization

Biggest advancement in query processing in years! Data moves in batch through query plan operators Highly efficient algorithms Better parallelism

QP Vector Operators

Batch object Column vectors

bitmap of qualifying rows

Clustered & Updatable

101 Million Row T able + Index Space

[VALUE]GB

[VALUE]GB [VALUE]GB [VALUE]GB [VALUE]GB [VALUE]GB TABLE WITH CUSTOMARY INDEXING TABLE WITH CUSTOMARY INDEXING (PAGE COMPRESSION) TABLE WITH NO INDEXING TABLE WITH NO INDEXING (PAGE COMPRESSION) TABLE WITH COLUMNSTORE INDEX CLUSTERED COLUMNSTORE

How It Works

CREATE CLUSTERED COLUMNSTORE

Organizes and compresses data into ColumnStore

Partition
ColumnStore Deleted Bitmap

BULK INSERT
Creates new ColumnStore Row Groups

INSERT
Rows are placed in the Row Store (heap) When Row Store is big enough, a new ColumnStore Row Group is created

DELETE
Rows are marked in the Deleted Bitmap

UPDATE
Delete plus insert

Most data is in ColumnStore format

Row Store

Whats New?

SQL Server 2014

Support for all flavors of JOINs
OUTER JOIN Semi-join: IN, NOT IN

UNION ALL Scalar aggregates Mixed mode plans Improvements in bitmaps, spill support,

Whats New?

Adds an additional layer of compression on top of

the inherent compression used by columnstore Shrink on-disk database sizes by up to 27%
during rebuild

Compression applies per partition and can be set either during index creation or

Insert & Updating Data

Load Sizes

Bulk insert
Creates row groups of 1Million rows, last row group is probably not full But if <100K rows, will be left in Row Store

Insert/Update
Collects rows in Row Store

Tuple Mover
When Row Store reaches 1Million rows, convert to a ColumnStore Row Group Runs every 5 minutes by default Started explicitly by ALTER INDEX <name> ON <table> REORGANIZE

ColumnStore Enhancements Summary

Whats being delivered: Clustered & Updateable In-Memory Columnstore ColumnStore archive option for data compression Global batch aggregation Main benefits: Real-time super-fast data warehouse engine Ability to continue queries while updating without the need to drop and recreate index or partition switching Huge disk space saving due to compression Ability to compress data 5X 15X using archival per-partition compression Better performance and more efficient (less memory) batch query processing using batch mode rather than row mode

 Whats being delivered: Usage of non-volatile drives (SSD) to extend buffer pool NUMA aware Large page and BUF array allocation Main benefits: BP Extension for SSDs
Improve OLTP query performance with no application changes No risk of data loss (using clean pages only) Easy configuration optimized for OLTP workloads on commodity servers (32GB RAM) Example: ALTER SERVER CONFIGURATION SET BUFFER POOL EXTENSION ON (FILENAME = 'F:\SSDCACHE\EXAMPLE.BPE, SIZE = 50 GB)

Scalability improvements for systems with >8 sockets

Query Tree

Command

Cmd Parser
Query Plan

TDS TDS

Optimizer

Query Result Sets Execut or

SNI

Relational Engine
Data

Protocol Layer
Plan Cache

Results

Transaction Log
GetPage

Access Methods Transactio n Manager Data Files

Cached Pages

D
Data Cache

Buffer Manager Buffer Pool Read I/O

Storage Engine
Write I/O

 www.yammer.com/serversmschile
Nuestra comunidad de It Pros & Desarrolladores Microsoft Chile

Tech Series
Fecha Jueves 8 Agosto Hora Temas 18:30-20:30 Nuevas capacidades de SQL Server 2014 Azure SQL Reporting: Escenarios de uso para ambientes Jueves 22 Agosoto 18:30-20:30 de BI 100% Cloud. Arquitectura de integracin de aplicaciones hibridas con Martes 27 Agosto 18:30-20:30 Azure Biztalk Services Windows Server 2012 R2: Revisin tcnica del nuevo Martes 3 Sept. 18:30-20:30 release. Performance, xVelocity, Indexing - SQL Server 2012 Jueves 5 Sept. 18:30-20:30 Best Practices Monitoreo de Servicios en la nube con System Center Martes 10 Sept. 18:30-20:30 Operations Manager Redes en Windows Azure: VPN, Traffic Manager, etc... Jueves 26 Sept. 18:30-20:30 para qu sirve cada uno y cmo se consume. Remote Desktop Services: Escenarios de uso para Marte 1 Octubre. 18:30-20:30 ambientes de escritorio centralizados/virtualizados.