SQL Server Transaction

Log Internals
Denzil Ribeiro
 About me
Denzil Ribeiro
Sr. Dedicated Premier Field Engineer at Microsoft
Prior Escalation Engineer @ CTS
SQL PFE Blog
Session Assumptions & Considerations

• Wish we had more than an hour…

• Today we will focus on:

• Transaction Logging Components
• Logging and Recovery
 A Brief Transaction Log
Introduction…
• Captures changes that occur in the database
• Changes always logged here first, later are persisted in
the data files
• Implements Atomicity & Durability ACID properties
• Written to sequentially, usually read sequentially
• You ONLY need one of these
 Transaction Log File Structure
• The physical Log file is composed of VLFs
• Log Blocks contain Log Records

VLF’s comprising of Log Blocks

Version File Seq No is a unique 4 byte

File Seq No monotonically increasing number
Size assigned to each log file.

Log File Header

 Log Blocks
• The unit of physical commit to a log file
noRecords, blockSize,
• Contains a Header, Log Records, and a Slot
prevBlockSize, ...
Array
Record 1
• Each block ranges in size from 512 bytes to
60K Record 2

Record 3
 Log Records

• An atomic database change

o Uniquely identified by a Log Sequence Number (LSN)

• Not only associated with committed transactions

o Contains information to Redo or Undo a transaction
o Generated irrespective of recovery model
 Log Record Structure Varying Num of
Elemets
Ex: Lock
PageID Slot Alloc Prev collection
Unit Page
LSN

Log Record Header Page Log Structure Row Log Structure Variable Log Data
Optional Optional Optional
For Update of Rows

Leng LSN Flags XDES Log OP Context

LOP_FORMAT_PAGE,
th LOP_INSERT_ROWS,
LOP_DELETE_ROWS,
LOP_BEGIN_XACT,
LOP_Expunge_Rows
 Demo

Fn_dblog & fn_dump_dblog

Dropped tables
 Log Sequence Number
• Uniquely identifies a Log Record
• Monotonically increasing
• VLF Number: Log Block Offset: Slot Number
o Each time a VLF is reused, it is given a new file sequence no
 Virtual Log Files (VLF)
• The unit of truncation of the transaction log file
• Generated at log file creation or log file growth
Log File Growth VLFs Added

<= 64MB 4

> 64 MB AND <= 1 GB 8

> 1 GB 16
How many VLFs do you need?
• It Depends!
• Too many can lengthen database restore time
• Too few may make it difficult to truncate the log
• We now alert in SQL 2012 if you have too many!
 VLF Reuse
• A VLF can be reused if:
o No active transactions are contained in the VLF or previous VLFs
o No “un-replicated” transactions are contained in the VLF

• Occurs at Checkpoint in the Simple Recovery Model

• Otherwise, ONLY occurs at log backup

 Demo

VIRTUAL LOG FILES

The circular nature of the Log file
VLF Fragmentation
 Log Flushes
Current Buffer

Current Buffer

Current Buffer
 Transaction Log Limits
• Size
o SQL 2012/SQL 2008 - limit 3840K
o SQL 2005 - limit of 480K at any given time
• Number of Outstanding IOs
o SQL 2012 – 112 outstanding IO’s ( 64 bit)
o Prior - 32 outstanding IO ( 64 bit), 8 on 32 bit
• Once Limits hit, we will wait for outstanding IO to
complete before issuing any new IO.
o Will wait on LOGMGR_FLUSH
 Example Limits ( SQL 2008)
• Size Limit : 3840K
• (9.6MB Log Bytes Flushed/sec)/(5000 Log Flushes/sec) = 1.9K per flush
• (32 Outstanding I/O’s)*(1.9K per Flush) = ~60K “in-flight” at any given time (far
below the 3840K limit)

WRITELOG
• Waiting on IO to Log file, disk
can’t keep up or a slow disk
• Check Log disk counters
• Check sys.dm_io_pending_ios
Log Wait Types
LOGBUFFER
• Waiting on a free log buffer, all
log buffers in use, most often
due to them not being flushed
fast enough.
LOGMGR_RESERVE_APPEND
• Tried to grow the log file but
couldn’t, waiting on log
truncation. Check
sys.databases
log_reuse_wait_desc
 Demo

Log Flushing
 Delayed Durability (SQL 2014)
ALTER DATABASE SET DELAYED_DURABILITY
={DISABLED|ALLOWED|FORCED}
• Disabled – Normal behavior durability guaranteed.
• Allowed – Allowed at the DB Level, Transaction has to specify durability
options, default is a durable transaction.
• COMMIT TRAN……. WITH (DELAYED_DURABILITY=ON)
• FORCED – Changes default durability for the DB to “delayed”. Can be useful
for Applications bottlenecked on Log IO, that can tolerate some Data loss on a
failure.

• Durability automatically managed by System on Log cache filling up.

• Can be forced with a durable Transaction or by sp_flush_log manually.
• Recoverability point now becomes DurableLSN
 Checkpoint
Minimizes the REDO portion of recovery

Writes modified (dirty) buffer pages to disk (data file)

o Writes ALL dirty pages to disk – transaction state does not
matter
• Can be very IO intensive

• Does not remove pages from the buffer pool

o Lazy Writer does this when there is pressure
 Checkpoint Options
• Use ‘recovery interval’ to set at the instance level
• In SQL Server 2012, can set at the database level
o TARGET_RECOVERY_TIME db option
Checkpoint IO
 Minimally Logged Operations
• For some operations, SQL only tracks that the event
happened
o Typically only allocation level changes (PFS, GAM, SGAM, etc)
•
Can make operations faster than in full recovery
o Caveat: Data pages flushed to the data file prior to transaction commit

• Likely makes log growth smaller, but NOT log backups

 Demo

Minimal Logging
Checkpoint Logging
 Phases of Recovery

1. Analysis 2. Redo 3. Undo

• Read the log and find • Replays all committed • Rolls back active
active transactions but un-reflected transactions
and dirty pages transactions • Starts at oldest active
• Usually starts at last • Database is available transaction
checkpoint in Enterprise Edition
• Finds where to start at this point
the Redo process
 Demo

Database Recovery
 Why is my log file getting so big?
• Very common problem with customers
• Adding a log file on another drive is sometimes needed
• Use log_reuse_wait_desc field in sys.databases for more info

log_reuse_wait_desc
Nothing Database Mirroring
Resolution: Checkpoint Replication

• Fix underlying problem Log backup Database snapshot

creation
• Shrink transaction log
• Grow log to appropriate size Active backup or restore Log Scan
Active Transaction Other
Questions?