A Te c h n i c a l J o u r n a l f o r t h e S Q L S e r v e r C e n t r a l .

c o m a n d PA S S C o m m u n i t i e s
January/February 2005

Approaching In This Issue:

Best SQL Server 2005 7

Yukon Features for DBA’s

By Brian Knight

What’s New in SQL 11

Integration Services
By Allen Mitchell and Darren Green

Paging Result Sets 14

Using Server 2005’s
ROW_NUMBER
By Joseph Sack

SQL Server 2005 16

Name Resolutions -
Owners & Schemas
By Wendy Sue Williams

The Next Evolution 18

of Management Objects
By Steve Jones

SQL Server Metadata 20

By Anthony Bressi

Yukon Encryption 22
By K. Brian Kelley

PASS
Introducing Database 31
Mirroring
By Rick Heiges, PASS Director of Membership

Strengthening 34
European
Presence in 2005
By Professional Association for SQL Server
“ Ihasneed to know how my database
been accessed and modified.

With Entegra, I have

a complete picture of
who did what when! ”
®
Lumigent Entegra TM
sets a whole
new standard for data integrity. It’s the only
comprehensive data auditing solution focused at
the data level — giving you greater visibility into
your database activity. It monitors data access
modifications, and changes in schema and
permissions — it even tells you who’s viewed
what data. Entegra gives you the insight you
need without the hassle of re-coding application
logic or developing and maintaining triggers.
With end-to-end management, collection, and
reporting capabilities, you’ll have a complete
picture of who did what when to your data.

With Entegra:

• Monitor and audit your database activity

without triggers or application modifications
• Automatically capture, store, and report
changes in permissions, schema, and
data modifications
• Audit SELECT statements to know who’s FREE White Paper
viewed your data
• Store all data in a single, centrally- Learn how you can benefit from monitoring
managed repository and auditing database activity without the
use of triggers and application modifications.
Download your free white paper today at
www.lumigent.com/go/sqlknow.

Copyright © 2004 Lumigent Technologies, Inc. All rights reserved. Lumigent, Entegra and the Lumigent Logo are trademarks or registered trademarks of Lumigent Technologies, Inc.
 Editorial

Approaching Yukon
A publication of The Central Publishing Group It’s now known as SQL Server 2005, though I
Managing Editor have to admit that I like Yukon much better. How
Susan Page interesting would it be if the code names were
the actual names of the products? Wishful
Technical Editors:
thinking, but I’d sure like to install SQL Server
Trey Johnson Mark McCorkindale
Randy Lee Anthony Virgil Yukon or SQL Server Shiloh or even SQL Server
Jeff Niblack Sean Lambert Sphinx rather than SQL Server 2000. It makes as
Typesetting, Layout and Printing:
much sense as the current naming. 2000 is v8,
Paramount Printing 2005 is v9?!??!!
Copy Editor This issue marks a change for SQL Server
Susan Page
Standard. Not only is this the start of our second
Advertising: year, but we’ve also made a change in the way
Advertising@sqlserverstandard.com
we build the magazine. When we started this last
Subscription & Address Change:
year, we looked at lots of layout ideas and
http://www.sqlserverstandard.com
designs. Since artistic ability isn’t really one of
Feedback:
editor@sqlserverstandard.com
our talents, we let someone else choose the look,
and then scrambled to get content for the feel. A
Coyright:
year of scrambling has left us looking like most
Unless otherwise noted, all programming code and articles
in this issue are the exclusive copyright of the Central
other magazines. A theme and an article or two
Publishing Group. Permission to photocopy for internal to support it, but mostly we’re filling space with
personal use is granted to the purchaser of this magazine. whatever interesting articles we can find. And
SQL Server Standard is an independant publication and is we’re trying to put something in each magazine
not affiliated with Microsoft Corporation, Microsoft for everyone.
Corporation is not responsible in any way for the editorial
policy or other contents of this publication. SQL Server, No longer. This year our goal, and an ambitious
ADO.NET, Windows, Windows NT, Windows 2000 and Visual one at that, is to dedicate each magazine to a
Studio are registered trademarks of Microsoft Corporation.
Rather than put a trademark symbol in each occurance of
theme and try to fill it with articles that support
other trademarked name, we state that we are using the that theme. As you might have guessed, January
names only in an editorial fashion with no intent of is devoted to Yukon and we have 6 articles, all on
infringement of the trademark. Although all reasonable that very topic to get you ready for the next
attempts are made to ensure accuracy, the publisher does release of SQL Server. And we’ve got similar
not assume any liability for errors or omissions anywhere in
this publication. It is the reader’s responsibility to ensure that
plans for the remainder of the year; each issue
the procedures are acceptable in the reader’s environment devoted to a topic: replication, performance, just
and that proper backup is created before implementing any to name a few.
procedures.
Hopefully you’ll enjoy it and find some value in it.
SQLServerCentral.com Staff:
Brian Knight, President We’re trying something new here at
Steve Jones, Chief Operating Officer SQLServerCentral.com and we’d love to have
Andy Warren, Chief Technology Officer some feedback on whether this meets your
needs, if you like it, any improvements you’d like
to see, or even those topics that interest you.
Design Layout and Printing by: Send a note to articles@sqlservercentral.com
and we’ll take a look and see what we can do.
- Steve Jones

904 448 1700 • 5299 St. Augustine Road, Jacksonville, FL • www.printparamount.com

JANUARY/ FEBRUARY, 2005

3
 SQLsuite
THE PROFESSIONAL DBA’S CHOICE

tools for
your SQL Server Enterprise

SQL diagnostic manager

Real-time performance monitoring and diagnostics

SQLsafe
Enterprise-scale backup and recovery

SQLconfig
Change and configuration management

SQLtool
Powerful Web-based administration

SQLschedule
Enterprise-class job management

DTx
Easy-to-use data transformation and migration

For free trials visit www.idera.com

or call toll free 877.GO.IDERA
 Table of Contents

A Te c h n i c a l J o u r n a l f o r t h e S Q L S e r v e r C e n t r a l . c o m a n d P A S S C o m m u n i t i e s

Best SQL Server 2005 Features for DBA’s 7

By Brian Knight

What’s New in SQL Integration Services 11

By Allen Mitchell and Darren Green

Paging Result Sets 14

Using Server 2005’s ROW_NUMBER
By Joseph Sack

SQL Server 2005 Name Resolutions - Owners & Schema 16

By Wendy Sue Williams

The Next Evolution of Management Objects 18

By Steve Jones

SQL Server Metadata 20

By Anthony Bressi

Yukon Encryption 22
By K. Brian Kelley

PASS
Introducing Database Mirroring 31
By Rick Heiges, PASS Director of Membership

Strengthening European Presence in 2005 34

By Professional Association for SQL Server

JANUARY/ FEBRUARY, 2005

 Best SQL Server 2005 Features
for DBA’s By Brian Knight

“ There are more new features than I can ever

mention in this article, so I’ll touch on the high points.
”
You’ve probably already read lots about the SQL Your central tool now for management is the
Server 2005’s programmability and business intelligence Microsoft SQL Server Management Studio (seen in
enhancements, but what’s in it for the DBA? In this article, Figure 2). This tool is best compared with Enterprise
I’ll show you a few of my favorite features in SQL Server Manager. Instead of using the MMC console that you
2005. As you can imagine from reading some of the other would see in Enterprise Manager though, Microsoft has
Yukon articles in this issue, the trainers are licking their chosen a new path with a light version of Visual Studio
chops right now with anticipation of training the new crop 2005. The environment is built for enterprise management
of SQL Server users since there will be such a learning of hundreds of databases on a single server. For example,
curve in this release of SQL Server. There are more new when you expand the Databases tree, the tool begins to
features than I can ever mention in this article, so I’ll touch query for the database list and you can move on to the
on the high points. I’ll dive deeper into each feature on next task while it finishes the retrieval. Traditionally in SQL
SQLServerCentral.com over the next few months. Server 2000, you would have to wait while the database
or table list completed retrieving. In large environments,
New Management Environment
this could take 5-10 minutes.
Everything you know about the management tools in
SQL Server 2000 can be thrown out in SQL Server 2005,
with the exception of Profiler (who has also received some
enhancements as well). One of the new minor tool
additions is SQL Computer Manager. This tool replaces
much of what you would previously do in the Services
applet in Control Panel and adds quite a bit more
functionality. For example, you can start and stop the
services and change the password that is used to start
them much like the Services applet. You can also change
the port that SQL Server is listening or create aliases here
much like the Client and Server Network Configuration
programs in SQL Server 2000. Essentially, it combines all
the old service and network management applications
Figure 2: Microsoft SQL Server Management Studio
into one central MMC interface.
Everything Online
One of the most important features for an active
database environment is keeping everything online. While
SQL Server 2000 goes a long way to doing this, you
cannot do a number of things online. SQL Server 2005
fixes that immensely by allowing you to restore databases
while users are still connected to the database, querying.
This option is available in Enterprise, Developer and
Figure 1: SQL Computer Manager Evaluation editions of SQL Server for file and file group

JANUARY/ FEBRUARY, 2005

7
restores. With this feature only the data that is being
restored at that point-in-time is unavailable. The rest of
the database is still accessible. This same type of feature
is also available for index rebuild operations and DBCC
commands. You will be able to rebuild a clustered index
while making active modifications to the data.
Data Partitioning
A common problem for DBA’s has been handling
large tables. A large table can be challenging to insert
large amounts of data into or delete chunks of data. Let’s
take a real-world example that most of us have
encountered in an environment that happens to come in
the AdventureWorks database.
You are a DBA who has a large transaction table
called TransactionHistory and you archive transactions
older than 2 years off to the TransactionHistoryArchive
table once a month. Since you have 40 million records in
the TransactionHistory table, it takes hours to copy the
millions of records from table to table, even with the
clustered index on TransactionDate. This would be a
perfect scenario for data partitioning.
You would have partitions on both the
TransactionHistory and TransactionHistoryArchive tables
on the TransactionDate column by month. Before you can
move the data using partitions, you must ensure two core
things: the schema between the two tables must match
and there must be an empty partition on the target table
to receive the data. Once you’re ready, you would issue a
series of ALTER TABLE commands to migrate the data
between the two tables. The thing that makes this run so
fast (instantaneously) is that you’re not moving the
physical data pages or rows. Instead, you’re only pointing
the meta data of the records for that date range to be
owned by the other table.
Database Mirroring and Snapshots
Database mirroring is one of the most anticipated
high-availability features in SQL Server 2005 and is
covered in much more detail later in this issue of SQL
Server Standard. It is much like log shipping, where the
transaction log is restored to a secondary site
automatically. In database mirroring, the secondary site
(called a mirror) receives the transaction log backups
from the primary site (called the principal). There is also a
third server called the witness that monitors the availability
of each server in the relationship. Optionally the witness
can be consolidated onto the primary or secondary server
but it’s not recommended since it creates a single point of
failure.
8
There are some key differences between database
mirroring and log shipping. Think of database mirroring as
an enhanced version of log shipping. In database
mirroring, if the principal site fails for whatever reason, the
mirror will immediately (within a second or two) take over
ownership of the data and bring itself out of perpetual
recovery mode. Thanks to a new revision in MDAC,
clients will then begin to redirect themselves over to the
mirrored site automatically. Traditional log shipping
required that you build this functionality into your
application and the failover was manual. Another
difference between this and log shipping is that you
cannot use the mirrored database for reporting as in log
shipping since it’s in recovery mode. There are some
workarounds for this that you can use by using snapshots.
A great benefit to database mirroring is that you can
have geographically dispersed sites in a partnership. For
example, my home office in Jacksonville can send the
data to our disaster recovery site in Chicago and all of my
applications will automatically move over as soon as I take
the database down for maintenance. One negative about
database mirror or log shipping is you will double your
space requirements since you are mirroring a database to
a different server.
Snapshots
A database snapshot as its name implies is a
snapshot of the database as a given point in time. A
snapshot is read-only and static. Once a snapshot is
created data page changes that have occurred to the
database since the change are tracked. Essentially, a
change log that contains the data page that was changed
and what the original data page looked like is kept in the
snapshot. Because of this, initially, the snapshot is quite
small, containing very little data. Over time, though, this
can cause a space problem for a server as more and more
data is updated in the database’s tables.
To create a snapshot in T-SQL, you can use the
CREATE DATABASE syntax as shown below:
CREATE DATABASE AdventureWorks_DecReport ON
( NAME = AdventureWorks_Data, FILENAME =
‘C:\Program Files\Microsoft SQL
Server\MSSQL.1\MSSQL\Data\AdventureWorks_DecRepo
rt.ss’ )
AS SNAPSHOT OF AdventureWorks;
GO
 After the snapshot is created, you can connect to it as
if it were a physical database. After you’re connected, you
can query it in the same manner that you do a physical
database with the exception of not being able to do
UPDATE, INSERT or DELETE statements. Any user who
has RESTORE permission can also use a snapshot as a
source for the restoration.
RESTORE DATABASE database FROM DATABASE_SNAPSHOT
= AdventureWorks_DecReport
To delete the snapshot, you must only issue a
standard DROP DATABASE command and specify the
snapshot name as shown here:
DROP DATABASE AdventureWorks_DecReport
Maintenance Plans
Maintenance plans in SQL Server 7.0 and 2000 were
a blessing for DBA’s who wanted the basic maintenance
to be performed on a regular basis for your server.
Maintenance plans to me always seem like a great feature
but lacked in some functionality to make it my dream tool.
For example, you couldn’t customize a maintenance plan
to meet your own standards. Maintenance plans in SQL
Server 2005 have gone the extra mile. When you start, it
asks you what type of tasks you’ll want to accomplish in
your maintenance plan (shown in Figure 3).
Figure 3: Maintenance plan configuration.=
After you answer a series of questions, it will create
the maintenance plan as a DTS package (shown in Figure
4). The nice thing about having the maintenance plan
created as a DTS package is it allows for customization.
For example, if I’d like to be notified if a backup fails but I
don’t necessarily care if the update statistics didn’t create,
JANUARY/ FEBRUARY, 2005
then I could customize the DTS package to do just that.
You can also have the package call another SQL Server
job after the maintenance is complete or execute T-SQL.
You can reorder the way your maintenance workflow
occurs also.
Figure 4: Maintenance plan as a dts package.=
Security Enhancements
This broad category represents a lot of the work that
was done in SQL Server 2005 for the administrator. The
world is a lot different place than it was 5 years ago, and
we must dead-bolt the database door locks to make sure
hackers stay out and that we’re compliant with new
regulations like Sarbanes-Oxley. For more information on
the security enhancements in SQL Server 2005, see Brian
Kelley’s article “First Look at Yukon Security” in the
January 2004 issue of SQL Server Standard. Here are the
highlights, though.
An important change that will help SQL Server
comply with new government regulations is the ability for
the DBA to force the user to change his password every
so many days based on a server policy. A user can
potentially get locked out of the SQL Server if he does not
change his password within the set amount of days.
A constant nagging frustration for DBA’s is how SQL
Server 2000 binds users to schemas. In SQL Server 2000,
if your user bknight creates an object called History, the
table may be called bknight.History. In this case though,
you cannot remove the bknight user until you change the
object owner. By doing this, though, you reset the
permissions on the object, which creates a DBA
headache.
In SQL Server 2005, schemas can be unbound for the
user. You can create a schema, for example, called
9
Accounting and then a table that is owned
by the schema may be called
Accounting.History. The nice thing about
this implementation is that a database role
can own the schema so you’re not tied to a
given user. Finally, you’ll now find that
system objects are a member of the sys
schema.
SQL Profiler
and Database Tuning Advisor
Index Tuning Wizard has a new name
and new functionality. It is now called the
Database Tuning Advisor (DTA) and now
has the ability to give index table partition
recommendations. It’s also highly
configurable to give the DBA a more
control of the recommendations to make Figure 5: The new Profiler event selection process.
sure it doesn’t waste your time.
When you first open Profiler, you may think that Conclusion
nothing has changed. You quickly realize how much If you haven’t done so already, make sure you
better it is. When creating a new trace, you’re given a update your SQL Server Beta 2 install to be the
matrix of events and columns that you can choose latest Community Technology Preview (CTP). As I
between. SQL Server 2000 would let you choose a write this, the latest release was in December of
column for an event that was not a match. For example, 2004. There are significant features that are
the Audit Logout event has no TextData. In SQL Server cleaned up in each preview release. By Beta 3, the
2005, you can’t make that mistake as you can see in features will begin to be locked down and will be
Figure 5. much cleaner. The latest release is much faster
and more bug-free than previous betas of SQL
My favorite feature in Profiler now is the ability to
Server 2005.
tie Profiler to System Monitor results. In other
words, you can select a line in Profiler and
potentially see what that query did to your CPU
and memory of the server visually on the same
Brian Knight is a SQL Server MVP and MCDBA. He is the
screen. You can now trace events in Analysis
co-founder of SQLServerCentral.com and works at Allstate
Services and output the results of a trace to XML (more of a title to co. name and your title?) as the database
for portability. Finally, you can also aggregate your manager. Brian is the author of several books and can be
trace on given keys instead of having to export the reached at bknight@sqlservercentral.com.
data into SQL Server to perform that function.

10
 What’s New in SQL Server
Integration Services? By Allan Mitchell/Darren Green

“ It will quickly become apparent that IS is not

just an extension of DTS, but a complete and
total rewrite from the ground upwards.
”
Introduction empty package within the new package designer. One of
Microsoft SQL Server Data Transformation Services the most important and dramatic changes in the way we
(DTS) made its debut appearance with the release of build packages becomes apparent from the tabs at the top
SQL Server 7.0, and gained even more functionality with of the design sheet, Control Flow and Data Flow, which
the release of SQL Server 2000, but with SQL Server provide two distinct design surfaces. (See Control Flow &
2005 we have SQL Server Integration Services (IS) Data Flow tabs at the top of the central design window in
instead. It will quickly become apparent that IS is not just Figure 1.) Control Flow hosts tasks and related workflow
an extension to DTS, but a complete and total rewrite structures, including precedence constraints and
from the ground upwards, and this no doubt played an containers. The Data Flow Task, also known as the
important part in Microsoft’s decision to select a new pipeline, is where the product really begins to show its
name for the functionality. The depth of the changes will strength. The focus is on true Extract, Transform & Load
become apparent as we try to illustrate some of the new (ETL) through a single high performance pass over the
features and functionality, but also the paradigm change, data, as opposed to the many systems currently
requiring not only changes in our practical implemented more as ELT. If nothing else does, the
implementations but also our mindsets for designing richness of the pipeline will clearly indicate Microsoft’s
solutions in IS compared to DTS. commitment to making IS an enterprise class ETL product.
One of the first changes you will notice across SQL
Server 2005 is the new toolset, and so with the demise of
Enterprise Manager, the Business Intelligence
Development Studio (BIDS) is the new home for the IS
development environment. (Figure 1) The Visual Studio
style environment by its very nature offers several new
features to help develop and manage our packages,
including solution and project-based development,
integration with source control systems such as Microsoft
Visual SourceSafe, and interactive debugging of package
execution. Highlights of debugging include the ability to
set breakpoints at various stages of task execution and
Watch functionality allowing you to view the values of
variables at run-time. Whilst the concept of global
variables is still available, the term “global” is no longer Figure 1: the BIDS Environment
always accurate as variables can now be scoped to a
specific container. Containers, themselves another new Data Flow
concept, provide structure and base functionality for For each Data Flow task in a package there is a
almost all IS objects, from packages through to tasks. separate work surface accessed through the Data Flow
Once you have opened the BIDS and started a new tab. You can perform a simple transfer of data from a
Data Transformation Project, you will be presented with an single source to a single destination, or more advanced

JANUARY/ FEBRUARY, 2005

11
flows, covering multiple sources and destinations, and perform the extra processing when needed to maximize
merge or split the data flow paths between these performance.
multiple connection adapters, transforming the data
Expressions in Workflow
along the way.
The connection objects or connection managers as Workflow very simply is the “how and why” of how we
they are now called are held in the separate Connections get from one task to another in our package. In DTS we
window, but are consumed by tasks and source or have very rudimentary functionality here. For example, if
destination adapters alike. you want to execute a task based on the result of one of
n upstream components that are joined by workflow
The transformations themselves are now easily
constraints to your task this takes a lot of “glue” code,
reusable components that are dropped on to the design
since DTS constraints must all be satisfied. Whilst you can
sheet in a similar manner to tasks, albeit on the Data Flow
improve on basic constraints by using Workflow Scripts,
design surface as opposed to the Control Flow. An
these are not always easy to write and are hidden away
example of a new transformation is the Derived Column
such that they can easily be forgotten and making
component, which will replace much of the custom
maintenance difficult. In IS the level of control you get is
ActiveX script used previously, and instead relies on the
quite extensive and presented in a friendlier and more
new expression syntax implemented throughout IS. Whilst
structured manner. Below is a very short but “thirst
this is initially a learning curve as the language is neither
quenching” list of these new features:
T-SQL nor an existing scripting language, it emphasises
the more structured nature of the product, and perhaps • Constraint
most importantly it was designed for performance. The • Expression
language itself is based on C, and covers most of the • Expression OR Constraint
simple functions you may have used previously. If this • Expression AND Constraint
structured approach is too restrictive there is still the
Where we have multiple constraints going to the
highly flexible Script Component, which brings the power
same task we can choose further from:
of VB.Net into your transformation without going as far as
writing a full custom transformation component. • Logical AND (All constraints must evaluate to true)
There are many other new components available such • Logical OR (Any of the constraints must evaluate to
as the ground breaking Fuzzy Lookup or Term Lookup true)
components which bring true language independent
fuzzy logic and text mining capabilities respectively, right
to the heart of your ETL processing.
One of the most difficult areas of ETL development
and management is around what to do when it goes
wrong. The new logging infrastructure is covered in more
detail below, but from a Data Flow perspective the most
significant advance is the error output. Most components,
source and destination adapters as well as
transformations, implement the error output. These
components allow you to route any rows that fail to
process correctly to that component’s error output path.
This immediately allows you to see which rows caused the
problem, an often asked question today. The simplest Figure 2
thing you can do would be to write the failed rows to a file
A simple article illustrating one of the last points is
for closer examination later, but perhaps more useful
available here Introduction To Expressions On A Workflow.
would be for the data flow to have some additional
components that tried to fix the problem and then merge Property Expressions
the now clean rows back into the main flow. Whilst you Do you remember the good old days, when if we
could code defensively, if the failure rate is only a handful wanted to set the connection string of a text file to a
of rows out of several million, you would only want to dynamic value at execution time in our package, we had

12
to add a lot of glue code? (How can I change the filename
for a text file connection). Well, a lot of those needs are
gone. Now we have a truly amazing piece of functionality
that you can make as expressive as you want or as simple
as you want. Property Expressions can be placed on a
container so this means you can place them on objects
such as packages, tasks, Foreach Loop enumerators, For
Loop enumerators, Sequence containers, event handlers,
connection managers, and log providers to name but a
few. So what do they do? Well, they evaluate an
expression at runtime and substitute the expression for
the value to which it is mapped. The following image
shows how to find property expressions on a Text File
connection manager.

Figure 3 Figure 4
Once you have The good thing about the dialog is that we can click
selected the expressions on the ellipses button to the side of the expression text
tab you are offered the box and we are offered the chance to build our
chance to choose the expression using an editor. The language used is pretty
property you wish to set intuitive if you have spent anytime using IS and
and the value to which you expressions in other parts of the product. Let’s have a
wish to set it. look, then, at a couple of examples. The first example
shows us using the expression syntax to derive a value.
The second expressions shows us using a system variable
Control Flow Vs Data Flow value to use in our expression.
The introduction of the distinct Control Flow and Data “c:\\MyLoadFile_” +
Flow design surfaces is a striking and potentially confusing (DT_WSTR,4)DATEPART(“Year”,Getdate()) +
change for existing DTS users. The key point to remember (DT_WSTR,2)DATEPART(“Month”,Getdate()) +
is that the Control Flow hosts tasks. To help illustrate this, (DT_WSTR,2)DATEPART(“Day”,Getdate()) + “.txt”
here is a sample of tasks, some of which will already be
“c:\\MyLoadFile_” + @[System::MachineName] +
familiar to DTS users. “.txt”
• Bulk Insert Task • File System Task
We have tried to give you a very brief overview of
• Data Flow Task • Send Mail Task some of the conceptual changes as well as the practical
• Execute Package Task • WMI Event Task ones that we think will make your ETL processes
• Execute SQL Task • XML Task
considerably easier. If you can relate to some of the
improvements described above, then there are probably
another ten that we didn’t have space for this time round.
The Data Flow task is the replacement for the old Data
This article is based on SQL Server 2005 Beta 2 and Beta
Pump Task. However, that is where the similarity ends.
3 (Yukon).
Inside the Data Flow you can place sources and
destinations, collectively called adapters. You can then
manipulate the flow of the data between adapters by
Allan and Darren are both Microsoft SQL Server MVPs,
using transformations. To try and illustrate what you can specialising in Data Transformation Services and now
do in between here is a small selection of the embracing SQL Server Integration Services.
transformations available.
They run a successful consultancy, Konesans Ltd, providing
• Aggregate • Fuzzy Lookup support and design expertise to a variety of clients in a
• Conditional Split • Merge Join number of countries as well as providing community
• Data Mining Query • Pivot support through the Microsoft newsgroups and the popular
• Derived Column • Union All SQLDTS.com and SQLIS.com websites.

13
JANUARY/ FEBRUARY, 2005
 Paging Result Sets Using
SQL Server 2005’s ROW_NUMBER By Joseph Sack

“ After years of creative workarounds and code bloat,

Microsoft has finally given us a ROWNUM equivalent,
the ROW-NUMBER function, introduced in SQL Server2005

Have you ever needed to return a row number END

”
column in your query result set? Or how about writing a ELSE
BEGIN
stored procedure that pages through a result set,
SELECT @BeginRow = ((@Page-1) * @Size + 1),
returning data in 10 row blocks? @EndRow = ((@Page-1) * @Size) + @Size
If you work with Oracle, then you already have a built- END
in solution, the ROWNUM function. And now, after years DECLARE @Stores TABLE
of creative workarounds and code bloat, Microsoft has (Name nvarchar(50) NOT NULL,
finally given us a ROWNUM equivalent, the ContactType nvarchar(50) NOT NULL,
ROW_NUMBER function, introduced in SQL Server RowNum int IDENTITY(1,1))
2005. INSERT @Stores
(Name, ContactType)
I’ve been waiting for this function for some time now. SELECT Name, ContactType
My job often involves writing stored procedures that FROM Sales.vStore
return data to website search applications. The web ORDER BY Name
pages don’t show the entire query result set on a single SELECT Name,
page, but instead present results in smaller, ordered, ContactType,
paged chunks. One technique for doing this in SQL RowNum
Server 2000 was as follows: FROM @Stores
WHERE RowNum BETWEEN @BeginRow AND @EndRow;
1. Create a temporary table or temporary table GO
variable with an IDENTITY column. The To display the first 10 rows of the first page of your website
IDENTITY column serves as a row number. search results, you would execute the following:
2. Populate the table with an ordered result set.
3. Query the table using the IDENTITY column,
EXEC Sales.uspRetrieveStores_OLDStyle 1, 10
qualifying which rows I wish to see.
To display the second page of rows, rows 11 through
The following procedure demonstrates the SQL Server 2000 20, you would execute:
technique of paging through the Sales.vStore view (available
in the AdventureWorks database in SQL Server 2005, Beta EXEC Sales.uspRetrieveStores_OLDStyle 2, 10
2). It takes two parameters, @Page and @Size, which is Now in SQL Server 2005, ROW_NUMBER (syntax
used to determine which rows from the result set should be
below) allows you to query an ordered result set, as well
retrieved:
as qualify which rows are returned, all without creating an
GO explicitly populated table:
CREATE PROCEDURE Sales.uspRetrieveStores_OLDStyle
@Page int, ROW_NUMBER ( ) OVER ( [ < partition_by_clause > ] <
@Size int order_by_clause> )
AS Before I review the stored procedure equivalent, we’ll
DECLARE @BeginRow int first review the various behaviors of the ROW_NUMBER
DECLARE @EndRow int function. In my first example of using ROW_NUMBER,
IF @Page = 1 the query returns the name, contact type, and row
BEGIN number for stores in the AdventureWorks database.
SELECT @BeginRow = 1, @EndRow = @Size
14
SELECT Name, ContactType, Row_Number()
OVER (ORDER BY Name) ‘RowNum’
FROM Sales.vStore
Notice the ORDER BY clause is embedded within the
syntax of the function, ordering the results by the “name”
column. This ORDER BY is a required clause and is used
to determine the order in which rows appear. The column
you ORDER BY doesn’t, however, effect the sequential
numbering if you have a redundant field.
For example, this next query orders by name, but
qualifies a store that appears twice in the result set. Even
though the name ‘Advanced Bike Components’ is
duplicate, you’ll see that the row number still increments
from 1 to 2.
SELECT Name, ContactType, Row_Number()
OVER (ORDER BY Name) ‘RowNum’
FROM Sales.vStore
WHERE Name = ‘Advanced Bike Components’
ROW_NUMBER also has a partitioning feature that
allows you to restart the sequential row numbering based
on the grouping of a specific column (either the same
column as your ORDER BY or a different column). This
next example partitions data by ContactType. Numbering
begins with “1” for the ContactType of “Order”, and
increments until row 272. It then restarts at “1” for the
ContactType of “Purchasing Agent” until row 246, when it
starts again at “1” for “Purchasing Manager”.
SELECT Name, ContactType, Row_Number()
OVER (PARTITION BY ContactType ORDER BY Name)
‘RowNum’
FROM Sales.vStore
Now to the SQL Server 2005 equivalent of the paging
procedure (we’ll review the elements of this procedure
afterwards):
CREATE PROCEDURE Sales.uspRetrieveStores
@Page int,
@Size int
AS
DECLARE @BeginRow int
DECLARE @EndRow int
IF @Page = 1
BEGIN
SELECT @BeginRow = 1, @EndRow = @Size
END
ELSE
BEGIN
SELECT @BeginRow = ((@Page-1) * @Size + 1),
@EndRow = ((@Page-1) * @Size) + @Size
END;
JANUARY/ FEBRUARY, 2005
WITH CTE_Stores AS
(SELECT Name, ContactType, Row_Number()
OVER (ORDER BY Name) ‘RowNum’
FROM Sales.vStore)
SELECT Name,
ContactType,
RowNum
FROM CTE_Stores
WHERE RowNum BETWEEN @BeginRow AND @EndRow;
GO
After compiling the stored procedure, you can view
for example, the second page of results:
EXEC Sales.uspRetrieveStores 2, 10
I threw in something we haven’t discussed yet: the
Common Table Expression (CTE). A CTE is a temporary
named result set which can be used within the execution
scope of a single SELECT, INSERT, UPDATE, or DELETE
query. We use a CTE because we cannot use the
ROW_NUMBER() function directly in the WHERE
clause. Doing so would result in the following error:
Msg 4108, Level 15, State 1, Line 2
Windowed functions can only appear in the select or
order by clauses.
Instead, if you wish to designate which rows are
returned, you must encapsulate your query within a CTE:
WITH CTE_Stores AS
(SELECT Name, ContactType, Row_Number()
OVER (ORDER BY Name) ‘RowNum’
FROM Sales.vStore)
SELECT Name,
ContactType,
RowNum
FROM CTE_Stores
WHERE RowNum BETWEEN 1 AND 10
Another special consideration involves the CTE itself.
When using a CTE within a batch of statements, the
statement proceeding the CTE definition must be followed
by a semicolon.
BEGIN
SELECT @BeginRow = ((@Page-1) * @Size + 1),
@EndRow = ((@Page-1) * @Size) + @Size
END;
What about performance considerations? Until SQL
Server 2005 RTM is released, we should avoid making
any concrete comparisons. However, I did compare
(because I couldn’t resist) both versions of the procedure
using SET STATISTICS IO, and found that the I/O results
were almost identical. I’ll be interested to see how
ROW_NUMBER performs with much larger result sets
15
and more complicated queries in Beta 3 and RTM.
In the meantime, I’ve added ROW_NUMBER at the
top of my favorite improvement lists, and will definitely be
converting over my stored procedures to this new
technique next year.
SQL Server 2005
Name Resolutions - Owners & Schemas
“ A Schema is simply a logical partition
of a database’s namespace.
In SQL Server 2000, all objects technically have a 4-
part name (remember server.database.owner.object ?) So
– ignoring the server.database components for the
purposes of this article, and concentrating on
owner.object – let us look at the case of a developer
named Ana, who had the db_ddladmin role in a database.
Ana creates a procedure called uspAddCustomer
with the command:
CREATE PROCEDURE uspAddCustomer AS...
This results in the creation of an object called:
Ana.uspAddCustomer
This can create security and/or name resolution
problems if any objects referenced in the procedure (eg.
the Customer table) are owned by dbo, or others. The
general recommendation in SQL Server 2000 is to have
dbo own everything. Ownership can be changed using
the sp_changeobjectowner procedure, or – preferably -
Ana can create her procedure using:
CREATE PROCEDURE dbo.uspAddCustomer AS....
As a result of the dbo’s usual ownership of all objects,
some developers and DBA’s take shortcuts writing code.
(Not you, of course. You’ve always followed the best
practice of qualifying objects with owners!) Unless the
application requires cross-database or cross-server
development, you may have some code that looks like:
CREATE TABLE mytable (col1 INT NOT NULL, col2
INT NULL)
instead of:
CREATE TABLE dbo.mytable (col1 INT NOT
NULL, col2 INT NULL)
16
Joseph Sack, MCDBA, is a SQL Server consultant based in
the Twin Cities. He is the author of SQL Server 200 Fast
Answers for DBA’s and Developers (Apress) and the
co-author of Beginning SQL Server 2000 DBA: Novice to
Professional (Apress). He can be contacted via his website,
http://www.JoeSack.com.
By Wendy Sue Williams
”
In SQL Server 2005, ownership is separate from
schema membership. Objects still have a 4-part name;
however, what was previously the “owner” part now
reflects the object’s schema as shown in:
server.database.schema.object. Qualifying your object
names becomes imperative when you use schemas. Let’s
take a look at what schemas are, and how they work.
A schema is simply a logical partition of a database’s
namespace. For example, if I have a Training database, I
may have objects related to the “people” aspect of my
business: trainers, employees, students, companies,
contact information. I can logically group these objects
together by creating them in the same schema.
Schemas work similarly to file storage on a hard disk. If I
create a file, for example “ServerDocumentation.doc”, I’ll
normally put it in a folder, let’s say “d:\articles”, making the
fully qualified name of the file includes the path:
“d:\articles\ServerDocumentation.doc.” I am the owner,
but ownership is not reflected in the path. I can secure the
folder by using NTFS permissions, which may allow
certain individuals to view the documents in that folder,
and I can set permissions on an individual file within that
folder, for when the folder level permissions are not
appropriate.
Back on the SQL Server, your objects are going to be
like the files; your schema will be analogous to the path.
By creating a schema, with, for example:
CREATE SCHEMA Person
then placing objects in that schema, eg:
CREATE TABLE Person.Address (PersonId INT
IDENTITY....
 you’ll be able to grant access to all objects in that
schema to a set of users. Eg:
GRANT EXECUTE,INSERT,DELETE,UPDATE,SELECT ON
SCHEMA::person TO [HR]
You can also continue to set permissions at the table
or view level, as appropriate.
For example, in the Training database, I have two
schemas: Person and Course. Within the Person schema,
I have the tables and views related to all “people”
elements of my business, such as the Address table, the
Instructors table, and the Students table. These tables are
referred to as Person.Address, Person.Instructors, and
Person.Students. In the Course schema, I have
Course.Vendors, Course.CourseTitles and
Course.Categories. ie:
Schema: Person Schema: Course
Tables: Address Tables: Vendors
Instructors CourseTitles
Students Categories
A CustomerService user will need to look up course
information to give to students, so I can grant
CustomerService the SELECT permission to the Course
schema. Eg:
GRANT SELECT ON SCHEMA::Course TO
CustomerService
This gives the CustomerService users permission to
select on all three tables in that schema. The same role
will need to enroll new students, so they’ll need the
INSERT permission on the Person.Students table, but
only SELECT permission on the Person.Instructors table.
I’ll need to assign permissions on the tables directly, much
as is done in SQL Server 2000. Eg:
GRANT INSERT ON Person.Students TO
CustomerService
GRANT SELECT ON Person.Instructors TO
CustomerService
From the permissions aspect, schemas provide a
much needed method of simplifying the assignment of
permissions. Often, an application will have several related
tables, to which users need the same level of permission.
However, using multiple schemas introduces object name
resolution issues. Let’s take a look at some code.
In SQL Server 2000, if all objects were owned by dbo,
my CustomerService users could execute:
JANUARY/ FEBRUARY, 2005
SELECT... FROM mytable ...
…without qualifying the table name.
In SQL Server 2005, users can be assigned a default
schema, which will result in similar behavior. I assign the
CustomerService members to the Person schema, and
ensure the CustomerService role has the SELECT
permission on that schema; as a result, the following
statement will run for that user without error:
SELECT first_name, last_name FROM Students
However, when the CustomerService rep runs:
SELECT course_name FROM CourseTitles
…the statement will fail. Why? For objects that are not
qualified, SQL Server 2005 first looks in the user’s default
schema (Does Person.CourseTitles exist?), then in the
“dbo” schema (Does dbo.CourseTitles exist?). In order
for my CustomerService users to access the CourseTitles
table, I’ll have to ensure that the table name is always
qualified with its schema. ie:
SELECT course_name FROM Course.CourseTitles
In addition, as long as they reside in separate
schemas, objects can have the same name. Just like I can
have two files on my hard drive named “README.TXT,”
so long as they are stored in different directories, I can
have two views called vMyView, as long as they are
located in different schemas.
SQL Server 2005 does maintain a default “dbo”
schema. This will make porting existing applications to
SQL Server 2005 easier. However, as you work with new
applications from this point forward, you’ll see how
valuable multiple schemas are. When you start using
schemas other than the default, you’ll need to ensure the
use of objects’ two-part name (schema.object). Even if
you’re not using SQL Server 2005 yet - and even if you
never intend using schemas other than dbo - you might
want to ensure your organization continues the best
practice of using those two part names in your code.
Wendy Sue Williams has worked with SQL Server since
1996 as a developer and a DBA. As a Microsoft Certified
Trainer, she currently trains for a CPLS in New Jersey, and
recently presented a session at the SQL PASS 2004
conference in Orlando, FL.
17
 The Next Evolution
of Management Objects
“ In SQL Server 2005, DMO has been
abandoned in favor of SMO.
SQL Server introduced the ability to
programmatically manage the database server with the
introduction of Distributed Management Objects
(DMO) in SQL Server v6.0. The objects were
substantially enhanced with each version, always
maintaining backward compatibility. That become
slightly confusing for developers when there were both
Server and Server2 objects and which methods could be
used with which, but for the most part it worked.
Now in SQL Server 2005, DMO has been
abandoned in favor of SMO. SQL Server Management
Objects replace the distributed management objects as
well as enhancing the functionality that is available when
programming applications to manipulate the server.
While there are many changes to the management
framework, the objects are still compatible with SQL
Server 7.0 and SQL Server 2000.
Everything is .NET
In keeping with the .NET enabling of all their
technologies, The SMO framework is implemented as a
.NET assembly. Just as you access the CLR from within
SQL Server, you can include all DMO functionality in
your .NET applications as well. The SMO namespace is
Microsoft/SqlServer.Smo and is located by default
in the c:\program files\Microsoft SQL
Server\90\SDK\Assemblies folder. It is installed with the
client tools and requires the CLR runtime to be installed
as well.
One change from DMO is the replication objects are
not a part of the SMO framework. Instead, there are
Replication Management Objects (RMO) that exist as
well. These enable you to manage all aspects of
replication, including the new parallel snapshot
preparation, initialize a subscription from a backup,
subscribing to data from other databases (like Oracle),
etc. A separate namespace exists for these objects
(Microsoft.SqlServer.Replication) located in the
Microsoft.SqlServer.Rmo.dll, another .NET assembly.
Never fear, however; not everything is .NET. There
18
By Steve Jones
”
are COM wrappers around the SMO classes to enable
you to work in an unmanaged environment. The
SQLSMO.dll file is located with the other assemblies and
a SQLSMO.tlb file exists that shows SQLSMO and its
lower objects in the object browser. This allows you to
continue to work with SMO with VBScript or other
legacy methods of COM programming. An example is
shown in Figure 1.
set oServer = CreateObject(“SQLSMO.Server”)
oServer.Name = “MyServer”
Wscript.Echo oServer.Name & “ “ &
oServer.Information.VersionString
Figure 1
Two Classes
There are two main categories of classes in SMO:
instance classes and utility classes. The instance classes
are for the main objects that you are used to dealing
with: the servers, databases, tables, views, stored
procedures, triggers, etc. These are in the familiar
hierarchy that you are used to: servers at the top,
databases under them, tables under databases, etc. As
with DMO, if there are multiple objects below a class
they are in a collection Otherwise, there is a single object
as the child.
The utility classes exist to perform specific tasks.
There is the Transfer class, the Backup and Restore
classes, and the Scripter class.
The transfer class contains the functionality that
existed in the old DMO Transfer objects as well as a few
enhancements. In SMO, however, the Transfer object
uses DTS to transfer data. The Backup and Restore class
handles all the functionality you need to perform any
type of backup or restore task that you might need. Each
Backup or Restore object represents the functionality
that you might need to handle a specific task. An
example of this would be Figure 2, which shows how
you might perform a backup using VB.NET. The Scripter
class is used to handle scripting of objects. In DMO,
there was a method for various objects that allowed you
to script the current object. In 2005, there is a separate
object that discovers the object and its dependencies,
generates a list of objects that need to be scripted based
on the discovery, and then the scripts are generated and
the resulting script returned.
Dim oServer as Server
oServer = New Server
Dim oBackup as New Backup
oBackup.Action = BackupActionType.Database
oBackup.BackupSetName = ‘Sample Backup’
oBackup.Database = ‘Northwind
Figure 2. Backing up a Database using VB.NET
Optimized Instantiation
These are a couple of very fancy words that describe
a performance enhancement that was implemented in
SMO. This means that as the programmer you can control
the level or amount of instantiation that occurs when you
create an object. In DMO, all objects in a collection had to
be instantiated, which could eat up lots of resources. Even
if you never intended to reference some of the objects,
they would all be created. In SMO, you can partially
instantiate a collection, or even properties. It’s all up to
you for each server. It’s not all good, however. Delaying
the instantiation when you are planning on referencing
more objects or properties isn’t good for performance.
You’ll create lots of round trips to the server as you
instantiate items.
There are 3 levels of instantiation that you can set.
Uninstantiated, nothing loaded, or full instantiated,
everything loaded (like DMO) are the most common
levels that most programmers are familiar with. There’s
also partial instantiation, which doesn’t load objects that
are not directly referenced. Once the object is referenced,
then it is fully instantiated.
New Features
The new features in SQL Server 2005 have
corresponding items in SMO as well. HTTP and SOAP
requests can be handled by EndPoints objects. There are
new objects inside the database object to work with Full
Text Search. The database object can work with snapshot
isolation and the new row level versioning feature. The
ability to spread tables and indexes across file groups has
corresponding SMO objects. The new XML features,
including schema namespaces and indexes, are
represented in SMO. Almost everything in the server that
you can work with has a corresponding SMO
counterpart. In addition, the WMI provider objects are
JANUARY/ FEBRUARY, 2005
now wrapped inside SMO to allow you to work directly
with them.
Capture Execution is one of the really interesting new
features. Instead of having your application actually
submit statements to the server for execution, they can be
captured using SMO. Suppose you have a section of your
code that creates a database or table, adds an index,
populates data, for example, in an installation routine.
After testing, you can actually use SMO to capture this as
a script for later execution, or on a separate server.
3 Card Monty
Hopefully it doesn’t feel like that, but with the evolution
of the management objects, SMO has moved a number of
things around. Books Online has a short list of what’s been
moved, though keep in mind that this is a Beta product. It’s
entirely possible that more changes will be made in the
RTM version. Look for a new article then :).
Conclusion
While I normally don’t like wholesale changes
between versions, there are times that it’s warranted and
this is one of them. With most everything else being
rewritten, the CLR added in, and more, SQL Server 2005
needed a change from DMO instead of adding Server3
objects and more. I am glad that backwards functionality
is retained, especially through SQL Server v7 when DMO
got very popular. It’s also great to see that COM
functionality is still available, since I find quick and dirty
VBScript command files to be an easy way to handle
many administrative tasks.
If you use DMO, I think you’ll be excited and find the
changes well worthwhile. It also appears the rewriting
scripts from DMO to SMO should be fairly
straightforward. In many cases, changing from a “2”
object to the single, i.e. Server2 to Server, should allow
many of your scripts to work. I’m really looking forward to
Beta 3 and RTM and hopefully much more detailed
documentation on the objects. As of now, many of them
are not documented in Books Online. Look for an updated
version of this article at SQLServerCentral.com once the
product releases.
Steve Jones has been working with SQL Server since 1992 in
a variety of companies from small start-ups to large Fortune
500 companies. His primary area of expertise has been in
the Operations part of IT managing hundreds of databases.
He currently works full-time for SQLServerCentral.com
managing all aspects of their publishing business.
19
 Accessing SQL Server Metadata By Anthony Bressi

“ Microsoft has been encouraging users

to use information_Schema views
”
In this article we will look at some of the methods FOR
that are available for accessing SQL Server metadata. Select sysobjects.name
The term ‘metadata’ in this article is primarily used to From sysobjects, sysprotects
describe database schema information, but most Where sysprotects.uid = 0
methods described below also allow you to gather AND xtype IN (‘X’,’P’)
AND sysobjects.id = sysprotects.id
statistical information, such as the number of rows in a
and protecttype <> 206 —can be revoked
table. We will primarily concentrate on options to use in
Order by name
persistent code, but will first start at the top. OPEN getRS
System Tables FETCH NEXT FROM getRS INTO @sysName
WHILE (@@FETCH_STATUS <> - 1)
This is probably the most common method that
BEGIN
DBA’s use to access metadata. I am a fan of this method IF (@@FETCH_STATUS <> -2)
because it’s quick, straight forward, and can be BEGIN
accessed using TSQL ‘Select’ statements. This is a good Set @tsql = ‘Revoke Exec On ‘ +
way to access metadata if you need a one-time look at @sysName + ‘ From public’
the data; i.e., your TSQL code will not persist past your Exec dbo.sp_executesql @tsql
current user session in Query Analyzer. But because the END
structure of system tables can change from release to FETCH NEXT FROM getRS INTO @sysName
release, it is <b>not</b> a recommended method to END
use within persistent code. If you are looking for a CLOSE getRS
simple TSQL way to access metadata for a project such DEALLOCATE getRS
as a software application or Internet application you will
INFORMATION_SCHEMA Views
be better off using INFORMATION_SCHEMA Views
instead, which is the next method we will look at. Retrieves database schema information using
familiar TSQL ‘Select’ constructs. The schema
—This example will revoke execute permissions information that is returned is based on the current
for the public group user’s permissions, in other words, <b>the results set
— on all of our stored procedures and extended will only contain data about database objects that you
stored procedures have access to</b>. INFORMATION_SCHEMA
enables you to retrieve metadata about Tables, Stored
— First we’ll use our sys tables to get the Procedures, Views, User Defined Functions, User
procedures that can be revoked, Defined Data Types, Databases, and more. Support for
— then loop through them and revoke them INFORMATION_SCHEMA began with SQL Server 7.0
and carried over to both 2000 and 2005. Because these
— PLEASE EXCUSE THE CURSOR, but in the example
views are system table-independent, Microsoft has been
it helps convey what
encouraging users to use INFORMATION_SCHEMA
— we are trying to do instead of basing code on system tables directly if
DECLARE @sysName varchar(128) possible (Note: some usage statistics that are available
Declare @tsql nvarchar(255) through the system tables are not available through
DECLARE getRS CURSOR INFORMATION_SCHEMA).

20

—Get Information About Tables
Select * from INFORMATION_SCHEMA.TABLES where
TABLE_TYPE = ‘BASE TABLE’
— returns information for each table
—Return column info for each table
Select * from INFORMATION_SCHEMA.TABLES st
Inner Join INFORMATION_SCHEMA.COLUMNS sc
On st.TABLE_NAME = sc.TABLE_NAME
where TABLE_TYPE = ‘BASE TABLE’
—Get Information About Stored Procedures
Select * from INFORMATION_SCHEMA.Routines
—returns stored procedures and functions
Select * from INFORMATION_SCHEMA.PARAMETERS
—returns parameter information for user-defined
functions and stored procedures
—Get Information About Views
Select * from INFORMATION_SCHEMA.VIEWS
—returns information for each view
Select * from
INFORMATION_SCHEMA.VIEW_TABLE_USAGE
—returns each view and the tables it is using
SQL Server Distributed Management
Objects (SQLDMO)
If you are building a software application that needs
metadata from a SQL Server 7.0 or 2000 database, this is
the way to go. In fact, SQL Server 2000 Enterprise
Manager uses SQLDMO for most of its actions.
Database information is exposed nicely through an easy
to use COM object library that can be used in any COM
aware language such as Visual Basic, and can also be
used in .Net - although it requires using Interop. While
using SQLDMO you work in a hierarchical manner to
access the data that you want. For example, a Server
contains a collection of databases, a database contains a
collection of tables, and each table contains a collection of
columns. Microsoft SQLDMO documentation states that
applications using SQLDMO can perform all functions
performed by SQL Server 2000 Enterprise Manager,
making this a powerful but easy-to-use way to access
metadata. SQLDMO is supported by SQL Server 7.0 and
2000. It can be used with SQL Server 2005, but was not
updated to support 2005 specific features. SQLDMO is
bundled up in sqldmo.dll so you simply need to add a
reference to this object in your application. If you installed
your SQL Server files to the default location on your PC,
the file should be located in: C:\Program Files\Microsoft
SQL Server\80\Tools\Binn. SQLDMO is an ideal choice
JANUARY/ FEBRUARY, 2005
when you need to write persistent code for an application
that interacts with SQL Server 7 or 2000 metadata. A
sample SQLDMO example is available in the download.
SQL Management Objects (SMO)
SMO replaces SQLDMO in SQL Server 2005. As
noted above, SQLDMO is compatible with SQL Server
2005 but no new features have been added. SMO was
modeled with SQLDMO in mind and Microsoft has called
it a “logical continuation” to SQLDMO and explicitly
stated that they have incorporated objects similar to
SQLDMO objects when possible. SMO makes 150 new
classes available to users. SMO is backwards compatible
with SQL Server 2000 and 7.0 and is 100% managed
code. Like SQLDMO, SMO is very easy to work with,
especially if you are using Visual Studio 2005.
Unfortunately, if you want the convenience of
intellisense and IDE debugging while writing your SMO,
you will need to use Visual Studio 2005. The Visual Studio
2003 IDE will not let you add a reference to the SMO
managed library through its GUI (you can still use a
‘Using’ statement and include SMO but you lose
intellisense and debugging from within the IDE). You can
still build an SMO application with VS 2003 or even
notepad but you will need to make your references while
using the command line compiler (you will need to use
the compiler in your Framework 2.0 folder, not your
Framework 1.1 compiler).
Conclusion
There are many ways to access SQL Server
metadata, and we covered some of the most common
methods in this article. In addition, we exposed some
pros and cons of those methods which can hopefully
help you make the decision of which tool to use for the
task at hand. There are still yet other alternatives to use
to access SQL Server metadata, from ADO.Net to TSQL
functions like DATABASEPROPERTY() (see Books
Online -> meta data -> functions for more TSQL
metadata functions). Ultimately, you may need to do
some additional research to determine the right tool for
whatever your needs may be, but I hope that this article
has helped you get on your way.
Anthony Bressi is owner of Agilist Technologies Inc. which
specializes in software for SQL Server Database
Administrators and SQL Server developers. Mr. Bressi has
over 8 years of hands-on experience in the Microsoft SQL
Server development environment.
21
 Yukon Encryption
“ If this stays in the final product, It’ll be one more thing
DBA’s can check off their wish list.
One of the items I have heard asked for in SQL
Server is the ability to do encryption at the database
level. In SQL Server 2000 the only option is to use a
third party product as there is no built-in capability to
take raw data and store it in encrypted form. With SQL
Server 2005 Beta 2, however, Microsoft introduced
encryption at the database level. If this feature stays in
the final product, it’ll be one more item SQL Server
DBAs can check off their wish lists. In this article I’ll look
at why encryption at the database level is a much
requested feature, how it currently works in beta 2,
some thoughts on how to use encryption, and a few
issues to consider when implementing encryption
within SQL Server.
Passwords, Sensitive Data, and Things
Folks Shouldn’t See
SQL Server databases are used to store all sorts of
things from library lists to passwords to IDs to credit
card numbers. Auditors tend to cast a doubtful eye
when bits of sensitive information like passwords, IDs,
and credit card numbers are available in plain view,
even if it is only to the DBAs. Passwords tend be a big
problem because people have a tendency to re-use
passwords. Therefore, the password a user has for an
application may very well be the same password the
user has for their Windows login. This is the type of
situation that causes the auditor’s pencil to start
scribbling furiously. Other types of data we would do
well to encrypt are things like salary figures in the HR
database. Day-to-day DBA operations wouldn’t need
the unencrypted data therefore there’s no reason to
display it. If the encryption capabilities in Beta 2 roll
forward into the released product, we’ll have a native
solution for both of these types of scenarios and many
more we’ve faced. Some of our options in the beta are
extensive. Others are extremely quick and simple.
Encryption/Decryption By Pass Phrase
SQL Server 2005 Beta 2 provides the option of
encrypting using a pass phrase. This is the simplest of
the encryption mechanisms and managing the pass
phrase is left to the implementer. If you want to use
22
By K. Brian Kelley
”
encryption via a pass phrase, you’re on your own to
secure the pass phrase used. There are two new
functions to support this mechanism of encryption:
EncryptByPassPhrase( <pass phrase>, <plain text>
)
DecryptByPassPhrase( <pass phrase>, <encrypted
text> )
EncryptByPassPhrase() takes the pass phrase and
the text to be encrypted and produces a varbinary
result. DecryptbyPassPhrase() takes the pass phrase
and the encrypted text in varbinary format and
produces the original plain text. Here is an example of
how both are used:
DECLARE @plain nvarchar(4000)
DECLARE @key nvarchar(4000)
DECLARE @cipher varbinary(4000
DECLARE @decrypt nvarchar(4000)
SET @plain = ‘This is a standard phrase in
plaintext’
SET @key = ‘EncryptWithMe’
SET @cipher = EncryptByPassPhrase(@key, @plain)
SET @decrypt = DecryptByPassPhrase(@key,
@cipher)
SELECT 1, ‘Plain:’, @plain UNION
SELECT 2, ‘Cipher: ‘, @cipher UNIO
SELECT 3, ‘Decrypt: ‘, @decrypt
ORDER BY 1
Building the Example
Going beyond the simple pass phrase encryption,
we’ll need to build a few structures to demonstrate how
the various other options for encryption work. The build
script creates two server principals (logins), TestUser1
and TestUser2, a database, and a table we’ll use to store
our encrypted data. It also maps the two server
principals to two database principals (users), creates a
role, and makes the database principals members of the
role. Finally, the script gives direct SELECT and INSERT
access to the table via the role.
CREATE LOGIN TestUser1 WITH PASSWORD =
‘MyPassword1!’
CREATE LOGIN TestUser2 WITH PASSWORD =
‘MyPassword2!’
GO
CREATE DATABASE Sandbox
GO
USE Sandbox
GO
CREATE USER TestUser1
CREATE USER TestUser2
CREATE ROLE EncryptionUsers
EXEC sp_addrolemember EncryptionUsers,
TestUser1
EXEC sp_addrolemember EncryptionUsers,
TestUser2
GO
CREATE TABLE dbo.ExampleTable (
ExampleID int IDENTITY,
EncryptMethod varchar(50),
PlainData nvarchar(1000),
CipherData varbinary(1000) )
GO
GRANT SELECT, INSERT ON dbo.ExampleTable TO
EncryptionUsers
GO
The Database Master Key
SQL Server 2005 Beta 2 comes with the ability to
completely manage the keys and provide for encryption
almost completely behind the scenes. It does so using
what is called a database master key. To understand how
this comes into play, let’s take a step back and talk a bit
about the service master key and the encryption
hierarchy.
When you install SQL Server 2005, a symmetric key
(a single stream of bits which can be used to both
encrypt and decrypt data) is created for the SQL Server
service. This service master key is used to encrypt a
whole host of important and sensitive things such as
linked server passwords, connection stings, and mapped
account credentials. It is also used to encrypt any and all
database master keys. A query against the
sys.symmetric_keys view in the master database will
reveal the existence of this key (name of
##MS_ServiceMasterKey##).
When a database master key is created, it too is a
symmetric key. The purpose of a database master key is
to encrypt the certificates and keys within the database if
no other encryption mechanism is specified. If we want
SQL Server to handle key management, the database
master key is a requirement.
By default, SQL Server will take the database master
key, encrypt that with the service master key, and store
this result within private structures in the master
database. SQL Server takes the database master key
encrypted with the password you’ve specified (more on
this shortly) and stores this in the given database the key
applies to. You can verify its existence by querying
JANUARY/ FEBRUARY, 2005
sys.symmetric_keys in the given database and looking for
a certificate with the name of
##MS_DatabaseMasterKey##. The version stored in
the master database allows SQL Server to open the
database master key automatically (part of allowing SQL
Server to completely handle key management). The
version stored in the database is then used for
encryption/decryption as necessary.
Creating:
To create the master key, the following syntax is used:
CREATE MASTER KEY ENCRYPTION BY PASSWORD =
‘<password>’
For the examples, make sure you are in the Sandbox
database and enter something similar to the following:
CREATE MASTER KEY ENCRYPTION BY PASSWORD =
‘OneRingToRuleThemAll!’
If we try to create a certificate or key without a
database master key and we haven’t specified some
other means to encrypt (such as a password), SQL
Server will return an error. For example, the following
code to create a certificate will fail:
CREATE CERTIFICATE ExampleCert
WITH SUBJECT = ‘This fails without the db
master key’
The error message received is the this:
Msg 15581, Level 16, State 1, Line 2
Please create a master key in the database or
open the master key in the session before
performing this operation.
This goes back to who is handling key management.
If we don’t specify an encryption mechanism, we are
telling SQL Server we want it to handle the management.
Therefore, the database master key is needed.
Dropping:
Eventually you may want to remove the database
master key. To do so, just execute the following in the
proper database.
DROP MASTER KEY
However, if any keys or certificates are encrypted
with the database master key, you’ll receive an error like
the following:
Msg 15580, Level 16, State 1, Line 1
Cannot drop Master Key since Certificate
‘’NorthwindCert’’ is encrypted by that.
23
 These must be removed before the master key can In the example I’ve given (after the key was
be dropped. regenerated), I’d open the database master key with the
following command:
Changing the Password:
OPEN MASTER KEY USING PASSWORD =
There may come a time where you’ll need to
‘MyPrecious...’
regenerate, or reset, the master key. The following
regenerates the master key with a new password: With the database master key opened, I can now
encrypt using the service master key. This doesn’t affect
ALTER MASTER KEY REGENERATE WITH PASSWORD =
the encrypted version by the password, but it does
‘<new password>’
create the encrypted version stored in the master
Continuing with the example, you can reset the database. Now SQL Server is back in control of key
Sanbox database master key by executing the following management, at least as far as any keys encrypted using
command in the database. the database master key.
ALTER MASTER KEY REGENERATE WITH PASSWORD = ALTER MASTER KEY ADD ENCRYPTION BY SERVICE
‘MyPrecious...’ MASTER KEY
You should only do this, however, when you have a
lot of spare cycles on your server or you have no other Is the Database Master Key Necessary?
choice. SQL Server will decrypt any keys encrypted with Strictly speaking, the database master key is not
the master key and then encrypts them again with the necessary. If you choose to encrypt using passwords (at
new master key. As a result, the whole process can be least initially), the database master key doesn’t have to
exist at all. This is actually an option if the level of trust
very resource intensive. Of course, if the master key has
been compromised, you may not wait until you have low doesn’t allow for the DBAs to control the keys. So long
CPU-usage on your SQL Server. as the keys within the database are encrypted using the
database master key, the DBA has potential access to all
Preparing to Detach the Database: of them. If, however, keys are encrypted using
If you need to detach the database, you’ll want to passwords, the DBA will have to know the password to
remove the encryption from the service master key. Keep unlock the key. An example I’ll use later has an
in mind that when the database master key was created asymmetric key encrypted with a password being used
you also specified a password. By removing the service to encrypt a symmetric key. Therefore, not every level has
master key encryption you can then detach the database to have a password, but the initial one does. However, if
without losing the database master key and any keys you you go down this road, then you’ll have to build your own
encrypted with it. Here’s how: mechanisms for key management. If you are concerned
about someone being able to see the password using
ALTER MASTER KEY DROP ENCRYPTION BY SERVICE Profiler or a server-side trace, the contents are masked.
MASTER KEY
Certificates
Steps After Attaching the Database: SQL Server 2005 Beta 2 has the ability to generate
Once you’ve re-attached the database, you’ll self-signed certificates. Certificates contain the public
probably want to re-encrypt with the service master key key of a person, device, or service. Data can be
for the new SQL Server. This will allow SQL Server to encrypted using the public key and only someone
open the database master key automatically and keeps possessing the private key can decrypt the data again.
key management totally within the control of SQL Server. Certificates also have a range for which each one is valid.
Normally SQL Server is able to open the database master Should a certificate be used before or after the validity
key. However, we removed the service master key period, most applications will throw an error. Should a
encryption before detaching the database, thereby certificate be used to represent something other than
breaking SQL Server’s ability to open the database what it was created for, most applications will again
master key automatically. Until SQL Server is able to throw an error. A good example is (at the time of this
encrypt the database master key using the service writing) if you try to navigate to the website
master key, you must first open the database master key https://www.dshield.org. You should get a message
manually. You can do so by specifying the password. revealing the certificate is actually for
OPEN MASTER KEY USING PASSWORD = ‘<password>’ https://secure.dshield.org. Here we see an example of
where the identity on the certificate doesn’t match the
24
name of the web site.
The certificates created by SQL Server comply with
the X.509v3 certificate standard. In addition to using
certificates for encryption within the database, SQL
Server can also use certificates for SSL connections as
well as for use with Service Broker. However, I’ll only
concentrate on their use for encryption within the
database.
Creating:
Creating a certificate follows this syntax (without
syntax for Service Broker use):
CREATE CERTIFICATE <certificate name>
[AUTHORIZATION <user>]
{ ENCRYPTION_PASSWORD = ‘<new password>’
FROM <certificate source>
| If the certificate was encrypted with the database
WITH <certificate options>
}
The certificate source is a file, an executable, or an
assembly. I’ll focus strictly on the WITH clause option.
The certificate options are defined as:
SUBJECT = ‘<certificate subject name>’
[, START_DATE = ‘<mm/dd/yyyy>’]
[, EXPIRY_DATE=’<mm/dd/yyyy>’]
[, ENCRYPTION_PASSWORD = ‘<password>’]
The only required field is SUBJECT. Therefore we can
create a certificate simply by doing the following:
CREATE CERTIFICATE ExampleCert1
WITH SUBJECT = ‘Example Certificate 1’
Since no encryption password was specified, the
database master key will be used to encrypt the private
key. This allows SQL Server to manage access to the
private key. I’ve also not specified authorization.
Authorization determines who will own the certificate. I
do not have to specify a user as the owner of a certificate.
There is another means if I want to give that user access
to the certificate and its private key. I’ll demonstrate this
shortly.
Here is a second certificate, but this one is created
with the expiry date and the password set. Since a
password is used here, SQL Server cannot perform key
management with this certificate. Instead, if you want to
decrypt data encrypted with the certificate’s private key,
you’ll also have to provide the password used to secure
the private key. I’ll show this when I show how to encrypt
and decrypt data using certificates.
CREATE CERTIFICATE ExampleCert2
JANUARY/ FEBRUARY, 2005
WITH SUBJECT = ‘Example Certificate 2’,
ENCRYPTION_PASSWORD = ‘GandalfTheGrey’,
EXPIRY_DATE = ‘12/31/2010’
Changing the Properties:
There are only a limited number of things you can
change with a certificate. Two things you can’t change
are the valid range of the certificate the certificate
subject. You can, however, change the private key as well
as the password the private key is encrypted with. I’ll only
focus on the changing password option:
ALTER CERTIFICATE <certificate name>
WITH PRIVATE_KEY (
[, DECRYPTION_PASSWORD = ‘<old password>’]
master key, the decryption_password is not needed. In
the example I used above, however, I did specify a
password for ExampleCert2, so I’ll need to specify the old
password in order to set a new one:
ALTER CERTIFICATE ExampleCert2
WITH PRIVATE_KEY (
ENCRYPTION_PASSWORD = ‘GandalfTheWhite’,
DECRYPTION_PASSWORD = ‘GandalfTheGrey’ )
Dropping:
When a certificate is no longer useful, whether it’s
due to the fact that it’s expired, it has been compromised,
or the data is no longer being encrypted, dropping the
certificate is simple:
DROP CERTIFICATE <certificate name>
Granting Access to a Certificate:
I mentioned earlier you could give access to a
certificate without making a user an owner of the
certificate. SQL Server 2005 completely restructures the
permissions hierarchy from SQL Server 2000 and also
introduces new options such as CONTROL. CONTROL
basically gives all permissions and is everything short of
actual ownership. CONTROL is what is needed to use a
certificate. The permissions hierarchy changes are
beyond the scope of this article so I’ll just provide two
examples on how to grant access to the certificates
created earlier, one for each created principal. If you want
to understand more about changes to permissions, refer
to that topic in the SQL Server 2005 Beta 2 Books Online.
25
GRANT CONTROL ON CERTIFICATE::ExampleCert1 TO
TestUser1
GRANT CONTROL ON CERTIFICATE::ExampleCert2 TO
TestUser2
Encrypting/Decrypting Data with Certificates:
Here’s where the rubber meets the road. Having
certificates doesn’t do us any good if we can’t use them
to encrypt and decrypt data. In SQL Server 2005 there
are two functions in order to do just that:
EncryptByCert( <certificate ID>, <plain text> )
DecryptByCert( <certificate ID>, <encrypted
text>
[, <password>] )
The password is only necessary if the certificate’s
private key was encrypted using a password If the
certificate’s private key was encrypted using the
database master key, only the certificate ID and the
encrypted text is necessary with DecryptByCert(). The
certificate ID can be determined from the certificate
name using another function:
Cert_ID(‘<certificate name>’)
One thing to understand about DecryptByCert() is
the result from the function is of the varbinary type.
Therefore, the return value has to be changed to a
useable form with the Convert() function. This is true of
all of the Decrypt functions.
Putting all of it together, here’s an example of
inserting encrypted data into the table and then
retrieving it back out in decrypted form. If you attempt to
use a certificate you don’t have access to when inserting
the data, a null value will be inserted instead (providing
the column allows for nulls). You will likewise receive a
null value should you try to decrypt with a certificate you
don’t have access to. To see how access to the
certificates work, execute the statements both as
TestUser1 and TestUser2 and note the differences.
DECLARE @Plain nvarchar(1000)
SET @Plain = ‘Example Plaintext’
INSERT INTO dbo.ExampleTable
(EncryptMethod, PlainData, CipherData) VALUES
(‘ExampleCert1’, @Plain,
EncryptByCert(Cert_ID(‘ExampleCert1’), @Plain))
INSERT INTO dbo.ExampleTable
(EncryptMethod, PlainData, CipherData) VALUES
(‘ExampleCert2’, @Plain,
EncryptByCert(Cert_ID(‘ExampleCert2’), @Plain))
SELECT
26
ExampleID, EncryptMethod, PlainData,
CipherData,
CASE
WHEN EncryptMethod = ‘ExampleCert1’ THEN
CONVERT(nvarchar(max),
DecryptByCert(Cert_ID(‘ExampleCert1’),
CipherData))
WHEN EncryptMethod = ‘ExampleCert2’ THEN
CONVERT(nvarchar(max),
DecryptByCert(Cert_ID(‘ExampleCert2’),
CipherData,
N’GandalfTheWhite’))
END DecryptedData
FROM dbo.ExampleTable
Asymmetric Keys
Asymmetric keys are similar to certificates in that
there is a public and private key pair. However,
asymmetric keys don’t have a validity period and with
respect to SQL Server aren’t specifically used to identify
a person, device, or service. Asymmetric keys tend to be
used more to encrypt symmetric keys than data. I’ll
explain this in more detail when I talk about where and
how to encrypt.
Creating:
Creating an asymmetric key is similar to a certificate,
but there are different options, such as the encryption
algorithm. There is also the possibility of specifying an
asymmetric key source outside of SQL Server (path,
executable, or assembly, as with the certificate),
however, I will look at the case where the asymmetric key
is generated entirely in SQL Server:
CREATE ASYMMETRIC KEY <key name> [AUTHORIZATION
<user>]
WITH ALGORITHM = <encryption algorithm>
[ ENCRYPTION BY PASSWORD = ‘<password>’]
The available encryption algorithms are RSA_512,
RSA_1024, and RSA_2048. If the ENCRYPTION BY
PASSWORD clause is not specified, the database master
key is used to encrypt the private key. Here are two
examples, the first encrypting with the database master
key and the second with a password I’ve specified.
CREATE ASYMMETRIC KEY AsymmKey1 AUTHORIZATION
TestUser1
WITH ALGORITHM = RSA_1024
CREATE ASYMMETRIC KEY AsymmKey2 AUTHORIZATION
TestUser2
WITH ALGORITHM = RSA_1024
ENCRYPTION BY PASSWORD = ‘TheShire’ CASE
WHEN EncryptMethod = ‘AsymmKey1’ THEN
Altering: CONVERT(nvarchar(max),
As with certificates, the password encrypting the DecryptByAsymKey(AsymKey_ID(‘AsymmKey1’),
private key can be changed. The only difference in syntax CipherData))
is ASYMMETRIC KEY instead of CERTIFICATE as in the
WHEN EncryptMethod = ‘AsymmKey2’ THEN
following example:
CONVERT(nvarchar(max),
ALTER ASYMMETRIC KEY AsymmKey2
WITH PRIVATE_KEY ( DecryptByAsymKey(AsymKey_ID(‘AsymmKey2’),
ENCRYPTION_PASSWORD = ‘MasterFrodo’, CipherData,
DECRYPTION_PASSWORD = ‘TheShire’ ) N’MasterFrodo’))

Dropping: END DecryptedData

Dropping an asymmetric key is also similar to the FROM dbo.ExampleTable
syntax for a certificate. Another example:
DROP ASYMMETRIC KEY ASymmKey2 Symmetric Keys
A symmetric key is one which is used both for
Encrypting/Decrypting Data with Asymmetric Keys: encryption and decryption. Symmetric key algorithms
Like with certificates, there are functions to encrypt also tend to be fast relative to asymmetric key algorithms
and decrypt data using asymmetric keys. There is also a and are often chosen when large amounts of data have
function to find the asymmetric key ID, much like there is to be encrypted and decrypted quickly. SQL Server 2005
one to determine the certificate ID. Here is the syntax for provides support for symmetric keys using a variety of
those functions: encryption algorithms.

EncryptByAsymKey( <key id>, <plain text> ) Creating:

DecryptByAsymKey( <key id>, <encrypted text> [,
‘<password>’] )
AsymKey_ID(‘<key name>’)
As with the decryption by certificate, the resulting
value is of varbinary type and will need to be converted.
Here is the example using asymmetric keys for
encryption and decryption. Again, the first asymmetric
key used the database master key and the second
asymmetric key used a specified password:
DECLARE @Plain nvarchar(1000)
SET @Plain = ‘Example Plaintext’
INSERT INTO dbo.ExampleTable
(EncryptMethod, PlainData, CipherData) VALUES
(‘AsymmKey1’, @Plain,
EncryptByAsymKey(AsymKey_ID(‘ASymmKey1’),
@Plain))
INSERT INTO dbo.ExampleTable
(EncryptMethod, PlainData, CipherData) VALUES
(‘AsymmKey2’, @Plain,
EncryptByAsymKey(AsymKey_ID(‘ASymmKey2’),
@Plain))
SELECT
ExampleID, EncryptMethod, PlainData,
CipherData,
JANUARY/ FEBRUARY, 2005
Creating a symmetric key is similar to creating a
certificate or asymmetric key. The big difference is a
symmetric key can have a certificate, an asymmetric key,
another symmetric key, or a password to encrypt the
symmetric key. The syntax for creating a symmetric key
is:
CREATE SYMMETRIC KEY < key name> [AUTHORIZATION
<user>]
WITH <key options>
ENCRYPTION BY <encryption mechanism>
The only key option I’ll look at is the algorithm. The
following symmetric key encryption algorithms are
available: DES, TRIPLE_DES, RC2, RC4, DESX,
AES_128, AES_192, and AES_256. The encryption
mechanisms are more varied:
CERTIFICATE <certificate name>
|
PASSWORD = ‘<password>’
|
SYMMETRIC KEY <key name>
|
ASYMMETRIC KEY <key name>
Unlike certificates and asymmetric keys, an
encryption mechanism must be specified. I’ve providede
27
two examples. The first uses an asymmetric key (created
earlier) and the second uses a password:
CREATE SYMMETRIC KEY SymmKey1 AUTHORIZATION
TestUser1
WITH ALGORITHM = AES_256
ENCRYPTION BY ASYMMETRIC KEY AsymmKey1
CREATE SYMMETRIC KEY SymmKey2 AUTHORIZATION
TestUser2
WITH ALGORITHM = TRIPLE_DES
ENCRYPTION BY PASSWORD = ‘GatesOfMordor’
Altering:
Altering a symmetric key is different from both
certificates and asymmetric keys. Like with a database
master key (which is a symmetric key), mechanisms of
encryption can be added and removed. The syntax is:
ALTER SYMMETRIC KEY <key name>
{ ADD ENCRYPTION BY <encryption mechanism> |
DROP ENCRYPTION BY <encryption mechanism> }
The encryption mechanisms are the same for
CREATE SYMMETRIC KEY. Multiple encryption
mechanisms can be present at the same time. However,
before any changes can be made to a symmetric key
(and before a symmetric key can be used, period), the
symmetric key must be opened. The syntax to open a
symmetric key is:
OPEN SYMMETRIC KEY <key name>
USING <encryption mechanism>
Here is an example where the second symmetric key
is opened and an asymmetric key is added as an
encryption mechanism.
OPEN SYMMETRIC KEY SymmKey2
USING PASSWORD = ‘GatesOfMordor’
EncryptByKey( <key GUID>, <plain text> [, <col
name> | <col value>] )
DecryptByKey( <encrypted text> [, <col name> |
<col value>] )
Key_GUID(‘<key name>’)
The <col name> or <col value> is an optional
parameter. It’ll be hashed and concatenated with the
plain text before encryption or used to validate the data
during decryption. Its sole purpose is data integrity. I did
not use this optional parameter in the examples. I’ve
included a second opening of SymmKey2 demonstrating
how to open a symmetric key encrypted with an
asymmetric key having a password.
OPEN SYMMETRIC KEY SymmKey1
USING ASYMMETRIC KEY AsymmKey1
OPEN SYMMETRIC KEY SymmKey2
USING PASSWORD = ‘GatesOfMordor’
OPEN SYMMETRIC KEY SymmKey2
USING ASYMMETRIC KEY AsymmKey2
WITH PASSWORD = ‘MasterFrodo’
DECLARE @Plain nvarchar(1000)
SET @Plain = ‘Example Plaintext’
INSERT INTO dbo.ExampleTable
(EncryptMethod, PlainData, CipherData)
VALUES
(‘SymmKey1’, @Plain,
EncryptByKey(Key_GUID(‘SymmKey1’), @Plain))
INSERT INTO dbo.ExampleTable
(EncryptMethod, PlainData, CipherData)
VALUES
(‘SymmKey2’, @Plain,
EncryptByKey(Key_GUID(‘SymmKey2’), @Plain))

ALTER SYMMETRIC KEY SymmKey2 SELECT

ADD ENCRYPTION BY ASYMMETRIC KEY AsymmKey2 ExampleID, EncryptMethod, PlainData,
CipherData,
Dropping: CASE
Dropping a symmetric key is similar to dropping a WHEN EncryptMethod = ‘SymmKey1’ THEN
certificate or an asymmetric key: CONVERT(nvarchar(max),
DecryptByKey(CipherData))
DROP SYMMETRIC KEY SymmKey2 WHEN EncryptMethod = ‘SymmKey2’ THEN
CONVERT(nvarchar(max),
Encrypting/Decrypting Data with Symmetric Keys: DecryptByKey(CipherData))
To encrypt or decrypt data with a symmetric key, it END DecryptedData
must be opened. Once that is accomplished, you can use FROM dbo.ExampleTable
the specified symmetric key algorithms. They are similar
to the other two encryption/decryption methods: Where and How to Encrypt?
The decision of trust is an important one when

28
deciding where to apply the encryption and whether or
not to use the database master key. SQL Server provides
data encryption at two different “trust levels.” If DBAs are
trusted, SQL Server can handle all of the key
management using the database master key. This is by
far the simplest method of implementing encryption in
the database. Asymmetric keys and certificates can be
accessed automatically and symmetric keys should be
opened fairly easily as well.
If, however, due to business or other reasons even
the DBAs shouldn’t be able to decrypt the data, then the
use of keys with passwords effectively keeps them out
from within SQL Server itself. Even profiler won’t reveal
important contents of key commands. An administrator,
however, can hook up a debugger, and setting the
proper breakpoints, see a query come through and see
the password. Of course, that’s true of any server. Using
SQL Server’s native encryption, these are the two options
provided.
The only other option is to build in encryption into the
application and send encrypted data to SQL Server. This
is considered a third “trust level” because it should be
completely independent of the DBAs and the SQL Server
platform itself.
Use of a Hybrid Encryption Scheme
In comparison of performance, symmetric key
algorithms tend to be substantially faster and if you’re
dealing with a lot of conversions (as with a database),
symmetric keys offer the best performance. However,
symmetric keys are also weaker than asymmetric keys for
the simple fact that once I have the key, I can encrypt and
decrypt the data at will. With an asymmetric key pair, I
can expose the public key, which you can use to encrypt
data intended for me. You can then send the encrypted
data over a non-secure channel without worrying about
compromise. Once I receive it, I can decrypt using my
private key. No one else can unless they’ve managed to
get my private key
However, if we have a lot of data to send back and
forth, it’s better to use a symmetric session key and then
encrypt it using my public key. I decrypt the session key
and then we encrypt all data using that session key.
We’ve been able to exchange a symmetric key and we’ve
not had to send it in the clear over a non-secure channel.
Therefore both of us possess the symmetric key securely
and are able to leverage the faster speed of the
symmetric encryption algorithms. These hybrid
cryptosystems are actually fairly popular. This is the
principle behind Secure Sockets Layer (SSL) as well as
Encrypting File System (EFS): A symmetric key is
encrypted with an asymmetric key pair and the data is
JANUARY/ FEBRUARY, 2005
encrypted with the symmetric key.
The holder of the private key can decrypt the
symmetric key and then use that symmetric key to
decrypt the data. It also makes a good practice for SQL
Server. Create an asymmetric key and then create a
symmetric key using the asymmetric key as the
encryption mechanism. Encrypt the data with the
symmetric key and you have a good balance of both key
security and performance.
Issues to Consider
When implementing encryption, several issues must
be addressed. The first is the one of key management. If
SQL Server is not handling key management, you must
come up with a secure mechanism to store passwords
and allow an application to access keys. If a critical key is
compromised, so is the data. Therefore, protecting the
keys is almost as important as protecting the data itself.
The second main issue is performance. This
consideration has two prongs: performance loss due to
cycles spent encrypting and decrypting the data and
performance loss due to an inability to use indexes
effectively since the data is encrypted. Using a hybrid
cryptosystem like I mentioned above is probably the best
solution for the first prong. However, an expectation of
slightly slower performance should be expected. With
the second prong, it’s a matter of if the encrypted data is
used to filter or sort a result set. If not, there shouldn’t be
a big performance problem. If, however, the encrypted
data is used to do either of those two functions, indexes
are effectively out. Expect a slightly to significantly worse
performance as a result.
Conclusion
Native encryption built into SQL Server 2005 looks
very promising. This addresses an item on the wish list of
a number of DBAs and Microsoft looks to have
implemented a reasonable solution. Certificates,
asymmetric keys, and symmetric keys are all supported
within SQL Server and each encryption mechanism has
its own set of encryption/decryption functions. There are
trust and performance issues that must be addressed
before implementing a solution, but overall SQL Server
2005 gives us a lot of flexibility to come up with workable
solutions for our environments. I hope encryption
remains and makes it into the final product.
Brian Kelley is an Enterprise Systems Architect with AgFirst
Farm Credit Bank and the regular security columnist for
SQLServerCentral.com. He is also the author of “Start to
Finish Guide to SQL Server Performance Monitoring” from
NetImpress and a member of the GSEC Advisory Board. You
can contact him at SQLServerCentral.com.
29
PASS_0111204_SummitAd 12/7/04 2:35 PM Page 1

Register Now for the

2005 PASS Events

2005 PASS European Conference 2005 PASS Community Summit

11–13 May 2005
Hilton Munich Park
Munich, Germany
Register by 28 February 2005 for only 995€!
• Benefit from two days filled with highly technical sessions
presented by Microsoft, MVP’s and highly regarded industry
professionals, focusing on three tracks: Data Warehousing
and Business Intelligence, Database and Application
Development, and DBA/Enterprise Database Administration
and Deployment.
• Hear from members of the SQL Server development team at
Microsoft on the hottest topics to date including:
• What’s New for Developers in SQL Server 2005 –
Gert Drapers, Microsoft
• Upgrading to SQL Server 2005 Management Tools –
Euan Garden, Microsoft
• Reporting Services Enhancements for SQL Server 2005 –
Jason Carlson, Microsoft
• View the latest products and solutions at the Vendor Fair
designed specifically for users in the Europe, Middle East and
Africa (EMEA) region.
September 27– 30, 2005
Gaylord Texan Resort & Convention Center
Grapevine (Dallas), Texas USA
Register by June 30, 2005 for $1295!
• Be a part of the LARGEST event exclusively dedicated to
Microsoft SQL Server education and training.
• Get the most comprehensive coverage of SQL Server 2005
anywhere while you develop and expand your knowledge
of SQL Server 2000.
• Meet and network with members of the Microsoft SQL
Server development team, SQL Server MVP’s and other SQL
Server professionals.
• Check out the world's premier SQL Server vendors at the
largest, most comprehensive tradeshow dedicated to
SQL Server.
Visit www.sqlpass.org for more information
or to register for the 2005 PASS Events.
One-year complimentary PASS membership is also
included. Register today!

Present a Technical Session and Receive a
Complimentary Registration and Worldwide
Industry Recognition.
If you would like to share your expertise at the 2005
European Conference, please submit your proposal at
www.sqlpass.org. This is a great opportunity to solidify
your reputation as a SQL Server expert.
Looking for a way to freshen up on your
SQL Server skills before attending one of the
PASS Conferences?
The 2004 PASS Community Summit technical sessions have all
been compiled on a CD-ROM with full desktop capture, MP3
audio, live demos, speaker PowerPoint presentations and
much more! Order your copy today at www.sqlpass.org.

ORKING TOG
R S W ET
HE
S E
U R

d
E

u
ca it
tio un
m
n u Networking u Com

www.sqlpass.org
 PASS
About the Author:
Rick Heiges is a Database
Consultant with Scalability Experts.
Prior to joining Scalability Experts,
he was on the full-time faculty at
High Point University specializing
in Database Coursework. He also
serves as Director of Membership
of PASS and founded the local SQL
Server User Group in High Point,
North Carolina. He is currently
under contract with McGraw-
Hill/Irwin to write a textbook on
SQL Server for college-level classes.
JANUARY/ FEBRUARY, 2005
The Definitive Global Community
for SQL Server Professionals
Introducing Database
Mirroring
Database Mirroring is one of the
coolest new features in SQL
Server 2005. What does
Database Mirroring do for you?
Database Mirroring provides an
up-to-date copy of your data on
another server. But it provides
more than just a warm and
fuzzy feeling that your data is in
more than one place for safety.
It also provides another High
Availability or Disaster Recovery
option for SQL Server as well as
an ability to do some reporting
from the mirror server.
Microsoft offers Clustering as its
current “native” solution for
High Availability in SQL Server
2000. Clustering is not going
away in SQL Server 2005 and it
is receiving some enhancements
which will not be discussed
here. Database Mirroring has
been described as the evolution
of Log Shipping and has also
been referred to as Transaction
Shipping. Log Shipping is NOT
going away in SQL Server 2005,
but it is not getting any new
major features other than bug
fixes.
How it works….
The server that contains the
“real” database will be called
ServerA and will initially fill the
“principal” role. The server
containing the “mirror” of the
database will be called ServerB
and will initially fill the “mirror”
role. There is also a third server
which we will call ServerW and
serve in the role of “witness” in
the High Availability scenarios.
• When a transaction from a
client hits the principal (step
#1), it is recorded in the
transaction log as it
normally would (step #2).
• While it is making the
change in the data file (step
#3), it sends the transaction
(step #3) to the mirror
server (ServerB).
• The mirror will then record
the transaction in its log file
(step #4) and then start to
write the change in its data
file (step #5) while sending
an acknowledgement (step
#5) back to the principal.
• Once the principal receives
the acknowledgement, the
transaction is committed
(step #6) and the data is
mirrored successfully.
• There is no data that gets
31
 transmitted to or through
the witness during this
process.
Server Roles…
The principal server handles
your client connections and
allows access to the database. It
is essentially this same server
setup used today in your
applications. The mirror server is
constantly restoring / redoing
the transaction log from the
principal server. This way it has
an up-to-date copy of the
database at all times. The
clients cannot access the mirror
server directly unless a failover
occurs. When a failover occurs,
the mirror server becomes the
principal server. In reality, clients
never access the server when it
is in the mirror role. The witness
server is there for one reason
and one reason only. The
reason is to provide automatic
failover. The witness monitors
the other servers and confirms
connectivity and roles being
played by each server. By having
the witness, there can be a
quorum. It takes at least two
servers to decide which server
will play the principal role; this
prevents the possibility of two
servers assuming the same role
at the same time. If the witness
did not exist, there might be a
possibility that both ServerA
and ServerB assume the
principal role if one of them had
a brief network connection
outage. The witness is there to
vote in this process. This means
that the server must talk to at
least one other server before
32
assuming the principal role. The
witness server is simply another
instance of SQL Server and can
act as the witness for more
than one mirroring session. The
role of the witness takes up very
little in the way of resources
and is not a single point of
failure. If the Witness crashes,
it would simply mean that
automatic failover is not
possible although manual
failover is possible. The witness
may live on the same server as
the principal or the mirror, but
that is not recommended and
would be considered a “worst”
practice.
Benefits
Database mirroring provides for
two basic benefits. It provides
for high availability or disaster
recovery. The high availability
scenario means that you have
automatic failover and you can
lose the Principal server and still
be available to your clients. This
means that the safety level is set
to “full”. Although database
mirroring may be set up with
the servers not physically close
to one another, having the
servers physically close helps to
eliminate network latency. If
the purpose of your Database
Mirroring session is to provide
High Availability and the servers
experience network latency, the
performance of your system
will be affected. The Disaster
Recovery scenario (also referred
to as asynchronous scenario) is
used where the mirror server is
used as a warm standby server.
In this scenario, the servers are
more likely to be physically far
apart from one another. The
safety level is set to “off”.
Failover to the mirror server can
only occur if the DBA initiates it.
Because the safety is off, there
is a chance for some data loss
on a very small scale. This is
because the principal server
does not wait for an
acknowledgement from the
mirror that the transaction has
been received before
acknowledging to the client
that the transaction has
successfully been committed.
This scenario fits in well for
disaster recovery. There is also
no need for a witness in the
asynchronous scenario.
Setting it Up…
Database mirroring is initially set
up by backing up the data to be
mirrored on the principal server
and restoring it to the mirror
server. Once the full backup,
differential backup (if any), and
transaction log backups have
been applied to the mirror
server, the commands to
establish the partnership
between the principal and the
mirror may be issued. Once the
commands are issued, the
principal starts sending the
latest updates since the last
transaction log back up to the
mirror server. For a small- to
medium-size database, this
works quite well. However, a
large database may take an
extraordinary amount of time to
initially set up due to the volume
of data. In the case of the VLDB,
clustering may be a much better
 solution. During this period of
initial setup, the database will
have a session state of
synchronizing. This means that
the database is working its way
to the synchronized state.
Synchronized state occurs when
all updates from the principal
have been sent and
acknowledged that was sent by
the principal server. There is also
a suspended session state which
means that the database is
behind because it was
suspended by the DBA, but
communication with the
partners is still possible. The last
state is called a disconnected
state. This means that the
partner cannot communicate
with the other partner. Only in
the SYNCHRONIZED state, you
are safe from data loss. The
three other states indicate that
you are exposed to possible
data loss.
This is the basic syntax for
establishing a mirroring session
once the database is restored
with NO RECOVER on the
mirror server. More information
may be obtained in BOL.
ALTER DATABASE <database>
SET PARTNER =
<server_network_address>
This command is run on both
the principal and mirror servers.
This syntax promotes simplicity.
If mirroring is for high
availability, another T-SQL
command is issued to the
“ALTER DATABASE” syntax to
specify the Mirror server.
JANUARY/ FEBRUARY, 2005
Reporting for Duty….
Another really cool feature in
SQL Server 2005 is the
Database Snapshot. Basically,
this feature allows a complete
copy of a database to exist at
the time it commands to create
the snapshot. It results in a very
space-efficient, read-only copy
of the database. We will not
fully explore this feature here,
but it can be very useful in
gaining another benefit. A
database snapshot maybe
created on the mirror server.
This allows read-only access to
the data stored on the mirror
server at the point in time when
the commands were issued to
create the database snapshot.
Once a transaction is processed
after the command has been
issued, the database snapshot is
no longer up-to-date. But, this
does allow for the mirror server
to also be used as a reporting
server. However, putting more
stress on the mirror server can
cause a slowdown of
processing which may affect
the mirroring session. This is
because it may affect the speed
in which the transaction is
recorded which would in turn
affect the speed of the
acknowledgement back to the
principal server. Remember, the
principal does not acknowledge
the transaction until the Mirror
responds that the transaction
has been successfully recorded
when the safety is set to “full”.
Client Concerns
The discussions so far have
been focused on the database.
Failover can occur over and
over again, but what about the
client applications? Clients can
connect to the principal
database either by using the
latest MDAC or by a slight
modification of code (about 10
lines) concerning the
connection. Essentially, the
client tries to connect to the
principal. If it fails, it attempts
connection to the secondary
server which will be the new
principal if failover has
occurred. At no time can a
client connect to a database on
a server in the mirror role. The
servers that participate in
mirroring have their own IP
address and do not share a
third IP address, as is the case in
clustering.
Summary
The Database Mirroring feature
on SQL Server 2005 provides a
much simpler method of
obtaining High Availability or
Disaster Recovery than before.
Although there are still many
good reasons to consider a
clustering solution, mirroring
does not require any special
hardware or shared storage to
complete the setup. In
addition to these reasons, a
mirroring session may also
provide access to data for
reporting purposes on the
mirrored server. At the time of
this article, there was no
decision of which version of SQL
Server this would appear in.
33
 PASS Reflects on
Significant Global
Expansion in 2004:
Looking Forward to
Further Strengthening its
European Presence in 2005
by: Professional Association
for SQL Server
The 2004 PASS Board of
Directors is proud to announce
the success of a continued
push to serve the needs of
global users of SQL Server
technology. Since it’s inception
in 1999, PASS has ensured
that SQL Server users’
educational needs have been
met both through a global
community structure as well as
a network of local chapters.
Overall growth in Chapters has
grown to over 80 worldwide,
with our total number of
Chapters at the end of 2002
more than doubling in 2003
and 2004 alone. “ In addition
to significant growth in the
United States, PASS has
successfully extended its
community and benefits to
SQL Server users throughout
the world” explains PASS
President, Kevin Kline, “in part
due to our extremely
dedicated members who help
drive the process as well as
through our close cooperation
with Microsoft.”
401 North Michigan Avenue
Chicago, Illinois 60611- 4267
34
Extending from across the
United States to Western and
Eastern Europe, PASS Official
Chapters have recently sprung
up in Germany, Norway, the
United Kingdom, Denmark and
Bosnia & Herzegovina. This
growth in Europe has helped to
fuel PASS’ educational
offerings in the region, from
hosting a SQL Server Academy
in Dubrovnik, Croatia in June of
2004, to recently announced
plans to build upon that effort
with the 2005 PASS European
Conference, 11-13 May in
Munich, Germany, at the Hilton
Munich Park.
Europe is not the only success
story in terms of PASS’ global
expansion. With affiliated
groups located in Australia and
Japan, growth in Asia has
experienced a tremendous
surge of energy in India.
Realizing that India’s rapidly
growing marketplace of SQL
Server developers and users did
not yet have a user group
dedicated to SQL Server
education and networking,
PASS member Rushabh Mehta
worked with Kurt Allebach, the
chapter liaison on the PASS
Board of Directors, and Joe
Homnick, a Microsoft Regional
Director, to find a way to serve
this need. With help from
Microsoft Regional Director,
Phone: 312.527.6742
Fax: 312.245.1081
E-mail: passhq@sqlpass.org
Web: www.sqlpass.org
Sanjay Shetty, the PASS
Chapter network in India now
extends across multiple cities in
India, including Bangalore,
Hyderbad, Mumbai,
Trivandrum and Pune, and
Chapter meetings are held on a
monthly basis in each city.
“PASS is a savior for all of those
in the SQL Server community,
especially DBAs, OLAPs and
SQL Developers who previously
had nowhere to turn. PASS has
played a crucial role in
providing an opportunity for
this segment of the [SQL
Server] community to interact
and grow,” comments Shetty.
Some of the proposed plans for
the future include growth into
additional cities throughout
India where there is a
significant SQL Server
development community,
hopes to expand into other
APAC countries and plans for a
regional 3-5 day joint event
with both the INETA and PASS
communities.
For additional information on
PASS and the benefits of
becoming a member, please
visit the PASS Web site at
www.sqlpass.org, or contact
PASS Headquarters at
312.527.6742.
 SQL Server Standard May/June, 2004

36