I have received call from my DBA friend who read my article SQL SERVER 2005 Introduction to Partitioning.

g. He suggested that I should write simple tutorial about how to horizontal partition database table. Here is simple tutorial which explains how a table can be partitioned. Please read my article Introduction to Partitioning before continuing to this article. Step 1 : Create New Test Database with two different filegroups I have written tutorial using my C: Drive, however to take advantage of partition it is recommended that different file groups are created on separate hard disk to get maximum performance advantage of partitioning. Before running following script, make sure C: drive contains two folders Primary and Secondary as following example has used those two folder to store different filegroups.
USE GO --Step 1 : Create New Test Database IF EXISTS SELECT FROM WHERE name DROP DATABASE GO CREATE DATABASE ON (NAME='TestDB_Part1', FILENAME= 'C:\Data\Primary\TestDB_Part1.mdf', SIZE=2, MAXSIZE=100, FILEGROWTH=1 FILEGROUP (NAME = FILENAME 'C:\Data\Secondary\TestDB_Part2.ndf', SIZE = MAXSIZE=100, FILEGROWTH=1 GO Master; with two different filegroups. ( name sys.databases N'TestDB') TestDB; TestDB PRIMARY

), TestDB_Part2 'TestDB_Part2', = 2, );

Step 2 : Create Partition Range Function Partition Function defines the range of values to be stored on different partition. For our example let us assume that first 10 records are stored in one filegroup and rest are stored in different filegroup. Following function will create partition function with range specified.
USE GO --CREATE AS VALUES GO TestDB; Step 2 PARTITION : RANGE Create FUNCTION Partition Range TestDB_PartitionRange LEFT Function (INT) FOR (10);

on image to see larger image Step 3 : Attach Partition Scheme to FileGroups Partition function has to be attached with filegroups to be used in table partitioning. In following example partition is created on primary and secondary filegroup.
USE GO --CREATE AS TO GO TestDB; Step 3 : Attach Partition PARTITION SCHEME PARTITION ([PRIMARY], Scheme to FileGroups TestDB_PartitionScheme TestDB_PartitionRange TestDB_Part2);

Click

Step 4 : Create Table with Partition Key and Partition Scheme The table which is to be partitioned has to be created specifying column name to be used with partition scheme to partition tables in different filegroups. Following example demonstrates ID column as the Partition Key.
USE GO --Step CREATE (ID Date ON GO TestDB; 4 : Create Table INT TestDB_PartitionScheme with Partition TABLE Key NOT and Partition Scheme TestTable NULL, DATETIME) (ID);

Step 5 : (Optional/Recommended) Create Index on Partitioned Table This step is optional but highly recommended. Following example demonstrates the creation of table aligned index. Here index is created using same Partition Scheme and Partition Key as Partitioned Table.
USE GO --TestDB; Step 5 : (Optional/Recommended) Create Index on Partitioned Table

CREATE ON ON GO

UNIQUE

CLUSTERED TestDB_PartitionScheme

INDEX

IX_TestTable TestTable(ID) (ID);

Step 6 : Insert Data in Partitioned Table Insert data in the partition table. Here we are inserting total of 3 records. We have decided that in table partition 1 Partition Key ID will contain records from 1 to 10 and partition 2 will contain reset of the records. In following example record with ID equals to 1 will be inserted in partition 1 and rest will be inserted in partition 2.
USE GO --INSERT VALUES INSERT VALUES INSERT VALUES GO TestDB; Step INTO INTO INTO 6 : TestTable TestTable TestTable Insert (ID, Date) (ID, (ID, Date) Date) Data ---in Inserted Inserted Inserted Partitioned Table in Partition 1 (1,GETDATE()); in Partition 2 (11,GETDATE()); in Partition 2 (12,GETDATE());

Step Query
USE GO --SELECT FROM GO

7 TestTable
Step 7

: and

Test see the

: Test

Data values
Data

from inserted
from

in

TestTable TestTable.
TestDB; TestTable * TestTable;

Step 8 : Verify Rows Inserted in Partitions We can query sys.partitions view and verify that TestTable contains two partitions and as per Step 6 one record is inserted in partition 1 and two records are inserted in partition 2.
USE GO --SELECT FROM WHERE GO TestDB; Step 8 : Verify Rows Partitions * sys.partitions OBJECT_NAME(OBJECT_ID)='TestTable'; Inserted in

Partitioning table is very simple and very efficient when used with different filegroups in different tables. I will write very soon more articles about Table Partitioning. If you need any help in table partitioning or have any doubt, please contact me and I will do my best to help you. Reference : Pinal Dave (http://blog.SQLAuthority.com)

Share this: Facebook Digg StumbleUpon Reddit Print

SHOPPING SHORTCUTS

Ads by Google

JDBC Driver SQL Server

Type-4 Compliant Java / SQLServer Downlad Now. Free Evaluation! www.datadirect.com/JDBC-SQL

Posted in Database, Pinal Dave, SQL, SQL Authority, SQL Data Storage, SQL Index, SQL Performance, SQL Query, SQL Scripts, SQL Server, SQL Tips and Tricks, T SQL, Technology | Tagged Partitioning | 79 Comments
Like Be the first to like this post.

79 Responses
1. on January 26, 2008 at 3:10 pm | Reply SQL SERVER - 2005 - Introduction to Partitioning Journey to SQL Authority with Pinal Dave [...] SQL SERVER 2005 Database Table Partitioning Tutorial How to Horizontal Partition Database Tabl [...]

2. on February 15, 2008 at 10:00 am | Reply hi..

pramod bhat

how to increase performance of a table which containes data in croresalso the query which is used to retrieve data is large/big. plz help me out thanks, pramod

3. on March 5, 2008 at 4:22 pm | Reply Hi Can anyone please help me out,

Abhishek Doyle

Does PARTITION TABLE help to improve performance in terms of INSERTION into a TABLE ? I have two windows services, which pick the MESSAGE from MSMQ and inserts the data into a TABLE of SQL Server 2005, execution of any heavy query hampers insertion process of my services and pileup gets started at MSMQ end.

Would it be really useful to me if i create two partitions into my table for respective windows services ? NOTE : Table contains billions of record

4. on March 11, 2008 at 8:34 pm | Reply Thanks for the article.. got a question.

Kenneth Leong

With a table with horizontal partition in place (say by id) if the select statement in your store proc does not have a where clause of account id, then that store proc wont benefit from the partitioning right ? ie. no perf gain?

5. on March 19, 2008 at 9:21 pm | Reply

Thien Nguyen

Is it possible for an admin to move a partition from one server to another server as part of archiving? From all the reading, it does not seem to be possible. Thanks.

6. on April 22, 2008 at 11:31 am | Reply

Ashley James

Is it possible to create a partition using multiple columns in SQL Server 2005?

7. on June 25, 2008 at 3:57 am | Reply

Sameer

We have OLTP environment with 50,000 concurrent user and considering table partition option due to CPU bottleneck. We dont have nay memory issue on system but as User grows, CPU increases. Since we are moving data between partitions and we use data using Partition column, wont our CPU on Select statements on table come down and improve CPU? Got confused from article as it says there wont be significant gain if its CPU issue. Please clarify. Thanks!!

8. on July 2, 2008 at 12:38 pm | Reply

rrblr

We have 2 tables in our DB Schema. Table1 is partioned by col1 and table2 is partitioned by col2. now we have to insert values to table1 and table2 inside one transaction. we are getting error saying transaction cannot be used across multiple partitions. Any thoughts on using transaction across multiple partitions thanks

9. on July 22, 2008 at 12:07 am | Reply Hi Dave,

Ritesh

I wanted to check if it possible to create a Table Partition on the column which is not a primary key. Can you please advise. Thanks in Advance. Regards, Ritesh

10. on July 23, 2008 at 11:08 pm | Reply

ali

Before creating a partition schema and function, how can I check if they already exists???

11. on September 8, 2008 at 10:09 am | Reply

Mustaque Ahemed

The article about the partitioning is very helpful. I need your help to partition the existing table. thanks

12. on September 9, 2008 at 1:08 am | Reply

Scott

Hello, I have a table with 100 million rows of values ( id, date ,value) .. 90% of the time only the current month data is accessed. Nightly 50,000 rows are inserted with yesterdays value. In the past, i would create a history table and archive the older data.. is partioning by date a good solution?

13. on October 4, 2008 at 12:34 pm | Reply Hi, Need your help for partion my table

maheshkumar

14. on October 4, 2008 at 5:03 pm | Reply

i want to know how i can create a partition for my existing table? the sample you provided creates the table

15. on October 20, 2008 at 10:52 am | Reply

AQ

I still need to know How I can partition my existing database. I released my application online before a week, and now I have more than 2 million records in my table. Can you lead me how I can partition my existing huge table.

16. on November 12, 2008 at 8:12 pm | Reply

Naren Sangu

Can we create partition table with Primary Key column other than Partition key column

17. on November 22, 2008 at 7:31 am | Reply Hi,

COBRASoft

I have several tables with more than 2 million records. It is about invoices, invoicedetail, invoicedescription (multi-language) and invoicepayments. I get time-outs everywhere in my software due to this huge tables. Any idea if partitioning will solve this? Do you have an idea how I could partition these? Greetings, Sigurd

18. on December 1, 2008 at 12:12 pm | Reply

Andy

Can you tell me how to convert an existing table into a partition table? I have some tables that need to be partitioned due to the huge amount of data stored.. Thanks in advance :)

19. on December 4, 2008 at 3:34 pm | Reply

Andrei

@ Andy Daves article is excellent, I follow the instructions step by step and works great. Rename the your table, follow all the steps described above and at the end just insert the data from the old renamed table into newly created (the one that you just partitioned).

20. on December 19, 2008 at 8:59 am | Reply Hi

Hooman

I have a very large table containing almost more than 100 million records and when I want to do a search performance is very low, so I decided to use partitioning and create 4 partitions based on a CHAR field in the table. Do you think this increases search performance? The DB is already created and it has one file group, what can I do about that? Could you please let me know?Thanks Regards, Hooman

21. on December 19, 2008 at 2:21 pm | Reply

Hooman

Hi I created 5 partitions for my table as the following code: CREATE PARTITION FUNCTION ParitioningTest_PartitionRange(nchar(4)) AS RANGE LEFT FOR VALUES (CARD, CBO, CHEQ, CUST, TELL) CREATE PARTITION SCHEME ParitioningTest_PartitionScheme AS PARTITION ParitioningTest_PartitionRange TO ([FG_CARD], [FG_CBO], [FG_CHEQ],[FG_CUST], [FG_TELL], [FG]); Now I dont know how to move records from Transaction_Summary table to this partitions. I should mention that all this partitioning is for this Transaction_Summary which has more than 100 million records and there in a CHAR(4) field in this table called Channel which doesnt have an index(there is an id column which is primary key and I want to do the partitioning based on this filed, I know if that filed was a primary ke I could say Drop index Transaction_Summary_IXC on Transaction_Summary with (Move To [ParitioningTest_PartitionScheme] (Channel) ) But I dont know what to do in this case, could you please help me? Regards, Hooman

22. on December 21, 2008 at 1:35 pm | Reply Hi pinal, We are facing some problems in our company.

Ajay

we have a very huge dataset ~100M and we have used partitioning for the same. All the data was supposed to be residing in the partitions created according to the statecodes but the size of primary mdf file has increased to a bulk. Can you suggest a solution or reason for.

Regards Ajay

23. on January 6, 2009 at 4:25 pm | Reply

Phani

Can i create a partition with multiple columns as my partition key. If Yes, can you give an example for me. Iam in a pressing need for that please do reply

24. on January 8, 2009 at 10:20 am | Reply Hi Pinal,

Anoop Paul

This article is excellant. Thanks for this. I have a very huge table and i wish to partition this table based on a date field with out affecting other tables. How can i partition this table which has millions of records? Any impacts on partitioning a table which has data? Thanks In Advance, Anoop

25. on January 8, 2009 at 10:33 am | Reply Hooman,

Manu

Did u succeed in your project , Can you please share your knowledge ?

26. on February 16, 2009 at 6:10 pm | Reply

vinkas

C:\Data\Primary\TestDB_Part1.mdf, SIZE=2, MAXSIZE=100, FILEGROWTH=1 ), FILEGROUP TestDB_Part2 (NAME = TestDB_Part2, FILENAME = C:\Data\Secondary\TestDB_Part2.ndf, i have one doubt in the post. Here TestDB_Part2.ndf is .mdf or .ndf?

27. on February 17, 2009 at 11:31 am | Reply

Prasanna

I have doubt on this, how do partitioning on the existing tables? you have partitioned for new table but how do we know this table gets filled in future? I am not clear this point (New table partitioning) can u clear me out this? Thanks in advance

28. on February 18, 2009 at 7:29 pm | Reply

Mike

Is there a way to automate backup and testing restore process for all partitioned databases on a server. I have databases partioned for each month and i would like to backup only recent 3 years of data and the rest would like to backup only once in FULL. is this possible?

29. on February 20, 2009 at 2:47 am | Reply Hi, My Database is a Decision Support System.

Mounika

I have table that contains 100 Million records, and each day we bulk copy and insert 300,000 more records. The main problem we are facing is as per our logic, the data for the past 25 days is only required when performing other business logic. We do not want to have archiving and moving old data to history tables, so any given day all the data for the last 2 years exist. Now my problem is when creating indexes I dont need the index to be on the previous months rather than I need the index if it can cover the last 1 month of Date range. If I do table partitioning how will be my performance increase or is there any other way round for this problem.

30. on March 1, 2009 at 7:06 pm | Reply

Yaser

hi, i need help please can any body help, i want to retrieve data from one table, between two given dates, if these dates are of different years example 2008 & 2009, then im getting error. im converting with the convert function with the code 101. any help in this regard will be help full.

31. on March 6, 2009 at 3:08 am | Reply

Minli

I like to know How I can partition my existing database. Thanks

32. on March 6, 2009 at 10:46 am | Reply @Milni, To Start, I would say

Imran Mohammed

Create new datafiles in same or different hard disk to improve IO Performance ( if it is raid 5 or 10 then for DR ).

Again each datafile can have multiple file groups, you can have similar objects created under one filegroup, like all tables in one filegroup and all Indexes in other file group ( this gives you oppurtunity to take backup of individual filegroup, also improves performance ..

33. on April 1, 2009 at 6:42 pm | Reply

Qainan

I would like to know, that i have 22 million records and i want to increase performance of application which design is better for me.

34. on April 14, 2009 at 10:42 am | Reply

Husain

I have crores of records in a table and lakhs or record are added everyday. Will partitioning improve performance drastically and if yes how many partitions would be most suitable to have on these many number of records.

35. on May 16, 2009 at 9:46 pm | Reply

Dan

The question asked about queries not using the partion key was good I just dont see an answer. Do youhave to query by the partition key to get the benefits of the partitions?

36. on June 20, 2009 at 8:05 am | Reply Hi Pinal Dave,

Xuan Thang

I have a very big database, so it takes me about 1 hours to run one report. Can I use partion to divide my database to many smaller partition by every month? Can we store the data of one month in one partition? Example: partition_2009_01: will store all data which insert in January. partition_2009_02: will store all data which insert in February. Thanks and Regards.

37. on August 13, 2009 at 7:57 pm | Reply Hi Pinal,

Brijesh

I have a requirement where a table on its own is 600GB and the data is kept for 30 days. Every weekend there is a job which triggers 40 others bulk insert jobs which pumps data into the database from different locations. Since there is no common factor to create the partition I have decided to create it based on day example Every months has got 1,2,3,4.30 or 31 so through of creating 31 file groups and attached 31 file ndf files. I am not sure if this can be done as I tried couple of times but the partition cannot pick up values from the getdate() field Any help would be really appropriated! Thanks Brijesh

38. on October 5, 2009 at 7:16 pm | Reply

kumar

i have more then 5 rows in one table(i,e table1) i need to copy the rows into other table (ie table2) which have its own rows in it.

when i try INSERT tbl_Education ( Standard, School, Board ) SELECT Standard,School, Board FROM tbl_Prev_Education then it is just inserting one row (if tbl_Prev_Education has one row) when tbl_Prev_Education has more then one row then it is not inserting in to tbl_Education table anyone plz help me

39. on October 29, 2009 at 10:51 pm | Reply

subbarao

Hi, I have table with 6 months data(11 lacs). On this table we are performing grouping. For that the query taking around 2 sec. This table already has the index. So to improve the performance I am trying to implement the partition based on date field. I want to partition the table into month wise. will it help to improve the performance?

40. on November 7, 2009 at 1:30 pm | Reply

Shital

Hello pinal, I have been create a partition on existing table. this table having 10 to 12 indexes. so which index I would be drop and recreate. pleases replay on mail id. Thanks Shital

41. on December 3, 2009 at 2:15 pm | Reply

oguzkaygun

Hello where are answers about questions related to this topic ? Thanks

42. on December 3, 2009 at 2:32 pm | Reply Hello

oguzkaygun

if i create new coulums and new sql query (select, insert, update or delete) after i have do partition, Should i do partition again. Or my partition will see my new coulums and my new sql query. Also i can create new table too. Thanks

43. on January 20, 2010 at 9:52 pm | Reply

Imran Yasir

Hi Pinal,How can I mention 50% records instead of mentioning first 10 records should store in one filegroup and rest should be stored in different filegroup? If you dont know exactly how many records will be inserting into partioned table then you cant say first 10 records,then i would like to menion % instead of fix numbers. Please can you reply with answere. Imran

44. on February 7, 2010 at 4:15 pm | Reply Hi,

Sanjay

I have gone through your tutorial and havinf sme question. I have one table where its aroung 500 million records now after reading your article i want o use partition.I have two date columnn in this table one is Recd_Date and another is Entered_Date.Now i want to create partition based on entered date and that should be monthly.I am afraid to do that becauase this table is currently using for production. And i do not have any file group in the database so needyour suggestion on this. Sanjay

45. on February 24, 2010 at 11:36 am | Reply hi Pinal,

Rupa

I found the data partitioning article interesting and tried to implement for my appliaction. The impletmentation discussed in this article is done. My doubt is regarding the deletion part, in the site http://msdn.microsoft.com/enus/library/ms345146(SQL.90).aspx , after creating a staging table the older data is switched and later the stagign table is dropped Can you suggest any alternative mthod available to drop partion with lesaa complexity. Thanks in advance Rupa

46. on March 31, 2010 at 9:35 pm | Reply Hi,

Jonas

To gain perf. improvement you would need to have an index on the partition key as Pinal Dave statues. The queries would need to be using that column as well. Even though switching partition in and out of partition table to staging tables are quite complicated it s not too complicated. And its REALLY fast! Because its just a meta data change.

So as one of you asked. I would remove archival data to other server by one of the following alternatives, depending on network capacity among other parameters. 1) switching the old partitions to a staging table. 2) Copying the staging table data to 2.1a) another staging db on same server 2.1b) another db on the other archival server 3) if 2.1b then done. If 2.1a make backup of staging db, move backup file to other server, restore it there to a staging db on that server, move data to archival db between db on hist.server

47. on April 6, 2010 at 3:58 pm | Reply

Brijesh

I have brought a file group offline in SQL Server 2005, Now I dont know how to bring it back online ? Please help, due to this I am unable to take the backup of that particular file group.

48. on April 6, 2010 at 11:26 pm | Reply Hello Brijesh, Use the following command:

Pinal Dave

RESTORE DATABASE database_name FILEGROUP=filegroup_name Regards, Pinal Dave

49. on April 7, 2010 at 3:17 pm | Reply Hello Pinal Dave,

Tipu

Nice to read you article. I have created partation after reading your article. But i need some advice or code by which i can create Dynamic Partation. Like if in one table i have 4 month data in a year then i should have 4 partation .After one month if one more month is add in the table then it dynamically add one more partation and also if any previous month data is not aviliable or get deleted then remove that partation from database. Please help me to achive this.

50. on April 12, 2010 at 10:39 pm | Reply Hello sir,

Mihir

Your Article On partitions is excellent. Its really Helpful me. I just want to know sir, I have one Question. How can i Partition On Two different On same table For Example I Want To Create two Partition. 1)On year Column means (1994 to 1999) Inside it another partition 2) On Car Column means(Toyota,Hyundai,Bmw) Pls if possible reply soon Thank you Sir

51. on April 14, 2010 at 1:11 am | Reply Hi Pinal,

Suresh

I have renamed a table which was involved in table partition, After rename, partition function points to new name. Will there be any issues in long run during switchout or drop old partition when a table is renamed. I havent tested it as the table has more than 100 million records

Please suggest. Regards Suresh kumar

52. on April 14, 2010 at 11:39 am | Reply

Kunal

Hello Pinal Dave, As per your article what will happned if i create ID as primary key. As i executed SELECT * FROM sys.partitions WHERE OBJECT_NAME(OBJECT_ID)=TestTable; after making ID as primary key i found that one more partiton increased also number of rows in newely add partition is total count of table .Is it correct ? if not then how do we implement Unique cluster index on primary key with partition. Please advice.

53. on May 25, 2010 at 5:41 pm | Reply

jagannadh

hi, i am having table with 1 lakh records, i want to do partition on this table based on the column with name status. i want to move all separated and active records to two different partitions. please guide me how to do this ? thanks, jagan

54. on June 7, 2010 at 2:26 pm | Reply i m running the code for partition as

Rino

use master go create database testdb on primary (name=test_part1, filename=c:\partitionpractice\test_part1.mdf,size=4mb,maxsize=100,filegrowth=1), filegroup testdb_part2 (name=testdb_part2,filename=c:\partitionpractice\testdb_part2.mdf,size=2mb,maxsize= 100,filegrowth=1) go use Northwind create partition function partfunc1 (int) as range left for values(10) use Northwind create partition scheme mypartschm as partition partfunc to ([primary],testdb_part2) after runing the partition scheme i get the error As Msg 208, Level 16, State 58, Line 2 Invalid object name testdb_part2.

55. on June 7, 2010 at 2:28 pm | Reply i m running the code for partition as use master go create database testdb on primary

Rino

(name=test_part1, filename=c:\partitionpractice\test_part1.mdf,size=4mb,maxsize=100,filegrowth=1), filegroup testdb_part2 (name=testdb_part2,filename=c:\partitionpractice\testdb_part2.mdf,size=2mb,maxsize= 100,filegrowth=1) go use Northwind create partition function partfunc1 (int) as range left for values(10) use Northwind create partition scheme mypartschm as partition partfunc to ([primary],testdb_part2) after runing the partition scheme i get the error As Msg 208, Level 16, State 58, Line 2 Invalid object name testdb_part2. pls reply to this comment as the above comments mail id is Wrong

on July 2, 2010 at 4:06 pm | Reply

Naidu

More imp You created database TestDB and you are searching in the Northwind for the existence of your filegroup. and Please change the Partition schema definition to use the Partition Function. Instead of using partfunc (which is not exists) to partfunc1 (Which you created).

56. on July 13, 2010 at 1:58 am | Reply

Vijaya

When I partitioned a table, I noticed the below: 1. The Primary key should include the partition key on a Partitioned Table. 2. But if I am creating this Primary key on the Primary Filegroup or not on the partition, it is letting me do so without including the partition key. Is this a good practice to have index/ Primary key not on Partition. Please advice.

57. on July 22, 2010 at 4:53 am | Reply

Bhaktapur

For Horizontal Table Partitioning, please refer to this article at http://msdn.microsoft.com/enus/library/ms345146%28SQL.90%29.aspx#sql2k5parti_topic27. This article explains each steps. I found it very helpful.

58. on August 3, 2010 at 4:14 pm | Reply

andrea

Hi! We have a very large table partitioned by month. Current month is partitioned by days, older only by months. Partitioning is automated. all works smoothly. One month-partition holds about 500 mio. records. Older partitions are read-only. We want to get rid of the oldest one. I mean not switch to an empty one, we really and completely want it GONE without a trace. do we really need to copy all data into an empty, stand-alone table??? Is there no faster way to drop the partition/filegroup? Anything? Thank you so much for your ideas! Andrea

on August 10, 2010 at 12:32 am | Reply

siri

Andrea, Will you please let me know how you guys implemented the below automation process? As i need to implement the same in my business. Please reply back ASAP. Current month is partitioned by days, older only by months.Partitioning is automated. Thanks in Advance.

59. on August 26, 2010 at 10:05 am | Reply Hi

Faisal

How can i do the table partitioning on the existing database.

on October 29, 2010 at 5:22 am | Reply Hi Faisal, Is it on existing table?? if so let me know

Siri

60. on October 18, 2010 at 1:54 pm | Reply Your artical is very helpful. thanks,

Mahesh

61. on October 27, 2010 at 5:23 pm | Reply hi pinal

Sharath

we are planning the partitioning in our databasebut we are not clear about the performance benefits..can u please show how and where can i see those performance benifits..? regards Sharath

62. on November 4, 2010 at 9:13 am | Reply Hi

bookworm

I wish to get your article on how to partition based on day such as 1,2,331 and use the ring structure rotating window and do not required merge and split. the data will be kept for 1 month, and i am thinking this structure meets my requirement as I can age the data in each partition which consists of 1 day of data before the next month comes. But the problem is the date field in the table is not use in most of the user query but there are other non cluster index (composite indexes) in the exisitng huge table. Will this a tradeoff? example: table 1 id1 (PK) filed1 filed2 filed3 filed4 update_dm insert_dm non cluster index is id1, field1,field2

but i will age the data based on insert_dm user will query the data based on non cluster indexes. ETL will base on insert_dm too. the data is huge and over 20 billions of rows per month, so, do you think, if I partition it based on day of the insert_dm, and use the ring structure, it is the a good option. any suggestion and example will be great.

63. on January 4, 2011 at 2:21 am | Reply Awesome example clear and concise

Rob

64. on January 6, 2011 at 2:13 pm | Reply

Kasper Bengtsen

Be aware that under certain conditions partitioning will not be an option due to the following bug in the query optimizer: http://connect.microsoft.com/SQLServer/feedback/details/240968/partition-table-usingmin-max-functions-and-top-n-index-selection-and-performance The bug results in certain queries performing a complete index sort, rather than just using the known order of an index to determine the result. This is very unfortunate if your table contains billions of rows. Examples of simple queries affected by the bug: A. SELECT MAX(ColumnN) FROM PartitionedTable (with a non-clustered aligned index on ColumnN) B. SELECT TOP 10 ColumnX, ColumnY, ColumnZ FROM PartitionedTable ORDER BY ColumnX, ColumnY, ColumnZ (with a clustered aligned index on ColumnX, ColumnY, ColumnZ) In our case the above queries resulted in a difference in execution duration of milliseconds vs. hours, for the exact same queries, on non-partitioned and partitioned tables respectively. :(

As far as I know Microsoft has no plans to fix this bug in current releases of SQL Server, and has no plans to fix it in the next major either. So for an enterprise solution that allows for you to: -prioritize data in the same table on physical storage (because all data has to be available online, but only some is frequently queried) -keep the physical prioritization transparent to the layers above the database (because you already have existing applications using the database, and/or is using ORMs in the data access layer) -keep your indexes aligned to utilize PARTITION SWITCH (because your maintenance window for moving less relevant data to low priority storage is limited) -use TOP N/ORDER BY style queries on a large amount of rows (because who doesnt :) ) you will have to look elsewhere.

65. on January 11, 2011 at 3:31 am | Reply Hi ALL, using: Sqlserver 2008 enterprise edition

sneha

we have database partitioned basing on date [one partition for each month]. Now i want to shrink all the data files [one file per partition] in partitions except the current month partition, that is i have to shrink all the data files in partitions till dec 2010. I am trying to get the partition name ,file group name, file name and the partition range value from below system views , but cannot find the approprtiate column to join sys.partition_range_values sys.partition_schemes sys.partition_parameters sys.partition_functions from sys.partitions. My main requirement: to retreive database file names in the file groups till last month. Please help me. Thanks in advance

66. on January 15, 2011 at 3:47 pm | Reply Hello Pinal,

PSA

I want to partition my 500 GB database into multiple LUNS, the method would be like group of tables Like: All tables related to PMS should be in one LUN All tables related to PMO in another All tables related to RND in another so on. Currently DB in two files on one filegroup . Could you please suggest the steps. Thanks in advance

67. on January 18, 2011 at 5:00 pm | Reply hi,

anil joshi

i have a large table and have thousands of rows, i want to partition this table but the problem is this table has lot of references with other tables if i re-create this table using partition schema then i lost all references so my question is without affecting anything can i put this table in a partition?

68. on January 18, 2011 at 5:06 pm | Reply Hi,

anil joshi

it is possible to create two partition on a single table?

69. on January 25, 2011 at 3:08 pm | Reply

Zulfiqar

Dear Pina, How to revert back from Partioning to Single Level Database. Example: DB Name Shop c:\data\shop.mdf (200MB) d:\data\tran.ndf (100MB) clustered index e:\data\hist.ndf (50MB) clustered index l:\data\shop_log.ldf. I have to convert my database to c:\data\shop.mdf l:\data\shop_log.ldf I have done every thing..like I removed clustered index and and shrink file but when i go for deletion it says Can not delete file or filegoup because it is not empty. please help me.. in a step by step.

70. on January 25, 2011 at 3:10 pm | Reply Just to sya there is no partioning only FileGroups.

Zulfiqar

71. on February 11, 2011 at 11:46 pm | Reply Hi Pinal how to select from indivisual partition. I mean i have one table that has four partitioning. Year1,Year2,Year3,Year4 how should I, select only one year1

Zulfiqar

72. on February 18, 2011 at 9:59 am | Reply hi, how can you resolve error like this. thanks.

joel

ALTER TABLE SWITCH statement failed. The specified partition 13 of target table jfcdw_prd.dbo.FactSales_CK must be empty

73. on February 21, 2011 at 7:12 pm | Reply Interesting thread

Mark Whitfield

I have a requirement to set-up an MS SQL Server 2005 (or 2008) database to hold significant amounts of transaction data. One table needs to hold 26 Gigabytes of data and the other, 3 Gigabytes. This is for data querying of the transaction data. First question is Can it be done? in MS SQL Server. Would I need to partition the tables over a number of hard drives to make this work? What typical hardware configuration/approach would be recommended if the answer is YES? Thanks in advance folks for any help, much appreciated Regards Mark.

74. on March 4, 2011 at 12:07 am | Reply

Will

Thank you for the excellent articles. These have been a life saver several time. I have partitioned my table by a DateTime field RecordTime and have partitioned it by MONTH

Two questions: 1. Is there a query to retrieve the field name on which a table was partitioned ? 2. Is there a query to retrieve the partition ranges? In my example I am expecting to see the list of current partition ranges like Jan2011, Feb2011 Thanks. Will

75. on March 9, 2011 at 3:49 pm | Reply

sharmi

how to create a partition function. And give the syntax pls

76. on April 20, 2011 at 8:57 am | Reply

Kevin

HI PINALKUMAR DAVE, When I created Partition for my database then : Partition function can only be created in Enterprise edition of SQL Server. Only Enterprise edition of SQL Server supports partitioning. How can I solve with this situation? please explain to me I am very appreciated ! Kevin