SharePoint Patching: Notes

From the Field

Sam Hassani, Premier Field Engineer, Microsoft UK

Agenda
The story of SharePoint patching
Answering the what do I need
question
Dismantle a released update
Update scenarios
Planning and upgrading
Monitoring and troubleshooting

Terminology

Type of Updates
Released
Hotfix
COD Critical on Demand
Public Hotfix e.g. US DST Hotfix 941422
Post Service Pack rollup
Rollup of hotfixes build during SP-lock timeframe when
service pack is locked to not accept new fixes

Cumulative Update
Since December * rolling 2 month basis
Broadly affecting support issues

Service Pack
Service Packs include all of the updates for MOSS plus deliver
important customer-requested stability and performance
improvements, while incorporating further enhancements to
user security.

Build Numbers

12
12

00

6219
6219

Major
Major

Minor
Minor

Build
Build

Major = Office 12 : SharePoint 2007

Minor = RTM
Build = Internal build number

4518 = RTM
6219 = SP1
6320(WSS)/6322(MOSS) = Infrastructure Update
6341 = February Cumulative Update

Revision = Update type

1000
1000
Revisio
Revisio
nn

1000 = Service Pack

30xx = Private build
5000 = Cumulative Update
500x = COD Build

Check Your Build How?

WSS Check Version OWSSVR.DLL

WSS Localised Check Installed KB Numbers
Content DB Versions Table

MOSS Check Version

Microsoft.SharePoint.Portal.DLL
MOSS Localised Check Installed KB Numbers
Content DB Versions Table

DLC Check Version Microsoft.Office.Policy.DLL

OSRV Check Version of

Microsoft.Office.Server.DLL

Check Your Build How?

DEMO Check Your Build

Patching Post IU

http://support.microsoft.com/kb/953749/

Improvements to the time that is required to update and

upgrade Windows SharePoint Services sites

PSConfig no longer updates Site Collection Version number

unless a Schema update is required Significant
Performance Improvement

February Cumulative Updates for WSS and MOSS

http://support.microsoft.com/kb/961756
http://support.microsoft.com/kb/961755

Latest service pack + latest WSS Cumulative Update

+latest MOSS Cumulative Update = completely up-todate.

How it used to be....

How it was - MOSS
CoreSer
ver GLB

Build

CoreSer
ver Loc

6324.500
0
6327.500
0
6331.500
0

Jul
y
Au
g
Oc
t

955593

Build

STSSTSGLB
LOC
Jul 95559
y
4
Au
957109
g
Oc 95769
t 1

DLCLoc

pjsrv
wfeglb

pjsrva
pp-glb

956056
957693

Ifswf
e
-glb

Xlsrva
pp
-glb

955586 95558
6
95339
7
958567

How it was - WSS

6324.50
00
6327.50
00
6332.50
00

DLCGLB

958569

95769
6

95769
4

The December CU and

Beyond.....
How it is- MOSS
Build

CoreSer CoreSer
ver -glb ver -loc

6324.50 Jul 955593

00
y
6327.50 Au 956056
00
g
6331.50 Oct 957693
958567
00
6335.50 De 960011
960011
How
it is-cWSS
00
6341.50
Fe STS961755 STS961755
Build
02
b
glb
loc
6324.50 July 955594
00
6327.50 Au
957109
00
g
6332.50 Oct 957691
00
6335.50 De 960010 960010
00
c
6341.50 Feb 961756 961756

DLCGLB

DLC
-loc

pjsrv
wfeglb

pjsrva Ifswf
pp-glb
e
-glb

Xlsrva
pp
-glb

955586 95558
6
95339
7
958569
960011 96001
1
961755 96175
5

95769
6
96001
1
96175
5

95769
4
96001
1
96175
5

9600
11
9617
55

96001
1
96175
5

Whats In the MSP?

Orca
Part of the Windows development tools
portion of the SDK
http://msdn2.microsoft.com/en-us/libr
ary/aa370557.aspx
Open MSI file
View MSP as transform

Whats In the MSP?

DEMO Dismantling a
Patch

Typical Scenarios

New
Farm

Updating to...
......Minimum Build
......Recommended
Build
......Latest and
Greatest Build

Existing
Server in
Existing
Farm

New
Server in
Existing
Farm

Updating To ....

Minimum required build:

The oldest supported build
Currently this is SP1 for WSS 3.0/MOSS 2007
Once a Service Pack is released, the n-1 version is
supported for 1 year after Service Pack release.
After this time, patches for n-1 version will not be shipped

Microsoft recommended build:

Product groups pubic recommendations:
http://blogs.msdn.com/sharepoint
The current recommended build is the Infrastructure
Update

Latest and greatest build:

Running with maximum number of fixes

Culture Shift
Which build should I be on?
Minimum required
Microsoft recommended
Latest and greatest

"If it ain't broke don't fix it attitude is

becoming less prevalent (based on Case
Histories)
Software vendors are more diligent leading
to increased consumer trust

Version Distribution
SP1 + IU

Improved Support from PSS

RTM

SP1 + Dec CU

New Farm
What is the best way to build it?
Slipstream it
Any package can be slipstreamed - extract the
MSPs into the update folder
Benefits of fixes in the provisioning and
configuration wizard code paths.
When you configure CA it is configured with the
fixes and updates already in place.
Long term is no different to manual updates but
is more efficient

New Server in Existing

Farm
Adding a new server to an existing farm:
Replacement
Scale-out process

New server must be same version as existing

servers
How do we achieve this?
Slipstream installation
Manually install patches

Do the above before running configuration

wizard to join the farm

Existing Server in Existing

Farm
Patching an existing server in an existing
farm:
New Microsoft recommendations
To correct an error that has been experienced (per
a KB article)
To be running the most recent code

How do we achieve this?

Manually install patches

Do the above before running configuration

wizard

Slipstream Creation
Requirement for deploying on Windows
Server 2008
Deployment guides
Windows SharePoint Services 3.0
http://technet2.microsoft.com/windowsserver
/WSS/en/library/91649a7e-6b5a-4e5a-9ee5-519
51f4b857f1033.mspx

Office SharePoint Server 2007

http://technet2.microsoft.com/Office/en-us/li
brary/f484f5f2-35bb-4d70-bf56-dd1c4c287c72103
3.mspx

Slipstream Creation

Pre Upgrade Steps

Take a complete farm backup and verify it
restores
It is not supported to restore an earlier version onto a
newer version e.g. RTM to SP 1

Backup any SharePoint out of box files you may

have customized
Run orphan repair tool,
http://support.microsoft.com/kb/923904

Defragment databases,
http://support.microsoft.com/kb/943345

Ensure adequate disk space exists on each

server
Installation of the new binaries,
and upgrade logging will take disk space

Pre Upgrade Steps

Load balance sites across databases if needed
Whitepaper released at SP 1, Performance
Recommendations for Storage Planning and Monitoring,
http://go.microsoft.com/fwlink/?LinkID=105623&clcid=0
x409

Verify no upgrade Timer Jobs are currently

running
In place upgrade needs to complete before installing an
update
Gradual upgrade needs to complete current site
upgrade before installing an update

Take a full farm backup immediately after you

know the update has applied successfully

Deployment Overview
All updates follow the same steps and
procedures
Make sure the account you will be installing
the update has adequate permissions
Mechanics
Install the update on the SharePoint server(s)
Installs the updated binaries

Run SharePoint Configuration Wizard or psconfig.exe

Updates the web applications, databases, and versions
The database upgrade is the longest running piece

Until the database upgrade is complete access to

content is blocked to prevent corruption

Deployment Options In Place

Single server Basic installation
Automatic updates

In place installation for a farm

Install the update on each server until the following
dialog appears

Click OK on the dialog on the first server

Wait until it has completed
Continue to each of the remaining servers

Deployment options DB attach

Document all databases assigned to each web
application
Remove all of the databases from the web
applications
Install the update on each server following the in
place method
Re-attach the content databases to respective web
applications
Note: Attaching the content database has been
reported to be faster and give more flexibility and
control than normal in place upgrading of a farm

Minimising Downtime
High performance db attach
MSIT Scenario:
4 Terabytes of data to upgrade
80 content databases at 50GB per database

Upgrade Procedure:
1.
2.
3.
4.
5.
6.

Create the 4 farms (Identical target patch version)

Backup databases
Attach databases to upgrade farms (in parallel)
Verify upgrade has completed successfully
Detach databases from upgrade farms (in parallel)
Attach upgraded databases to production farm

Complete with quadrupled throughput!

Minimising Downtime
Duplicate Read-Only Farm
Internet Scenario:
24x7 Internet Presence Required

Upgrade Procedure:
1.
2.
3.
4.
5.
6.
7.

Create 2 farms (Identical original patch version)

Backup databases
Attach databases to target farm
Change routing / DNS
Verify access to Read Only farm
Upgrade production farm
Switch back to Upgraded farm

Complete with continued uptime!

Sequence of Installation
WSSv3 Updates must be applied before the
MOSS 2007 Updates
FAQ Can I install WSS and MOSS updates and run the
configuration wizard once?
FAQ Can I uninstall any update if there are problems?

Upgrade Order

Web01 = COMPLETE + Central Admin

Web01 = WFE + Central Admin

Upgrade this server first

Upgrade index01 first if it is also a WFE

since it has the most roles/services
If index01 does not have WFE enabled
upgrade web01 first

Multiple Central Admin

Upgrade 1st Central Admin before

2nd

Multiple Query/Index

If Search/Index contains Central Admin

upgrade it first Otherwise, whichever
server has the most roles/services is
before others

Verify Update was Successful

The Upgrade Log file
Finished upgrading SPFarm Name=<Configuration
Database>
In-place upgrade session finishes. Root object =
SPFarm=<Configuration Database>, recursive = True.
0 errors and 0 warnings encountered.
If you cannot find the entries look for Exception or
Error, correct the problem and run the configuration
wizard or psconfig.exe again

Check build version

Monitoring and Troubleshooting

Most support issues reported on deploying updates relates
to failures running the Configuration wizard
SharePoint Products and Technologies Configuration wizard
psconfig.exe

Monitoring and Troubleshooting

Log files
PSCDiagnostics log files, e.g.
PSCDiagnostics_1_21_2008_14_27_10_84_4533747
83.log
Upgrade log files, e.g. Upgrade.log
Event Logs
PSCDiag and Upgrade log files are stored in:
c:\Program Files\Common Files\Microsoft
Shared\web server extensions\12\LOGS

Monitoring and Troubleshooting

?
?
psconfig -cmd upgrade -inplace b2b -wait
-force

Monitoring and Troubleshooting

Upgrade Log
Refreshing the upgrade log
Sequences and Actions
http://support.microsoft.com/kb/948780/en-us

FAQ What is SyncUpgradeTimerJob: sleeping for

10 seconds

Monitoring and Troubleshooting

Sequences can repeat in certain scenarios
Multiple content databases
Multiple web applications

If the SPTimerv3 account can not write to the

default Upgrade.log, it may write to
\Documents and Settings\SPTimerv3
account\
Local Settings\Temp\Upgrade.log
The Upgrade.log file is cumulative

Common Failures
Issue: Server Error when accessing SharePoint
site

Cause: WFEs are upgraded but Databases are

not
Resolution: Detach/reattach version
mismatched database(s).

Common Failures
Issue: PSConfig fails with inadequate install
permissions
Cause: Not logged in with adequate
permissions to install the update
Overall #2 SP1 deployment failure

Resolution: Ensure necessary permissions

when deploying any update
Administrators on the WFEs
Administrators on the SQL Server or granted
the database role db_owner on all SharePoint
databases

Common Failures
Issue: PSConfig fails during site collection upgrade
sequence

Cause:
[NewWebParts] [12.1.6301.0] [DEBUG] [1/10/2009
8:15:40 AM]: Begin Upgrade()
....
....
[SiteSequence] [ERROR] [1/10/2009 8:15:40 AM]:
Unknown server error number: d

Resolution:.
Disable ForeFront Security for SharePoint
Run PSconfig, or detach/reattach content
databases

Common Failures
Issue: PSConfig fails at Step 8
Cause:
[SPManager] [ERROR] [12/19/2008 10:46:02 AM]: Upgrade
[SPSearchDatabase Name=OSST_SEARCH
Parent=SPDatabaseServiceInstance Name=sql08,1433]
failed.
...
...
[SPManager] [ERROR] [12/19/2008 10:46:02 AM]: Object
reference not set to an instance of an object.

Resolution : Corrupt dll in Dec CU [Build 3334]

Final Thoughts...
Understand what you have...
Understand where you want to be...
While we cannot upgrade a live farm
we do have processes that will move
us closer to the nirvana of a zero
downtime upgrade