Eat That Frog

Mx000 Server User Guide
Version Number: 2.0
Author: Kevin Hickman
Document Date: 11th August 2009
© Sun Microsystems, 2009

Table of Contents
0. Document Control.............................................................................................................................................................6
0.1. Revision History.......................................................................................................................................................6
0.2. Document References.............................................................................................................................................6
1. Introduction.......................................................................................................................................................................7
2. Overview...........................................................................................................................................................................8
2.1. Product Line.............................................................................................................................................................8
2.2. Meet the processors................................................................................................................................................9
2.2.1. Olympus Processor.........................................................................................................................................9
2.2.2. Jupiter Processor............................................................................................................................................9
2.3. Meet the Servers....................................................................................................................................................10
2.4. Solaris Support......................................................................................................................................................10
3. Servers...........................................................................................................................................................................11
3.1. Sun SPARC Enterprise M3000 Server..................................................................................................................11
3.1.1. M3000 Overview...........................................................................................................................................11
3.1.2. M3000 Broad Specs......................................................................................................................................11
3.1.3. M3000 Front View.........................................................................................................................................12
3.1.4. M3000 Rear View..........................................................................................................................................12
3.1.5. M3000 Top View...........................................................................................................................................13
3.2.1. M4000 Overview...........................................................................................................................................14
3.2.2. M4000 Broad Specs......................................................................................................................................14
3.2.3. M4000 Front View.........................................................................................................................................15
3.2.4. M4000 Rear View..........................................................................................................................................15
3.2.5. M4000 Top View...........................................................................................................................................16
3.3.1. M5000 Overview...........................................................................................................................................17
3.3.2. M5000 Broad Specs......................................................................................................................................17
3.3.3. M5000 Front View.........................................................................................................................................18
3.3.4. M5000 Rear View..........................................................................................................................................18
3.3.5. M5000 Top View...........................................................................................................................................19
3.4.1. M8000 Overview...........................................................................................................................................20
3.4.2. M8000 Broad Specs......................................................................................................................................20
3.4.3. M8000 Front View.........................................................................................................................................21
3.4.4. M8000 Rear View..........................................................................................................................................22
3.5. Sun SPARC Enterprise M9000/M9000+ Server....................................................................................................23
3.5.1. M9000/M9000+ Overview.............................................................................................................................23
3.5.2. M9000 Expansion Cabinet (M9000+)............................................................................................................23
3.5.3. M9000-32 Broad Specs................................................................................................................................24
3.5.4. M9000-64 Broad Specs................................................................................................................................24
3.5.5. M9000 Front View.........................................................................................................................................25
3.5.6. M9000 Rear View..........................................................................................................................................26
3.5.7. M9000+ Front View.......................................................................................................................................27
3.5.8. M9000+ Front View (No Cables)...................................................................................................................27
3.5.9. M9000+ Rear View........................................................................................................................................28
4. Components...................................................................................................................................................................29
4.1. Hot Plug.................................................................................................................................................................30
4.2. Hot Swap...............................................................................................................................................................30
4.3. PSU........................................................................................................................................................................30
4.4. FANU.....................................................................................................................................................................31
4.4.1. Form Factor Servers.....................................................................................................................................31
4.4.2. Data Center Servers.....................................................................................................................................31
4.4.3. Air Flow.........................................................................................................................................................31
4.5. MBU.......................................................................................................................................................................32
4.6. Backplane..............................................................................................................................................................33
4.6.1. BP_A.............................................................................................................................................................33
4.6.2. BP_B.............................................................................................................................................................33
4.7. XBU........................................................................................................................................................................33
4.8. CLKU.....................................................................................................................................................................33
4.9. CPU & Memory......................................................................................................................................................34
4.9.1. Memory Subystem........................................................................................................................................35
4.9.1.1. SC.........................................................................................................................................................35
4.9.1.2. MAC......................................................................................................................................................35
4.9.1.3. DIMM's..................................................................................................................................................36
Project : 30024250 Page 2 of 162 Author : Kevin Hickman

Ref: /2.0 Copyright © Sun Microsystems, 2009 Issue Date : 11th August 2009
Sun Confidential: Internal Only


4.9.2. CPUM............................................................................................................................................................37
4.9.3. MEMB............................................................................................................................................................38
4.9.3.1. FF Memory Configuration Rules..........................................................................................................39
4.9.4. CMU..............................................................................................................................................................40
4.9.4.1. DC Memory Configuration Rules..........................................................................................................41
4.9.5. Memory Interleaving......................................................................................................................................42
4.9.6. Memory Mirror Mode.....................................................................................................................................43
4.9.7. CPU Operational Modes...............................................................................................................................44
4.10. I/O........................................................................................................................................................................45
4.10.1. PCIe & PCI-X..............................................................................................................................................46
4.10.2. IOU..............................................................................................................................................................47
4.10.2.1. M4000/M5000 I/O Unit.......................................................................................................................48
4.10.2.2. M8000/M9000 I/O Unit.......................................................................................................................48
4.10.2.3. PCI Cassette......................................................................................................................................49
4.10.3. External I/O Expansion Unit........................................................................................................................50
4.11. XSCFU (eXtended System Control Facility Unit).................................................................................................51
4.11.1. Service Processor or System Controller.....................................................................................................51
4.11.2. XSCFU Key Functions................................................................................................................................51
4.11.3. XSCF (XCP) F/W Version...........................................................................................................................52
4.11.4. XSCFU Layout............................................................................................................................................52
4.11.4.1. M4000/M5000.....................................................................................................................................52
4.11.4.2. M8000/M9000.....................................................................................................................................53
4.11.5. Service Processor Networks.......................................................................................................................54
4.11.5.1. External Administration Network........................................................................................................54
4.11.5.2. Domain to Service Processor Comms Protocol (DSCP)....................................................................55
4.11.5.3. Internal Service Network (ISN)...........................................................................................................56
4.12. Operator Panel (OPNL).......................................................................................................................................57
4.13. Sun Rack Hardware.............................................................................................................................................58
5. Concepts........................................................................................................................................................................59
5.1. Partitioning.............................................................................................................................................................59
5.2. PSB (Physical System Board)...............................................................................................................................59
5.3. XSB (eXtended System Board).............................................................................................................................60
5.3.1. Unusable XSB's............................................................................................................................................61
5.3.2. Uni-XSB.........................................................................................................................................................62
5.3.3. Quad-XSB.....................................................................................................................................................63
5.3.4. XSB Best Practice.........................................................................................................................................64
5.4. DCL (Domain Control List).....................................................................................................................................65
5.4.1. DCL Policy....................................................................................................................................................65
5.4.1.1. Floating board option............................................................................................................................65
5.4.1.2. Omit-memory option.............................................................................................................................65
5.4.1.3. Omit-I/O option.....................................................................................................................................65
5.5. LSB (Logical System Board)..................................................................................................................................66
5.6. Domains.................................................................................................................................................................68
6. Device Paths..................................................................................................................................................................69
6.1. I/O Device Paths....................................................................................................................................................69
6.2. CPU Device Paths.................................................................................................................................................71
6.2.1. How to locate the PSB..................................................................................................................................72
6.2.1.1. Find the LSB.........................................................................................................................................72
6.2.1.2. Find the CPU........................................................................................................................................72
6.2.1.3. Find the XSB........................................................................................................................................72
6.2.1.4. Find the PSB........................................................................................................................................72
6.2.2. M4000/M5000 CPU Path Example...............................................................................................................73
6.2.3. M8000/M9000 CPU Path Example...............................................................................................................77
7. Operation........................................................................................................................................................................81
7.1. Connectivity...........................................................................................................................................................81
7.1.1. Connecting to the XSCF...............................................................................................................................81
7.1.2. Login to the XSCF.........................................................................................................................................81
7.1.3. XSCF Commands.........................................................................................................................................82
7.1.3.1. Display Commands..............................................................................................................................82
7.1.3.2. Command Line Editing.........................................................................................................................83
7.1.3.3. Command Help.....................................................................................................................................83
7.1.3.4. Man Pages...........................................................................................................................................83
7.2. Display...................................................................................................................................................................84
7.2.1. Platform.........................................................................................................................................................84
7.2.1.1. showhostname.....................................................................................................................................84
7.2.1.2. shownetwork.........................................................................................................................................84



7.2.1.3. showroute.............................................................................................................................................84
7.2.1.4. showdscp..............................................................................................................................................84
7.2.1.5. showsunmc...........................................................................................................................................85
7.2.1.6. shownameserver..................................................................................................................................85
7.2.1.7. showdate..............................................................................................................................................85
7.2.1.8. showhardconf.......................................................................................................................................86
7.2.1.9. showenvironment.................................................................................................................................88
7.2.2. Domain..........................................................................................................................................................89
7.2.2.1. showboards..........................................................................................................................................89
7.2.2.2. showfru.................................................................................................................................................89
7.2.2.3. showdcl.................................................................................................................................................90
7.2.2.4. showdomainstatus................................................................................................................................90
7.2.2.5. Displaying Device Information..............................................................................................................91
7.3. Control...................................................................................................................................................................92
7.3.1. Platform.........................................................................................................................................................92
7.3.1.1. Reboot the XSCF.................................................................................................................................92
7.3.2. Domain..........................................................................................................................................................93
7.3.2.1. Start the Domain..................................................................................................................................93
7.3.2.2. Domain Console...................................................................................................................................93
7.3.2.3. Logout & Disconnect............................................................................................................................93
7.3.2.4. showdomainmode................................................................................................................................94
7.3.2.5. setdomainmode....................................................................................................................................94
7.3.2.6. setdomparam.......................................................................................................................................95
7.3.2.7. Stop the Domain...................................................................................................................................96
7.3.2.8. Send a Break........................................................................................................................................98
7.3.2.9. Force a Domain panic (reset domain)..................................................................................................99
8. Administration...............................................................................................................................................................100
8.1. Platform................................................................................................................................................................100
8.1.1. Configuring the XSCF.................................................................................................................................100
8.1.1.1. Login as the default User...................................................................................................................100
8.1.1.2. Setup XSCF Administrative User.......................................................................................................101
8.1.1.3. setautologout......................................................................................................................................101
8.1.1.4. setupplatform......................................................................................................................................102
8.1.1.5. sethttps...............................................................................................................................................103
8.1.1.6. restoredefaults....................................................................................................................................104
8.1.2. Component Replacement...........................................................................................................................105
8.1.2.1. showlocator........................................................................................................................................105
8.1.2.2. setlocator............................................................................................................................................105
8.1.2.3. FANU Replacement............................................................................................................................106
8.1.2.4. PSU Replacement..............................................................................................................................107
8.1.3. F/W Upgrade...............................................................................................................................................108
8.2. Domain.................................................................................................................................................................110
8.2.1. Create a Domain.........................................................................................................................................110
8.2.1.1. Simple (Uni-XSB) Domain..................................................................................................................110
8.2.1.2. Divided (Quad-XSB) Domains............................................................................................................113
8.2.1.3. Memory Mirror Domain.......................................................................................................................116
8.2.2. Remove a Domain......................................................................................................................................117
9. Dynamic Configuration.................................................................................................................................................119
9.1. Annuity Supported DR Operations......................................................................................................................119
9.1.1. XSCF...........................................................................................................................................................119
9.1.2. Solaris.........................................................................................................................................................119
9.2. DR Commands....................................................................................................................................................119
9.2.1. Display Commands.....................................................................................................................................119
9.2.2. Operation Commands ................................................................................................................................119
9.3. Adding an XSB.....................................................................................................................................................119
9.4. Adding a PCI Card from the Domain...................................................................................................................121
10. Housekeeping.............................................................................................................................................................122
10.1. Reporting...........................................................................................................................................................122
10.1.1. Configure SMTP........................................................................................................................................122
10.1.2. Configure Mail Reporting..........................................................................................................................122
10.2. Archiving ...........................................................................................................................................................123
10.2.1. setarchiving...............................................................................................................................................123
10.2.2. showarchiving...........................................................................................................................................123
10.3. Auditing..............................................................................................................................................................124
10.3.1. Audit Records............................................................................................................................................124
10.3.2. Audit Records............................................................................................................................................124



10.3.3. Audit Events..............................................................................................................................................124
10.3.4. Audit Classes............................................................................................................................................125
10.3.5. Audit Policy...............................................................................................................................................125
10.3.6. setaudit......................................................................................................................................................126
10.3.7. showaudit..................................................................................................................................................126
10.3.8. viewaudit...................................................................................................................................................127
10.4. XSCF Backup & Restore...................................................................................................................................128
10.4.1. Automatic Backup.....................................................................................................................................128
10.4.2. Manual Backup.........................................................................................................................................128
10.4.2.1. dumpconfig.......................................................................................................................................128
10.4.2.2. restoreconfig.....................................................................................................................................128
11. Monitoring...................................................................................................................................................................129
11.1. Sun Management Center (SunMC)...................................................................................................................129
11.1.1. Configuring the SunMC Server.................................................................................................................130
11.1.1.1. Copy the SunMC Add-On S/W to the SunMC Server......................................................................130
11.1.1.2. Unzip and Extract the SunMC Add-On S/W.....................................................................................130
11.1.1.3. Install the SunMC Add-On S/W........................................................................................................130
11.1.2. Configuring the XSCFU.............................................................................................................................131
11.1.3. Configuring the Mx000 Domains...............................................................................................................132
11.1.3.1. Copy the SunMC Add-On S/W to the domain..................................................................................132
11.1.3.2. Unzip and Extract the SunMC Add-On S/W.....................................................................................132
11.1.3.3. Install the SunMC Add-On S/W........................................................................................................132
11.1.4. Configure the Mx000 XSCFU and Domains on the SunMC Server..........................................................134
12. Troubleshooting..........................................................................................................................................................135
12.1. Platform..............................................................................................................................................................135
12.1.1. snapshot....................................................................................................................................................135
12.1.1.1. SSH Target Mode.............................................................................................................................135
12.1.1.2. USB Device Mode............................................................................................................................135
12.1.1.3. Examples..........................................................................................................................................136
12.1.1.4. snapshot & Sun Explorer..................................................................................................................137
12.1.2. Data Collection..........................................................................................................................................138
12.1.2.1. showstatus........................................................................................................................................138
12.1.2.2. showlogs...........................................................................................................................................138
12.1.2.3. showlogs error..................................................................................................................................138
12.1.2.4. showlogs event.................................................................................................................................138
12.1.2.5. showlogs power................................................................................................................................138
12.1.2.6. showlogs env....................................................................................................................................139
12.1.2.7. showlogs monitor.............................................................................................................................139
12.1.2.8. showmonitorlog................................................................................................................................139
12.1.2.9. showlogs console.............................................................................................................................139
12.1.2.10. showlogs ipl....................................................................................................................................139
12.1.2.11. showlogs panic...............................................................................................................................139
12.1.3. Fault Manager...........................................................................................................................................140
12.1.3.1. fmdump.............................................................................................................................................140
12.1.3.2. fmadm...............................................................................................................................................141
12.1.3.3. fmstat................................................................................................................................................141
12.1.4. Diagnostics................................................................................................................................................142
12.1.4.1. testsb................................................................................................................................................142
12.1.5. Service & Escalation Mode.......................................................................................................................143
12.1.5.1. Service Mode....................................................................................................................................144
12.1.5.2. Escalation Mode...............................................................................................................................145
12.2. Domain...............................................................................................................................................................146
12.2.1. DSCP........................................................................................................................................................146
12.2.2. SunVTS.....................................................................................................................................................147
13. Appendix.....................................................................................................................................................................148
13.1. PSU Failure Example.........................................................................................................................................148
13.2. F/W Upgrade Example......................................................................................................................................151
13.3. Install the SunMC Add-On S/W.........................................................................................................................156
13.4. Bookmarks.........................................................................................................................................................160
13.4.1. Documentation..........................................................................................................................................160
13.4.2. Support......................................................................................................................................................160
13.4.3. Training.....................................................................................................................................................161
13.4.4. Sales.........................................................................................................................................................161
13.4.5. Various......................................................................................................................................................162


0. Document Control
0.1. Revision History
Version Author Reason For Issue Date
1.0 Kevin Hickman Draft for Review 12/02/09
2.0 Kevin Hickman Update after Review 11/08/09
Removal of Client specific references
0.2. Document References

Ref Referenced Item
1 See “Bookmarks Section” at the end of this Document


1. Introduction
This document is Sun Confidential and is only to be used by Sun badged engineers who meet the pre-requisites
listed below.
This guide has been written to introduce the concepts of, and to demonstrate basic operation and administration
functions of the five joint Sun-Fujitsu OPL (Oympus Product Line) Servers, namely the M3000, M4000, M5000, M8000
and M9000 SPARC Enterprise Servers.
Based on the dual-core SPARC64 VI processor, the quad-core SPARC64 VII processor and the Solaris Operating
System (OS), the OPL (Olympus Product Line) SPARC Enterprise servers are ideal platforms for high-availability,
mission-critical enterprise applications including large-scale databases, BIDW, ERP and CRM.
The guide aims to provide the user with enough information to meet the following objectives:
• Describe the architecture, functional goals, and reliability, availability, and serviceability (RAS) features of the
Sun SPARC Enterprise MX000 servers
• Identify system components by location, layout, and function
• Perform configuration and platform administration methodologies supported by Sun on the Sun SPARC
Enterprise MX000 servers
• Be familiar with and use troubleshooting tools and techniques available within the Solaris OS
• Configure and manage the platform using service processor (XSCF) utilities
• Initialize and configure the Solaris OS on a domain
• Define and edit nonvolatile random access memory (NVRAM) parameters
• Perform flash programmable read-only memory (PROM) image updates
• Create, manage, and delete multiple domains using hardware, system configuration, and domain component
lists (DCLs)
• Use OpenBoot programmable read-only memory (PROM) commands to gather system information and
interpret results
To make the most use out of the guide, the user should already be able to perform the following functions without
assistance:
• Provide administration support for the Solaris OS

• Understand the fundamentals of data storage administration
• Understand the fundamentals of Transmission Control Protocol/Internet Protocol (TCP/IP) networking and
administration
• Understand general enterprise server concepts
• Troubleshoot server and network system software and hardware


2. Overview
2.1. Product Line
The MX000 servers are the result of an alliance between Sun Microsystems and Fujitsu. These servers are Sun
and Fujitsu’s mid-range and high-end server models which have been jointly designed and manufactured by
Sun and Fujitsu. They run the Solaris 10 OS and are SPARC v9-compliant for binary compatibility with SPARC-
based Sun Fire™ and Fujitsu PRIMEPOWER servers and are fully compliant with Sun’s standards for software,
storage, and services. This alliance entails shared investment of MX000 servers’ product development costs
and has allowed Sun to invest more heavily in chip multithreading (CMT) technology for future enterprise
computing products.
• FF (Form Factor) is the code name for the systems from the Olympus Product Line. The FF product
family comprises 6U (FF1/M4000) and 10U (FF2/M5000) rack optimized servers with dual core
SPARC64 VI processors or quad core SPARC64 VII processors.
• The SPARC Enterprise M8000 and M9000 comprise the DC (Data Center) product line. Together, FF
and DC servers form the Olympus Product Line (OPL). OPL and LE (SPARC Enterprise T1000 &
T2000) form the Advanced Product Line (APL)
Advanced Product Line

SPARC Enterprise Servers
Niagara Product Line OPL (Olympus Product Line)
LE ("Low End") FF ("Form Factor") DC ("Data Center")
LE1 ( SPARC Enterprise FF1 ( SPARC Enterprise

DC1 ( SPARC Enterprise M8000 )
T1000 ) M4000 )
LE2 ( SPARC Enterprise FF2 ( SPARC Enterprise DC2 and DC3 (32 CPU and 64 CPU SPARC
T2000 ) M5000 ) Enterprise M9000 )
Note:
• FF1 (M4000) replaces the V490

• FF2 (M5000) replaces the E2900, E4900 and V890
• DC1 (M8000) replaces the E6900
• DC3 (M9000) replaces the E20K
• DC3 (M9000+) replaces the E25K


2.2. Meet the processors

The Servers can comprise 2 processor types:
• The dual-core SPARC64 VI processor (Olympus)
• 2.1Ghz on Form Factor Servers

• 2.28Ghz or 2.4Ghz on Data Center Servers
• The quad-core SPARC64 VII processor (Jupiter)

• 2.52Ghz on Data Center Servers
Sun SPARC Enterprise M4000, M5000, M8000, and M9000 servers utilize SPARC64 VI and SPARC64 VII
processors developed by Fujitsu. The SPARC64 VI dual-core, multithreaded processor takes advantage of
90nm technology while the SPARC64 VII processor provides a quad-core implementation with a faster clock
speed and a reduction in size using 65 nm fabrication. In all Servers, except the M3000, these CPU's can be
mixed.
For reference, there are several features of the Fujitsu SPARC VI+ Processors which you may come across in
the documentation:
• Chip Multi Processing (CMP), which provides multiple cores per CPU
• Chip Multi Threading (CMT), which provides multiple threads per core
• Vertical Multi Threading (VMT) technology, which allows multiple threads to run in parallel
Note
• The M3000 only supports the SPARC64 VII (Jupiter CPU).

• The previous Fujitsu CPU, the SPARC 64 IV & SPARC 64 V were called Panther.
2.2.1. Olympus Processor

The Olympus Product Line was originally based on the SPARC64 VI processor (Olympus).
The CPUs have two physical cores and each core has two strands with VMT structures. This allows four
threads to run in parallel. The two strands that belong to the same physical core share most of the
physical resources, while the two physical cores do not share physical resources except the Level 2 (L2)
cache and system interface.
The SPARC64 VI processor consists of Dual Core Processor with 2 Vertical Threads Per Core. It has a
Frequency Range of 2.15-2.4GHz at Launch and Instruction and Data Caches of 128KB each Per Core.
There is 5/6MB On Chip L2 Cache Per Chip and it supports and Out of Order Execution capability.
2.2.2. Jupiter Processor

SPARC64 VI was followed by the SPARC64 VII processor (Jupiter ). A Quad Core Processor, it provides
a performance increase of approximately 2x over SPARC64 VI.
The new SPARC64 VII processor consists of four SPARC V9 cores with two simultaneous threads per
core. It's built with an advanced 65nm Cu 10-layer process technology for high speed and density, and is
100% binary compatible with the SPARC64 VI process so there is no need to recompile.
All OPL systems support the Jupiter processor. The OPL systems also support Mixed CPU speeds, but
the XSCF require firmware version 1070 or higher to run Jupiter.


2.3. Meet the Servers

The OPL Line of Servers comprises the following models:
• Sun SPARC Enterprise M3000 Server (Ikkaku)

• Sun SPARC Enterprise M4000 Server (FF1)
• Sun SPARC Enterprise M5000 Server (FF2)
• Sun SPARC Enterprise M8000 Server (DC1)
• Sun SPARC Enterprise M9000-32 & M9000-64 Servers (DC 2/3)
Server Max No. Max. No. Max No. Max No. Of Max No. of Max No. Max No.
of of of RAM Internal External of I/O of
Procs Cores Threads (GB) I/O Slots I/O Units Slots Domains
M3000 1 4 8 32 4 Not Supported 4 1
M4000 4 16 32 128 5 2 25 2
M5000 8 32 64 256 10 4 50 4
M8000 16 64 128 512 32 8 112 16
M9000-32 32 128 256 1024 64 16 224 24
M9000-64 64 256 512 4096 128 16 288 24
Notes:
• Maximum RAM capacity for Form Factor Servers is calculated based on 4-Gbyte DIMM
• Maximum RAM capacity for Data Center Servers is calculated based on 8-Gbyte DIMM
• Maximum CPU statistics for all Servers is based on the quad-core SPARC64 VII CPU (Jupiter)
In Summary, including the Entry Level M3000, there are 5 Servers in the OPL Product Range. For reference, the
next generation of Sun Servers will be project SuperNova which uses the “Rock” CPU which is due in July 2009.
2.4. Solaris Support

The OPL product line (M4000 - M9000) supports Solaris 10_u3 11/06 and above:
• OS Support for SPARC64 VI: Solaris 10_u3 (11/06) or higher

• OS Support for SPARC64 VII: Solaris 10_u4 (8/07) or higher
Notes:
• The M3000 only supports SPARC64 VII CPU's and therefore requires Solaris 10_u4 (8/07) or higher
• In order to support COD and DR you will need Solaris 10_u4 8/07. In the M3000 server, the COD
function is not available.


3. Servers
3.1. Sun SPARC Enterprise M3000 Server
3.1.1. M3000 Overview
The SPARC Enterprise M3000 (code named Ikkaku.) is a single socket (single processor ) SPARC64 VII
based 2RU server which offers extreme RAS in a 2RU footprint. Hot swappable power supplies, fans and
disk drives along with XSCF, automatic recovery and instruction level retry, Extended ECC memory
protection, and Fault Isolation (Chip,processors, I/O, ASICS) rich check/recovery mechanisms in the
processor eliminate planned or unplanned downtime.
The SPARC Enterprise M3000 server maximum configuration is equipped with the 2.52 GHz SPARC64
VII quad-core processor, up to four 146 GB SAS disks, 32 GB of memory, and one, two-lane SAS port for
additional storage connectivity. Additionally, the M3000 is outfitted with four low profile PCIe slots, one
RCI, one USB, one Serial, two LAN 0 & 1 connections, two UPC, and four Gigabit Ethernet slots.
The advanced design of the Sun SPARC Enterprise M3000 server architecture reduces the number of
working parts eliminating multiple points of failure. And because it is a SPARC64 based server, it can be
upgraded to the next generation processor within the same box.
3.1.2. M3000 Broad Specs

• Code Name Ikkaku
• 1 Domain
• 1 XSCFU (eXtended System Control Facility Unit) Service Processor
• 2 RU Server (22kg)
• 1 CPU - SPARC64 VII quad-core 2.52GHz, 4 cores and 8 threads
• 32GB RAM (8 DIMM Slots using 4GB DIMMs)
• 4 PCIe Low Profile slots, no hot plug
• 4 x 2.5" (146gb) SAS hot swap HDD
• 1 Internal DVD Drive
• 4 On board GBE
• 2 PSU (n + 1), optional dual feed, hot swap
• n + 1 Cooling Fan, hot swap
• Solaris 10_u4 (8/07) or higher
• No IO Expansion Unit support

• COD function is not available.
• Memory Mirroring not supported


3.1.3. M3000 Front View
3.1.4. M3000 Rear View


3.1.5. M3000 Top View



The 6RU high, M4000 (code named FF1) Server moves you up to 2.4 GHz quad-core SPARC64 VII
processing across 16 cores, or four dual-core 2.15 Ghz SPARC64 VI processing across 8 cores.
Combined with the Solaris 10 OS, you get two dynamic domains and up to thousands of Solaris
Containers per system. You can configure the Sun SPARC Enterprise M4000 server with up to 5 I/O
slots with 4 PCIe (PCI Express) slots each and 1 PCI-X (PCI eXtended) on one I/O tray. Up to 25 PCIe
and PCI-X slots with the optional External I/O Expansion Unit.
Note that the OPL product line (M4000 - M9000) only supports Solaris 10_u3 11/06 and above. In order to
support COD and DR you will need Solaris 10_u4 8/07

Built for data center RAS demands: End-to-end ECC protection, ECC and Extended ECC protection for
memory, memory mirroring, and predictive self healing, guaranteed data path protection, and more. An
all-new, faster backplane interconnect gives you 32 GB/second speeds and acts as a common
interconnect between all chips, maximizing address and data bandwidth on one interconnect.

• Code Name FF1
• 2 Domains
• 6 RU server (84kg)
• 2 CPUM (CPU Modules – 2 CPU sockets per board)
• 4 CPUs - SPARC64 VI dual-core and/or SPARC64 VII quad-core (2.1 or 2.4 GHz),
therefore up to 16cores and 32 threads
• 4 MEMB (Memory Boards)
• 128GB RAM (32 DIMM Slots using 4GB DIMMs)
• 1 IOU (5 cassettes)
• 4 PCIe slots and 1 PCI-X
• Up to 2 External I/O Expansion Units
• Up to 25 PCIe or PCI-X slots with optional I/O Expansion Units
• 1 CD-RW/DVD-RW drive unit,
• 2 Internal 2.5" (146GB) SAS HDDs
• 1 tape drive unit (optional)
• 2 built in Gigabit Ethernet ports
• 2 PSU (n + 1) Hot-Swap Single Phase
• 4 fans per server for cooling.
• Two 172-mm fans (FAN_A). One fan is redundant
• Two 60-mm fans (FAN_B). One fan is redundant)




3.2.5. M4000 Top View



You can configure the Sun SPARC Enterprise M5000 (code named FF2) with up to eight quad-core 2.4
GHz SPARC64 VII processors or eight dual-core 2.15 GHz SPARC64 VI processors, and up to 32 cores
for 64 simultaneous thread. Up to 10 I/O slots with 8 PCIe slots each and 2 PCI-X (4PCIe and 1 PCI-X
per I/O tray). Up to 50 PCIe and PCI-X slots with the optional External I/O Expansion Unit.

Built for datacenter RAS demands: End-to-end ECC protection, ECC and Extended ECC protection for

• Code Named FF2
• 4 Domains
• 10 RU server (125kg)
• 4 CPUM (CPU Modules – 2 CPU sockets per board)
• 8 CPUs - SPARC64 VI dual-core and/or SPARC64 VII quad-core (2.1 or 2.4 GHz),
therefore up to 32 cores and 64 threads
• 8 MEMB (Memory Boards)
• Up to 256 GB RAM (64 DIMM Slots using 4GB DIMMs)
• 2 IOU (10 cassettes)
• 8 PCIe slots and 2 PCI-X
• Up to 50 PCIe or PCI-X slots with optional I/O Expansion Units
• 1 CD-RW/DVD-RW drive unit,
• 4 Internal 2.5" (146GB) SAS HDDs
• 1 tape drive unit (optional)
• 4 built in Gigabit Ethernet ports
• 4 PSU (n + 2) Hot-Swap Single Phase
• Four 172-mm (FAN_A) fans per server for cooling. Two fans are redundant




3.3.5. M5000 Top View



The Sun SPARC Enterprise M8000 (codenamed as DC1 - DataCenter1) is the 16 processor member of
the Sun-Fujitsu joint APL product line. The M8000 server can use the dual-core (each core with 2
threads) SPARC64 VI or the quad-core SPARC64 VII (each core with 2 true SMT threads) processors.
The server is ideal for databases and computationally intensive application.
The Sun SPARC Enterprise M8000 servers can support up to 16 dynamic domains. For pure flexibility,
we extended the domain partitioning granularity down to one processor chip, four DIMMs and two I/O
slots.

The Sun SPARC Enterprise M8000 server features the option to mix and match up to 16 powerful quad-
core SPARC64 VII processors running at 2.4-2.52 GHz or 16 dual-core SPARC64 VI processors and
comes preinstalled with Solaris 10 OS, providing binary compatibility and maximum investment
protection.
Built for data center RAS demands: End-to-end ECC protection, ECC and Extended ECC protection for
The I/O system interface features hot-swappable I/O Units (IOUs), each of which provides eight PCIe
slots and four 2.5-inch SAS disk drive bays. Fully configured, the server supports up to 16 internal SAS
boot disks, 32 internal PCIe slots, and up to 512 GB of memory. With the optional external I/O expansion
unit, you can increase the number of supported I/O slots to a maximum of 112 PCIe and PCI-X slots.

• Code Named DC1
• 16 Domains
• 1 Cabinet (700kg)
• 4 CMU (CPU Memory Boards) – 4 CPU sockets and 32 DIMM Slots per board
• 16 CPUs - SPARC64 VI dual-core and/or SPARC64 VII quad-core (2.28, 2.4 GHz or 2.52),
• 512gb RAM (128 DIMM Slots using 4GB DIMMs)
• 4 IOU (8 PCIe slots & 4 Internal HDD)
• 16 Internal HDD
• 32 Internal PCIe slots
• Up to 112 PCI or PCI-X slots with the optional External I/O Expansion Unit
• 12 fan units (N+1 redundant), 4 x 172 mm (FAN_A) fans, 8 x 60 mm (FAN_B) fans
• Option Dual Power Feed (DPF) Single Phase Power Expansion Unit (7RU)
• Can be housed in the Base Cabinet
• Optional power expansion cabinet required for 3-phase power distribution (350kg)






3.5. Sun SPARC Enterprise M9000/M9000+ Server

3.5.1. M9000/M9000+ Overview
The SPARC Enterprise M9000 is 32/64 Socket based server comes in 2 variations - a 32 socket (M9000-
32) & a 64 socket (M9000-64). The M9000-32 is also referred to as the M9000 (code named DC2), while
the M9000-64 is also referred to as the M9000+ (code named DC3). Basically, an M9000-64 is 2 M9000-
32's cabled together. The M9000 32/64 use the dual core (each core with 2 VMT threads) SPARC64 VI
or Quad Core (each core with 2 SMT threads) SPARC64 VII processors.

The Sun SPARC Enterprise M9000 server features the option to mix and match up to 64 quad-core
SPARC64 VII processors or dual-core SPARC64 VI processors, with up to 256 cores, and 512 threads
per system
• M9000-32 : Upto 32 SPARC64 VI/SPARC64 VII processors (64/128 cores), 24 hardware

Domains,
2 TB memory, 32 Internal SAS disks, 64 PCI-E slots, DVD/DAT support
• M9000-64 : Upto 64 SPARC64 VI/SPARC64 VII processors (128/256 cores), 24 hardware
Domains,
4 TB memory, 64 Internal SAS disks, 128 PCI-E slots, DVD/DAT support
The Sun SPARC Enterprise M9000-32/64 server can support up to 24 dynamic domains regardless
whether it is connected to the expansion cabinet. For pure flexibility, we extended the domain partitioning
granularity down to one processor chip, four DIMMs and two I/O slots. An all-new, faster backplane
interconnect gives you 368 GB/second speeds and acts as a common interconnect between all chips,
maximizing address and data bandwidth on one interconnect.
The I/O system interface features hot-swappable I/O Units (IOUs), each of which provides eight PCIe
slots and four 2.5-inch SAS disk drive bays. Fully configured, these servers support up to 32 or 64
internal SAS boot disks, 64 or 128 internal PCIe slots, and up to 1TB or 2TB of memory (with 4GB DIMM).
With the optional external I/O expansion unit, you can increase the number of supported I/O slots to a
maximum of 224 slots (M9000-32) or 288 slots (M9000-64)
3.5.2. M9000 Expansion Cabinet (M9000+)

The M9000 server with Expansion Cabinet is two M9000 servers bolted together, also know as the
M9000+ or M9000/64).
The M9000 server with Expansion Cabinet includes a total of 64 interconnect cables between the two
frames. Of the total number of cables, 62 of the cables are data and clock cables, one is an XSCFU_B
board cable, and one is an XSCFU_C board cable. Although the system cabinets are mostly the same,
there are small differences between the two sides. Therefore, one side is referred to as the main cabinet
and the other side is called the expansion cabinet.


3.5.3. M9000-32 Broad Specs

• Code Named DC2
• 1 Cabinet (940kg)
• 24 Domains
• 2 XSCFU_B (eXtended System Control Facility Unit) Service Processor
• 1TB RAM (256 DIMM Slots using 4GB DIMMs)
• 32 Internal HDD
• 16 fan units (N+1 redundant), 16 x 172 mm fans
• 15 PSU (N+1 redundant)
3.5.4. M9000-64 Broad Specs

• Code Named DC3
• 2 Cabinets (1880kg)
• 24 Domains
• 2 XSCFU_B (eXtended System Control Facility Unit) Service Processor
• Redundant XSCFU_B boards in the main cabinet. The XSCFU_C boards expansion
cabinet do not have any intelligence, they are used as a repeater to keep both
cabinets synchronized.
• XBU (System Crossbar) uses 16 individual XBU boards
• 8 XBU boards are plugged into the main cabinet’s centerplane
• 8 XBU boards are plugged into the expansion cabinet’s centerplane.
• The 2 sets of 8 expander boards are interconnected with 64 data and clock cables.
• 2TB RAM (512 DIMM Slots using 4GB DIMMs)
• 64 Internal HDD
• 32 fan units (N+1 redundant), 32 x 172 mm fans
• 30 PSU (N+1 redundant)






3.5.7. M9000+ Front View
3.5.8. M9000+ Front View (No Cables)


3.5.9. M9000+ Rear View


4. Components
The Mx000 Server Range was introduced in the earlier Overview Section, but before we start looking at the
components, here follows a recap of the characteristics:


4.1. Hot Plug

The M-Series Server Range allow you can insert and remove certain components while the server is running,
these components are deemed to be hot pluggable. The component which can be removed or inserted depends
on the type of Server. If a component is not hot-pluggable (hot-FRU), it is deemed cold-pluggable (cold-FRU).
On the M3000/4000/5000 )Mid-range) Servers, System Boards are not hot-pluggable and therefore the platform
needs to be powered off before a component can be replaced. On the M8000/M9000 Servers, the system
Boards can be replaced while the Platform is powered on. PCI cards for certain PCIe (PCI Express) and PCI-X
(PCI eXtended) controllers are hot-pluggable.
The Fan Units and Power Supplies on all M-Series Servers are Hot Pluggable meaning that these components
can be removed and inserted (replaced) without the need to power the Platform off.
4.2. Hot Swap

If a component is hot-pluggable it does not necessarily mean that the component can be configured and
unconfigured into or out of the Solaris O/S with shutting it down. If the O/S supports the feature to configure and
unconfigure the component, it is deemed hot-swappable.
Dynamic Reconfiguration (DR) is the process of replacing components while the O/S is running . PCI cards can
be replaced using the Solaris cfgadm command, If System Boards are hot-pluggable, they must be replaced
using the addfru, deletefru or eplacefru command from the XSCF.
At time of writing, the project on which I was working did not use Hot Swap and therefore this User Guide does
not cover the removal of components using DR.
4.3. PSU
Power supply units are hot replacement components on all M-Series Servers. Server PSU configurations are as
follows:
• M3000 - 2 Redundant Hot-Swap Single Phase Power Supplies

• M4000 - 2 Redundant Hot-Swap Single Phase Power Supplies (1 + 1 Redundant)
• M5000 - 4 Redundant Hot-Swap Single Phase Power Supplies (2 + 2 Redundant)
• M8000 - 9 Redundant Hot-Swap Single Phase Power Supplies (N + 1 Redundant)
• M9000/32 - 15 Redundant Hot-Swap Single Phase Power Supplies (N + 1 Redundant)
• M9000/64 - 30 Redundant Hot-Swap Single Phase Power Supplies (N + 1 Redundant per cabinet)
Power is supplied by redundant hot-swap power supplies, helping to support continued server operation even if a
power supply fails. Since the power units are hot-swappable, removal and replacement can occur while the
system continues to operate.
• M3000: can loose 1 out of 2 PS, regardless VAC input value.

• M4000: can loose 1 out 2 PS, if running on 200VAC input.
• M5000: can loose any 2 out of 4 PS, if running on 200VAC input.
Note:
• Power Supplies must be replaced by running replacefru from the XSCF

• The dualpowerfeed option on a FF must be left disabled (default)
XSCF> showdualpowerfeed
Dual power feed is disabled.


4.4. FANU
Sun SPARC Enterprise M4000, M5000, M8000, and M9000 servers comprise redundant cooling configurations.
If a single fans fails, the XSCF detects the failure and switches the remaining fans to high-speed operation to
compensate for the reduced airflow. Each Server can operate normally under these conditions, allowing ample
time to service the failed unit. All fan units are hot pluggable, therefore replacement of fans units can occur
without interrupting application processing.
Note: Fan Trays must be replaced by running replacefru from the XSCF
More detail can be found in the Internal Sun Document Sun SPARC Enterprise[R] Mx000 server fan/fantray
redundancy and thermal environments
4.4.1. Form Factor Servers

There are 2 Fan Types available in the Form Factor (M4000/M5000) Severs:
• FAN_A (172mm Fan)
The 172-mm fan module is the primary cooling system. The 172-mm fan module is in both
midrange servers.
• FAN_B (60mm Fan)
The 60-mm fan module is for additional cooling in the SPARC Enterprise M4000 server.
4.4.2. Data Center Servers

There are 2 Fan Types available in the Data Center (M8000/M9000) Severs. Each FAN unit consists of
two or three cooling fans. FAN units have an N+1 redundancy configuration, and active maintenance can
be performed on them. Each cabinet contains multiple FAN units so that the cooling specifications are
satisfied even when one individual fan inside the FAN units fails.
• FAN_A (3-FAN Unit)
Used in both the M8000 & M9000 Servers
• FAN_B (2-FAN Unit)
Used only in the M8000 Server
4.4.3. Air Flow

The Air Flow on the OPL Servers is as follows:
• M4000 (FF1)
The Air Flow is from Front to Rear
• M5000 (FF2)
The Air Flow is from Front to Rear
• M8000 (DC1)
The Air Flow is from Bottom to Rear
• M9000/M9000+ (DC2/DC3)
The Air Flow is from Bottom to Top


4.5. MBU
On a Form Factor Server, The CPUM's and MEMB's plug into slots on the MBU (Mother Board or Main Board).
The MBU provides an Interconnect between CPU, Memory and I/O through the System Controller (SC) ASIC's
housed on the MBU. Both the M4000 and the M5000 have a single MBU but they both take a different number
of components:
• The M4000 MBU accepts up to 2 CPUM's, which allows up to 4 CPUs and 4 MEMB's which allow up to
32 DIMM's.
• The M5000 MBU accepts up to 4 CPUM's, which allows upto 8 CPUs and 8 MEMB's which allow up to
64 DIMM's.
Note that the High End Servers do not have any MBU's, instead they have several Backplane's.
FF MBU (Mother Board)


4.6. Backplane
The DataCenter Servers (M8000/M9000) do not have a MBU, instead they have a Backplane.
4.6.1. BP_A
The M8000 has an Active Backplane with integrated Cross Bar (XB) providing the Clock (CLK)
4.6.2. BP_B
The M9000 has a Passive Backplane and additional Clock Boards (CLKU) which provide the Clock for
the ASIC's
4.7. XBU
The XBU (Cross Bar Unit) is only found on the M9000 and they provide connectivity to the M9000 Expansion
Cabinet which forms the M9000+. There are 8 XBU's in each cabinet which are connected by 64 Cables.
4.8. CLKU
CLKU (Clock Board Units) are only found in the M9000 and provide the Clock for the ASIC's. On an M8000 the
Clock (CLK) is integrated on the Active Backplane
To provide increased reliability, most of the internal parts of Sun SPARC Enterprise M9000 server clock chip are
redundant. In addition, there are two sources of clock signal and a dual signal line is implemented between the
clock chip and the system boards. By implementing redundant bus routes, the Sun SPARC Enterprise M9000
server can automatically restart in the event one route fails. The surviving route is used to continue operation.


4.9. CPU & Memory

The Servers can comprise 2 processor types:
• The dual-core SPARC64 VI processor (Olympus)

• 2.28Ghz or 2.4Ghz on Data Center Servers
• The quad-core SPARC64 VII processor (Jupiter)

• 2.52Ghz on Data Center Servers
The Form Factor Servers house CPU's and RAM on separate boards. CPU's are housed on the CPU Module
(CPUM) containing two CPU sockets which can host either SPARC64 VI or VII processor chips. On the
M4000 and M5000, there is no RAM on the CPUM, this is located on the Memory Board (MEMB) which can
contain up to 32 Memory DIMM's.
The Data Center Servers have CPU & RAM installed on the Same CMU (CPU Memory Board Unit)
Note: CPU & Memory information can be seen from the XSCFU by running the showhardconf command
Mx000 Server CPU & Memory Capacity Table
Server No. of No. of No. of Max No. Max. No. of Max No. Max
CPUM MEMB CMU of Cores of RAM (GB)
Boards Boards Boards Procs Threads
M3000 N/A N/A N/A 1 4 8 32
M4000 2 4 N/A 4 16 32 128
M5000 4 8 N/A 8 32 64 256
M8000 N/A N/A 4 16 64 128 512
M9000-32 N/A N/A 8 32 128 256 1024
M9000-64 N/A N/A 16 64 256 512 2048
Notes:
• Maximum RAM capacity for Form Factor Servers is calculated based on 4-Gbyte DIMM
• Maximum RAM capacity for Data Center Servers is calculated based on 4-Gbyte DIMM
• Maximum CPU statistics for all Servers is based on the quad-core SPARC64 VII CPU (Jupiter)


4.9.1. Memory Subystem

The memory subsystem of Sun SPARC Enterprise M4000, M5000, M8000, and M9000 servers are built
with reliability in mind. ECC protection is implemented for data stored in main memory, and the following
advanced features foster early diagnosis and fault isolation that preserve system integrity and raise
application availability.
• Memory patrol — Memory patrol periodically scans memory for errors. This proactive function
prevents the use of faulty areas of memory before they can cause system or application
errors, improving system reliability.
• Memory Extended-ECC —The memory Extended-ECC function of these servers provides

single-bit error correction, supporting continuous processing despite events such as burst
read errors that are sometimes caused by memory device failures. This feature is similar to
IBM’s chipkill technology.
• Memory mirroring — Memory mirroring is an optional, high-availability feature appropriate for

execution of applications with the most stringent availability requirements. When memory
mirroring mode is enabled on Sun SPARC Enterprise M4000, M5000, M8000, and M9000
servers, the memory subsystem duplicates the data on write and compares the data on read
to each side of the memory mirror. In the event that errors occur at the bus or DIMM level,
normal data processing continues through the other memory bus and alternate DIMM set.
Memeory Mirroring is covered in more detail in a alter section of this User Guide.
The memory subsystem on the Sun SPARC® Enterprise Mx000 (OPL) Servers is a complex
architecture. It is physically composed of 3 main components :
4.9.1.1. SC
The System Controller (SC) is a multi-function ASIC which directs traffic between local CPUs,
memory, I/O subsystems, and interconnect paths and provides cache coherency control.
There are 4 SC units on the CMU of the M8000/M9000 systems and 2 SC units on the MBU of
the M4000/M5000 systems. The physical addressing of all memory on each SB or CMU is
divided equally across the SCs on that board. The real memory is controlled by the MAC
(Memory Access Controller) under the SC.
4.9.1.2. MAC
The Memory Access Controller (MAC) is the chip which controls the memory access according to
the direction from the System Controller. The MAC is mounted on the CMU of the M8000/M9000
systems and on the MEMB of the M4000/M5000 systems.


4.9.1.3. DIMM's
The Servers support the following DIMM (Dual Inline Memory Module) sizes:
• 1GB DDR2 1-Rank DIMM (Form Factor & Data Center Servers)
• 2GB DDR2 1-Rank or 2-Rank DIMM (Form Factor & Data Center Servers)
• 4GB DDR2 2-Rank DIMM (Form Factor & Data Center Servers)
Note:
• DDR = Double Data Rate

• 2-Rank DIMMS are sometimes referred to as “stacked” DIMMS
• 8GB DIMM's are expected to be available in Q1 2009 (check the SSE Handbook!).
The showhardconf command displays information about the of DIMM's as follows:
XSCF> showhardconf
SPARC Enterprise M5000;
+ Serial:BCF0904031; Operator_Panel_Switch:Locked;
+ Power_Supply_System:Single; SCF-ID:XSCF#0;
+ System_Power:On; System_Phase:Cabinet Power On;
Domain#0 Domain_Status:OpenBoot Execution Completed;
MBU_B Status:Normal; Ver:0201h; Serial:BC090206E7 ;

+ FRU-Part-Number:CF00541-0478 07 /541-0478-07 ;
+ Memory_Size:64 GB;
<... output deleted ...>
MEMB#0 Status:Normal; Ver:0101h; Serial:BF0850R2AW ;
MEM#0A Status:Normal;
+ Code:ce0000000000000001M3 93T5750EZA-CE6 4145-4550e4c1;
+ Type:2B; Size:2 GB;
The memory DIMM types displayed by showhardconf can be identified as follows:
• Type 1A = 1GB / rank1

• Type 1B = 1GB / rank2
Additional information on Memory Configuration can be found by clicking in the following links:
• Sun Doc 235101 - Mx000 (OPL) Servers Memory Configuration Rules

• Mx000 (OPL) ServersMemory configuration, restrictions and considerations


4.9.2. CPUM
Each CPUM can contain 2 CPU sockets, he CPUM's are cold-FRU removal components:
• M4000 (FF1)
• 2 CPUM x 2 CPU = 4 CPU

• 4 CPU x 4 Core = 16 Cores
• 16 Cores x 2 threads = 32 Threads
• M5000 (FF2)
• 4 CPUM x 2 CPU = 8 CPU

FF CPUM (CPU Module)


4.9.3. MEMB
Each memory board provides a MAC (Memory Address Controller) and eight DIMM slots. The MEMBs
are cold-FRU replacement components. The entire system must be powered down and the power cords
disconnected to replace the memory boards.
FF MEMB (Memory Board)
On the M4000/M5000 servers, the memory is physically implemented as follows:



In summary, the FF Server Memory Configuration is as follows:
• M4000 (FF1)
• 4 MEMB x 8 DIMM Slots = 32 DIMM Slots

• 32 DIMM Slots x 4GB DIMM = 128GB RAM
• M5000 (FF2)
• 8 MEMB x 8 DIMM Slots = 64 DIMM Slots

4.9.3.1. FF Memory Configuration Rules

Within a XSB (UNI or QUAD), the following rules apply:
• All DIMMs within a Group (A or B) must be of the same size / rank
• The size of the DIMMs in Group B must be smaller than or equal the size of the DIMMs
in Group A.
• The number of DIMMs in Group B must be either 0 or the same as the number of
DIMMs in Group A
• Half populated MEMB, with only Group A populated are supported.
• In UNI-XSB mode, MEMBs must be installed in powers of 2 (1,2 or 4). 3 MEMBs is not
possible, this is due to the single interleaving scheme used on an XSB.
• It is permitted to have 3 MEMBs on a PSB, as long as the PSB is configured in QUAD-

XSB mode.
• The minimum requirement for UNI-XSB mode is CPUM#0 / MEMB#0 (and CPUM#2 /
MEMB#4)
• A CPU on a CPUM without its associated MEMB and vice-versa cannot be used in
QUAD-XSB mode.
• The “setupfru” command will report “Operation has completed. However, a

configuration error was detected”. If you configure a PSB in QUAD-XSB mode and one
of the QUAD-XSB is made of a CPU and no corresponding MEMB or vice-versa. In
this case, "showboards" will report the QUAD-XSB as "Unmount".
• XSBs can have different memory configurations.
• Only identically configured QUAD-XSBs can be reconfigured into one UNI-XSB
• Filling group B does not increase the interleaving factor, thus does not increase
memory performance. Only the memory capacity is increased.


4.9.4. CMU
The High-End Data Center Servers have a CPU/Memory Board Unit (CMU) which is equipped with
multiple CPU modules and memory. There are currently two types of CMUs available:
• CMU Type-A – A fully populated CMU board; contains four CPUs (8 or 16 cores)
• CMU Type-B – A partially populated CMU board; contains two CPUs (4 or 8 cores)
DC Server “Type A Board”



Data Center Server CPU & Memory Specs are as follows:
• M8000 (DC1):
• 4 CMU x 4 CPU = 16 CPU

• 4 CMU x 32 DIMM Slots = 128 DIMM Slots

• M9000 (DC2):


• 256 DIMM Slots x 4GB DIMM = 1024GB RAM (1TB)
• M9000+(DC3):


• 512 DIMM Slots x 4GB DIMM = 2048GB RAM (2TB)
4.9.4.1. DC Memory Configuration Rules

Memory slots on the CMU are divided into two groups, Group-A and Group-B. The following rules
must be followed when populating memory in the MX000 high-end servers:
• All DIMMs within a Group (A or B) must be of the same size / rank
• The size of the DIMMs in Group B must be smaller than or equal the size of the DIMMs
in Group A.
• The number of DIMMs in Group B must be either 0 or the same as the number of
DIMMs in Group A
• Memory must either be fully populated (32 DIMMs) or half populated (16 DIMMs).
• If populating half the memory, install DIMMs in Group-A first.
• Mixing DIMM sizes within the same group is not supported. If populating DIMMs of
different sizes, make sure to install the larger value DIMMs in Group-A slots.


4.9.5. Memory Interleaving

OPL automatically configures memory interleaving to obtain the best performance. Interleaving takes
place within a XSB (UNI or QUAD). All DIMMs within a XSB are included in a single interleaving scheme,
there is only one interleave factor defined inside a XSB. This has the consequence that the number of
DIMMS in a Group can only be 4, 8 or 16 and that the number of DIMMs in Group B must be either 0 or
the same as the number of DIMMs in Group A Populating Group B does not affect interleaving or
mirroring, it only increases the memory capacity.
Interleave factor = 2 4 DIMMS in Group A optionally 4 DIMMs in UNI-XSB or

Group B QUAD-XSB
Interleave factor = 4 8 DIMMS in Group A optionally 8 DIMMs in UNI-XSB

Group B
Interleave factor = 8 16 DIMMS in Group A optionally 16 DIMMs in UNI-XSB

Group B
Memory Interleaving can be seen by running the prtdiag command as follows, in this example, 4-way
interleaving:
# prtdiag
System Configuration: Sun Microsystems sun4u Sun SPARC Enterprise M5000 Server
System clock frequency: 1012 MHz
Memory size: 16384 Megabytes
==================================== CPUs ====================================
CPU CPU Run L2$ CPU CPU

LSB Chip ID MHz MB Impl. Mask
--- ---- ---------------------------------------- ---- --- ----- ----
00 0 0, 1, 2, 3 2150 5.0 6 146
00 1 8, 9, 10, 11 2150 5.0 6 146
00 2 16, 17, 18, 19 2150 5.0 6 146
00 3 24, 25, 26, 27 2150 5.0 6 146
============================ Memory Configuration ============================
Memory Available Memory DIMM # of Mirror Interleave

LSB Group Size Status Size DIMMs Mode Factor
--- ------ ------------------ ------- ------ ----- ------- ----------
00 A 8192MB okay 1024MB 8 no 4-way
00 B 8192MB okay 1024MB 8 no 4-way


4.9.6. Memory Mirror Mode

Memory can be configured in mirror mode for higher reliability. (except M3000). When memory mirroring
mode is enabled, the memory subsystem duplicates the data on write and compares the data on read to
each side of the memory mirror. In the event that errors occur at the bus or DIMM level, normal data
processing continues through the other memory bus and alternate DIMM set. In mirrored mode, both the
the memory capacity and the the interleave factor are halved. Memory Mirroring is configured by the
Administrator on each XSB by running the setupfru command as follows:
XSCF> setupfru -m y -x 1 sb 0
In Sun SPARC Enterprise M4000 and M5000 servers, memory is mirrored within the same memory
module, using the common memory address controller (MAC) Application Specific Integrated Circuit
(ASIC).
M4000 /M5000 server memory mirroring architecture
On Sun SPARC Enterprise M8000 and M9000 servers, memory is mirrored across adjacent MAC ASICs
to increase reliability. However, in a Quad-XSB configuration, paired DIMMs are split across different Sun
SPARC Enterprise M8000 and M9000 Quad-XSBs. As such, you can not configure Memory Mirroring on
the High End M8000/M9000 Server XSB in Quad-mode.
M8000/M9000 server memory mirroring architecture


4.9.7. CPU Operational Modes

A SPARC Enterprise M4000/M5000/M8000/M9000 server domain runs in one of the following CPU
operational modes:
• SPARC64 VI Compatible Mode – All processors in the domain – which can be SPARC64 VI
processors, SPARC64 VII processors, or any combination of them – behave like and are treated
by the OS as SPARC64 VI processors. The new capabilities of SPARC64 VII processors are not
available in this mode.
• SPARC64 VII Enhanced Mode – All boards in the domain must contain only SPARC64 VII
processors. In this mode, the server utilizes the new features of these processors.
The operation mode can be changed using the setdomainmode(8) command, which is covered later in
this User Guide. To check the CPU operational mode, execute the prtdiag(1M) command (below) on the
Solaris OS. If the domain is in SPARC64 VII Enhanced Mode, the output will display SPARC64-VII on the
System Processor Mode line. If the domain is in SPARC64 VI Compatible Mode, nothing is displayed on
that line.
# prtdiag
==================================== CPUs ====================================

--- ---- ---------------------------------------- ---- --- ----- ----
00 0 0, 1, 2, 3, 4, 5, 6, 7 2400 5.0 7 145
00 1 8, 9, 10, 11, 12, 13, 14, 15 2400 5.0 7 145
00 2 16, 17, 18, 19, 20, 21, 22, 23 2400 5.0 7 145
00 3 24, 25, 26, 27, 28, 29, 30, 31 2400 5.0 7 145

--- ------ ------------------ ------- ------ ----- ------- ----------
========================= IO Cards =========================
LSB Name Model

--- -------------- ------------
00 scsi LSI,1064
00 network N/A
00 network N/A
00 SUNW,qlc QLE2462
00 SUNW,qlc QLE2462
00 SUNW,qlc QLE2462
00 SUNW,qlc QLE2462
00 network SUNW,pcie-qg
==================== Hardware Revisions ====================
System PROM revisions:

----------------------
OBP 4.24.10 2008/08/12 19:52
=================== Environmental Status ===================
Mode switch is in UNLOCK mode
=================== System Processor Mode ===================
SPARC64-VII mode


4.10. I/O
The MX000 servers provide I/O functionality using a component called the IOU (I/O Unit). Each IOU has one I/O
controller, which manages four PCI buses. The IOU also contains cassettes that support PCI cards.
In order to facilitate hot-plug of PCIe (PCI Express) and PCI-X (PCI eXtended) adapter cards, Sun SPARC
Enterprise M4000, M5000, M8000, and M9000 servers utilize PCI cassettes. PCI cards which support PCI Hot
Plug can be mounted by administrators into a PCI cassette and inserted into an internal PCI slot or External I/O
Expansion Unit of a running server.
A summary of the Mx000 Server I/O Configuration is as follows:
Server No. of IOU No. Of Max No. of Max No. Max No.
(I/O Units) Internal External of I/O of
I/O Slots I/O Units Slots Domains
M3000 N/A 4 Not Supported 4 1
M4000 1 5 2 25 2
M5000 2 10 4 50 4
M8000 4 32 8 112 16
M9000-32 8 64 16 224 24
M9000-64 16 128 16 288 24


4.10.1. PCIe & PCI-X

The OPL Servers support 2 different types of PCI Technology, PCIe (PCI Express) and PCI-X (PCI
eXtended). The Form Factor Servers come with both PCI-X and PCIe slots, however the Data Center
Servers come only with PCIe. The M4000, M5000, M800 & M9000 servers can also support additional
PCIe & PCI-X expansion cards through the optional External I/O Unit.
• PCI-X (PCI eXtended) was developed by HP and IBM to improve PCI Bandwidth and is
backward compatible with existing PCI cards. The PCI-X implementation on OPL Servers allows
a clock speed of 133MHz, providing 1Gbyte/sec bandwidth.
• PCIe is a serial implementation of PCI & PCI-X which provide 8 lanes on the OPL Servers. Each
lane allows 250Mbytes/sec uplink and 250 Mbytes/sec downlink (therefore 500Mbytes/sec) and
therefore a total of 4Mbytes/sec Bandwidth with all 8 lanes. In fact, PCI Express was developed
to accommodate high-speed interconnects such as Fibre Channel, InfiniBand, and Gigabit
Ethernet.
The following table summarises PCI Technology Bandwidth:
PCI Bandwidth


4.10.2. IOU
I/O is added to the SPARC Enterprise M-series servers through assemblies called I/O Units (IOUs) that
contain a number of slots for PCI cards.
The Form Factor Servers share common I/O Technology with 1 or 2, 5 Slot IOU. The M4000 has one
IOU that provides four PCIe (PCI Express) slots and one PCI-X (PCI eXtended) slot. The IOU also has
several on-board devices that don’t take up any slots: two 1 gigabit Ethernet ports, a controller for two
Serial Attached SCSI (SAS) 2.5-inch disk drives, and a serial ATA (SATA) controller for the DVD and
optional DAT drive. The drives are installed in the system chassis, but the IOU is needed to provide
connectivity to the rest of the system.
The Sun SPARC Enterprise M5000 server can house up to two of these IOUs. The second IOU
contains the same 5 slots (4 PCIe & 1 PCI-X) and is required for connectivity to the third and fourth
built-in disk drives. There is a fixed relationship between the IOU and specific CPU/memory modules.
The first IOU is associated with CPUM 0 and 1, and MEMB 0 through MEMB 3. The second IOU is
associated with CPUM 2 and 3, and MEMB 4 through MEMB 7.
The M8000 and M9000 Data Center Server IOUs have eight PCIe slots (no PCI-X) and can hold up to
four 2.5-inch SAS disks in a 2 x 2 configuration. Unlike the IOUs in the Sun SPARC Enterprise M4000
and M5000 servers, there are no on-board controllers. The IOU Adapter card (IOUA) provides two 1
gigabit Ethernet ports and a SAS controller for up to two drives. IOUA card can be placed in any of the
even numbered PCIe slots. However, an IOUA is required in slot 0 in order to utilize disks 0 and 1, and
slot 4 for disks 2 and 3. The system chassis contains a DVD drive and optional DAT drive. An IOU with
an IOUA is required to access these peripherals.


4.10.2.1. M4000/M5000 I/O Unit
4.10.2.2. M8000/M9000 I/O Unit


4.10.2.3. PCI Cassette

PCI Cards on both the Form Factor (M4000 & M5000) and DataCenter (M8000/M9000) Servers
are housed in a Hot Swap PCI Cassette within the IOU. Although Hot replacement of the PCI
Card is covered in the Dynamic Reconfiguration section of this User Guide, we should note that
Annuity does not support removal of a PCI Card from a running Domain. To remove a PCI
Card, arrange an O/S outage and then shutdown the O/S before removing the card.
FF Server PCI Cassette
DC Server PCI Cassette


4.10.3. External I/O Expansion Unit

The External I/O Expansion Unit provides a host server with additional slots for PCI cards. Additional
PCIe or PCI-X slots can be added to the server by installing a PCIB-EX card into a PCIe slot and
connecting an external I/O Expansion Unit. The Unit has capacity to take 2 I/O Boats, each able to
support 6 PCI I/O Slots as follows:
• The single I/O boat configuration provides six slots for I/O cards which require 1 internal I/O
slot
• The optional two I/O boat configuration (dual boat) provides twelve slots. Each of the I/O
boats requires its own link kit, so the host server must have two I/O slots available for this
purpose.
Some additional fact about the External I/O Expansion Unit:
• It is 4U high and weighs approximately 81 pounds (36.8 kg).

• There are two power supply units (PSUs) for redundancy.
• There are two types of I/O boat, PCI-X and PCI Express. PCI cards are not interchangeable
between the two types of boats.
• A PCI-X I/O boat has six PCI-X sockets and one link socket.
• A PCI Express I/O boat has six PCI Express sockets and one link socket.
• The PCI-X I/O boat accepts PCI-X cards and some older types of PCI cards.
• The PCI Express I/O boat accepts PCI Express cards up to x8 lanes wide. PCI
Express x16 cards do not fit in this boat.
• PCI card slots are hot-pluggable.
• On the M4000/M5000, the PCIB-EX uplink card can go into any PCIe (not PCI-X) slot
• On the M8000/M9000, the PCIB-EX uplink card can only go into odd numbered slots.
If an I/O Unit is attached, you would see the output similar to that below from the following commands on
the XSCF:
XSCF> showhardconf
PCI#2 Status:Normal; Name_Property:; Card_Type:DownLink;
+ Ver:25h; Serial:XF08MP; Type:Optic;
+ Connection:IOX@X2K6/IOB0;
+ FRU-Part-Number:CF00501-7040 08/5017040-08;
IOX@X2K6 Status:Normal; Serial:XCX2K6;
IOB0 Status:Normal; Serial:XE02VG; Type:PCI-Express;
LINK Status:Normal; Ver:25h; Serial:XF08RK; Type:Optic;
SLOT1 Name_Property:SUNW,emlxs;
SLOT2 Name_Property:SUNW,emlxs;
SLOT3 Name_Property:network;
SLOT4 Name_Property:network;
PS0 Status:Nornal; Serial:DD3227;
PS1 Status:Normal; Serial:DD3254;
XSCF> ioxadm -v list

Location Type FW Ver Serial Num Part Num State
IOX@X05E IOX 1.5 XCX05E 5016937-03 On
IOX@X05E/PS0 A195 - DD0219 CF00300-1701 A1 /3001701-04 On
IOX@X05E/PS1 A195 - DD0197 CF00300-1701 A1 /3001701-04 On
IOX@X05E/IOB0 PCIX - XX00C8 5016938-04 On
IOX@X05E/IOB0/LINK OP 1.6 XF00VG 5017040-03 On
IOU#0-PCI#2 OP 1.6 XF00WD 5017040-03 On


4.11. XSCFU (eXtended System Control Facility Unit)

The eXtended System Control Facility Unit (XSCFU) is a Service Processor that continuously monitors the
system hardware configuration and health, domain configuration and status, error monitor, and notification on
the Mx000 OPL Servers.
In the Sun Fire Servers, this unit was commonly called the System Controller, but on an OPL Server, the System
Controller is a completely different component to the service processor (or XSCFU). On the OPL Servers, a
System Controller is a multi-function ASIC which directs traffic between local CPUs, memory, I/O subsystems,
and interconnect paths.
The XSCF is the firmware running on the Service Processor in the server. The board with the installed XSCF
firmware is called the XSCFU (also referred to as the "XSCF Unit") or Service Processor. So, what will we call
it? Well, we shall refer to it as the Service Processor or XSCFU in this document.
The XSCFU can configure the system to not use the broken part of the chip, and while the system remains
available, allowing you to schedule maintenance at a convenient time without incurring costly system down time.
The XSCFU enables standard control and monitoring functionality through the network.
The M3000, M4000 & M5000 use a single XSCFU, but the M8000 & M9000 use a redundant configuration of
XSCFU's, thereby providing high system reliability. The XSCFU that controls the server is called the Active
XSCFU, while the other XSCFU acts as a backup and is called the Standby XSCFU. The Active XSCFU and the
Standby XSCFU monitor each other, and if an error is detected, they determine when a failover switching to
Active or Standby should be performed.
4.11.1. Service Processor or System Controller

So, in summary, the Service Processor (or XSCFU) must not be confused with a System Controller. On
an OPL Server, the System Controller (SC) is a multi-function ASIC. The SC directs traffic between local
CPUs, memory, I/O subsystems, and interconnect paths. There are 4 SC units on the CMU of the
M8000/M9000 systems; 2 SC units on the MBU (Mother Board) of the M4000/M5000 systems.
The eXtended System Control Facility Unit (XSCFU) is a Service Processor that continuously monitors
the system hardware configuration and health, domain configuration and status, error monitor, and
notification on the Mx000 OPL Servers.
4.11.2. XSCFU Key Functions

The key functions of the XSCFU are summarised below:
• Monitoring the Server Status and RAS Function (Fault Management)

• XSCF Shell and XSCF Web
• XSCF Unit Diagnosis
• Initial System Configuration Function
• XSCF User Account Control
• Security
• Power Control for the Server System and Domains
• Support of Hot-Swapping of Components
• Component Configuration Recognition and Temperature/Voltage Monitoring
• Internal Cabinet Configuration, Recognition, and Domain Configuration Control Functions
• Dynamic Reconfiguration Function
• Console Redirection Function
• Capacity on Demand Function
• Functions for Monitoring and Notification During Operation
• Hardware Fault Information Collection (Hardware Log Collection)
• Firmware Update Function


4.11.3. XSCF (XCP) F/W Version

The XSCF version command displays the comprehensive firmware version (XCP version) of the XSCF
firmware and POST/OpenBoot PROM firmware. XCP is an XSCF Control Package that includes the
programs which control the hardware components making up the OPL Server.
XSCF> version -c xcp

XSCF#0 (Active )
XCP0 (Current): 1080
XCP1 (Reserve): 1080
There is a firmware upgrade procedure to upgrade to a later (or earlier) version. The XCP F/W contains
updated functionality and bug fixes. The current latest XCP version (March 2009) is version 1081
4.11.4. XSCFU Layout

4.11.4.1. M4000/M5000
The entry level Form Factor Servers (M4000/M5000) have a single, removable service processor
(XSCFU), pictured below. The Serial Port is marked No. 2, and the the NIC's are marked No. 4 &
5. Note that NIC 0 (xscf#0-lan#0) is the right hand port as you look at it.
The M3000 XSCFU is not removable, but looks and functions like the Form Factor XSCFU below.


4.11.4.2. M8000/M9000
The High End Datacenter Servers (M8000/M9000) have a pair of removable service processor
(XSCFU), pictured below. The Serial Port is marked No. 6, and the the NIC's are marked No. 3 &
4. Note that NIC 0 (xscf#0-lan#0) is the left hand port as you look at it.


4.11.5. Service Processor Networks

The service processor makes use of as many as three networks to handle communications between
components of the MX000 servers, as well as provide external access. The three networks are the:
• XSCF External Administration Network

• Domain to Service Processor Communications Protocol (DSCP)
• XSCF Internal Service Network (ISN) for Service Processor (active) to Service Processor
(standby) communication for the high-end servers
Note – The MX000 servers are shipped without any of the networks configured.
4.11.5.1. External Administration Network

The XSCF External Administration Network connects the service processors to the customer
network for administrative access to the service processor(s), such as access to the CLI or BUI.
The XSCF External Administration Network makes use of several interfaces to handle the
communication for the service processors. Each service processor has two network ports for
external communication to
the customer network. The high-end service processors provide floating IP addresses in case of
service processor failover.
The External Administration Network interfaces can be seen below:
Note
• The XSCF0 interfaces apply to both the mid-range and high-end servers. The XSCF1 and
lan#0 & lan#1 interfaces apply only to the high-end servers.
• Each service processor should be configured so that IP addresses for the network
interfaces are on two different subnets. The IP address associated with xscf#0-lan#0
should be on a different subnet than xscf#0-lan#1.


4.11.5.2. Domain to Service Processor Comms Protocol (DSCP)

The DSCP provides an internal and secure point-to-point protocol (PPP) based communication
link between the Solaris OS domains and the Service Processor. This is not a general use
network; it is intended to be used by DR, the Fault Management Agent, and NTP.
The DSCP network is comprised of:
• One IP address dedicated to the active service processor

• One IP address per domain dedicated to the DSCP network
The DSCP Interfaces can be seen below:
The DSCP Network
As you will see in Administration Section of this User Guide, DSCP is configured from the XSCF
using either the setupplatform or setdscp commands and from the domain by ensuring that the
dscp SMF Service is running. More details can be found in the Troubleshooting Section.
Note
• The DSCP network should only be configured when there are no domains running. If a
change is made to the DSCP network while a domain is active, the domain must be
rebooted before the service processor can communicate with it.


4.11.5.3. Internal Service Network (ISN)

The XSCF ISN provides an internal and secure PPP-based communication link between the
service processors in a high-end MX000 server. In a highend server, one service processor is
designated as active, while the other service processor is designated as standby.
The XSCF ISN allows the active service processor to exchange pertinent system management
information with the standby service processor so that if a failure occurs, the standby service
processor can take over as the new active service processor without any interruption of domain
service. Network Interfaces The XSCF Internal Service Network consists of one ISN interface for
each service processor, and provides for a private internal communication.
The ISN Interfaces can be seen below:


4.12. Operator Panel (OPNL)

When no remote connection is available, the Operator Panel (OPNL) is used to start or stop the server. The
Operator Panel displays three LED status indicators, a Power switch, and a Security Keyswitch.
• On the M4000 & M5000, the panel is located on the upper right front of the server.
• On the M8000 & M9000, the panel is located on the upper centre front of the Server
The Security Mode switch can be put into one of 2 settings using the OPNL key. The recommended setting for
all Mx000 Servers is Locked. Both the XSCP showhardconf command and the Solaris prtdiag command display
the keyswitch setting.
• Locked - This mode sets the server for normal operation, enabling error notifications.
• Service - This mode disables error notification as components are removed.
The 3 LED status indicators on the operator panel provide the following information:
• General system status

• System problem alerts
• Location of the system fault
The picture below is of a Data Center Operator Panel, which is horizontal (the Form Factor Operator Panels are
Vertical):
When the server is running, the Power and XSCF STANDBY LEDs (green) should be lit and the CHECK LED
(amber) should not be lit. If the CHECK LED is lit, search the system logs to determine what is wrong.


4.13. Sun Rack Hardware

Sun will supply 3 different racks which can house the Form Factor Servers(Sun Rack 1000-42 is recommended):
• Sun Rack 1000-38

• Sun Rack 1000-42
• Sun Rack 900
The Sun Rack 1000 hardware is designed to enclose Sun servers, storage systems, and qualified third-party
products. The Sun Rack 1000 hardware supports as many as three M5000 and six M4000 servers. It has a
redundant power configuration with two 32A IEC 309 five pin for 230/400 VAC three phase, pre-installed with the
rack
Rack 1000 Key features:
• 6 x M4000 (6U) Servers

• 3 x M5000 (10U) Servers
Sun Rack 1000 Hardware With Sun SPARC Enterprise M4000 and M5000 Servers


5. Concepts
5.1. Partitioning
OPL servers have a unique partitioning feature that allows the Administrator to configure a Physical System
Board (PSB) into 1 or 4 eXtended System Boards (XSB) using the setupfru command on the XSCF. These can
then be allocated to domains by running the setdcl & addboard commands
Unlike the Sun Fire range of Servers (F3800, F4800, F6800, F12K/F15K or E20K/E25K), an OPL Server needs
to be partitioned by the Administrator before any domains can be created. Basically, this means that the Server
must be configured so that the required number of domains can be created with the required resources (CPU,
RAM & I/O). If this is not done correctly, we may not be able to create the required number of domains, or the
domains created could have less resources than required. If it was configured incorrectly, you would then need
to remove any domains you have created, then remove the incorrect partitions and try again!
So in summary, Partitioning, or the creation of XSB's, is performed by the Administrator by logging into the
XSCF as a user with appropriate privileges and by then running the setupfru command. At the end of this
section, we will hopefully understand the configuration rules and options for the OPL Server range:
5.2. PSB (Physical System Board)

A PSB is a grouping of CPU, Memory and optional I/O on OPL Servers (M4000 to M9000). PSB's are
automatically configured on the OPL Servers, the Administrator does not need to run any commands to create
them as they always exist. All we need to do is make sure that the Hardware is in the correct places to ensure
that it can be allocated to the domains when they are created.
A Physical System Board (PSB) consists of up to 4 CPU's, 32 DIMM's and optional I/O. For example, on an fully
populated M4000, the 2 CPUM (CPU Modules) provide us with 4 CPU's and the 4 MEMB (Memory Boards)
provide us with 32 DIMMS, as shown in the diagram below (no I/O in this example):
A single PSB (PSB0) on an M4000
Therefore, on an M4000, it is true to say that all these resources are automatically grouped into a single PSB:
• PSB0 = (CPUM0 & CPUM1) + (MEMB0, MEMB1, MEMB2 & MEMB3)
If we only had 1 x CPUM or 2 x MEMB, all resources would still be allocated to 1 PSB, however, it would not be a
fully populated PSB.
On an M5000, all resources would be grouped into 2 PSB's

The next step is to partition the Server. This is done by dividing a PSB into either 1 or 4 XSB's (eXtended
System Boards) through the Partitioning feature of these Servers.


5.3. XSB (eXtended System Board)

Before you can create a domain, you need to partition the hardware resources (PSB and IOU) into XSB's, this is
done by the Administrator from the Active XSCF by running the setupfru command. Once the XSB's have been
created using setupfru, the Administrator then needs to allocate them to the domain(s) using setdcl and
addboard.
A PSB (and any I/O slots in the associated IOU, if present) that is logically configured into one XSB is called a
Uni-XSB, whereas a PSB that is logically divided into four boards is called a Quad-XSB. Each resulting part of
the divided PSB (either 1 part or all 4 parts) is called an (XSB, regardless of it's size. These XSBs can be
combined freely to create domains, subject to certain restrictions.
A domain requires, at the very least, a CPU, some memory and some I/O (boot disk & network) to be of any use.
You create a domain by assigning XSB's to it, but first you must create the XSB's. You will also see that I/O
resources on an IOU are tied to a specific PSB, and that CPU, Memory, & I/O resource are automatically
assigned to a XSB.
The minimal hardware required for a functioning XSB is to have at least 1 CPU and at least 4 DIMM's, the
maximum hardware which will be allocated to an XSB is 4 CPU's and 32 DIMM's. Therefore, a domain can
consists of 1 or more XSB,s but at least 1 of them must have CPU, memory and I/O. You will see, as we move
on, that I/O is not always allocated to an XSB, but that is OK. Provided the XSB has a valid configuration of CPU
and Memory, it can still be allocated to a domain.
A Domain can comprise of a combination of Uni-XSB and Quad-XSB Boards.


5.3.1. Unusable XSB's

Because of the way that the Platform divides the XSB, there must be a valid CPU and Memory
configuration in each XSB for it to be usable. The minimum configuration for an XSB to be usable is 1
CPU and at least 1 Group of Memory (4 DIMM's in Group A).
• If there is CPU and no Memory, then the resulting XSB is not usable
• If there is Memory and no CPU, then the resulting XSB is not usable
If an XSB does not meet minimum configuration requirments, it will appear as status “unmount” in the
Test column when you run showboards, as follows:
XSCF> setupfru -x 4 sb 0
Operation has completed. However, a configuration error was detected.
XSCF> showfru sb 0
Device Location XSB Mode Memory Mirror Mode
sb 00 Quad no
XSCF> showboards -av

XSB R DID(LSB) Assignment Pwr Conn Conf Test Fault COD
---- - -------- ----------- ---- ---- ---- ------- -------- ----
00-0 SP Unavailable n n n Unknown Normal n
00-2 SP Unavailable n n n Unmount Normal n
So, in summary, a PSB can be divided into uni-XSB (default) or quad-XSB from the XSCF using the setupfru
command. The resulting objects are referred to as XSB's


5.3.2. Uni-XSB
When a PSB (and any I/O slots in the associated IOU, if present) is configured as a Uni-XSB, this results
in a single XSB. All resources associated with that PSB and IOU are placed in a single XSB and can only
be allocated to one domain, not shared between domains.
For example, the M4000 only supports one domain when the PSB is in the Uni-XSB configuration. In this
case, all components of the single PSB (PSB0) are allocated to a single XSB (XSB 00-0), which in turn is
allocated to a single domain (DID 0).
A Uni-XSB on an M5000 (comprising all resources from PSB0 and IOU0)
The above example would be setup from the XSCF by typing the following command. The output will be
discussed in a later section.
XSCF> showfru sb 0
sb 00 Uni no


5.3.3. Quad-XSB
When a PSB (and any I/O slots in the associated IOU, if present) is configured as a Quad-XSB, this
results in 4 XSB's. As, the minimal hardware required for a functioning XSB is to have at least 1 CPU and
at least 4 DIMM's, so each of the 4 resulting XSB's must have this minimum configuration to be usable.
In this case, a PSB is broken down into smaller parts which can then be allocated to multiple domains.
Provided they meet the minimum usable requirements, each of the 4 resulting XSB's can be assigned to
a domain. All resources associated with resulting XSB and can only be allocated to one domain, not
shared between domains.
For example, the M4000 only supports one domain when the PSB is in the Uni-XSB configuration, but it
can support 2 Domains when in Quad-XSB configuration. In this case, all components of the single PSB
(PSB0) are divided between the resulting 4 XSB's. However, only 2 domains can be created because
only 2 of the resulting XSB's will have I/O slots allocated.
Probably the most important thing to know when configuring Quad-XSB is that for a resulting XSB to be
usable, a CPU must be present and there must be at least 4 DIMMS in Group A of the Memory Board at
that location. If there is no CPU present, the Memory is not usable, even if it exists. If there is no
Memory, the CPU at that location is not usable, even if it exists.
A Quad-XSB on an M4000 (comprising all resources from PSB0 and IOU0)
The above example would be setup from the XSCF by typing the following command. The output will be
discussed in a later section.
XSCF> showfru sb 0
sb 00 Quad no


5.3.4. XSB Best Practice

Try to avoid mixed configurations. It is advisable to have all identical MEMBs in a PSB of a M4000 /
M5000. This allows to change between UNI-XSB and QUAD-XSB mode without loosing maximum
performance or having systems containing unusable hardware.
If the capability to switch between UNI-XSB and QUAD-XSB mode is desired, then both the rules for UNI-
XSB and those for QUAD_XSB must be followed. The supported M8000 / M9000 configurations enforce
this.
UNI-XSB mode allows higher memory performance because it can use higher interleave factors. If all
resources of a PSB are needed in one domain, then it is advisable to configure the PSB in UNI-XSB
mode.
Memory mirroring is an effective way to protect against memory failures. Configuring an XSB for memory
mirroring halves the available memory capacity. It also halves the interleave factor. Memory mirroring
should therefore best be used in UNI-XSB mode.


5.4. DCL (Domain Control List)

The Domain Control List (DCL) for each configured domain is managed by the XSCF. It is a list of XSB's that
are assigned or will be assigned to a domain. The DCL for each domain contains information about the
registered XSB's as well as configuration and status information about the domain (e.g whether itn is running or
not).
The DCL provides a mechanism for the platform administrator to control which XSB resources can be used in a
particular domain. It is possible for an XSB to appear in multiple domain's DCL, allowing the XSB to be moved
between domains, for example, using Dynamic Configuration (DR). An XSB is can be active in only one domain
at at a time.
However, registering an XSB with the DCL also performs another very important function, it allows us to assign a
Logical System Board (LSB) against the XSB. This LSB number (0 – 15) is used in any component device
name, either through the OBP or Solaris.
A single XSB can belong to the DCL of multiple domains, but can not be in more than one domain at a time. A
good example of this would be if you wanted to frequently move an XSB between 2 domains. The XSB would
exist in the DCL for both Domains and could be moved as required using either static configuration, or perhaps
Dynamic Reconfiguration, if supported.
5.4.1. DCL Policy

The setdcl command provides users with some options to avoid the complexities of reconfiguration and
memory allocation with the Solaris OS. You can set up these options using the XSCF shell or XSCF Web
as follows:
5.4.1.1. Floating board option

The floating board option controls where the permanent kernel memory gets allocated. When
deleting a system board that contains the kernel memory, the OS is temporarily suspended. The
floating board option allows you to set the priority of kernel loading into the memory of each
system board. By setting this option on an XSB, you will ensure that permanent memory is not
assigned to the board.
This option is only relevant if you intend using Dynamic Reconfiguration (DR) to remove the XSB.
At the time of writing (Feb 2009) and for the foreseeable future, Annuity does not support the use
of DR to remove boards.
5.4.1.2. Omit-memory option

When the omit-memory option is enabled, the memory on a system board cannot be used in the
domain. Even when a system board actually has memory, this option enables you to make the
memory on the system board unavailable through an operation to add or move the system board.
This option can be used when the target domain needs only the CPU (and not the memory) of the
system board to be added.
For diagnosis and management of a system board, memory must be mounted on the system
board even if the omit-memory option is enabled. The value of this option is “true” (omit memory)
or “false” (do not omit memory). The default value is “false”.
5.4.1.3. Omit-I/O option

The omit-I/O option disables the PCI cards, disk drives, and basic local-area network (LAN) ports
on a system board to prevent the target domain from using them. Set this option to “true” if the
domain needs to use only the system board’s CPU and memory. Set this option to “false” if the
domain needs to use the system board’s PCI cards and I/O units. The value of this option is “true”
(omit I/O units) or “false” (do not omit I/O units). The default value is “false”.
We have previously seen a fully populated M4000 used as an example to demonstrate allocating all resources
from PSB0 and IOU0 to a single Uni-XSB (called 00-0). Building on that example, we register the XSB with LSB
0 with the DCL for Domain 0, and then configure it not to use memory for LSB#00 of domain 0:
XSCF> setdcl -d 0 -a 00=00-0

XSCF> setdcl -d 0 -s no-mem=true 00


5.5. LSB (Logical System Board)

At the time when an XSB is added to a Domain Control List, you also assign an LSB “number” (0 – 15) to the
XSB, which in turn is used tell the OS how to use the resources contained within the XSB (the LSB assignments
have a direct relationship to CPU numbering, memory location, I/O device paths, etc.)
Each domain has its own set of LSB assignments. Unlike an E25K for example, where every device in the frame
has a unique device name dependent on the System or I/O Board on which it reside, the device names on the
Form Factor and Data Center Servers is dependent on the LSB No. which is assigned to the XSB.
In short, the specific purpose of the LSB layer is “tell” the OS how you want it to “see” the resource, though the
actual “resource” assigned to the domain is done at the XSB layer. In summary, there are two steps:
• Assign the XSB resource to an LSB to tell the OS how you want the system to display/use/see the
resource if/when it is configured into the domain (by using the setdcl command)
• Assign the XSB resource to the domain to tell the domain to use the resource (by using the addboard
command)
The nice thing about the "abstraction" that the LSB provides is that it allows us to logically move an XSB between
domains without having to physically move them. We do this by assigning the XSB to multiple DCL's using the
same LSB No. for each one.
The LSB information for a domain can be seen by running the showdcl command. The following output builds on
the example where we want to create a single domain on a fully populated M4000 with a single Uni-XSB:
• The LSB can be seen both from the XSCF (showdcl below)
XSCF> showdcl -v -d 0
DID LSB XSB Status No-Mem No-IO Float Cfg-policy
00 Running FRU
00 00-0 False False False
01 -
02 -
03 -
04 -
05 -
06 -
07 -
08 -
09 -
10 -
11 -
12 -
13 -
14 -
15 -
• ..and from the Solaris Domain (prtdiag below):
# prtdiag
==================================== CPUs ====================================

--- ---- ---------------------------------------- ---- --- ----- ----
00 0 0, 1, 2, 3, 4, 5, 6, 7 2400 5.0 7 145
00 1 8, 9, 10, 11, 12, 13, 14, 15 2400 5.0 7 145
00 2 16, 17, 18, 19, 20, 21, 22, 23 2400 5.0 7 145
00 3 24, 25, 26, 27, 28, 29, 30, 31 2400 5.0 7 145
01 0 32, 33, 34, 35, 36, 37, 38, 39 2400 5.0 7 145
01 1 40, 41, 42, 43, 44, 45, 46, 47 2400 5.0 7 145
01 2 48, 49, 50, 51, 52, 53, 54, 55 2400 5.0 7 145
01 3 56, 57, 58, 59, 60, 61, 62, 63 2400 5.0 7 145




--- ------ ------------------ ------- ------ ----- ------- ----------
========================= IO Cards =========================
LSB Name Model

--- -------------- ------------
00 scsi LSI,1064
00 network N/A
00 network N/A
00 SUNW,qlc QLE2462
00 SUNW,qlc QLE2462
00 SUNW,qlc QLE2462
00 SUNW,qlc QLE2462
01 scsi LSI,1064
01 network N/A
01 network N/A
01 SUNW,qlc QLE2462
01 SUNW,qlc QLE2462
01 SUNW,qlc QLE2462
01 SUNW,qlc QLE2462

----------------------
OBP 4.24.10 2008/08/12 19:52
Mode switch is in UNLOCK mode
=================== System Processor Mode ===================
SPARC64-VII mode


5.6. Domains
A domain is an independent system resource that runs its own instance of the Solaris OS. Operations in one
domain are not affected by operations in another domain. Domains can be used to perform different types of
processing activity. For example, one domain can be used to test new applications, while another domain can be
used for production purposes.
Server Max No. of

Domains
M3000 1
M4000 2
M5000 4
M8000 16
M9000-32 24
M9000-64 24
So, why is the M9000-64 limited to only 24 domains if It has 64 cpus? Actually, this has nothing to do with the
number of cpus. The system uses the JBUS to address domains. JBUS uses a 5-bit protocol. This means there
are a maximum of 25 or 32 possible domains. HOWEVER, the system reserves 8 domains to be used in running
post if an IOU is inserted into a domain. This leave 24 domains for the Administrator to use.
Domains are created by assigning XSB's which were previously configured by running the setupfru command
and registered with a DCL by running setdcl command. The following output builds on the example where we
want to create a single domain on a fully populated M4000 with a single Uni-XSB:

---- - -------- ----------- ---- ---- ---- ------- -------- ----
00-0 SP Available n n n Unknown Normal n
XSCF> addboard -c assign -d 0 00-0

XSB#00-0 will be assigned to DomainID 0. Continue?[y|n] :y
XSCF> showboards -a -v
---- - -------- ----------- ---- ---- ---- ------- -------- ----
00-0 00(00) Assigned y y y Passed Normal n
Note that the documentation tells us that the XSB assigment state must be “Available” to be able to assigne an
XSB to a domain using addboard, but while writing this User Guide, I found that as long as the IDI(LSB) Couln
was “SP”, it did not matter if the assigment state was “Available” or “Unavailable”, it worked with either.
Once a domain has been created, it needs to be powered on so that POST can run and the OBP can be loaded,
after which we will want to start a console to access the OBP as follows:
XSCF> poweron -d 0
XSCF> console -d 0
Boards can also be removed from Domains. The FF & DC OPL Servers allow you to remove a board from a
Domain which is powered off (Static) and one which is powered on and running Solairs (Dynamic), both of which
are covered later in this document:
XSCF> poweron -d 0
XSCF> deleteboard -c unassign 00-0

XSB#00-0 will be unassigned from domain immediately. Continue?[y|n] :y

---- - -------- ----------- ---- ---- ---- ------- -------- ----
00-0 SP Available n n n Passed Normal n


6. Device Paths
The following section explains how to locate devices to a particular slot on an a physical CPU, Memory or I/O Board and
provides some working examples to check your understanding of I/O and CPU Device Paths for both FF and DC
Servers. By the end of the section you will have seen that the LSB number you choose for your domain is actually used
by the OBP & Solaris when identifying CPU's and I/O Devices, and that the LSB No. along with the DID allows you to
locate a device.
Some additional references to help with understanding Device Paths can be found as follows (these can also be found
in the Bookmarks Section of this User Guide):
• Sun SPARC® Enterprise M4000 and M5000 Server Device Paths

• Sun SPARC® External I/O Expansion Unit Device Paths
6.1. I/O Device Paths

In the following example on a fully loaded M5000, we have registered LSB#0 against XSB#00-0 & LSB#1 against
XSB#01-0 in DID#0. Note that in the resulting device names, the LSB#0 is “hidden”, while the LSB#1 is visible.
Basically, look for the first number after the '/pci@' but before the comma, and the next number after the
comma (e.g /pci@0,600000). If the 2nd number is a 600000 or 700000 then we can tell the LSB#0 of the
board which this device belongs too:
• If the number is a 2-digit number (e.g 10 or 20), the first digit is the LSB# (e.g 1 or 2)
• If the number is a single digit number (e.g 0 or 2), then there is a “hidden” 0 and this is LSB#0
There are many ways to view I/O Device Names, a few of them are listed below:
• OBP
{0} ok show-nets
a) /pci@0,600000/pci@0/pci@8/pci@0/network@2,1
b) /pci@0,600000/pci@0/pci@8/pci@0/network@2
q) NO SELECTION
Enter Selection, q to quit: q
{0} ok show-devs
<... stuff deleted...>
/pci@0,600000/pci@0
/pci@0,600000/pci@0/pci@9
/pci@0,600000/pci@0/pci@8
/pci@0,600000/pci@0/pci@8/pci@0,1
/pci@0,600000/pci@0/pci@8/pci@0
/pci@0,600000/pci@0/pci@8/pci@0/network@2,1
/pci@0,600000/pci@0/pci@8/pci@0/network@2
/pci@0,600000/pci@0/pci@8/pci@0/scsi@1
/pci@0,600000/pci@0/pci@8/pci@0/scsi@1/disk
/pci@0,600000/pci@0/pci@8/pci@0/scsi@1/tape
{0} ok show-disks
a) /pci@0,600000/pci@0/pci@8/pci@0/scsi@1/disk
q) NO SELECTION
Enter Selection, q to quit: q
{0} ok boot
Boot device: /pci@0,600000/pci@0/pci@8/pci@0/scsi@1/disk@0,0:a File and args:
SunOS Release 5.10 Version Generic_138888-02 64-bit
Copyright 1983-2008 Sun Microsystems, Inc. All rights reserved.
Use is subject to license terms.



• Solaris
# cat /etc/path_to_inst
#
# Caution! This file contains critical kernel state
#
"/iscsi" 0 "iscsi"
"/pseudo" 0 "pseudo"
"/scsi_vhci" 0 "scsi_vhci"
"/options" 0 "options"
"/ramdisk-root" 1 "ramdisk"
"/pci@8,4000" 0 "pcicmu"
"/pci@8,4000/ebus@1" 0 "ebus"
"/pci@8,4000/ebus@1/serial@14,400000" 0 "su"
"/pci@8,4000/ebus@1/scfc@14,200000" 0 "scfd"
"/pci@8,4000/ebus@1/panel@14,280030" 0 "oplpanel"
"/pci@0,600000" 0 "px"
"/pci@0,600000/pci@0" 0 "pxb_plx"
"/pci@0,600000/pci@0/pci@8" 1 "pxb_plx"
"/pci@0,600000/pci@0/pci@8/pci@0" 0 "px_pci"
"/pci@0,600000/pci@0/pci@8/pci@0/network@2" 0 "bge"
"/pci@0,600000/pci@0/pci@8/pci@0/network@2,1" 1 "bge"
"/pci@0,600000/pci@0/pci@8/pci@0/scsi@1" 0 "mpt"
"/pci@0,600000/pci@0/pci@8/pci@0/scsi@1/sd@0,0" 1 "sd"
"/pci@0,600000/pci@0/pci@8/pci@0,1" 1 "px_pci"
"/pci@0,600000/pci@0/pci@9" 2 "pxb_plx"
"/pseudo-console" 0 "oplmsu"
"/pseudo-mc@200,200" 0 "mc-opl"
"/pci@1,700000" 1 "px"
"/pci@2,600000" 2 "px"
"/pci@3,700000" 3 "px"
"/ramdisk-bootfs" 0 "ramdisk"
# echo | format
Searching for disks...done
AVAILABLE DISK SELECTIONS:

0. c0t0d0 <SUN146G cyl 14087 alt 2 hd 24 sec 848>
/pci@0,600000/pci@0/pci@8/pci@0/scsi@1/sd@0,0
Specify disk (enter its number): Specify disk (enter its number):
In the format example above, we use a fully loaded M5000 which comprises 2 Uni-XSB's (the showdcl &
prtdiag is a real view of the configuration). The device names can be read as follows:
• Disk 0 = /pci@0,600000
There is only one digit after the 'pci@' but before the comma, that number is a zero (0). The
numbers after the comma are 60000 so fall within the 600000 or 700000 range.
Therefore in this case, Disk 0 has a “hidden” zero (0) and this device is on LSB#0
• Disk 1 = /pci@10,600000
There are 2 digits after the 'pci@' but before the comma, and they are one-zero (10). The
numbers after the comma are 60000 so fall within the 600000 or 700000 range.
Therefore in this case, Disk 1 is on LSB#1


6.2. CPU Device Paths

We very rarely have to locate a CPU, but there are times when we need to do it. For example, there may be an
error reported in /var/adm/messages which references a strand of a CPU, and we want find out which PSB it is
on, and which CPU on the PSB.
It is worth noting that on a FF Server, the CPU is not an FRU and so the whole CPUM (less memory) may need
to be replaced. On a DC Server, the CPU is an FRU. Provided we know the DID of the domain a CPU is in, we
will able to locate it.
Remember that a CPU can be dual-core or quad-core, and that each core has 2 strands. A CPU strand can be
identified by a decimal number (e.g Solaris or XSCF) or by a hexidecimal number (e.g OBP). It is the strand
which supports a S/W thread and it is the strand which has a unique identifier.
Here are some examples of the same 4 CPU strands (0-4), from 2 dual-core CPU's, each with 2 strands, on a
CPUM which is part of a uni-XSB (00-0) in a Domain (DID 0) of an M5000:
• OBP
{0} ok show-devs
/cmp@400,0/core@1/cpu@1
• Solaris
# psrinfo
0 on-line since 02/27/2009 16:58:45
1 on-line since 02/27/2009 16:58:46
2 on-line since 02/27/2009 16:58:46
3 on-line since 02/27/2009 16:58:46
# prtdiag
System Configuration: Sun Microsystems sun4u Sun SPARC Enterprise M5000
==================================== CPUs ==================================

--- ---- -------------------------------------- ---- --- ----- ----
00 0 0, 1, 2, 3 2150 5.0 6 146
# cfgadm -al
SB0 System_Brd connected configured ok
SB0::cpu0 cpu connected configured ok
SB0::memory memory connected configured ok
<... stuff deleted ...>
• XSCF
XSCF> showdevices -d 0
CPU:
----
DID XSB id state speed ecache
00 00-0 0 on-line 2150 5
00 00-0 1 on-line 2150 5
00 00-0 2 on-line 2150 5
00 00-0 3 on-line 2150 5


6.2.1. How to locate the PSB

Here is an example of how we would locate a CPU (e.g decimal 43) on DID 0 of an M5000:
6.2.1.1. Find the LSB
Calculate this by taking the decimal CPU strand identifier (43), and divide by 32 discarding any
remainder, e.g:
CPU = 43
LSB = 43/2
LSB = 1.34375
LSB =1
6.2.1.2. Find the CPU
Calculate this by taking the decimal CPU strand identifier, and divide by 32, but this time we are
only interested in the remainder, so we use modular arithmatic. Once we have the remainder, we
divide by 8 and discard any remainder:
CPU = 43
A = 43 MOD 32
A = 11
CPU = 11/8
CPU = 1.375
CPU =1
6.2.1.3. Find the XSB
We now know the DID # (0), the LSB # (1) and the CPU # (1), so we now need to find the XSB
which will lead us to the PSB:
XSCF> showboards -d 0
---- - -------- ----------- ---- ---- ---- ------- -------- ----
XSB =1
6.2.1.4. Find the PSB
The 1st 2 digit's of the XSB (01-0), identify the PSB, so this is PSB 1
PSB =1
So, we can conclude that in this example, CPU # 43 (decimal) is actually CPU 1 on PSB 1. As
there are 2 CPU's per CPUM, this will be CPUM # 2.


6.2.2. M4000/M5000 CPU Path Example

The following output was taken from an M5000. Pick a CPU, the objective is to identify which CPUM it is
on and which CPU slot on the CPUM:
---- - -------- ----------- ---- ---- ---- ------- -------- ----
XSCF> showdcl -a
DID LSB XSB Status
00 Running
00 00-0
---------------------------
01 Powered Off
00 01-0
CPU:
----
00 00-0 0 on-line 2150 5
00 00-0 1 on-line 2150 5
00 00-0 2 on-line 2150 5
00 00-0 3 on-line 2150 5
00 00-0 8 on-line 2150 5
00 00-0 9 on-line 2150 5
00 00-0 10 on-line 2150 5
00 00-0 11 on-line 2150 5
00 00-0 16 on-line 2150 5
00 00-0 17 on-line 2150 5
00 00-0 18 on-line 2150 5
00 00-0 19 on-line 2150 5
00 00-0 24 on-line 2150 5
00 00-0 25 on-line 2150 5
00 00-0 26 on-line 2150 5
00 00-0 27 on-line 2150 5
Memory:
-------
board perm base domain target deleted remaining
DID XSB mem MB mem MB address mem MB XSB mem MB mem MB
00 00-0 16384 1327 0x000003c000000000 16384
IO Devices:
----------
DID XSB device resource usage
00 00-0 sd1 /dev/dsk/c0t0d0s0 component of concat "/dev/md/dsk/d11"
00 00-0 sd1 /dev/md/dsk/d11 submirror of "/dev/md/dsk/d10"
00 00-0 sd1 /dev/md/dsk/d10 mounted filesystem "/"
00 00-0 sd1 /dev/md/dsk/d20 swap area
00 00-0 sd1 /dev/md/dsk/d50 contains soft partition(s)
00 00-0 sd1 /dev/md/dsk/d80 soft partition based on "/dev/md/dsk/d50"
00 00-0 sd1 /dev/md/dsk/d80 mounted filesystem "/var/crash"
00 00-0 sd1 /dev/dsk/c0t0d0s7 contains metadb(s)
00 00-0 sd0 /dev/dsk/c0t1d0s7 contains metadb(s)
00 00-0 bge0 SUNW_network/bge0 bge0 hosts IP addresses: 10.131.12.23,
10.131.99.23
00 00-0 bge1 SUNW_network/bge1 bge1 hosts IP addresses: 10.131.99.24



XSCF> console -d 0
# psrinfo
0 on-line since 02/27/2009 16:58:45
1 on-line since 02/27/2009 16:58:46
2 on-line since 02/27/2009 16:58:46
3 on-line since 02/27/2009 16:58:46
8 on-line since 02/27/2009 16:58:46
9 on-line since 02/27/2009 16:58:46
10 on-line since 02/27/2009 16:58:46
11 on-line since 02/27/2009 16:58:46
16 on-line since 02/27/2009 16:58:46
17 on-line since 02/27/2009 16:58:46
18 on-line since 02/27/2009 16:58:46
19 on-line since 02/27/2009 16:58:46
24 on-line since 02/27/2009 16:58:46
25 on-line since 02/27/2009 16:58:46
26 on-line since 02/27/2009 16:58:46
27 on-line since 02/27/2009 16:58:46
# prtdiag
==================================== CPUs ====================================

--- ---- ---------------------------------------- ---- --- ----- ----
00 0 0, 1, 2, 3 2150 5.0 6 146
00 1 8, 9, 10, 11 2150 5.0 6 146
00 2 16, 17, 18, 19 2150 5.0 6 146
00 3 24, 25, 26, 27 2150 5.0 6 146

--- ------ ------------------ ------- ------ ----- ------- ----------
========================= IO Cards =========================
LSB Name Model

--- -------------- ------------
00 scsi LSI,1064
00 network N/A
00 network N/A

----------------------
OBP 4.24.8 2008/04/17 18:20
Mode switch is in LOCK mode

# init 0
{0} ok



{0} ok show-devs
/os-io
/ramdisk-root
/pci@3,700000
/pci@2,600000
/pci@1,700000
/pci@0,600000
/pci@8,4000
/cmp@418,0
/cmp@410,0
/cmp@408,0
/cmp@400,0
/pseudo-mc@200,200
/nvram
/pseudo-console
/virtual-memory
/memory@m3c000000000
/aliases
/options
/openprom
/chosen
/packages
/pci@0,600000/pci@0
/pci@0,600000/pci@0/pci@9
/pci@0,600000/pci@0/pci@8
/pci@0,600000/pci@0/pci@8/pci@0,1
/pci@0,600000/pci@0/pci@8/pci@0
/pci@0,600000/pci@0/pci@8/pci@0/network@2,1
/pci@0,600000/pci@0/pci@8/pci@0/network@2
/pci@0,600000/pci@0/pci@8/pci@0/scsi@1
/pci@0,600000/pci@0/pci@8/pci@0/scsi@1/disk
/pci@0,600000/pci@0/pci@8/pci@0/scsi@1/tape
/pci@8,4000/ebus@1
/pci@8,4000/ebus@1/panel@14,280030
/pci@8,4000/ebus@1/scfc@14,200000
/pci@8,4000/ebus@1/serial@14,400000
/pci@8,4000/ebus@1/flashprom@10,0
/cmp@418,0/core@1
/cmp@418,0/core@0
/cmp@410,0/core@1
/cmp@410,0/core@0
/cmp@408,0/core@1
/cmp@408,0/core@0
/cmp@400,0/core@1
/cmp@400,0/core@0
/openprom/client-services
/packages/hsfs-file-system
/packages/ufs-file-system
/packages/obp-tftp
/packages/terminal-emulator
/packages/disk-label
/packages/deblocker
/packages/SUNW,builtin-drivers
/packages/SUNW,probe-error-handler



{0} ok cd /
{0} ok ls
f00f90ec os-io
f00f258c ramdisk-root
f00e31bc pci@3,700000
f00d709c pci@2,600000
f00caf74 pci@1,700000
f0098fb4 pci@0,600000
f00933dc pci@8,4000
f0091420 cmp@418,0
f008f464 cmp@410,0
f008d4a8 cmp@408,0
f008b4ec cmp@400,0
f008ae18 pseudo-mc@200,200
f007a570 nvram
f0072134 pseudo-console
f0043764 virtual-memory
f0043150 memory@m3c000000000
f002d464 aliases
f002d3ec options
f002d2a4 openprom
f002d230 chosen
f002d1b8 packages
{0} ok cd /
{0} ok cd cmp@418,0
{0} ok ls
f0092d1c core@1
f0092694 core@0
{0} ok cd core@0
{0} ok ls
f0092bf0 cpu@1
f0092afc cpu@0


6.2.3. M8000/M9000 CPU Path Example

The following output was taken from an M8000 which deliberately has a rather complicated configuration
with regard to XSB's and LSB's! Pick a CPU, the objective is to identify which CMU it is on and which
CPU slot on the CMU:
XSCF> showboards -a
XSB DID(LSB) Assignment Pwr Conn Conf Test Fault
---- -------- ----------- ---- ---- ---- ------- --------
01-0 07(01) Assigned y y y Passed Normal
XSCF> showdcl -a
DID LSB XSB Status
00 Running
02 01-2
---------------------------
01 Running
04 03-2
05 01-1
---------------------------
03 Running
02 01-3
12 03-0
---------------------------
07 Running
01 01-0
03 03-1
09 03-3
CPU:
----
07 01-0 32 on-line 2280 5
07 01-0 33 on-line 2280 5
07 01-0 34 on-line 2280 5
07 01-0 35 on-line 2280 5
07 03-1 104 on-line 2280 5
07 03-1 105 on-line 2280 5
07 03-1 106 on-line 2280 5
07 03-1 107 on-line 2280 5
07 03-3 312 on-line 2280 5
07 03-3 313 on-line 2280 5
07 03-3 314 on-line 2280 5
07 03-3 315 on-line 2280 5
Memory:
-------
board perm base domain target deleted remaining
07 01-0 16384 2150 0x0000038000000000 49152
07 03-1 16384 0 0x0000030000000000 49152
07 03-3 16384 0 0x0000018000000000 49152
IO Devices:
----------
07 01-0 sd1 /dev/dsk/c0t0d0s0 mounted filesystem "/"
07 01-0 sd1 /dev/dsk/c0t0d0s1 swap area
07 01-0 sd1 /dev/dsk/c0t0d0s1 dump device (swap)
07 01-0 bge0 SUNW_network/bge0 bge0 hosts IP addresses: 10.2.89.135
XSCF> console -d 7



# prtdiag
==================================== CPUs ====================================

--- ---- -------------------- ---- --- ----- ----
01 0 32, 33, 34, 35 2280 5.0 6 146
03 1 104, 105, 106, 107 2280 5.0 6 146
09 3 312, 313, 314, 315 2280 5.0 6 146
====================== Memory Configuration ======================
Memory Available Memory DIMM Number of

LSB Group Size Status Size DIMMs
--- ------ ------------------ ------- ------ ---------
01 A 8192MB okay 2048MB 4
01 B 8192MB okay 2048MB 4
03 A 8192MB okay 2048MB 4
03 B 8192MB okay 2048MB 4
09 A 8192MB okay 2048MB 4
09 B 8192MB okay 2048MB 4
========================= IO Cards =========================
LSB Name Model

--- -------------- ------------
01 scsi LSI,1064
01 network N/A
01 network N/A
LSB Name Model

--- -------------- ------------
03 scsi LSI,1064
03 network N/A
03 network N/A
LSB Name Model

--- -------------- ------------
09 scsi LSI,1064
09 network N/A
09 network N/A

----------------------
OBP 4.24.8 2008/04/17 18:20
Mode switch is in LOCK mode
# psrinfo
32 on-line since 03/10/2009 14:23:00
33 on-line since 03/10/2009 14:23:01
34 on-line since 03/10/2009 14:23:01
35 on-line since 03/10/2009 14:23:01
104 on-line since 03/10/2009 14:23:01
105 on-line since 03/10/2009 14:23:01
106 on-line since 03/10/2009 14:23:01
107 on-line since 03/10/2009 14:23:01
312 on-line since 03/10/2009 14:23:01
313 on-line since 03/10/2009 14:23:01
314 on-line since 03/10/2009 14:23:01
315 on-line since 03/10/2009 14:23:01
# init 0
{20} ok



{20} ok show-devs
/pci@97,700000
/pci@96,600000
/pci@33,700000
/pci@32,600000
/pci@11,700000
/pci@10,600000
/pci@98,4000
/pci@38,4000
/pci@18,4000
/cmp@538,0
/cmp@468,0
/cmp@420,0
/pseudo-mc@290,200
/pseudo-mc@230,200
/pseudo-mc@210,200
/nvram
/pseudo-console
/virtual-memory
/memory@m38000000000
/aliases
/options
/openprom
/chosen
/packages
/pci@96,600000/pci@0,1
/pci@96,600000/pci@0
/pci@96,600000/pci@0,1/network@1,1
/pci@96,600000/pci@0,1/network@1
/pci@96,600000/pci@0/scsi@1
/pci@96,600000/pci@0/scsi@1/disk
/pci@96,600000/pci@0/scsi@1/tape
/pci@32,600000/pci@0,1
/pci@32,600000/pci@0
/pci@32,600000/pci@0,1/network@1,1
/pci@32,600000/pci@0/scsi@1
/pci@10,600000/pci@0,1
/pci@10,600000/pci@0
/pci@10,600000/pci@0,1/network@1,1
/pci@10,600000/pci@0/scsi@1
/pci@98,4000/ebus@1
/pci@98,4000/ebus@1/panel@14,280030
/pci@98,4000/ebus@1/scfc@14,200000
/pci@98,4000/ebus@1/serial@14,400000
/pci@38,4000/ebus@1
/pci@38,4000/ebus@1/panel@14,280030
/pci@38,4000/ebus@1/scfc@14,200000
/pci@38,4000/ebus@1/serial@14,400000
/pci@18,4000/ebus@1
/pci@18,4000/ebus@1/panel@14,280030
/pci@18,4000/ebus@1/scfc@14,200000
/pci@18,4000/ebus@1/serial@14,400000
/cmp@538,0/core@1
/cmp@538,0/core@0
/cmp@468,0/core@1
/cmp@468,0/core@0
/cmp@420,0/core@1
/cmp@420,0/core@0



/openprom/client-services
/packages/obp-tftp
/packages/terminal-emulator
/packages/disk-label
/packages/deblocker
/packages/SUNW,builtin-drivers
/packages/SUNW,probe-error-handler
{20} ok


7. Operation
7.1. Connectivity
7.1.1. Connecting to the XSCF
You can login to the XSCF over the Serial Port or over the network, the same as an E25K Controller.
• If you login over the Serial Port, you may get the Solaris Root Console Login instead of the
XSCF login , do a # . and this should get you down to the XSCF> prompt, then exit. The
restriction with a serial port login is that only one person can use the XSCF at a time.
Without a network connection, we would not be able to monitor the XSCFU
$ ssh user@tc
> conn mx000-sc
• Alternatively, login directly to the XSCF over the network:
$ ssh mx000-xscf-userr@mx000-sc
7.1.2. Login to the XSCF

All OPL Server XSCF users should be logged in spasswd and can be found from the Infrastructure
servers as follows
$ spasswd xscf
The following example is of a connection to the Serial Port of the XSCF of an M5000. In this case, when
we connect to the XSCF, we get the XSCF login prompt. We login in, establish a console connection
with a domain, then logout again:
login: mx000-xscf-user
XSCF> showdomainstatus -a
DID Domain Status
00 -
01 -
02 -
03 -
XSCF> console -d 0
Connect to DomainID 0?[y|n] :y
c1718-3-93 console login:

# . (disconnect from console.)
• If there is already a console connection, you can view details as follows:
XSCF> who
USER TTY IDLE FROM HOST
m5000-admin pts/1 00:00m Mar 12 14:39 195.121.217.14
XSCF> showconsolepath -a
User DID ro/rw escape Date
m5000-admin 0 rw # Thu Mar 12 14:47:00 2009
XSCF> exit
login:
(CTL-SHIFT-6) then (X)

This will break out of a Cisco TC
cisco TC> disco

cisco TC> quit


7.1.3. XSCF Commands

7.1.3.1. Display Commands
You can do a <TAB> <TAB> at the XSCF> prompt and you will get the following:
XSCF> <TAB> <TAB>
Display all 143 possibilities? (y or n) y
addboard setcod showdevices
addcodlicense setdate showdomainmode
addfru setdcl showdomainstatus
adduser setdomainmode showdscp
applynetwork setdomparam showdualpowerfeed
cfgdevice setdscp showemailreport
clockboard setdualpowerfeed showenvironment
confdidr setemailreport showfru
console sethostname showhardconf
deleteboard sethttps showhostname
deletecodlicense setldap showhttps
deletefru setlocale showldap
deleteuser setlocator showlocale
disablemodes setlookup showlocator
disableuser setnameserver showlogs
dumpconfig setnetwork showlookup
enableescalation setntp showmodes
enableservice setpasswordpolicy showmonitorlog
enableuser setpowerupdelay shownameserver
escalation setprefetchmode shownetwork
exit setprivileges shownotice
flashupdate setrci showntp
fmadm setrcic showpasswordpolicy
fmdump setroute showpowerupdelay
fmstat setservicetag showprefetchmode
getflashimage setshutdowndelay showresult
ioxadm setsmtp showroute
man setsnmp showservicetag
moveboard setsnmpusm showshutdowndelay
nslookup setsnmpvacm showsmtp
password setssh showsnmp
ping setsunmc showsnmpusm
poweroff settelnet showsnmpvacm
poweron settimezone showssh
prtfru setupfru showstatus
rebootxscf setupplatform showsunmc
replacefru showaltitude showtelnet
reset showapcs showtimezone
resetdateoffset showarchiving showuser
restoreconfig showaudit snapshot
restoredefaults showautologout switchscf
sendbreak showboards testsb
service showcod traceroute
setaltitude showcodlicense unlockmaintenance
setapcs showcodusage version
setarchiving showconsolepath viewaudit
setaudit showdate who
setautologout showdcl
Alternatively, you can type part of a command and then do a <TAB> <TAB> at the XSCF>
prompt and it will list all available commands:
XSCF> delete <TAB> <TAB>
deleteboard deletecodlicense deletefru deleteuser


7.1.3.2. Command Line Editing

As XSCF Shell allows EMACS style Command Line Editing (same as bash) as follows:
• Up Arrow (CTL p) - Command History - Scroll backwards

• Down Arrow (CTL n) - Command History - Scroll forwards
• Left Arrow (CTL b) - Command Line Editing - Back one character
• Right Arrow (CTL f) - Command Line Editing - Forward one character
• CTL e - Command Line Editing - End of line
• CTL a - Command Line Editing - Start of line
• CTL d - Command Line Editing - Delete character
• CTL l - Command Line Editing - Redraw line
• CTL c - Command Line Editing - Interrupt. Cancel command
7.1.3.3. Command Help

Running any command with the “-h” option will show command syntax:
XSCF> console -h
usage : console -d domain_id [[-q]{-y|-n}] [-f|-r] [-s escapeChar]
console -h
7.1.3.4. Man Pages

The man(1) displays manual pages of specified XSCF shell command.
XSCF> man exit
System Administration exit(1)
NAME
exit - exit the XSCF shell
SYNOPSIS
exit
DESCRIPTION
The exit(1) command exits and closes the XSCF shell.
Privileges
No privileges are required to run this command.
Refer to setprivileges(8) for more information.
XCP Last change: June 2007 1


7.2. Display
7.2.1. Platform
The following commands display basic information about the platform configuration and status:
7.2.1.1. showhostname
Display XSCF hostnames
XSCF> showhostname -a
xscf#0:c1718-4-92-mgt-xscf0.ssclabs.net
7.2.1.2. shownetwork
The shownetwork command displays XSCF network configuration details:
XSCF> shownetwork -a
xscf#0-lan#0
Link encap:Ethernet HWaddr 00:21:28:25:DE:D8
inet addr:10.130.4.92 Bcast:10.130.255.255 Mask:255.255.0.0
UP BROADCAST RUNNING MULTICAST MTU:1500 Metric:1
RX packets:2182410 errors:0 dropped:0 overruns:0 frame:0
TX packets:41151 errors:0 dropped:0 overruns:0 carrier:0
collisions:0 txqueuelen:1000
RX bytes:136027156 (129.7 MiB) TX bytes:17908147 (17.0 MiB)
Base address:0xe000
xscf#0-lan#1
Link encap:Ethernet HWaddr 00:21:28:25:DE:D9
BROADCAST MULTICAST MTU:1500 Metric:1
RX packets:0 errors:0 dropped:0 overruns:0 frame:0
TX packets:0 errors:0 dropped:0 overruns:0 carrier:0
collisions:0 txqueuelen:1000
RX bytes:0 (0.0 B) TX bytes:0 (0.0 B)
Base address:0xc000
7.2.1.3. showroute
The showroute command displays XSCF routing details:
XSCF> showroute -a
Destination Gateway Netmask Flags Interface
10.130.0.0 * 255.255.0.0 U xscf#0-lan#0
default 10.130.0.1 0.0.0.0 UG xscf#0-lan#0
7.2.1.4. showdscp
The showdscp command displays the IP addresses assigned to the Domain to Service Processor
Communications Protocol (DSCP)
XSCF> showdscp
DSCP Configuration:
Network: 192.168.224.0
Netmask: 255.255.255.0
Location Address
---------- ---------
XSCF 192.168.224.1
Domain #00 192.168.224.2
Domain #01 192.168.224.3
Domain #02 192.168.224.4
Domain #03 192.168.224.5


7.2.1.5. showsunmc
This command is used to show the setup information and current status of the Sun Management
Center agent.
XSCF> showsunmc
Agent Status: Disabled, stopped
Setup Status: Not set up
SunMC Server: NOT_SET
Security Seed: maplesyr
SNMPv1 Community String: public
Agent Port: 1161
Host Trap Port: 162
Host Event Port: 163
SNMP Agent Port: 161
Domain Agent Port: 1161
7.2.1.6. shownameserver
The shownameserver(8) command displays the registered DNS servers in the XSCF network.
XSCF> shownameserver
nameserver 10.131.2.2
7.2.1.7. showdate
The showdate(8) command displays the date and time of XSCF.
XSCF> showdate
Tue Feb 24 04:55:47 GMT 2009


7.2.1.8. showhardconf
The showhardconf command displays a variety of information about the H/W configuration of the
server and also the status of each Domain.
Pay special attention to the output and look for an asterisk (*) in the first column of the output as
this indicates a problem which requires attention. An example of this is included at the end of this
document where we follow through a PSU problem on an M4000.
XSCF> showhardconf
Domain#0 Domain_Status:OpenBoot Execution Completed;

CPUM#0-CHIP#0 Status:Normal; Ver:0401h; Serial:PP090201GQ ;
+ FRU-Part-Number:CA06761-D202 D0 /375-3568-04 ;
+ Freq:2.400 GHz; Type:32;
+ Core:4; Strand:2;
+ Core:4; Strand:2;
MEM#0B Status:Normal;
+ Code:ce0000000000000001M3 93T5750EZA-CE6 4145-4550e4ee;
+ Code:ce0000000000000001M3 93T5750EZA-CE6 4145-4550e47e;
+ Code:ce0000000000000001M3 93T5750EZA-CE6 4145-4550e4eb;
+ Code:ce0000000000000001M3 93T5750EZA-CE6 4145-4550e4a4;
+ Code:ce0000000000000001M3 93T5750EZA-CE6 4145-4550e48f;
+ Code:ce0000000000000001M3 93T5750EZA-CE6 4145-4550e469;
+ Code:ce0000000000000001M3 93T5750EZA-CE6 4145-4550e49e;
DDC_A#0 Status:Normal;
DDC_B#0 Status:Normal;
IOU#0 Status:Normal; Ver:0101h; Serial:BF0848PGGR ;
DDCR Status:Normal;
PCI#1 Name_Property:SUNW,qlc; Card_Type:Other;
PCI#3 Name_Property:network; Card_Type:Other;
XSCFU Status:Normal,Active; Ver:0101h; Serial:BF0844MRXV ;
OPNL Status:Normal; Ver:0101h; Serial:BF0848P79J ;
PSU#0 Status:Normal; Serial:1357FYG-0843AD0061;



+ FRU-Part-Number:CF00300-2011 0202 /300-2011-02-02;
+ Power_Status:On; AC:200 V;
FANBP_C Status:Normal; Ver:0301h; Serial:BF0844MVD0 ;
FAN_A#0 Status:Normal;
Chassis Serial No.

The showhardconf command displays the Chassis Serial No. which can be broken down as
follows:
PPMYYWWSSS
PP is a factory code (place of manufacture)

M is a model code
YY is the last two digits of the year
WW is the week of the calendar year
SSS is a hexidecimal or alphanumeric sequence number
In the example above, serial number BCF0904031 indicates it is from week 04 of 2009
showhardconf -u
You can also view a formatted output with the -u option:
XSCF> showhardconf -u
SPARC Enterprise M5000; Memory_Size:64 GB;
+-----------------------------------+------------+
| FRU | Quantity |
+-----------------------------------+------------+
| MBU_B | 1 |
| CPUM | 4 |
| Freq:2.400 GHz; | ( 8) |
| MEMB | 4 |
| MEM | 32 |
| Type:2B; Size:2 GB; | ( 32) |
| DDC_A | 4 |
| DDC_B | 2 |
| IOU | 2 |
| DDC_A | 2 |
| DDCR | 2 |
| DDC_B | 2 |
| XSCFU | 1 |
| OPNL | 1 |
| PSU | 4 |
| FANBP_C | 1 |
| FAN_A | 4 |
+-----------------------------------+------------+


7.2.1.9. showenvironment
Display the intake air temperature and humidit, temperature sensor information, voltage sensor
information, and fan speed information about the system
• volt Displays the voltage information

• temp Displays the temperature information.
• Fan Displays the fan speed information
XSCF> showenvironment volt

MBU_B
1.0V Power Supply Group:1.000V
CPUM#0-CHIP#0
MEMB#0
IOU#0
12V Power Supply Group:12.260V
-12V Power Supply Group:-12.080V
FANBP_C
12.0V#0 Power Supply Group:11.930V
12.0V#1 Power Supply Group:12.120V
XSCF> showenvironment temp

Temperature:22.00C
MBU_B
CPUM#0-CHIP#0:35.40C
IOU#0:37.50C
IOU#1:35.50C
XSCF> showenvironment Fan

FAN_A#0:Low speed
FAN_A#0: 3133rpm
PSU#0
PSU#0:Low speed
PSU#0: 3276rpm
PSU#0: 3245rpm


7.2.2. Domain
The following commands display basic information about the domain configuration and status:
7.2.2.1. showboards
The showboards(8) command displays system board information including the domain ID of the
domain to which the target system board belongs and various kinds of system board status in list
format. It does not tell us the domain status (Running or OBP) or what mode the XSB's are in.
Note that showboards -a displays XSB information
• Display of information on all system boards
In this example, we have an M5000 with 2 XSB's. Domain 0 has 1 XSB assigned (XSB 00-0).

---- - -------- ----------- ---- ---- ---- ------- -------- ----
00-0 00(00) Assigned y y n Passed Normal n
01-0 SP Unavailable y n n Unknown Normal n
7.2.2.2. showfru
The showfru(8) command displays information about the PSB division type and memory mirroring
mode settings in list format. To change the PSB configuration, use the setupfru(8) command.
Remember that showfru -a sb displays PSB information
• An M4000 has only 1 PSB and would look like this:

XSCF> showfru -a sb
sb 00 Uni no
• An M5000 Can have 2 PSB's and would look like this:

XSCF> showfru -a sb
sb 00 Uni no
sb 01 Uni no


7.2.2.3. showdcl
The XSCF showdcl command displays domain information including the domain ID, configured
system board numbers, and domain status
• Show all DCL information
In this example, we have an M5000 with 1 Domain (DID 0) which is at the OBP (after POST). The
domain comprises 1 XSB (XSB 00-0). There are no other DCL's configured.
XSCF> showdcl -av

00 OpenBoot Execution Completed FRU
01 -
02 -
03 -
04 -
05 -
06 -
07 -
08 -
09 -
10 -
11 -
12 -
13 -
14 -
15 -
7.2.2.4. showdomainstatus
The showdomainstatus(8) command lists the domains in the system and their status. This
command displays the same domain status information as the showdcl(8) command.
• Display of information on all domains
In this example, we have an M5000 with 2 Domains. Domain 0 is at the OBP (after POST) and
Domain 1 is Running Solaris.
DID Domain Status
00 OpenBoot Execution Completed
01 Running
02 -
03 -


7.2.2.5. Displaying Device Information

The showdevices(8) command displays information about the physical devices including CPUs,
memory, and PCI cards mounted on system boards, and displays the hardware resources usable
with these devices in hardware resource format.
• Display of device information on XSB00-0
DID Domain Status
00 Running
01 -
XSCF> showdevices 00-0
CPU:
----
00 00-0 0 on-line 2400 5
00 00-0 1 on-line 2400 5
00 00-0 2 on-line 2400 5
00 00-0 3 on-line 2400 5
00 00-0 4 on-line 2400 5
00 00-0 5 on-line 2400 5
00 00-0 6 on-line 2400 5
00 00-0 7 on-line 2400 5
00 00-0 8 on-line 2400 5
00 00-0 9 on-line 2400 5
00 00-0 10 on-line 2400 5
00 00-0 11 on-line 2400 5
00 00-0 12 on-line 2400 5
00 00-0 13 on-line 2400 5
00 00-0 14 on-line 2400 5
00 00-0 15 on-line 2400 5
00 00-0 16 on-line 2400 5
00 00-0 17 on-line 2400 5
00 00-0 18 on-line 2400 5
00 00-0 19 on-line 2400 5
00 00-0 20 on-line 2400 5
00 00-0 21 on-line 2400 5
00 00-0 22 on-line 2400 5
00 00-0 23 on-line 2400 5
00 00-0 24 on-line 2400 5
00 00-0 25 on-line 2400 5
00 00-0 26 on-line 2400 5
00 00-0 27 on-line 2400 5
00 00-0 28 on-line 2400 5
00 00-0 29 on-line 2400 5
00 00-0 30 on-line 2400 5
00 00-0 31 on-line 2400 5
Memory:
-------
board perm base domain target deleted
remaining
00 00-0 32768 1624 0x000003c000000000 32768
IO Devices:
----------
00 00-0 sd2 /dev/dsk/c0t0d0s0 mounted filesystem "/"
00 00-0 sd2 /dev/dsk/c0t0d0s1 swap area
00 00-0 sd2 /dev/dsk/c0t0d0s1 dump device (swap)
00 00-0 bge0 SUNW_network/bge0 bge0 hosts IP addresses:
10.128.3.92


7.3. Control
7.3.1. Platform
7.3.1.1. Reboot the XSCF
The XSCF can be rebooted with no impact on the running domains using the rebootxscf
command
XSCF> rebootxscf
The XSCF will be reset. Continue? [y|n] :y


7.3.2. Domain
7.3.2.1. Start the Domain
---- - -------- ----------- ---- ---- ---- ------- -------- ----
00-0 * 00(00) Assigned y n n Passed Normal n
XSCF> showdcl -av

00 Powered Off FRU
02 -
03 -
04 -
05 -
06 -
07 -
08 -
09 -
10 -
11 -
12 -
13 -
14 -
15 -
XSCF> poweron -d 0
DomainIDs to power on:00
Continue? [y|n] :y
00 :Powering on
*Note*
This command only issues the instruction to power-on.
The result of the instruction can be checked by the "showlogs power".
XSCF> showlogs power

Date Event Cause DID Switch
Feb 24 04:54:45 GMT 2009 Domain Power Off Operator 00 Locked
Feb 24 05:06:26 GMT 2009 Domain Power On Operator 00 Locked
7.3.2.2. Domain Console

XSCF> console -d 0
pnXu571-ukXr console login:
7.3.2.3. Logout & Disconnect

#● (hash dot - Diconnect from console)
XSCF> exit


7.3.2.4. showdomainmode
The showdomainmode(8) command displays the modes of operation that is set for the specified
domain:
XSCF> showdomainmode -d 0
Host-ID :8525ded4
Diagnostic Level :max
Secure Mode :on
Autoboot :on
CPU Mode :auto
7.3.2.5. setdomainmode
The setdomainmode(8) sets the modes of operation for the specified domain. The modes of
operation for the specified domain include the following types:
• Diagnostics Level
OpenBoot PROM diagnostic levels. The default is standard.
• Secure Mode
Whether to enable or disable the host watchdog and suppress break signal reception. The
default of the host watchdog is enable and suppress break signal reception is enable.
• Autoboot
Whether to enable or disable the auto boot function used at domain startup. The default is
enable.
• CPU Mode
Way of determining the CPU operational mode mounted on the domain. The CPU
operational mode can be automatically determined at domain startup, or manually set to
the compatible mode. The default is to let it automatically determined at domain startup.
Example 1, force a domain to stop at the OBP after poweron:
XSCF> setdomainmode -d 0 -m autoboot=off

Diagnostic Level :max -> -
Secure Mode :off -> -
Autoboot :on -> off
CPU Mode :auto -> -
The specified modes will be changed.
Continue? [y|n] :y
configured.
Secure Mode :on
Autoboot :off (autoboot:off)
CPU Mode :auto
Example 2, Set the OpenBoot PROM diagnostic level to “none”:

XSCF> setdomainmode -d 0 -m diag=none
Diagnostic Level :min -> none
Secure Mode :on -> -
Autoboot :on -> -
CPU Mode :auto -> -
Continue? [y|n]:y
configured.
Diagnostic Level :none
Secure Mode :on (host watchdog: available Break-signal:non-receive)
Autoboot :on (autoboot:on)
CPU Mode :auto


7.3.2.6. setdomparam
The setdomparam(8) command rewrites OpenBoot PROM environment variables for a specified
domain. The following OpenBoot PROM environment variables can be specified.
• use-nvramrc?
Whether to execute the contents of the NVRAM at the boot or reboot of a domain.
• security-mode?
Firmware security level setting
• set-defaults
Whether to restore OpenBoot PROM environment variables to the settings at the time of
shipment from the factory
Example 1, set use-nvramrc? to false for DID 0.

XSCF> setdomparam -d 0 use-nvramrc
DomainIDs of domains that will be affected:00
OpenBoot PROM variable use-nvramrc will be set to false.
Continue? [y|n]:y
Example 2, set security-mode to none for DID 0.

XSCF> setdomparam -d 0 security-mode
OpenBoot PROM variable security-mode will be set to none.
Continue? [y|n]:y
Example 3, Initializes the OpenBoot PROM environment variables of DID 0 to factory settings
XSCF> setdomparam -d 0 set-defaults
All OpenBoot PROM variable will be reset to original default values.
Continue? [y|n]:y
Note
The domain needs to be powered off for this command to run

XSCF> showdomainstatus -d 0
DID Domain Status
00 Running
01 -
XSCF> setdomparam -d 0 use-nvramrc

This operation cannot be done for the domain which is powered on.
Try again after powering off the domain.


7.3.2.7. Stop the Domain

• Connect & Login to the XSCF
• Check the Domain Status

DID Domain Status
00 Running
01 Running
• Login to the Domain console & shutdown Solaris
Note: You should check to see if there are any Oracle DB's running first! If there are,
arrange for the DBA's to shut these down gracefully.
XSCF> console -d 0
c1718-3-93 console login: root

Password:
Last login: Tue Feb 24 04:55:55 on console
Feb 24 05:12:05 c1718-3-93 login: ROOT LOGIN /dev/console
Sun Microsystems Inc. SunOS 5.10 Generic January 2005
Sourcing //.profile-EIS.....
root@c1718-3-93 # shutdown -i 0
Shutdown started. Tue Feb 24 05:12:16 GMT 2009
Broadcast Message from root (console) on c1718-3-93 Tue Feb 24 05:12:16...

The system c1718-3-93 will be shut down in 1 minute
Do you want to continue? (y or n): y
Broadcast Message from root (console) on c1718-3-93 Tue Feb 24 05:15:28...
THE SYSTEM c1718-3-93 IS BEING SHUT DOWN NOW ! ! !
Log off now or risk your files being damaged
showmount: c1718-3-93: RPC: Program not registered

Changing to init state 0 - please wait
root@c1718-3-93 # Feb 24 05:15:39 c1718-3-93 ip: ip_create_dl: hw addr length
= 0
svc.startd: The system is coming down. Please wait.
svc.startd: 100 system services are now being stopped.
Feb 24 05:15:44 c1718-3-93 syslogd: going down on signal 15
ip_create_dl: hw addr length = 0
svc.startd: The system is down.
syncing file systems... done
Program terminated
{18} ok
• Once at the OBP, disconnect :
#● (hash dot)
{18} ok exit from console.

XSCF>
• Check Domain Status
DID Domain Status
00 Booting/OpenBoot PROM prompt
01 -
---- -------- ----------- ---- ---- ---- ------- --------
00-0 00(00) Assigned y y n Passed Normal



• From XSCF, power off the domain:
XSCF> poweroff -d 0
DomainIDs to power off:00
Continue? [y|n] :y
00 :Powering off
*Note*
This command only issues the instruction to power-off.

<... output deleted …>


7.3.2.8. Send a Break

By default, a domain will ignore a send break request. This is because the default is that the
Domain Secure Mode is on (showdomainmode) and the Panel Switch is in the Locked position
(showhardconf).
To be able to send a break signal to the Domain, we must either turn the Panel Switch to the
Service position, which will of course require access to the Computer Hall, or we can turn the
Domain Secure Mode off. However, turning the Domain Secure Mode to off does have an
unwanted side effect in that it also disables the host watchdog reset.
Host Watchdog
Based on communication between XSCF and a domain, the host watchdog function checks
whether the domain is alive (heart beat or alive check). XSCF periodically monitors the
operational status of Solaris OS, to detect the Solaris OS hang-up. When detected the
Solaris OS hang-up, XSCF generates a Solaris OS panic on the relevant domain.
Therefore, to workaround this, we temporarily disable the Domain Secure Mode so that we can
send the break signal to the domain, and then enable it again as follows:
XSCF> showdomainstatus -av

DID Domain Status
00 Running
01 -
Host-ID :8525ded5
Diagnostic Level :min
Secure Mode :on
Autoboot :on
CPU Mode :auto
XSCF> setdomainmode -d 0 -m secure=off

Diagnostic Level :min -> -
Secure Mode :on -> off
Autoboot :on -> -
CPU Mode :auto -> -
Continue? [y|n] :y
configured.
Secure Mode :off (host watchdog: unavailable Break-signal:receive)
CPU Mode :auto
XSCF> sendbreak -d 0
Send break signal to DomainID 0?[y|n] :y
XSCF> console -d 0
{22} ok #.
exit from console.
XSCF> setdomainmode -d 0 -m secure=on

Diagnostic Level :min -> -
Secure Mode :off -> on
Autoboot :on -> -
CPU Mode :auto -> -
Continue? [y|n] :y
configured.
CPU Mode :auto
Note
On a console connection the key combination of "ctrl" and "\" ( CTRL- \ )will result in ascii value
0x1c (decimal = 28), which will also send a break signal to the domain.


7.3.2.9. Force a Domain panic (reset domain)

The reset(8) command resets the specified domain. The following three levels of resetting can be
specified:
• por Resets the domain system.

• panic Instructs the Solaris OS of the domain to generate a panic.
The command is ignored if it is issued during power-off or shutdown.
• xir Resets the domain CPU.
In this example, we will force a panic on a running domain (DID 0)
XSCF> console -d 0
c1718-3-92 console login: #.
Password: exit from console.
DID Domain Status
00 Running
01 -
XSCF> reset -d 0 panic

DomainID to panic:00
Continue? [y|n] :y
00 :Panicked
*Note*
This command only issues the instruction to reset.
XSCF> console -d 0
100% done: 94631 pages dumped, compression ratio 7.51, dump succeeded
rebooting...
Resetting... #.
exit from console.
DID Domain Status
00 Powered Off
01 -


8. Administration
8.1. Platform
8.1.1. Configuring the XSCF
8.1.1.1. Login as the default User
To perform any configuration or administration on the XSCF, you need to login as a user with
administrative privileges. If you do not know the administrative user name or password, you may
need to login as the default user to setup one up or change the details.
By default, the XSCF comes with a couple of users configured (default and admin) which can be
used to login to the XSCF using the default account as follows:
• The best way to do this is to take your laptop into the Hall and connect directly to the back
of the Server
• You will need somebody in the Computer Hall standing next to the system during this
operation, you will also need a key for the Operator Panel (OPNL)
• The Cable you need is the same as you would use for a V210, V240, V440, V480 (9-pin
female ot RJ45 female adapter and a straight through ethernet cable)
• Plug the serial cable into the Serial port of the Active XSCF (the M4000 only has one
XSCF – check the XSCF description under the Architecture Section of this document)
• From your laptop, connect to the XSCF over the serial connection using any terminal
software (e.g Tera Term, HyperTerm or tip)
• Log in with the default user account. Follow the instructions to change the mode switch of
the operator panel. It will ask you to change the OPNL key position twice within one
minute, which is why you (or someone else) needs to be next to the Server.
For Example:
login: default
Change the panel mode switch to Service and press return...
Turn Key
Leave it in that position for at least 5 seconds. Change the panel mode
switch to Locked, and press return...
Turn Key
XSCF> showuser -l
User Name: xscf-admin
UID: 101
Status: Enabled
Minimum: 0
Maximum: 99999
Warning: 7
Inactive: -1
Last Change: Feb 25, 2009
Password Expires: Never
Password Inactive: Never
Account Expires: Never
Privileges: useradm
platadm
mode
fieldeng
auditadm
Note
If you are setting up an XSCF for the first time, it is possible to login across the Serial Port (through
a TC) as user admin. However, the admin user account is automatically disabled once someone
has either logged in as the default user or created a user account, so this user account will be of
little use once the Server is under live support:


8.1.1.2. Setup XSCF Administrative User

Once you have logged into the XSCF, either remotely over the network or as the default user over
the serial connection, you can check and setup users.
• Check to see if there are any users
XSCF> showuser -l
User Name: xscf-admin
UID: 101
Status: Enabled
Minimum: 0
Maximum: 99999
Warning: 7
Inactive: -1
Last Change: Feb 25, 2009
Password Expires: Never
Password Inactive: Never
Account Expires: Never
Privileges: useradm
platadm
mode
fieldeng
auditadm
Note, you can also query a specific user as follows:

XSCF> showuser xscf-admin
• If there is not an administrative user already setup then setone up as follows:
XSCF> adduser xscf-admin

XSCF> password xscf-admin
New XSCF password: xxxxxx
Retype new XSCF password: xxxxxx
XSCF> setprivileges xscf-admin platadm useradm auditadm fieldeng mode
• An alternative way to setup user accounts is as follows:
XSCF> setupplatform -p user

Do you want to set up an account? [y|n]: y
Username: xscf-admin
User id in range 100 to 65533 or leave blank to let the system
choose one:
Username: xscf-admin
User id:
Are these settings correct? [y|n]: y
XSCF> adduser myadminuser
XSCF> setprivileges myadminuser useradm platadm platop
XSCF> password myadminuser
New XSCF password: xxxxxx
Retype new XSCF password: xxxxxx
However, you will need to apply all required priviliges in an additional step if you do it this way
as follows:
XSCF> setprivileges xscf-admin platadm useradm auditadm fieldeng mode
8.1.1.3. setautologout
The setautologout(8) command sets the session timeout time of the XSCF shell.
XSCF> showautologout
30min
XSCF> setautologout -s 60
60min


8.1.1.4. setupplatform
Setupplatform is used to configure the XSCF. The following example was taken from an M5000
XSCF> setupplatform
Do you want to set up an account? [y|n]: n
Do you want to set up networking? [y|n]: y
Do you want to set up the XSCF network interfaces? [y|n]: y
Do you want to configure xscf#0-lan#0? [y|n]: y
xscf#0-lan#0 ip address? []: 10.130.4.93
xscf#0-lan#0 netmask? []: 255.255.0.0
xscf#0-lan#0 default gateway? []: 10.130.0.1
xscf#0-lan#0 ip address: 10.130.4.93
xscf#0-lan#0 netmask: 255.255.0.0
xscf#0-lan#0 default gateway: 10.130.0.1
XSCF> setnetwork xscf#0-lan#0 -m 255.255.0.0 10.130.4.93
XSCF> setroute -c del -n 0.0.0.0 -m 0.0.0.0 -g 10.130.0.1 xscf#0-lan#0
XSCF> setroute -c add -n 0.0.0.0 -m 0.0.0.0 -g 10.130.0.1 xscf#0-lan#0
Do you want to configure xscf#0-lan#1? [y|n]: n
lan#0 is already configured:
lan#0 ip address: 10.130.4.93
lan#0 netmask: 255.255.0.0
Do you want to configure lan#0? [y|n]: n
Do you want to configure lan#1? [y|n]: n
DSCP network is already configured:
DSCP network: 192.168.224.0
DSCP netmask: 255.255.255.0
Do you want to set up the DSCP network? [y|n]: n
Domain name service is already configured:
Primary DNS server ip address:
Secondary DNS server ip address:
Tertiary DNS server ip address:
Domain name: ssclabs.net
XSCF#0 hostname: c1718-4-93-mgt-xscf0
Do you want to set up the domain name service? [y|n]: y
Primary DNS server ip address? []: 10.131.2.2
Do you want a secondary DNS server? [y|n]: n
Do you want to specify a domain name? [y|n]: y
Domain name []: ssclabs.net
XSCF#0 hostname []: c1718-4-92-mgt-xscf0
Primary DNS server ip address: 10.131.2.2
Secondary DNS server ip address:
Tertiary DNS server ip address:
Domain name: ssclabs.net
XSCF#0 hostname: c1718-4-92-mgt-xscf0
Do you want to set up the network time protocol? [y|n]: n
The ssh service is: enabled
Do you want to set up ssh? [y|n]: n
The https service is: disabled
Do you want to set up https? [y|n]: n
Do you want to configure email reports? [y|n]: n
Do you want to apply the network changes? [y|n]: n
The applynetwork command must be run in order for network changes to
take effect
Do you want to set up the chassis altitude? [y|n]: n
Do you want to set up the XSCF time zone? [y|n]: n
You can also run the setuplatform command with the “-p part” option which specifies the particular
setting you want, instead of doing them all as follows:
• altitude Configures the chassis altitude.

• network Configures the XSCF network, DSCP, Domain Name Service,
NTP, SSH, https, and SMTP.
• timezone Sets the time zone for the XSCF. The time zone is chosen from
a list of time zones.
• user Creates a new local XSCF user account with platadm, platop
and useradm privileges.
For Example:
XSCF> setuplatform -p network


8.1.1.5. sethttps
The sethttps(8) command starts or stops the HTTPS service, which is used in the XSCF
network. Also, this command performs authentication-related settings for authentication used
in the HTTPS service.
It is arguable whether this protocol will be allowed through the firewalls on the NHS Project, but
it has been included in case it is ever required. The BUI Interface (Browser User Interface) is
extremely easy to use and gives you pretty much the same full functionality as the XSCF CLI.
The sethttps command syntax takes the following form:
sethttps -c selfsign arg1 arg2 arg3 arg4 arg5 arg6 arg7
Where the arguments take the form:
country state locality organization org-unit common e-mail
The following example worked on servers in both Sun GMP and Linlithgow when configured by
members of the site based support teams and the Bridge during the Mx000 TOI's on XCP 1080.
XSCF> sethttps -c selfsign UK Berks GMP Sun PS m4-01 an.other@sun.com

CA key and CA cert already exist. Do you still wish to update? [y|n] :y
Enter passphrase: 0000
Verifying - Enter passphrase: 0000
XSCF> showhttps
HTTPS status: disabled
Server key: installed in Feb 18 17:20:43 GMT 2009
CA key: installed in Feb 18 17:20:43 GMT 2009
CA cert: installed in Feb 18 17:20:43 GMT 2009
CSR:
-----BEGIN CERTIFICATE REQUEST-----
MIIBzjCCATcCAQAwgY0xCzAJBgNVBAYTAnVrMQ0wCwYDVQQIEwRiZXJrMQwwCgYD
VQQHEwNnbXAxDDAKBgNVBAoTA3NsczENMAsGA1UECxMEZnJlZDEdMBsGA1UEAxMU
YzE3MTgtNC05Mi1tZ3QteHNjZjAxJTAjBgkqhkiG9w0BCQEWFmFudGhvbnkubW9y
cmlzQHN1bi5jb20wgZ8wDQYJKoZIhvcNAQEBBQADgY0AMIGJAoGBAMuyBioUaUoo
/Jj+NUGbe2EzHLaSciwE2vs00hgrPOcNCPXtlkE47xoQNCJOw0QlxhFpGDvfApUu
cF6Um4kY3atU6s7aQczLh5d8/c230vZYvGHjGUaK5heuE5mgIag90M79aVnjhQCz
4eVwmsvY+hAJngN4qggIrKjV2LfZ0LH/AgMBAAGgADANBgkqhkiG9w0BAQQFAAOB
gQBAKO9pVb7Dr1UG5HGrFZudfa33i7fVWA/abTm308TvsTxfcyzD+fP3MqhCeNPV
lP00OYRPTF5Xgq4XROUmHA89s8ouYmdeqgg6XJpOJtX0ZNiqR7TIpJkO72VPCTTi
6YApLBH0ND8vrqaF6xQCcUugWRjl0vcagQmsD4MZ7UJGAQ==
-----END CERTIFICATE REQUEST-----
XSCF> sethttps -c enable

Continue? [y|n] :y
Please reset the XSCF by rebootxscf to apply the https settings.
XSCF> rebootxscf
To use this, now fire up a web browser and point it to the following URL:
• https://XSCF_IP_Address
Note that you may need to add the XSCF IP Address under the “No Proxy For” settings of your
browser, otherwise the your proxy may no nothing about the IP address you are trying to connect
to.


8.1.1.6. restoredefaults
The restoredefaults(8) command deletes the setting and the log information that stored in the
server or the XSCF unit, and restores it to the state as of the factory shipment.
The command syntax is “restoredefaults -c range”, where range is one of:
• xscfu Only restores XSCFU Log and OPNL Backup to factory default
• factory Restores all XSCF settings & OPNL to factory default
Note
You will need to physically power cycle the Server afterwards before it can be used. An example
of setting all XSCF settings & OPNL to factory default is as follows:
XSCF> restoredefaults -c factory
WARNING:
If this system does not have OPNL, this command will set all the user
settable XSCF configuration parameters to their default value as they
were set when the system was shipped out.
Furthemore, this command will delete all logs on both XSCFUs.
Check the man page of this command before you run it.
Continue?[yes/no](default no):yes
You must check the following points.
1. Have the ability to powercycle the system.

2. Have access to the serial console and hold the serial console of the
XSCFU to confirm the completion of the command.
If you answer "yes" this command will HALT the XSCFU when it compltetes.
You will need to powercycle the system after the XSCF BOOT STOP.
Do you really want to continue?
The initialization of XSCFU will be started.
XSCFU : all data clear
OPNL : all data clear (exclude SYSTEM ID data)
XSCF will be automatically rebooted. Afterwards,
XSCFU will be initialized.
The NVRAM setting of XSCFU#0 was completed.
XSCF shutdown request was completed.
execute S10ioxoffXSCF> -- complete
Apr 15 11:07:50 FF2-24-0 XSCF[104]: XSCF shutdown sequence start
<snip>....XSCF reboot..<snip>
***** WARNING *****

XSCF initialization terminate for XSCF/OPNL data clear.
execute "setdefaults factory" (AUTO)
setdefaults : FACTORY mode clear : start
setdefaults : FACTORY mode clear : DBS start
setdefaults : FACTORY mode clear : wait 20s for DBS initialization
initialize OPNL SEEPROM 1/16 -- complete
<snip>....more setdefaults output..<snip>
Please stand by while rebooting the system.

Restarting system.
XSCF uboot 01950000 (Apr 15 2007 - 11:08:18)
XSCF uboot 01950000 (Apr 15 2007 - 11:08:18)
SCF board boot factor = 4040
DDR Real size: 512 MB
DDR: 480 MB
XSCF BOOT STOP (recover by NFB-OFF/ON)
Physically Power Cycle the Server now!


8.1.2. Component Replacement

On the Form Factor Servers (M4000/M5000), the replacefru command allows the Engineer to perform Hot
Replacement of the both Fan Units (FANU) and Power Supply Units (PSU).
On the Data Center Servers (M8000/M9000), the replacefru command allows the Engineer to perform Hot
Replacement the CPU memory unit (CMU), I/O unit (IOU), FAN unit (FANU), Power Supply Unit (PSU),
XSCF unit (XSCFU) & DC to DC Converters (DDC - M8000 only).
Note that you must run replacefru before you remove any of these components, and in the case of the
CMU and IOU on the DC Servers, these components must be removed from the Domain before
replacement using either Static or Dynamic Reconfiguration (if supported)
The SAS Disks are Hot Plugable, but Solaris Administration (SVM/VxVM) will be required to ensure that
they are not in use when they are replaced.
There is also an LED locator light on the OPL Servers Operator Panel (OPNL) which you can turn on and
off manually from the XSCF to help locate a Server using the command setlocator.
8.1.2.1. showlocator
The showlocator(8) command displays the blink state of the CHECK LED on the operator panel,
one of the following states is displayed:
• Off Indicates normal operation.
• Blinking Indicates that the unit is a maintenance target.
• On Indicates that an error was detected in the main unit.
For example:
XSCF> showlocator
Locator LED status:Blinking
8.1.2.2. setlocator
The setlocator(8) command controls the blink state of the CHECK LED on the operator panel.
One of the following can be specified:
• blink Starts the CHECK LED blinking.
• reset Stops the CHECK LED blinking.
For example:
XSCF> showlocator blink
XSCF> showlocator reset


8.1.2.3. FANU Replacement

XSCF> replacefru
----------------------------------------------------------------------
Maintenance/Replacement Menu
Please select a type of FRU to be replaced.
1. FAN (Fan Unit)

2. PSU (Power Supply Unit)
----------------------------------------------------------------------
Select [1,2|c:cancel] :1
----------------------------------------------------------------------
Please select a FAN to be replaced.
No. FRU Status

--- --------------- ------------------
1. FAN_A#0 Normal
2. FAN_A#1 Normal
3. FAN_A#2 Normal
4. FAN_A#3 Normal
----------------------------------------------------------------------
Select [1-4|b:back] :1
You are about to replace FAN_A#0.

Do you want to continue?[r:replace|c:cancel] :r
Please confirm the Check LED is blinking.

If this is the case, please replace FAN_A#0.
After replacement has been completed, please select[f:finish] :f
Diagnostic tests for FAN_A#0 have started.

[This operation may take up to 3 minute(s)]
(progress scale reported in seconds)
0..... 30..done
----------------------------------------------------------------------
Status of the replaced FRU.
FRU Status
------------- --------
FAN_A#0 Normal
----------------------------------------------------------------------
The replacement of FAN_A#0 has completed normally.[f:finish] :f
----------------------------------------------------------------------
1. FAN (Fan Unit)

----------------------------------------------------------------------
Select [1,2|c:cancel] :c


8.1.2.4. PSU Replacement

Note that on the Form Factor Servers, even though the power supplies are hot-FRU replaceable,
one power supply must be active in the server at all times.
XSCF> replacefru
----------------------------------------------------------------------
1. FAN (Fan Unit)

----------------------------------------------------------------------
----------------------------------------------------------------------
Please select a PSU to be replaced.
No. FRU Status

--- --------------- ------------------
1. PSU#0 Normal
2. PSU#1 Normal
3. PSU#2 Normal
4. PSU#3 Normal
----------------------------------------------------------------------
You are about to replace PSU#0.

Please confirm the Ready LED is not lit and that the Check LED is
blinking.
If this is the case, please replace PSU#0.
Diagnostic tests for PSU#0 have started.

0..... 30..done
----------------------------------------------------------------------
Status of the replaced FRU.
FRU Status
------------- --------
PSU#0 Normal
----------------------------------------------------------------------
The replacement of PSU#0 has completed normally.[f:finish] :f
----------------------------------------------------------------------
1. FAN (Fan Unit)

----------------------------------------------------------------------


8.1.3. F/W Upgrade

An upgrade on a FF Server takes about 10 minutes and on a DC Server, about 20 minutes.
Although an Upgrade can be performed while all Domains are up and running, each domain will need to
be shutdown and powered off and on again for the OBP and POST changes to take effect.
If you are upgrading the XSCFU F/W as part of a patch upgrade, perform the F/W Upgrade on the
XSCFU first and then patch the domains afterwards. This will avoid the need for the Domains to be
shutdown twice.
There are several key steps to performing an XSCF Firmware Upgrade as outined below. However, there
is a full example of a F/W Upgrade in the Appendix Section of this document.
• Download the required F/W
You need to locate the correct F/W file for the target server. At the time of writing, XCP 1081 is the
latest version and therefore the required files are as follows:
Sun SPARC Enterprise M4000/M5000 Server(s) Firmware
• FFXCP1081.tar.gz (40.45 MB)
Sun SPARC Enterprise M3000 Server(s) Firmware
• IKXCP1081.tar.gz (40.44 MB)
Sun SPARC Enterprise M8000/M9000 Server(s) Firmware
• DCXCP1081.tar.gz (40.54 MB)
These can either be download from the Sun Website below, or taken from the EIS DVD
• https://cds.sun.com/is-bin/INTERSHOP.enfinity/WFS/CDS-CDS_SMI-
Site/en_US/-/USD/ViewProductDetail-Start?ProductRef=OPL-M4-5-8-9000-
XCP-1080-SP-G-F@CDS-CDS_SMI
• Once the F/W tar.gz file has been copied to a Server or laptop, it can be loaded onto the XSCFU
from the XSCF command line (CLI) by the getflashimage command using FTP, HTTP or HTTPS or
through the Browser User Interface (BUI).
XSCF> getflashimage ftp://user:password@10.1.0.3//export/FFXCP1072.tar.gz

0MB received
1MB received
47MB received
Download successful: 48596 Kbytes in 51 secs (962.818 Kbytes/sec)
Checking file...
MD5: 0109630ded7d0fd838d4ba8f77568803
XSCF> getflashimage -l
Existing versions:
Version Size Date
FFXCP1072.tar.gz 49762997 Thu Mar 19 15:17:03 GMT 2009
Note that before it can be downloaded, the Server or laptop needs to be setup so that files can be
remotely copied using the desired protocol.



• Once the F/W tar.gz file has been downloaded to the XSCFU, the F/W update can be applied:
XSCF> flashupdate -c check -m xcp -s 1072

XCP update is possible with domains up
XSCF> flashupdate -c update -m xcp -s 1072

Checking the XCP image file, please wait a minute
XCP update is started (XCP version=1072:last version=1081)
OpenBoot PROM update is started (OpenBoot PROM version=02030000)
OpenBoot PROM update has been completed (OpenBoot PROM version=02030000)
XSCF update is started (XSCFU=0,bank=0,XCP version=1072:last version=1081)
XSCF download is started (XSCFU=0,bank=0,XCP version=1072:last
version=1081, Firmware Element ID=00:version=01070001:last
version=01080001)
XSCF is rebooting to update the reserve bank
• After the XSCFU has rebooted, check that the updates have been applied and that the Current and
Reserve XCP are at the new version.
Allow 10 minutes after the XSCFU has rebooted for the changes to take effect as the XSCFU
Upgrade continues after the reboot. Example output can be found in the Appendix Section of this
document:
login: xscf-admin-user
password:
XSCF#0 (Active )


8.2. Domain
8.2.1. Create a Domain
8.2.1.1. Simple (Uni-XSB) Domain
Creating a Domain requires us to run several commands in order to completely configure and
assign all resources required to form a running domain as follows:
• setupfru Configure the XSB (Uni-XSB or Quad-XSB)

• setdcl Remove all DCL's from the Domain
• addboard Asign all resources (XSB) to the Domain
• poweron Poweron all resources assigned to the Domain
• console Connect to and shutdown the running Domain
The M4000 only supports one domain when the PSB is in the Uni-XSB configuration. In this case,
all components of the single PSB (PSB0) are allocated to a single XSB (XSB 00-0), which in turn is
allocated to a single domain (DID 0).
• Check the configuration before you start
DID Domain Status
00 -
01 -

---- - -------- ----------- ---- ---- ---- ------- -------- ----
XSCF> showdcl -av

XSCF> showfru sb 0
sb 00 Uni no
• Configure the XSB into Uni-mode (if required)
XSCF> showfru sb 0
sb 00 Uni no
---- - -------- ----------- ---- ---- ---- ------- -------- ----



• After you have configured the XSB, you can setup the Domain DCL's
XSCF> showdcl -v -d 0
00 Powered Off FRU
01 -
02 -
03 -
04 -
05 -
06 -
07 -
08 -
09 -
10 -
11 -
12 -
13 -
14 -
15 -
• After you have setup the DCL's, you can assign the boards to the domain


---- - -------- ----------- ---- ---- ---- ------- -------- ----
00-0 00(00) Assigned n n n Unknown Normal n
DID Domain Status
00 Powered Off
01 -
• Once the boards have been assigned to the domains, you can change some basic OBP
settings before powering it on:
Host-ID :8525ded4
Secure Mode :on
Autoboot :on
CPU Mode :auto

Autoboot :on -> off
CPU Mode :auto -> -
Continue? [y|n] :y
configured.
CPU Mode :auto



• Once a domain has been created, it needs to be powered on so that POST can run and the
OBP can be loaded as follows:
XSCF> poweron -d 0
Continue? [y|n] :y
00 :Powering on
*Note*

Feb 24 16:47:32 GMT 2009 System Power On Operator -- Locked
• Now start a console to access the OBP as follows:
XSCF> console -d 0
POST Sequence 01 CPU Check
POST Sequence 02 Banner
LSB#05 (XSB#00-0): POST 2.7.0 (2008/09/25 13:56)
POST Sequence 03 Fatal Check
POST Sequence 04 CPU Register
POST Sequence 05 STICK
POST Sequence 06 MMU
POST Sequence 07 Memory Initialize
POST Sequence 08 Memory
POST Sequence 09 Raw UE In Cache
POST Sequence 0A Floating Point Unit
POST Sequence 0B SC
POST Sequence 0C Cacheable Instruction
POST Sequence 0D Softint
POST Sequence 0E CPU Cross Call
POST Sequence 0F CMU-CH
POST Sequence 10 PCI-CH
POST Sequence 11 Master Device
POST Sequence 12 DSCP
POST Sequence 13 SC Check Before STICK Diag
POST Sequence 14 STICK Stop
POST Sequence 15 STICK Start
POST Sequence 16 Error CPU Check
POST Sequence 17 System Configuration
POST Sequence 18 System Status Check
POST Sequence 19 System Status Check After Sync
POST Sequence 1A OpenBoot Start...
POST Sequence Complete.
Sun SPARC Enterprise M5000 Server, using Domain console

Copyright 2008 Sun Microsystems, Inc. All rights reserved.
Copyright 2008 Sun Microsystems, Inc. and Fujitsu Limited. All rights
reserved.
OpenBoot 4.24.10, 32768 MB memory installed, Serial #86367956.
Ethernet address 0:21:28:25:de:d4, Host ID: 8525ded4.
{0} ok


8.2.1.2. Divided (Quad-XSB) Domains

Similar to creating a Simple Domain, we are required to several commands in order to completely
configure and assign all resources required to form a running domain as follows:
• setupfru Configure the XSB (Uni-XSB or Quad-XSB)

• addboard Asign all resources (XSB) to the Domain
• poweron Poweron all resources assigned to the Domain
However, in this example, we will split the fully loaded M4000 PSB into Quad-XSB configuration
and then create 2 domains. In this case, PSB (PSB0) will be split into 4 XSB's (XSB 00-0, 00-1,
00-2 & 00-3). XSB 00-0 will be assigned to Domain 0 (DID 0) and XSB 00-1 will be assigned to
Domain 1 (DID 1).
DID Domain Status
00 -
01 -

---- - -------- ----------- ---- ---- ---- ------- -------- ----
XSCF> showdcl -av

XSCF> showfru sb 0
sb 00 Uni no
• Configure the XSB into Quad-mode:
Operation has completed. However, a configuration error was detected.
XSCF> showfru sb 0
sb 00 Quad no

---- - -------- ----------- ---- ---- ---- ------- -------- ----


Setup DID 0
XSCF> showdcl -d 0 -v
00 Powered Off FRU
01 -
02 -
03 -
04 -
05 -
06 -
07 -
08 -
09 -
10 -
11 -
12 -
13 -
14 -
15 -

XSCF> showboards -d 0 -v
---- - -------- ----------- ---- ---- ---- ------- -------- ----

---- - -------- ----------- ---- ---- ---- ------- -------- ----
Host-ID :8525ded4
Secure Mode :on
Autoboot :on
CPU Mode :auto

Autoboot :on -> off
CPU Mode :auto -> -
Continue? [y|n] :y
configured.
CPU Mode :auto
XSCF> poweron -d 0
Continue? [y|n] :y
00 :Powering on
*Note*
XSCF> console -d 0
{0} ok


Setup DID 1
XSCF> showdcl -d 1 -v
01 Powered Off FRU
01 -
02 -
03 -
04 -
05 -
06 -
07 -
08 -
09 -
10 -
11 -
12 -
13 -
14 -
15 -

XSCF> showboards -d 1 -v
---- - -------- ----------- ---- ---- ---- ------- -------- ----
00-1 * 01(00) Assigned n n n Unknown Normal n

---- - -------- ----------- ---- ---- ---- ------- -------- ----
Host-ID :8525ded4
Secure Mode :on
Autoboot :on
CPU Mode :auto

Secure Mode :off -> -
Autoboot :on -> off
CPU Mode :auto -> -
Continue? [y|n] :y
configured.
CPU Mode :auto
XSCF> poweron -d 1
Continue? [y|n] :y
01 :Powering on
*Note*
XSCF> console -d 1
{0} ok


8.2.1.3. Memory Mirror Domain

In the previous examples, we created both a Simple Domain (using Uni-XSB) and Divided
Domains (using Quad-XSB), but in both cases we did not use Memory Mirroring.
In mirrored mode, as we saw in an earlier section,, both the the memory capacity and the the
interleave factor are halved. Memory Mirroring is configured by the Administrator from the
XSCF when running the setupfru command on each XSB. You can not configure Memory
Mirroring on the M3000 or on High End M8000/M9000 Servers on Quad-XSB's.
In the following example, we create a Simple Domain using a simple Uni-XSB but with Memory
Morroring enabled. Please note that to get the full benefit of memory mirroing, all XSB's which
are assigned to the domain need to have Memory Mirroring enabled.
DID Domain Status
00 -
01 -

---- - -------- ----------- ---- ---- ---- ------- -------- ----
XSCF> showdcl -av

XSCF> showfru sb 0
sb 00 Uni no
• Configuring the XSB into Uni-mode with Memory Mirroring enabled (Note the “-m y”
XSCF> setupfru -m y -x 1 sb 0
• Continue to create and start the Domain as before (Note Memory Mirror Mode = yes).
XSCF> showfru sb 0
sb 00 Uni yes
XSCF> showdcl -d 0
DID LSB XSB Status
00 Powered Off
00 00-0


---- - -------- ----------- ---- ---- ---- ------- -------- ----
DID Domain Status
00 Powered Off
01 -
XSCF> poweron -d 0
XSCF> console -d 0


8.2.2. Remove a Domain

Removing a Domain requires us to run several commands in order to completely release all assigned
resources as follows:

• poweroff Poweroff all resources assigned to the Domain
• deleteboard Remove all assigned resources from the Domain
This example shows how to remove a domain from a fully loaded M4000 with 1 running Domain using a
Uni-XSB.
XSCF> showboards -va

---- - -------- ----------- ---- ---- ---- ------- -------- ----
DID Domain Status
00 Running
01 -
XSCF> console -d 0
c1718-3-92 console login: root

Password:
root@c1718-3-92 # init 0
{10} ok #.
exit from console.
DID Domain Status
00 Booting/OpenBoot PROM prompt
01 -
XSCF> poweroff -d 0
DomainIDs to power off:00
Continue? [y|n] :y
00 :Powering off
*Note*
This command only issues the instruction to power-off.

...
---- -------- ----------- ---- ---- ---- ------- --------
00-0 00(00) Assigned n n n Passed Normal
XSCF> deleteboard -c unassign 00-0

XSB#00-0 will be unassigned from domain immediately. Continue?[y|n] :y

---- - -------- ----------- ---- ---- ---- ------- -------- ----



XSCF> showdcl -a
DID LSB XSB Status
00 Powered Off
00 00-0
XSCF> setdcl -d 0 -r 0 1
XSCF> showdcl -av


---- - -------- ----------- ---- ---- ---- ------- -------- ----


9. Dynamic Configuration
Dynamic Reconfiguration (DR) enables hardware resources such as processors, memory, and I/O to be added
and deleted even while the Solaris Operating System is running. DR has three basic functions;which are,
addition, deletion and move.
For DR to complete successfully, please note the following conditions:
• Due to diagnostic requirements, the DR function works only on boards that have at least one CPU and
memory.
• For a DR attach to be successful, Solaris needs to be running on the Domain
• On domains running in SPARC64 VII Enhanced Mode (prtdiag will tell you), you cannot use DR to add a
system board that contains any SPARC64 VI processors. To add a SPARC64 VI processor you must
power off the domain, change it to SPARC64 VI Compatible Mode, then reboot the domain.
9.1. Annuity Supported DR Operations

This User Guide does not cover the removal of components using Hot Swap as the project for which this
document was written did not use DR. Therefore, to remove any system board in an OPL server, you must turn
off the power to all domains using it, then replace the board without using the DR commands.
The project did, however, allow the addition of components using the following DR operations on OPL Servers:
9.1.1. XSCF
You can execute DR operations from the service processor by using the XSCF addboard command to
accomplish the following tasks:
• Board addition – Dynamically adds a system board to a domain or a new board to the system as
a whole
The DR operation to add a system board cannot be executed when the system board has only been
mounted. The DR operation is enabled by registering the system board in the DCL by using the XSCF
shell or XSCF Web. You must confirm that the system board to be added is registered in the DCL before
performing the DR
As well as adding system boards dynamically, DR also lets you order such reconfiguration to take place
the next time the affected domains are turned on or turned off, or the domain is rebooted. Use the
addboard command with the -c reserve option to specify this action.
9.1.2. Solaris
The cfgadm command can be run directly from the domain to get board status and to dynamically
configure PCI adapters only. The cfgadm command does not support DR operations for system boards.
9.2. DR Commands
XSCF shell commands for DR operations are classified into two types: DR display and DR operation commands.
A summary of the commands which can be used by Annuity during DR operations is as follows.
9.2.1. Display Commands

• showdcl Display the DCL and domain status.
• showdomainstatus Display domain status.
• showboards Display system board information.
• showdevices Display information about the CPUs, memory,and I/O devices
• showfru Display PSB configuration information.
9.2.2. Operation Commands

• setdcl Update and edit the DCL.
• setupfru Set the division type and memory mirror mode for a PSB.
• addboard Add a system board to a domain.
9.3. Adding an XSB

Use the addboard(8) command to add a system board to a domain or reserve the addition of a system board to a
domain based on the DCL. Use the showdcl(8) command to check whether a system board is registered in the
DCL. To register a system board in the DCL, use the setdcl(8) command.


It is advisable to have a console session running to the Domain in a seperate window when perform a DR attach
operation so that any errors or issues, such as a domain hang, can be observed and reported.
In this example, we will assign an additional Uni-XSB System Board (XSB 01-0) to Domain 0 while Solaris is
running:
1. Login to XSCF.
2. Check the status of the domain.
Execute the showdcl(8) command to display domain information, and then check the operation
status of the domain. Based on the operation status of the domain, determine whether to perform the
DR operation or change the domain configuration.
XSCF> showdcl -d 0
DID LSB XSB Status
00 Running
00 00-0
01 01-0
3. Check the status of the system board to be added.
Execute the showboards(8) command to display system board information, and then check the
status of the system board to be added and confirm its registration in the DCL.
XSCF> showboards -a
----------------------------------------------------------------
01-0 SP Available y n n Passed Normal
4. Add the new system board.
Execute the addboard(8) command to add the system board to the move-destination domain.
XSCF> addboard -c configure -d 0 01-0
5. Check the status of the domain and added system board.
When the addboard(8) command ends normally, execute the showdcl(8) command to check the
operation status of the domain, and then execute the showboards(8) command to check the status
of the added system board. If the addboard(8) command completes abnormally or leaves the board
in an unwanted status, investigate and escalate to SSC as appropriate:
XSCF> showdcl -d 0
DID LSB XSB Status
00 Running
00 00-0
01 01-0
------------------------------------------------------------------


9.4. Adding a PCI Card from the Domain

In this example, we will add (configure) a QLFC FC-HBA into a running Domain on the domain console
root@c1718-3-92 # cfgadm -al

Ap_Id Type Receptacle Occupant Condition
SB0::pci0 io connected configured ok
c0 scsi-bus connected configured unknown
c0::dsk/c0t0d0 disk connected configured unknown
c0::dsk/c0t3d0 CD-ROM connected configured unknown
c2 fc connected unconfigured unknown
iou#0-pci#0 unknown empty unconfigured unknown
iou#0-pci#1 fibre/hp connected configured ok
iou#0-pci#2 unknown disconnected unconfigured unknown
root@c1718-3-92 # cfgadm -c configure iou#0-pci#2
root@c1718-3-92 # cfgadm -al

Ap_Id Type Receptacle Occupant Condition
c0 scsi-bus connected configured unknown
c0::dsk/c0t0d0 disk connected configured unknown
c0::dsk/c0t3d0 CD-ROM connected configured unknown
iou#0-pci#0 unknown empty unconfigured unknown


10. Housekeeping
10.1. Reporting
10.1.1. Configure SMTP
The mail reporting function used by the XSCF can send messages to the administrator and has the
following features:
• Notification by e-mail of any component faults in a server. Even if a system failure or a serious
error that disables reboot occurs, an e-mail message is sent.
• POP or SMTP authentication for sending of e-mail.
The setsmtp command can also be used without options. This will invoke an interactive mode that asks
you to provide:
• Name of the mail server to use

• Port number to use (default is 25)
• Authentication mode (none, POP, or smtp-auth)
• Valid e-mail address as the reply-to address
The following example configures a mail server with POP authentication:

XSCF> setsmtp -s mailserver=10.4.1.1 -s auth=pop -s
port=25 -s user=jsmith -s password=jsmith -s
replyaddress=jsmith@company.com
The following example displays the SMTP configuration:

XSCF> showsmtp
Mail Server: 10.4.1.1
Port: 25
Authentication Mechanism: pop
POP Server:
User Name: jsmith
Password: ******
Reply Address: jsmith@company.com
10.1.2. Configure Mail Reporting

The setemailreport command configures e-mail reporting data for remote maintenance. The command
can be executed interactively or using options. After the configuration data is configured, it is used by the
fault management daemon to send e-mail reports, as required.
The following example enables e-mail reporting interactively:
XSCF> setemailreport
Enable Email Reporting? [no]:yes
Email Recipient Address :joe@10.6.15.50
Do you want to send a test mail now (Yes/No): no
The following example adds an e-mail recipient:

XSCF> setemailreport -a kathy@10.6.15.50
The following example displays the current e-mail reporting configuration:

XSCF> showemailreport
EMail Reporting: enabled
Email Recipient Address: joe@10.6.15.50,
kathy@10.6.15.50


10.2. Archiving
The persistent storage space on the service processor is limited. A portion of this space is set aside for logs,
such as audit logs and error logs. Due to the limited space, some logs can grow to the point where old log
entries must be overwritten or deleted to make space for new logs.
By default, log archiving is disabled. To use log archiving, you must configure an archive host, and then enable
log archiving on the service processor.
When enabled, log archiving periodically uses the secure copy program (scp) to transfer new log data to the
archive host. Log archiving also usessecure shell (ssh) to monitor the disk space consumed by archives. It
deletes older archives when necessary, so that the space consumed by the archives never exceeds the user-
configurable archive space limits.
For security reasons, log archiving does not automatically delete audit log archives. You can, however, still
manually delete audit log archives that are no longer needed.
10.2.1. setarchiving
The setarchiving command configures the log archiving. Setarchiving has many options, of which the
following are key:
• The ' -t user@host:directory' option sets the archive target

• The '-r' option reads the password used for ssh login. The setarchiving command displays a
prompt and reads the new password without echoing it to the screen.
The following example sets the archiving target system and password.
XSCF> setarchiving -t jsmith@somehost.company.com:/home/jsmith/logs -r

Enter ssh password for jsmith@somehost.company.com:[]
The following example enables log archiving.
XSCF> setarchiving enable
The following example disables log archiving.
XSCF> setarchiving disable
10.2.2. showarchiving
The showarchiving command displays the log archiving configuration and status.
XSCF> showarchiving
*** Archiving Configuration ***
Archiving state ---------- Disabled
Archive host ------------- Not configured
Archive directory -------- Not configured
User name for ssh login -- Not configured
Archive host fingerprint - Server authentication disabled
*** Connection to Archive Host ***

Latest communication ----- None
Connection status -------- None
AUDIT LOGS OTHER LOGS

---------- ----------
Archive space limit Unlimited 2000 MB
Archive space used Not monitored Not monitored
Total archiving failures 0 0
Unresolved failures 0 0
Display Archiving Error Information

XSCF> showarchiving -e
No archiving errors have occurred.


10.3. Auditing
An MX000 server logs all service processor events that could be relevant to security, such as system startup and
shutdown, user login and logout, and privilege changes. These events are tracked and implemented through:
• Audit records
• Audit trails
• Audit events
• Audit classes
• Audit policy
10.3.1. Audit Records

Audit records are stored in audit files on a 4-megabyte file system on the service processor. You cannot
change the size reserved for the audit files, but you can transfer the files manually to a remote location.
Because local space is limited to 4 megabytes, the partitions fill up quickly. If you do not configure the
audit policy to automatically transfer files to remote storage, you will have to intervene frequently or begin
to drop records. If you are unable to maintain consistent audit trails, the utility of the audit system is
limited. Typically, you either set up sufficient remote space and automatic transfers or disable the audit
capability.
10.3.2. Audit Records

A collection of audit records that are linked is called an audit trail. An audit trail can reveal suspicious or
abnormal patterns of system behavior, in addition to identifying which user was responsible for a
particular event.
• Audit records
• Audit trails
10.3.3. Audit Events

Audit events can be any of the following types:
• Changes to the service processor configuration, such as the IP address

• Requests to perform an operation on an object protected by the access control policy
• Tests of password strength
• Modifications to the access control attributes associated with an object – for example, which
domain a board is associated with
• Changes made to user security attributes – for example, password or privileges
• Reading information from the audit records (including unsuccessful attempts)
• Modifications to the audit policy
• Actions taken caused by exceeding the audit trail size threshold
• Actions taken due to audit storage failure
• Modifications made by administrators to the audit trail
• Changes to the system time
The data recorded for each event includes:
• Date and time of the event

• Type of event
• Who caused the event
• Outcome of the event (success or failure)


10.3.4. Audit Classes

Audit classes are categories for grouping and sorting audit events. The MX000 servers provide a
predefined set of audit classes – for example, login events and service-related events. You cannot define
additional audit classes or change the events in a class.
10.3.5. Audit Policy

The audit policy determines how the auditing feature is implemented by configuring the following
variables:
• Whether auditing is enabled or disabled

• The types of events that are audited
• Which users have their events audited
• Which remote directories are used for storing audit records
• The threshold of local capacity at which a warning is issued
• What action to take when both audit partitions are full
The default audit policy that is configured on the service processor is:
• Auditing is enabled.
• Records are dropped and counted when the audit trail is full.
• All events are enabled for auditing.
• The global user audit policy is enabled.
• Per-user audit policy for all users is enabled.
• Audit warning thresholds are set at 80 percent and 100 percent full.
• Email warnings are disabled.


10.3.6. setaudit
The setaudit command manages the collection of security-related data regarding the use of system
resources. This data can be used to assign responsibility for actions that have taken place on the system.
This command requires auditadm privileges to execute.
The following example enables and disables events:
XSCF> setaudit -e ENTER,EXIT=disable -e LOGOUT=enable

Auditing for Enter and EXIT events has been disabled.
Auditing for LOGOUT is enabled.
The following example changes the audit recording for various classes:
XSCF> setaudit -c LOGIN,AUDIT=disable

Auditing for LOGIN and AUDIT classes has been disabled.
The following example sets the warning thresholds:
XSCF> setaudit -t 50,75

Warnings will be sent at 50% capacity and 75% capacity.
10.3.7. showaudit
The showaudit command displays the log archiving configuration and status. This command requires
auditadm, auditop, or escalation privileges to execute.
The following example displays the current audit status:
XSCF> showaudit
Auditing: enabled
The following example displays the status for all events:
XSCF> showaudit -e all

Events:
AEV_AUDIT_START enabled
AEV_AUDIT_STOP enabled
AEV_ENTER_MODE enabled
AEV_EXIT_MODE enabled
AEV_LOGIN_BUI enabled
AEV_LOGIN_CONSOLE enabled
AEV_LOGIN_SSH enabled
...


10.3.8. viewaudit
The viewaudit command displays audit records. When invoked without options, viewaudit displays all
current local audit records. When invoked with options, viewaudit displays only the selected records. By
default, records are displayed in text format, one token per line, with a comma as the field separator. This
command requires auditadm or auditop privileges to execute.
The following example displays audit records for February 22, 2009:
XSCF> viewaudit -D 20080222

file,1,2009-02-19 11:12:24.528 -04:00,20090219151224.0000000000.kong-sc
The following example displays user audit records for a user named kevin:
XSCF> viewaudit -u kevin

file,1,2009-02-19 11:12:24.528 -05:00,20090219151224.0000000000.kong-sc
header,37,1,login - telnet,kong-sc,2009-02-19 11:31:09.659 -05:00
subject,1,kevin,normal,ssh 45880 kong-sc
command,showuser
platform access,granted
return,0
The following example displays audit records between the dates of January 12, 2009 and January 14,
2009:
XSCF> viewaudit -A 20090112 -B +2d

file,1,2009-01-12 16:11:52.785 -05:00,20090112155230.0000000000.kong-sc
subject,1,samantha,normal,ssh 51409 kong-sc
header,56,1,command - showldap,kong-sc,2009-01-13 21:15:12.416 -05:00
subject,4,samantha,normal,telnet 51409 sun.com
command,showldap
platform access,granted
return,0


10.4. XSCF Backup & Restore

There are several ways that the XSCF configuration is protected, and if you are running XCP 1080 or later, there
are some additional commands (dumpconfig & restoreconfig) which allow you to copy and restore the XSCF
configuration if required.
10.4.1. Automatic Backup

After the XSCF is set up, the settings are automatically saved in XSCF internally and in the operator
panel (OPNL). Once you have configured the XSCF, it requires no day-to-day management.
If the Service Processor is replaced, it will be auomatically comnfigured from the backup in the OPNL. If
there Service Processor has some pre-existing configuration and is not able to automatically confiigure
the Service Processor, it will display details to the serial port and request confirmation.
If the OPNL is replaced, it will automatically take a backup of the Service Processor once it has been
installed and the Platform powered on.
10.4.2. Manual Backup

To save/restore the XSCF configuration information, execute the dumpconfig(8) and the restoreconfig(8)
command in the XSCF Shell. When the dumpconfig command is executed, all XSCF configuration
information is saved at the specified location and when the restoreconfig command is executed, it is
restored from the specified location.
The XSCF configuration information can be saved in the device and can be restored using one of the
following two methods.
• The configuration information can be saved and restored when a USB device has been
connected to the USB connector mounted on the XSCF Unit front panel of the M4000/M5000/
M8000/M9000 servers or rear panel of the M3000 server.
• The configuration data is transmitted through the network with an encryption protocol.
Note
• The USB device should only be formatted using the FAT32 file system.
• Power off all domains when you restore the configuration information. Moreover, after
completing the command, turn the input power supply of the server off, then on.
10.4.2.1. dumpconfig
• Backup to a USB stick
XSCF> dumpconfig file:///media/usb_msd/backup-file.txt
• Backup to an FTP Server or laptop
XSCF> dumpconfig ftp://server/backup/backup-sca-ff2-16.txt
10.4.2.2. restoreconfig
• Shutdown & Power off all domains.
• Perform the restoreconfig(8) command specifying the target directory. e.g
XSCF> restoreconfig file:///media/usb_msd/backup-file.txt

XSCF> restoreconfig ftp://server/backup/backup-sca-ff2-16.txt
• Turn the input power supply of the server off, then on.
• Poweron and Boot all domains


11. Monitoring
The following section assumes you are running SunMC 3.6.1.
11.1. Sun Management Center (SunMC)

The add-on software for Sun SPARC Enterprise Mx000 servers adapts the core Sun Management Center
software to function with Sun SPARC Enterprise Mx000 servers.
Sun Management Center 3.6.1 Version 2 add-on software introduces support for the following systems:
• Sun SPARC Enterprise M4000/M5000 servers (midrange servers)

• Sun SPARC Enterprise M8000/M9000 servers (high-end servers)
This add-on software for Sun SPARC Enterprise Mx000 servers provides the following features:
• Hardware monitoring
• Power management
• Domain management
• Dynamic reconfiguration
• FRU replacement (system board)
A combination of the following modules need to be download and installed on the SunMC Server, the SunMC
Console and the Mx000 Domains.
The XSCF requires no Add-on installation as the SunMC Agent is provided as part of the XCP. The XCP
contains the necessary firmware required to properly operate the XSCFU and the server components at a very
low level.
Agent Modules Description
Plat Admin Module Platform administration module. Provides monitoring and active
SPARC Enterprise management capability for the Sun SPARC Enterprise Mx000 server.
Mx000 Resides on the Service Processor.
Domain Config Reader

Domain administration module. Provides monitoring for domains on Sun
SPARC Enterprise
SPARC Enterprise Mx000 servers.
Mx000
Domain Dynamic Reconfiguration (DR) module. Enables an

Domain DR SPARC
administrator to perform dynamic reconfiguration of boards on one
Enterprise Mx000
domain at a time.


11.1.1. Configuring the SunMC Server

11.1.1.1. Copy the SunMC Add-On S/W to the SunMC Server
Download the SunMC 3.6.1 Add-On S/W (Sun_MC_Addon-3_6_1_V3-GA.zip) to the SunMC
Server, either from the Sun Website (below) or copy it from the EIS DVD:
Site/en_US/-/USD/ViewProductDetail-Start?ProductRef=SunMC-3.6.1V3-G-
F@CDS-CDS_SMI
11.1.1.2. Unzip and Extract the SunMC Add-On S/W

There will be 2 subdirectories (disk1 & disk2). In the following examples, the S/W was extracted to
/export/downloads/sunmc3.6.1
11.1.1.3. Install the SunMC Add-On S/W

Detailed output from the es-inst command can be found in the Appendix Section of this document,
a summary, with responses, can be seen below:
mx000-uksr: cd /export/downloads/sunmc3.6.1
mx000-uksr: cd disk2/image
mx000-uksr: /opt/SUNWsymon/sbin/es-inst -S .
----------------------------------------------------------------------------
This script installs Sun (TM) Management Center 3.6.1
----------------------------------------------------------------------------
Installation files source directory:

/export/downloads/sunmc3.6.1/disk2/image/SunOS/sparc
Installation destination directory: /opt
Reading Add-On products from image. Please wait...

This can take approximately 2 minutes to complete.
Found: PltAdmin SPARC Enterprise Mx000

Found: Dom DR SPARC Enterprise Mx000
Found: DomMonit SPARC Enterprise Mx000
Checking for applicable products. Please wait...

This can take approximately 1 minute to complete.
Answer yes to all questions.


11.1.2. Configuring the XSCFU

Note that before the following commands can succeed, the platform should have been setup on the
network. Specifically, the hostname and DNS domain need to have been configured using
“setupplatform -p network”
XSCF> showsunmc
Agent Status: Disabled, stopped
Setup Status: Not set up
SunMC Server: NOT_SET
Agent Port: 1161
Host Trap Port: 162
XSCF> setsunmc enable

setsunmc: Agent enabled.
setsunmc: Note that this command will return before SunMC is
completely enabled.
XSCF> setsunmc -s 192.168.15.9

setsunmc: Updating Sun MC config file
XSCF> showsunmc
Agent Status: Enabled, running
Setup Status: Set up
SunMC Server: 192.168.15.9
Agent Port: 1161
Host Trap Port: 162
• It is not necessary to configure SNMP for SunMC to work, an example of an XSCFU which has not
has SNMP configured can be seen below:
XSCF> showsnmp
Agent Status: Disabled

Agent Port: 161
System Location: Unknown
System Contact: Unknown
System Description: Unknown
Trap Hosts: None
SNMP V1/V2c: None
Enabled MIB Modules: None


11.1.3. Configuring the Mx000 Domains

The SunMC Add-On S/W needs to be installed on all Mx000 Domains. This needs to be peformed after
the basic SunMC S/W has been installed and configured.
11.1.3.1. Copy the SunMC Add-On S/W to the domain

In the previous section, we copied the SunMC Add-On S/W (Sun_MC_Addon-3_6_1_V3-
GA.zip) from the Sun Website or the EIS DVD. Now we need to copy the same ZIP file to the
Mx000 domains prior to installation
11.1.3.2. Unzip and Extract the SunMC Add-On S/W

There will be 2 subdirectories (disk1 & disk2). In the following examples, the S/W was extracted to
/export/downloads/sunmc3.6.1
11.1.3.3. Install the SunMC Add-On S/W

Detailed output from the es-inst command can be found in the Appendix Section of this document,
a summary, with responses, can be seen below:
mx000-uksr: cd /export/downloads/sunmc3.6.1
----------------------------------------------------------------------------
----------------------------------------------------------------------------




The following Add-On Products are applicable for installation.

DomMonit SPARC Enterprise Mx000
Dom DR SPARC Enterprise Mx000
Select the products you want to install:
DomMonit SPARC Enterprise Mx000 (y|n|q) y
Dom DR SPARC Enterprise Mx000 (y|n|q) y
The following Add-On Products will be installed:

Do you want to proceed (y|n|q) y
Disk Space Checking....
You must perform setup before using Sun Management Center 3.6.1.
Do you want to run setup now (y|n|q) y
----------------------------------------------------------------------------
This script will help you to setup Sun (TM) Management Center 3.6.1.
----------------------------------------------------------------------------
Stopping platform component
Stopping agent component
Documentation of Sun Management Center is not available on

Sun Management Center CD image, For all Sun Management Center
documentation please visit :



http://docs.sun.com/app/docs/coll/810.8 website
Setting up Addon[s]...
---------------------------------------------------------------------
Starting Sun Management Center Mx000 Systems Domain Setup
---------------------------------------------------------------------
Additional Setup information is available in the

Sun Management Center 3.6.1 Version 2 Add-On Software Supplement for SPARC
Enterprise Servers.
The Domain Config Reader for Systems will collect

configuration and status information for your tape drives at regular
polling intervals. This can be disruptive to tape drive controllers
that do not allow concurrent access.
Would you like to disable this feature? [y|n|q] y

Added module em to /var/opt/SUNWsymon/cfg/base-modules-d.dat.
Added module Config-Reader-Opl to /var/opt/SUNWsymon/cfg/base-modules-d.dat.
Created deviceinfo.conf file
Creating Agent Update configuration file.
------------------------------------------------------------
Do you want to start Sun Management Center agent now (y|n|q) y

Starting Sun Management Center Agent...
Agent started successfully.
Setup of Sun Management Center is complete.

Setup logfile is : /var/opt/SUNWsymon/install/setup_mx000-
uksr.090210134856.20260
Install logfile is : /var/opt/SUNWsymon/install/install_mx000-

uksr.090210134353.16424
End of Installation
Exiting Sun Management Center installation.


11.1.4. Configure the Mx000 XSCFU and Domains on the SunMC Server
Make sure that the Domains and the XSCFU are in the SunMC Server /etc/hosts file and that they are
pingable. We also need to ensure that ssh is working.
For each Domain, add a new entry with the following details
Hostname: Mx000-hostname
IP Address: <IP Address>
Port: 1161


12. Troubleshooting
This section contains information which should assist you to troubleshoot problems from the Domains and the Service
Processor. Note that there are additional examples in the Appendix Section of this document for reference.
12.1. Platform
12.1.1. snapshot
The snapshot(8) command provides a data-collection mechanism that enables rapid, reliable, and
flexible retrieval of diagnostic information on the Service Processor. Snapshot(8) collects the following
data: Configuration, Environmentals, Logs, Errors, and FRUID information. It transfers data to the
specified destination.
Basically, a snapshot is like an explorer, but run on the XSCFU. We provide it a target Server, Laptop or
USB Stick and it will generate based on the host name and IP address assigned to the Service
Processor and the UTC time (in hours, minutes, and seconds) and date on the Service Processor at
the time snapshot is invoked
(e.g jupiter_10.1.1.1_2006-07-08T22-33-44.zip)
The snapshot command can be operated in one of the following modes:
• SSH target mode

• USB device mode
12.1.1.1. SSH Target Mode
The first mode is SSH target mode. The data collector runs in this mode when it is invoked with the
-t or -T option. In this mode, the data collector opens an SSH connection from the service
processor to th specified target (after appropriate authentication) and sends the zipped data
archive through the SSH connection to the target host. The transmission encryption in this mode is
provided by SSH.
12.1.1.2. USB Device Mode
The second mode is USB device mode. The data collector runs in this mode when it is invoked
with the -d flag. In this mode, the data collector’s output (which is the zip archive) is saved in a file
on a USB device. No transmission encryption occurs in this mode because the data stays local to
the service processor.


12.1.1.3. Examples
• The following example downloads the snapshot using a public key through SSH and specifies
the remote host:
XSCF> snapshot -t barry@10.6.15.50:/export/home/barry/logs -k download

Downloading Public Key from ‘10.6.15.50’...
Key fingerprint in md5: c9:e0:bc+b2:1a:80:29:24:13:d9:f1:13:f5:5c:2c:0f
Accept this public key (yes/no)? Y
Enter ssh password for user ‘barry’ on host ‘10.6.15.50’
Setting up ssh connection to remote host...
Collecting data into barry@10.6.15.50:/export/home/barry/logs/kongsp0_
10.6.15.41_2008-02-25T06-19-06.zip
Data collection complete.
• The following example downloads the snapshot using a public key through SSH and specifies
the remote host:
XSCF> snapshot -T -k download
Downloading Public Key from ‘10.6.15.50’...
Key fingerprint in md5: c9:e0:bc+b2:1a:80:29:24:13:d9:f1:13:f5:5c:2c:0f
Accept this public key (yes/no)? Y
Enter ssh password for user ‘matt’ on host ‘10.6.15.50’
Collecting data into matt@godzilla:/export/home/matt/logs/kongsp0_
10.6.15.41_2008-02-25T06-21-17.zip
• The following example logs files only (no command output), using no public key:
XSCF> snapshot -t gary@10.6.15.50:/export/home/gary/logs -k none -l
Enter ssh password for user ’gary’ on host ’10.6.15.50’
Log only mode. No commands will be collected.
Collecting data into gary@10.6.15.50:/export/home/gary/logs/kongsp0_
10.6.15.41_2008-02-25T06-24-57.zip


12.1.1.4. snapshot & Sun Explorer

snapshot can be run remotely from a Server which has ssh access to the XSCFU through Sun
Explorer.
• Edit the /opt/SUNWexplo/tools/xscfextended
Change:
COMMAND="snapshot -T -D /tmp"
To:
COMMAND="snapshot -y -T -D /export/Mx000/snaphot"
• Edit the /etc/opt/SUNWexplo/xscfinput.txt
# Input file for extended data collection

# Format is HOST USER [PASSWORD]
# Include password for telnet connection (not
recommended)
# Omit password for SSH connection using public key
authentication
# Explorer requires the mode of xscfinput.txt is 0400
or 0600
After the previous lines, add:
service_processor_ip user_acct user_password
For example:
10.6.15.21 installer installer
• Use the following command to run the Sun Explorer utility:

# /opt/SUNWexplo/bin/explorer -w !default,xscfextended
Note – The xscfextended option to Sun Explorer assumes that archiving is set up and works.


12.1.2. Data Collection

12.1.2.1. showstatus
The showstatus(8) command displays information about degraded units that are among the FRUs
composing the system.
XSCF> showstatus
No failures found in System Initialization.
12.1.2.2. showlogs
The showlogs command displays the contents of the specified log file. The output is displayed
using a timestamp, starting with the oldest date. The showlogs command can be run with a
combination of options (-d DID, -M, -v, -V)
• error Displays the error log. (sometimes includes scan log)

• power Displays the power log.
• event Displays the event log.
• env Displays the temperature and humidity record.
• monitor Displays the monitoring message log.
- Also see showmonitorlog(8)
• console Displays the console message log.
• panic Displays the panic message log.
• ipl Displays the IPL message log.
12.1.2.3. showlogs error
XSCF> showlogs error

Date: Feb 09 15:20:46 GMT 2009 Code: 40000200-cfffd701-0300003200000000
Status: Information Occurred: Feb 09 15:20:45.876 GMT 2009
FRU: /HDD#0,/IOU#0
Msg: Boot process failed
Date: Feb 09 15:30:46 GMT 2009 Code: 40000000-ba010000-0300003200000000
FRU: /IOU#0
12.1.2.4. showlogs event
XSCF> showlogs event

Date Message
Feb 20 10:34:38 GMT 2009 System powered on
Feb 20 10:35:33 GMT 2009 DomainID 0 state change (initialize phase started,
detail#0)
Feb 20 10:35:47 GMT 2009 DomainID 0: Reset released
<... output deleted> ...
Feb 20 10:38:39 GMT 2009 DomainID 0 state change (OpenBoot PROM complete, detail#8)
Feb 23 23:52:09 GMT 2009 DomainID 0 state change (boot process started, detail#9)
Feb 23 23:52:36 GMT 2009 DomainID 0 state change (system running, detail#10)
Feb 23 23:53:35 GMT 2009 PCI configuration is changed by PHP operation (DomainID 0)
12.1.2.5. showlogs power

Feb 20 10:21:20 GMT 2009 System Power Off Operator -- Locked
Feb 20 10:34:38 GMT 2009 System Power On Operator -- Locked


12.1.2.6. showlogs env
XSCF> showlogs env

Date Temperature Power
Feb 23 23:56:48 GMT 2009 22.00(C) System Power On
12.1.2.7. showlogs monitor

This is similar to the Platform Message Log on an E25K. All Platform messages of note are
logged here and therefore this should be one of the first places to lookm when troubleshooting
(along with the domain console log).
XSCF> showlogs monitor

Feb 20 10:34:38 c1718-4-92-mgt-xscf0 monitor_msg: SCF:System powered on
Feb 20 10:35:33 c1718-4-92-mgt-xscf0 monitor_msg: SCF:DomainID 0 state change
(initialize phase started, detail#0)
Feb 20 10:35:47 c1718-4-92-mgt-xscf0 monitor_msg: SCF:DomainID 0: Reset released
Feb 20 10:38:39 c1718-4-92-mgt-xscf0 monitor_msg: SCF:DomainID 0 state change (OpenBoot
PROM complete, detail#8)
Feb 23 23:52:10 c1718-4-92-mgt-xscf0 monitor_msg: SCF:DomainID 0 state change (boot
process started, detail#9)
Feb 23 23:52:36 c1718-4-92-mgt-xscf0 monitor_msg: SCF:DomainID 0 state change (system
running, detail#10)
12.1.2.8. showmonitorlog
showmonitorlog is like 'showlogs monitor', but it effictively performs a 'tail -f' on the monitor log
XSCF> showmonitorlog
12.1.2.9. showlogs console
XSCF> showlogs -d 0 console -M

DomainID: 00
Feb 23 23:51:46 GMT 2009 {0} ok boot
Feb 23 23:52:08 GMT 2009 Boot device:
/pci@0,600000/pci@0/pci@8/pci@0/scsi@1/disk@0,0:a File and args:
SunOS Release 5.10 Version Generic_137137-09 64-bit
Feb 23 23:52:16 GMT 2009 Copyright 1983-2008 Sun Microsystems, Inc. All rights reserved.
Feb 23 23:52:16 GMT 2009 Use is subject to license terms.
Feb 23 23:52:39 GMT 2009 Hostname: c1718-3-92
Feb 23 23:53:07 GMT 2009 NIS domain name is ssclabs.net
Feb 23 23:53:08 GMT 2009 Reading ZFS config: done.
Feb 23 23:53:31 GMT 2009
Feb 23 23:55:13 GMT 2009 c1718-3-92 console login:
12.1.2.10. showlogs ipl
XSCF> showlogs -d 0 ipl
12.1.2.11. showlogs panic
XSCF> showlogs -d 0 panic


12.1.3. Fault Manager

12.1.3.1. fmdump
The Fault Manager (FM) maintains two sets of logs (error and fault), and allows us to view the list
of system components that have been diagnosed as faulty by the XSCFU.
• Error Log
Records error telemetry, the symptoms of problems detected by the system
• Fault Log
Records fault diagnosis information; the problems believed to explain these

symptoms. By default, fmdump displays the contents of the fault log, which records
the result of each diagnosis made by the fault manager or one of its component
modules.
Each problem recorded in the fault log is identified by:
• The time of its diagnosis

• A Universal Unique Identifier (UUID) that can be used to uniquely identify this particular
problem across any set of systems
• A message identifier (MSG-ID) that can be used to access a corresponding knowledge
article located at the specified website.
For example, here we have the fdump output from a M5000 with a failed fan in a PSU (a more
detailed example of this failure can be found a few pages further on in this User Guide):
XSCF> fmdump
TIME UUID MSG-ID
Feb 09 15:20:46.7179 88f2ccb8-cf7d-4dae-ba65-4ffc1f655124 SCF-8001-KC
Feb 09 15:30:46.6667 554cc530-a17a-4d76-ac56-2685c1c97daf SCF-8002-FA
Feb 24 02:14:03.0020 bf74be0f-fea6-4299-8c04-7688870ea64b SCF-8005-YH
Feb 24 02:14:04.4976 7f2f29b1-811f-403c-a058-6088c5c0969b SCF-8005-YH
XSCF> fmdump -v -u bf74be0f-fea6-4299-8c04-7688870ea64b

TIME UUID MSG-ID
100% fault.chassis.device.fan.tooslow
Problem in: hc:///chassis=0/psu=0/fan=0

Affects: hc:///chassis=0/psu=0/fan=0
FRU: hc://:product-id=SPARC Enterprise M5000:chassis-
id=BCF0904031:server-id=c1718-4-92-mgt-xscf0:serial=1357FYG-0843AD006:part=CF00300-2011
0202 \300-2011-02-0:revision=02/component=/PSU#0
Location: /PSU#0
XSCF> fmdump -v -m
MSG-ID: SCF-8001-KC, TYPE: upset, VER: 1, SEVERITY: Critical
EVENT-TIME: Mon Feb 9 15:20:46 GMT 2009
PLATFORM: SPARC Enterprise M5000, CSN: BCF0904031, HOSTNAME: c1718-4-92-mgt-xscf0
SOURCE: sde, REV: 1.16
EVENT-ID: 88f2ccb8-cf7d-4dae-ba65-4ffc1f655124
DESC: Failure to boot Solaris on a domain via an internal disk drive on the SAS
controller.
Refer to http://www.sun.com/msg/SCF-8001-KC for more information.
AUTO-RESPONSE: Nothing is deconfigured.
IMPACT: The domain drops to the OpenBoot PROM 'ok' prompt.
REC-ACTION: The platform administrator should investigate the cause of the boot problem
and schedule a service action, if necessary.
Please consult the detail section of the knowledge article for additional information.


12.1.3.2. fmadm
The fmadm command can display the configuration of the Fault Manager itself, including the
module name, version, and description of each component module. Fault Manager modules
provide services such as automated diagnosis, self healing, and messaging for hardware and
software present on the system.
Certain functions of fmadm (e.g repair & unload) are only available if you request a password to go
into service mode (see service(8)), however, you can run 'fmadm config' (below)
XSCF> fmadm config

MODULE VERSION STATUS DESCRIPTION
case-close 1.0 active Case-Close Agent
emailprv 1.0 active opl emailprv plugin
event-transport 2.0 active Event Transport Module
faultevent-post 1.0 active Gate Reaction Agent for errhandd
flush 1.10 active Resource Cache Flush Agent
fmd-self-diagnosis 1.0 active Fault Manager Self-Diagnosis
iox_agent 1.0 active IO Box Recovery Agent
reagent 1.16 active Reissue Agent
sde 1.16 active Simple Diagnosis Engine
snmp-trapgen 1.0 active SNMP Trap Generation Agent
sysevent-transport 1.0 active SysEvent Transport Agent
syslog-msgs 1.0 active Syslog Messaging Agent
12.1.3.3. fmstat
The fmstat utility can be used to report statistics associated with the Fault Manager (fault manager
daemon), and its associated set of modules. You can use fmstat to view statistics for diagnosis
engines and agents that are currently participating in fault management
XSCF> fmstat
module ev_recv ev_acpt wait svc_t %w %b open solve memsz bufsz
case-close 14 0 0.0 3.1 0 0 0 0 686b 0
emailprv 14 0 0.0 7.5 0 0 0 0 0 0
event-transport 0 0 0.0 1.8 0 0 0 0 5.8K 0
faultevent-post 14 0 0.0 113.5 0 0 0 0 0 0
flush 28 0 0.0 0.2 0 0 0 0 0 0
fmd-self-diagnosis 0 0 0.0 1.6 0 0 0 0 0 0
iox_agent 0 0 0.0 1.6 0 0 0 0 0 0
reagent 0 0 0.0 1.6 0 0 0 0 0 0
sde 14 14 0.0 81.8 0 0 0 14 159K 0
snmp-trapgen 14 0 0.0 4.6 0 0 0 0 0 0
sysevent-transport 0 0 0.0 68.4 0 0 0 0 0 0
syslog-msgs 14 0 0.0 11.4 0 0 0 0 97b 0


12.1.4. Diagnostics
12.1.4.1. testsb
The testsb command performs an initial diagnosis of the specified PSB, you must have
platadm or fieldeng privileges to run testsb
The configuration of the PSB and operation of each device mounted on the PSB are checked.
After the diagnostics, the result is displayed. The PSB must not be configured in the domain, or the
domain in which the PSB configured must be powered off. The result also can be seen in "Test"
and "Fault" displayed by the showboards command.
In the following example, we have run testsb against PSB#0 on an M4000. Note that Domain 0 to
which XSB#0 is assigned is powered off:
DID Domain Status
00 Powered Off
01 -

---- - -------- ----------- ---- ---- ---- ------- -------- ----
XSCF> testsb 0
Initial diagnosis is about to start, Continue?[y|n] :y
SB#00 power on sequence started.
0..... 30.....end
Initial diagnosis started. [1800sec]
0..... 30..... 60...end
Initial diagnosis has completed.
SB power off sequence started. [1200sec]
0.end
SB powered off.
XSB Test Fault
---- ------- --------
00-0 Passed Normal
00-1 Passed Normal

---- - -------- ----------- ---- ---- ---- ------- -------- ----
00-0 * 00(01) Assigned n n n Passed Normal n


12.1.5. Service & Escalation Mode

Most of the tasks required to monitor and configure the MX000 servers are performed in normal user
mode on the service processor. There are, however, two other modes of operation that provide advanced
functionality to the user. These additional modes are:
• Service mode
• Escalation mode
Four steps are required to enter either service or escalation mode:
1. Obtain a valid password from an authorized service provider to execute the appropriate
enable command. You will be required to supply the serial number of the server as well as
the XSCF version number.
• Service Mode passwords can be generated form the following Sun Internal
Website:
• http://oplpass.sfbay
• Escalation Mode passwords can only be generated by PTS via an SSC Logged
call:
2. Enable the requested mode of operation using the appropriate enable command,
enableservice or enableescalation.
3. Provide the password obtained in Step 1 when prompted.
4. Use the mode password that is generated by the enable command to enter into the
appropriate mode, service or escalation.
Note – The service mode is intended to be used only by authorized personnel. The escalation mode is
only used during an escalation and with the guidance of engineering personnel.


12.1.5.1. Service Mode

The following example enables escalation mode:
1. Get the password generator key from http://oplpass.sfbay
2. Get the required information (XSCF Version and Serial No.) from the Server
XSCF> version -c xscf

XSCF#0 (Active )
01.08.0003(Reserve) 01.08.0003(Current)
XSCF> showhardconf -M
+ Serial:BCF071100U; Operator_Panel_Switch:Service;
3. Click on “Geneate Password”
It will ask for your SWAN login and Password, then display a Password Generation Grid
like the one below:
MODE : service
VERSION : 01.08.0003
SERIAL : BCF071100U
UTC DATE : 01/07/2009 09:50
DIET OBOE MOS SAN SHUN FROG

SKID RIFT VOID OBEY BOIL ELK
HALL CHAD WERE DAN RICE DAME
HEBE LORD BED COLD ADD RAKE
DEFY WANT ARTS VEDA LINK A
!!! This password is valid for 48 hours !!!
4. Run the enableservice command to generate the password key

XSCF> enableservice
Service Password:
DIET OBOE MOS SAN SHUN FROG
SKID RIFT VOID OBEY BOIL ELK
HALL CHAD WERE DAN RICE DAME
HEBE LORD BED COLD ADD RAKE
DEFY WANT ARTS VEDA LINK A
Mode password is: WET RUDE LILY
XSCF> service
Password: WET RUDE LILY
service> showmodes
Current mode: service
Enabled for: service
Expires: Fri Jan 9 10:06:11 2009
service> exit
In addition to the additional options and arguments to exisiting commands, Service Mode offers
the following additional commands on XCP 1081:
• clearfault
• confxscf
• enablecodboard
• erasenvram
• setdumphost
• showdumphost
• shownvram


12.1.5.2. Escalation Mode

The following example enables escalation mode:
XSCF> enableescalation
Escalation Password:
**** **** **** **** *** ***
**** **** **** *** **** ***
**** *** **** **** **** ***
*** **** *** **** *** ****
*** **** **** **** *** *
Mode password is: LOVE CAT NAPS
XSCF> escalation
Password: LOVE CAT NAPS
escalation> showmodes
Current mode: escalation
Enabled for: escalation
Expires: Fri Jan 9 10:06:11 2009
escalation> exit


12.2. Domain
12.2.1. DSCP
You can check that the Service Processor is able to communicate with the OPL Domains through the
DSCP Network as follows:
# ifconfig -a
<... stuff deleted ...>
sppp0: flags=10010008d1<UP,POINTOPOINT,RUNNING,NOARP,MULTICAST,IPv4,FIXEDMTU> mtu
1500 index 4
inet 192.168.224.2 --> 192.168.224.1 netmask ffffff00
ether 0
If there is no sppp0 interface configured, check to see if the required Services, specifically dscp, are
running as follows:
# svcs dscp
STATE STIME FMRI
online 14:40:37 svc:/platform/sun4u/dscp:default
# svcs dcs
STATE STIME FMRI
online 14:40:37 svc:/platform/sun4u/dcs:default
# svcs sckmd
STATE STIME FMRI
online 14:40:29 svc:/platform/sun4u/sckmd:default
If they are not running, check to see if there are any problems
# svcs -l dscp
fmri svc:/platform/sun4u/dscp:default
name DSCP Service
enabled true
state online
next_state none
state_time 12 March 2009 14:40:37 GMT
logfile /var/svc/log/platform-sun4u-dscp:default.log
restarter svc:/system/svc/restarter:default
contract_id 44
dependency require_any/error svc:/milestone/network (online)
dependency require_all/none svc:/system/filesystem/local (online)
dependency optional_all/none svc:/platform/sun4u/sckmd:default (online)
# tail /var/svc/log/platform-sun4u-dscp:default.log
[ Feb 27 16:45:07 Executing start method ("/lib/svc/method/svc-dscp start") ]
[ Feb 27 16:45:12 Method "start" exited with status 0 ]
[ Mar 11 15:28:17 Executing start method ("/lib/svc/method/svc-dscp start") ]
[ Mar 11 15:28:23 Method "start" exited with status 0 ]
[ Mar 12 14:40:32 Executing start method ("/lib/svc/method/svc-dscp start") ]
[ Mar 12 14:40:37 Method "start" exited with status 0 ]


12.2.2. SunVTS
SunVTS, Sun Validation Test Suite, tests and validates Sun hardware by verifying the connectivity
and functionality of hardware devices, controllers and peripherals
It is better to do this over a remote network login rather than on the console as SunVTS output will fill
up the SMS console logs
• ssh to the Server and su to root
• Enable rpcbind and start the SunVTS Program.
# svcadm -v enable -t /network/rpc/bind

# TERM=vt100 export TERM
# /usr/openwin/bin/resize
# /opt/SUNWvts/bin/sunvts -t
• To Configure and run SunVTS, do the following
a) Tab to "Test Groups"

b) Down arrow to “VRTS” or "SVM" (whichever appears first) and remove the * by
pressing the SPACE bar. Remove all other *'s except Processor and Memory.
c) Tab to top menu and use ARROW keys to highlight "set_options" then press
RETURN
d) Select "Test_Execution"
e) Arrow down to "Num. of Instances:[1-128]" and change to 16.
f) Arrow down to "Apply" and select and hit return.
g) Select "Start" from the top menu.
h) Select "Start"
i) Check the Error Column in the “Status View” window. There should be 0 errors
• To stop SunVTS, do the following
a) Select "Stop" then "Esc"

b) Select "Quit" from top menu.
c) Select "Quit UI & Kernel"
• Disable rpcbind
# svcadm -v disable /network/rpc/bind


13. Appendix
13.1. PSU Failure Example
The following example is of a PSU fan failure on an M5000. It is used to illustrate the use of the XSCFU
commands to identify and view information about a fault on the system:
XSCF> replacefru
----------------------------------------------------------------------
1. FAN (Fan Unit)

----------------------------------------------------------------------
----------------------------------------------------------------------
Please select a PSU to be replaced.
No. FRU Status

--- --------------- ------------------
1. PSU#0 Normal
2. PSU#1 Normal
3. PSU#2 Normal
4. PSU#3 Normal
----------------------------------------------------------------------
You are about to replace PSU#0.

Please confirm the Ready LED is not lit and that the Check LED is
blinking.
If this is the case, please replace PSU#0.
Diagnostic tests for PSU#0 have started.

0....done
[Warning:021]
Diagnostics have detected a hardware error.
This FRU cannot be used.
Please check the FRU.
Replacement operation is terminated.[c:cancel] :c
----------------------------------------------------------------------
1. FAN (Fan Unit)

----------------------------------------------------------------------
XSCF> showstatus
* PSU#0 Status:Faulted;
XSCF>
XSCF> fmdump
TIME UUID MSG-ID
Feb 09 15:20:46.7179 88f2ccb8-cf7d-4dae-ba65-4ffc1f655124 SCF-8001-KC
Feb 09 15:30:46.6667 554cc530-a17a-4d76-ac56-2685c1c97daf SCF-8002-FA
Feb 24 02:14:04.4976 7f2f29b1-811f-403c-a058-6088c5c0969b SCF-8005-YH



XSCF> fmdump -v -u bf74be0f-fea6-4299-8c04-7688870ea64b
TIME UUID MSG-ID
100% fault.chassis.device.fan.tooslow
Problem in: hc:///chassis=0/psu=0/fan=0

Affects: hc:///chassis=0/psu=0/fan=0
FRU: hc://:product-id=SPARC Enterprise M5000:chassis-id=BCF0904031:server-
id=c1718-4-92-mgt-xscf0:serial=1357FYG-0843AD006:part=CF00300-2011 0202 \300-2011-02-
0:revision=02/component=/PSU#0
Location: /PSU#0
XSCF> showhardconf
Domain#0 Domain_Status:Running;

+ Core:4; Strand:2;
IOU#0 Status:Normal; Ver:0101h; Serial:BF0848PGGR ;
DDCR Status:Normal;
PCI#3 Name_Property:network; Card_Type:Other;
XSCFU Status:Normal,Active; Ver:0101h; Serial:BF0844MRXV ;
OPNL Status:Normal; Ver:0101h; Serial:BF0848P79J ;
* PSU#0 Status:Faulted; Serial:1357FYG-0843AD0061;
+ Power_Status:Off; AC:200 V;
PSU#1 Status:Normal; Serial:1357FYG-0843AD002E;
PSU#2 Status:Normal; Serial:1357FYG-0843AD00F5;
PSU#3 Status:Normal; Serial:1357FYG-0843AD00C8;
FANBP_C Status:Normal; Ver:0301h; Serial:BF0844MVD0 ;



XSCF> showlogs error
Date: Feb 09 15:20:46 GMT 2009 Code: 40000200-cfffd701-0300003200000000
FRU: /HDD#0,/IOU#0
Date: Feb 09 15:30:46 GMT 2009 Code: 40000000-ba010000-0300003200000000
FRU: /IOU#0
Date: Feb 24 02:14:03 GMT 2009 Code: 80002000-c4ff0000-0104342200000000
Status: Alarm Occurred: Feb 24 02:14:02.112 GMT 2009
FRU: /PSU#0
Msg: Abnormal FAN rotation speed. Insufficient rotation
Date: Feb 24 02:14:04 GMT 2009 Code: 80002000-c4ff0000-0104342200000000
Status: Alarm Occurred: Feb 24 02:14:02.190 GMT 2009
FRU: /PSU#0
Msg: Abnormal FAN rotation speed. Insufficient rotation


13.2. F/W Upgrade Example

<Work in Progress>
XSCF> getflashimage ftp://user:password@10.1.0.3//export/FFXCP1072.tar.gz

0MB received
1MB received
47MB received
Download successful: 48596 Kbytes in 51 secs (962.818 Kbytes/sec)
Checking file...
MD5: 0109630ded7d0fd838d4ba8f77568803
XSCF> getflashimage -l
Existing versions:
Version Size Date
FFXCP1072.tar.gz 49762997 Thu Mar 19 15:17:03 GMT 2009
XSCF> flashupdate -c check -m xcp -s 1072

XCP update is possible with domains up
XSCF> flashupdate -c update -m xcp -s 1072

Checking the XCP image file, please wait a minute
XCP update is started (XCP version=1072:last version=1081)
OpenBoot PROM update is started (OpenBoot PROM version=02030000)
OpenBoot PROM update has been completed (OpenBoot PROM version=02030000)
XSCF update is started (XSCFU=0,bank=0,XCP version=1072:last version=1081)
version=01080001)
XSCF download has been completed (XSCFU=0,bank=0,XCP version=1072:last
version=01080001)
version=01080004)
version=01080004)
version=01080000)
version=01080000)
version=01080004)
version=01080004)
version=01080004)
version=01080004)
version=01080001)
version=01080001)
version=01080004)
version=01080004)
XSCF update has been completed (XSCFU=0,bank=0,XCP version=1072:last
version=1081)
XSCF is rebooting to update the reserve bank
XSCF> execute S10ioxoff -- complete
execute K000endMar 19 15:28:34 yama XSCF[104]: XSCF shutdown sequence start
-- complete



execute K100end -- complete
execute K101end -- complete
unmount /hcp0/linux -- complete
unmount /hcp0/scfprog -- complete
unmount /hcp0/gendata -- complete
unmount /hcp0/remcscm -- complete
unmount /hcp1/linux
unmount /hcp1/scfprog
unmount /hcp1/gendata -- complete
unmount /hcp1/remcscm -- complete
unmount /hcpcommon/setup -- complete
unmount /hcpcommon/obp -- complete
unmount /hcpcommon/tmp -- complete
unmount /hcpcommon/var
unmount /hcpcommon/scflog1 -- complete
unmount /hcpcommon/scflog2 -- complete
XSCF reset.
Please stand by while rebooting the system.(15)
Restarting system.
XSCF uboot 01070000 (Feb 8 2008 - 11:12:19)
XSCF uboot 01070000 (Feb 8 2008 - 11:12:19)
SCF board boot factor = 4040

DDR Real size: 256 MB
DDR: 224 MB
## Booting image at ff800000 ...

Image Name: XSCF kernel 01070001 2.6.11.12-s
Image Type: PowerPC Linux Kernel Image (gzip compressed)
Data Size: 1454556 Bytes = 1.4 MB
Load Address: 00000000
Entry Point: 00000000
Verifying Checksum ... OK
Uncompressing Kernel Image ... OK
## Loading RAMDisk Image at ff980000 ...
Image Name: XSCF rootfs 01070005 ,2008/08/15
Image Type: PowerPC Linux RAMDisk Image (gzip compressed)
Data Size: 5451038 Bytes = 5.2 MB
Load Address: 00000000
Entry Point: 00000000
Verifying Checksum ... OK
Loading Ramdisk to 0bacd000, end 0bfffd1e ... OK
Linux version 2.6.11.12-sec (gcc version 3.4.4) #1 Fri Apr 25 09:21:18 JST
2008
new message buffer at 0f700000 size 1048576
log_buf_len: 1048576
mpc85xx_cds_setup_arch
Built 1 zonelists
Kernel command line: root=/dev/ram rw console=ttyS0,9600
init=/sbin/init_change_root panic=1 mem=240M
OpenPIC Version 1.2 (1 CPUs and 44 IRQ sources) at fbe79000
PID hash table entries: 1024 (order: 10, 16384 bytes)
Dentry cache hash table entries: 32768 (order: 5, 131072 bytes)
Inode-cache hash table entries: 16384 (order: 4, 65536 bytes)
Memory: 218752k available (2184k kernel code, 664k data, 316k init, 0k
highmem)
Mount-cache hash table entries: 512 (order: 0, 4096 bytes)
Freeing initrd memory: 5323k freed
RAMDISK driver initialized: 16 RAM disks of 32768K size 1024 blocksize
i2c-algo-cpm: CPM2 I2C algorithm module version 0.1 (Mar 22, 2005)
FCC ENET Version 0.3
TCP established hash table entries: 8192 (order: 4, 65536 bytes)
TCP bind hash table entries: 8192 (order: 3, 32768 bytes)
ip_tables: (C) 2000-2002 Netfilter core team
arp_tables: (C) 2002 David S. Miller
EXT2-fs warning: checktime reached, running e2fsck is recommended
VFS: Mounted root (ext2 filesystem).
Freeing unused kernel memory: 316k init
switching initrd filesystem, ramdisk to tmpfs
SCF Linux Boot Script 2006/03/04 for ROM boot environment
fsl-sec2 hardware crypt accelerator model3a ver 0.02 enabled
XSCF initial process start (pid=104)



load /scf/modules/lites_ldrv.ko -- complete
load /scf/modules/drvscftrace.ko -- complete
load /scf/modules/sec2_rsa.ko -- complete
load /scf/modules/sec2_md5.ko -- complete
load /scf/modules/sec2_des.ko -- complete
load /scf/modules/sec2_arc4.ko -- complete
load /scf/modules/sec2_aes.ko -- complete
load /scf/modules/sec2_sha256.ko -- complete
load /scf/modules/sec2_sha1.ko -- complete
load /scf/modules/hw_random.ko -- complete
load /scf/modules/scsi_mod.ko -- complete
load /scf/modules/sd_mod.ko -- complete
load /scf/modules/usbcore.ko -- complete
load /scf/modules/ohci-hcd.ko -- complete
load /scf/modules/usb-storage.ko -- complete
load /scf/modules/drvbootfmem.ko -- complete
load /scf/modules/drvmbc.ko -- complete
load /scf/modules/drvnand.ko -- complete
load /scf/modules/drvffc.ko -- complete
load /scf/modules/drvffcint.ko -- complete
load /scf/modules/drvrtc.ko -- complete
load /scf/modules/drvxscfuart.ko -- complete
load /scf/modules/drvrci.ko -- complete
load /scf/modules/drvtimer.ko -- complete
load /scf/modules/drvsdic.ko -- complete
load /scf/modules/sm2s.ko -- complete
load /scf/modules/spkmdrv.ko -- complete
mount /dev/mtdblock0 -> /hcp0/linux -- complete
mount /dev/mtdblock1 -> /hcp0/scfprog -- complete
mount /dev/mtdblock2 -> /hcp0/gendata -- complete
mount /dev/mtdblock3 -> /hcp0/remcscm -- complete
mount /dev/mtdblock4 -> /hcpcommon/setup -- complete
mount /dev/mtdblock5 -> /hcpcommon/obp -- complete
mount /dev/mtdblock6 -> /hcpcommon/tmp -- complete
mount /dev/mtdblock11 -> /hcpcommon/var -- complete
mount /dev/mtdblock12 -> /hcpcommon/scflog1 -- complete
mount /dev/mtdblock13 -> /hcpcommon/scflog2 -- complete
execute S00mtab -- complete
execute S00portmap -- complete
execute S00tmpmake -- complete
execute S00varclean -- complete
execute S10ldconfig -- complete
execute S20cfgcreate -- complete
execute S50iptables -- complete
execute S60check -- complete
execute S99syslogd -- complete
initialize XSCF common database (STANDALONE) -- complete
start /scf/sbin/dbs (pid=291)
start /scf/sbin/ksyseventd (pid=292)
start /scf/sbin/syseventd (pid=297)
start /scf/init/scf_cmgrd (pid=305)
start /scf/init/scf_coremgr (pid=306)
start /scf/init/dualprocess (pid=307)
check SCF operation mode (for OPNL access) -- complete
initialize OPNL -- complete
check SCF operation mode -- complete (ACTIVE)
start /scf/sbin/dbs (pid=321)
initialize XSCF common database (OWN) -- complete
synchronize setup data (XSCF -> OPNL) -- complete
initialize XSCF common database (ACTIVE) -- complete
wait for database synchronization -- complete
execute S10restore_um -- complete
execute S11network.sheth0: PHY is Intel LXT972A (1378e2)
eth1: PHY is Intel LXT972A (1378e2)
ERROR: failed to configure the routeing.
The network system may not work correctly.
-- complete
execute S15ntpcheck.sh -- complete
execute S21daemons -- complete
execute S51scflog_SCFboot.sh -- complete
execute S82cod -- complete
execute S98bui -- complete
execute S99tmp_login -- complete
execute S00cmem_check -- complete
execute S18restorefmdlog -- complete
execute S19setfmurl -- complete
execute S29setfmconf -- complete



execute S50setupSCFboot.sh -- complete
execute S83setservicetagdata -- complete
start /scf/sbin/scf_panelmgr (pid=677)
start /scf/sbin/auditd (pid=678)
start /scf/sbin/cmd (pid=679)
start /scf/sbin/cmd_sub (pid=680)
start /scf/sbin/errhandd (pid=681)
start /scf/sbin/fjdrd (pid=682)
start /scf/sbin/frura (pid=683)
start /scf/sbin/mond (pid=684)
start /scf/sbin/mond (pid=685)
start /scf/sbin/powerd (pid=686)
start /scf/sbin/rci (pid=687)
start /scf/sbin/rci_dual (pid=688)
start /scf/sbin/reset (pid=689)
start /scf/sbin/resetra (pid=690)
start /scf/sbin/rsyncprocess (pid=691)
start /scf/sbin/scf_firmup_exed (pid=692)
start /scf/sbin/scf_hardlog (pid=693)
start /scf/sbin/scf_loganalyze (pid=694)
start /scf/sbin/sequence (pid=695)
start /scf/sbin/ttyd (pid=696)
start /scf/sbin/apcsd (pid=697)
start /scf/sbin/codd (pid=698)
start /scf/sbin/scf_firmup_ctld (pid=699)
start /scf/sbin/fmd (pid=700)
Mar 19 15:31:34 yama XSCF flashupdate[699]: XSCF download is started
(XSCFU=0,bank=1,XCP version=1072:last version=1081, Firmware Element
ID=00:version=01070001:last version=01080001)
start /scf/sbin/dscpd (pid=759)
start /scf/sbin/monitor_msg (pid=760)
start /scf/sbin/picld (pid=761)
start /scf/sbin/sunmcrund (pid=762)
start /scf/sbin/stdiscoverer (pid=763)
start /scf/sbin/stlistener (pid=764)
start /scf/sbin/dfrud (pid=765)
start /scf/sbin/iomgrd (pid=766)
start /scf/sbin/archd (pid=818)
wait for SCF_READY -- complete
execute S20setupSCFready.sh -- complete
execute S99rcclear -- complete
execute S10iobox_scan -- complete
execute S10restore_um -- complete
execute S84snmpd -- complete
execute S85sunmc_esd -- complete
execute S86limits -- complete
execute S87opnl -- complete
execute S99dr_recover.sh -- complete
XSCF Initialize complete.
Mar 19 15:32:13 yama XSCF[104]: XSCF Initialize complete.
login: Mar 19 15:32:42 yama XSCF flashupdate[699]: XSCF download has been
completed (XSCFU=0,bank=1,XCP version=1072:last version=1081, Firmware
Element ID=00:version=01070001:last version=01080001)
Mar 19 15:35:16 yama XSCF flashupdate[699]: XSCF download has been



Mar 19 15:39:25 yama XSCF flashupdate[699]: XCP update has been completed
(XCP version=1072)
login: xscf-admin-user
Password:
XSCF#0 (Active )


13.3. Install the SunMC Add-On S/W

Extract the files from the SunMC 3.6.1 Add-On ZIP file (Sun_MC_Addon-3_6_1_V3-GA.zip). Instructions for
downloadin this file can be seen in the Monitoring Section of this document.
----------------------------------------------------------------------------
----------------------------------------------------------------------------




The following Add-On Products are applicable for installation.

Select the products you want to install:
DomMonit SPARC Enterprise Mx000 (y|n|q) y
Dom DR SPARC Enterprise Mx000 (y|n|q) y
The following Add-On Products will be installed:

Do you want to proceed (y|n|q) y
Disk Space Checking....
Processing package instance <SUNWesado> from

</export/downloads/sunmc3.6.1/disk2/image/SunOS/sparc/Addons/OplDomDr/Solaris_10/Basic>
Sun MC Dom DR SPARC Enterprise Mx000 Agent Support(sparc) 3.6.1-v3,REV=2.10.2007.07.11

Using </opt> as the package base directory.
## Processing package information.
## Processing system information.
7 package pathnames are already properly installed.
Installing Sun MC Dom DR SPARC Enterprise Mx000 Agent Support as <SUNWesado>
## Installing part 1 of 1.
/opt/SUNWsymon/base/lib/sparc-sun-solaris2.10/libdr.so <symbolic link>
/opt/SUNWsymon/base/lib/sparc-sun-solaris2.10/libdr.so.1.0
/opt/SUNWsymon/base/lib/sparc-sun-solaris2.10/pkgdr.so <symbolic link>
/opt/SUNWsymon/base/lib/sparc-sun-solaris2.10/pkgdr.so.1.0
/opt/SUNWsymon/modules/cfg/dr-opl-d.flt
/opt/SUNWsymon/modules/cfg/dr-opl-d.x
/opt/SUNWsymon/modules/cfg/dr-opl-daps-images-j.x
/opt/SUNWsymon/modules/cfg/dr-opl-disk-images-j.x
/opt/SUNWsymon/modules/cfg/dr-opl-emptyslots-images-j.x
/opt/SUNWsymon/modules/cfg/dr-opl-family-j.x
/opt/SUNWsymon/modules/cfg/dr-opl-io-images-j.x
/opt/SUNWsymon/modules/cfg/dr-opl-m.x
/opt/SUNWsymon/modules/cfg/dr-opl-models-d.x
/opt/SUNWsymon/modules/cfg/dr-opl-pci-images-j.x
/opt/SUNWsymon/modules/cfg/dr-opl-scsi-images-j.x
/opt/SUNWsymon/modules/cfg/dr-opl-slots-images-j.x
/opt/SUNWsymon/modules/cfg/dr-shell.tcl
/opt/SUNWsymon/modules/sbin/dr-opl-availability.sh
/opt/SUNWsymon/modules/sbin/execute-dapsdr-cmd.sh
/opt/SUNWsymon/modules/sbin/execute-sbddr-cmd.sh
/opt/SUNWsymon/modules/sbin/finish-sbddr-cmd.sh



[ verifying class <none> ]
Installation of <SUNWesado> was successful.
Processing package instance <SUNWesoplda> from

</export/downloads/sunmc3.6.1/disk2/image/SunOS/sparc/Addons/OplDomAdmin/Solaris_10/Basi
c>
Sun MC DomMonit SPARC Enterprise Mx000 Agent Support(sparc) 3.6.1-v3,REV=2.10.2007.07.11

Installing Sun MC DomMonit SPARC Enterprise Mx000 Agent Support as <SUNWesoplda>
/opt/SUNWsymon/addons/OplDomAdmin/classes/SetupArchive.class
/opt/SUNWsymon/addons/OplDomAdmin/lib/locale/install/classes/OplDomAdmin.properties
/opt/SUNWsymon/addons/OplDomAdmin/sbin/addon_common.sh
/opt/SUNWsymon/addons/OplDomAdmin/sbin/es-setup-esoplda.sh
/opt/SUNWsymon/addons/OplDomAdmin/sbin/es-setup.sh
/opt/SUNWsymon/addons/OplDomAdmin/sbin/es-uninst-esoplda.sh
/opt/SUNWsymon/addons/OplDomAdmin/sbin/es-uninst.sh
/opt/SUNWsymon/addons/OplDomAdmin/sbin/opl-dmn-update-cfg.sh
/opt/SUNWsymon/base/cfg/base-Config-ReaderOpl-D.tcl
/opt/SUNWsymon/base/lib/sparc-sun-solaris2.10/libconfigdopl.so <symbolic link>
/opt/SUNWsymon/base/lib/sparc-sun-solaris2.10/libconfigdopl.so.1.0
/opt/SUNWsymon/base/lib/sparc-sun-solaris2.10/pkgconfigdopl.so <symbolic link>
/opt/SUNWsymon/base/lib/sparc-sun-solaris2.10/pkgconfigdopl.so.1.0
/opt/SUNWsymon/bin/configdopld
/opt/SUNWsymon/bin/configdopld.bin
/opt/SUNWsymon/modules/cfg/Config-Reader-Opl-d.def
/opt/SUNWsymon/modules/cfg/Config-Reader-Opl-d.flt
/opt/SUNWsymon/modules/cfg/Config-Reader-Opl-d.prc
/opt/SUNWsymon/modules/cfg/Config-Reader-Opl-d.rul
/opt/SUNWsymon/modules/cfg/Config-Reader-Opl-d.x
/opt/SUNWsymon/modules/cfg/Config-Reader-Opl-m.x
/opt/SUNWsymon/modules/cfg/Config-Reader-Opl-models-d.x
/opt/SUNWsymon/modules/cfg/Config-Reader-Opl-ruleinit-d.x
/opt/SUNWsymon/modules/cfg/Config-Reader-Opl-ruletext-d.x
/opt/SUNWsymon/modules/cfg/Config-Reader-Opl.schema
/opt/SUNWsymon/modules/sbin/Config-Reader-Opl-schema.sh
/opt/SUNWsymon/modules/sbin/Config-Reader-Opl-shell.sh
Installation of <SUNWesoplda> was successful.
Processing package instance <SUNWesopldi> from

</export/downloads/sunmc3.6.1/disk2/image/SunOS/sparc/Addons/OplDomAdmin/Solaris_10/Basi
c>
Sun MC DomMonit SPARC Enterprise Mx000 Server and Agent Support(sparc) 3.6.1-
v3,REV=2.10.2007.07.11
Installing Sun MC DomMonit SPARC Enterprise Mx000 Server and Agent Support as
<SUNWesopldi>
Installation of <SUNWesopldi> was successful.
Processing package instance <SUNWfplda> from

</export/downloads/sunmc3.6.1/disk2/image/SunOS/sparc/Addons/OplDomAdmin/Common/Basic>
French Sun Management Center Agent layer support for OPL domains(all) 3.6.1-



v3,REV=2007.07.23.17.12.36
Installing French Sun Management Center Agent layer support for OPL domains as
<SUNWfplda>
/opt/SUNWsymon/addons/OplDomAdmin/lib/locale/install/classes/OplDomAdmin_fr.properties
/opt/SUNWsymon/lib/locale/fr/LC_MESSAGES/SUNWesoplda.mo
/opt/SUNWsymon/lib/locale/fr_FR.ISO8859-15/LC_MESSAGES/SUNWesoplda.mo <symbolic link>
Installation of <SUNWfplda> was successful.
Processing package instance <SUNWjplda> from

Japanese Sun Management Center Agent layer support for OPL domains(all) 3.6.1-
v3,REV=2007.07.23.17.12.17
Installing Japanese Sun Management Center Agent layer support for OPL domains as
<SUNWjplda>
/opt/SUNWsymon/addons/OplDomAdmin/lib/locale/install/classes/OplDomAdmin_ja.properties
/opt/SUNWsymon/lib/locale/ja/LC_MESSAGES/SUNWesoplda.mo
Installation of <SUNWjplda> was successful.
Processing package instance <SUNWkplda> from

Korean Sun Management Center Agent layer support for OPL domains(all) 3.6.1-
v3,REV=2007.07.23.17.12.46
Installing Korean Sun Management Center Agent layer support for OPL domains as
<SUNWkplda>
/opt/SUNWsymon/addons/OplDomAdmin/lib/locale/install/classes/OplDomAdmin_ko.properties
/opt/SUNWsymon/lib/locale/ko/LC_MESSAGES/SUNWesoplda.mo
Installation of <SUNWkplda> was successful.
Processing package instance <SUNWcplda> from

Simplified Chinese Sun Management Center Agent layer support for OPL domains(all) 3.6.1-
v3,REV=2007.07.23.17.12.26
Installing Simplified Chinese Sun Management Center Agent layer support for OPL domains
as <SUNWcplda>



/opt/SUNWsymon/addons/OplDomAdmin/lib/local/install/classes/OplDomAdmin_zh_CN.properties
/opt/SUNWsymon/lib/locale/zh/LC_MESSAGES/SUNWesoplda.mo
Installation of <SUNWcplda> was successful.
Processing package instance <SUNWhplda> from

Traditional Chinese Sun Management Center Agent layer support for OPL domains(all)
3.6.1-v3,REV=2007.07.23.17.12.55
Installing Traditional Chinese Sun Management Center Agent layer support for OPL domains
as <SUNWhplda>
/opt/SUNWsymon/addons/OplDomAdmin/lib/local/install/classes/OplDomAdmin_zh_TW.properties
/opt/SUNWsymon/lib/locale/zh_TW/LC_MESSAGES/SUNWesoplda.mo
Installation of <SUNWhplda> was successful.
You must perform setup before using Sun Management Center 3.6.1.
Do you want to run setup now (y|n|q) y
----------------------------------------------------------------------------
This script will help you to setup Sun (TM) Management Center 3.6.1.
----------------------------------------------------------------------------
Stopping platform component
Stopping agent component
Documentation of Sun Management Center is not available on

Sun Management Center CD image, For all Sun Management Center
documentation please visit :
http://docs.sun.com/app/docs/coll/810.8 website
Setting up Addon[s]...
---------------------------------------------------------------------
Starting Sun Management Center Mx000 Systems Domain Setup
---------------------------------------------------------------------
Additional Setup information is available in the
Sun Management Center 3.6.1 Version 2 Add-On Software Supplement for SPARC Enterprise
Servers.
The Domain Config Reader for Systems will collect

configuration and status information for your tape drives at regular
polling intervals. This can be disruptive to tape drive controllers
that do not allow concurrent access.
Would you like to disable this feature? [y|n|q] y

Added module em to /var/opt/SUNWsymon/cfg/base-modules-d.dat.
Added module Config-Reader-Opl to /var/opt/SUNWsymon/cfg/base-modules-d.dat.
Created deviceinfo.conf file
Creating Agent Update configuration file.
------------------------------------------------------------
Do you want to start Sun Management Center agent now (y|n|q) y
Starting Sun Management Center Agent...
Agent started successfully.
Setup of Sun Management Center is complete.

Setup logfile is : /var/opt/SUNWsymon/install/setup_mx000-uksr.090210134856.20260
Install logfile is : /var/opt/SUNWsymon/install/install_mx000-uksr.090210134353.16424
End of Installation
Exiting Sun Management Center installation.


13.4. Bookmarks
13.4.1. Documentation
• docs.sun.com M3000
• http://docs.sun.com/app/docs/prod/sparc.m3k?l=en&a=view
13.4.2. Support
• Onestop M3000
• https://sunspace.sfbay.sun.com/display/Onestop/SPARC%20Enterprise%20M3000
• Onestop M4000
• Onestop M5000
• Onestop M8000
• Onestop M9000
• SSE M3000
• http://sunsolve.central.sun.com/handbook_internal/Systems/SE_M3000/SE_M3000.html
• SSE M4000
• SSE M5000
• SSE M8000
• SSE M9000


13.4.3. Training
• WZO-TE2001 Sun SPARC Enterprise Servers Technical Essentials

• https://learning.sun.com/display/courses/WZO-TE2001+Sun+SPARC+
+Enterprise+Servers+Technical+Essentials
• WZT-6774 Sun SPARC Enterprise M3000 Server Administration

• https://learning.sun.com/display/courses/WZT-
6774+Sun+SPARC+Enterprise+M3000+Server+Administration
• WZT-4110 Sun SPARC Enterprise MX000 Server Overview, Architecture, and Components
4110+Sun+SPARC+Enterprise+MX000+Server+Overview%2C+Architecture
%2C+and+Components
• WGS-PREX-4110 Sun SPARC Enterprise MX000 Server Overview and Architecture Assessment
4110+Sun+SPARC+Enterprise+MX000+Server+Overview%2C+Architecture
%2C+and+Components
• WXI-4110 Sun SPARC Enterprise MX000 Server Support Readiness Training Assessment
• https://learning.sun.com/display/courses/WXI-
4110+Sun+SPARC+Enterprise+MX000+Server+Support+Readiness+Training+Assessment
• WZT-5393 Sun External I O Expansion Unit Installation and Administration

5393+Sun+External+I+O+Expansion+Unit+Installation+and+Administration
• WXI-5393 Sun External I O Expansion Unit Installation and Administration Assessment

• https://learning.sun.com/display/courses/WXI-
5393+Sun+External+I+O+Expansion+Unit+Installation+and+Administration+Assessment
13.4.4. Sales
• sun.com M3000
• http://www.sun.com/servers/midrange/m3000/index.xml
• sun.com M4000
• sun.com M5000
• sun.com M8000
• http://www.sun.com/servers/highend/m8000/index.xml
• sun.com M9000
• http://www.sun.com/servers/highend/m9000/index.xml


13.4.5. Various
• Sun Management Center 3.6.1 Version 3 Add-On Software
Site/en_US/-/USD/ViewProductDetail-Start?ProductRef=SunMC-3.6.1V3-G-F@CDS-
CDS_SMI
• Blueprint - M-Series Servers Configuration Concepts

• http://wikis.sun.com/display/BluePrints/Sun+SPARC+Enterprise+M-
Series+Servers+Configuration+Concepts
• Mx000 Servers XSCF User's Guide

• http://docs.sun.com/app/docs/doc/819-6202-14?l=en
• Mx000 Servers Administration Guide

• Mx000 Servers Dynamic Reconfiguration (DR) User's Guide

• M4000/M5000 Servers Service Manual

• SUN SPARC® ENTERPRISE M4000, M5000, M8000, AND M9000 SERVER ARCHITECTURE
• http://www.sun.com/servers/sparcenterprise/SPARCEnt-Arch-Final.pdf
• EIS Checklists
• http://www.eis.central/checklists/checklists.html
• Sun Doc 235101 - Sun[TM] Sparc Enterprise Mx000 (OPL) Servers Memory Configuration Rules
• http://sunsolve.sun.com/search/document.do?assetkey=1-61-235101-
1&searchclause=235101
• Mx000 (OPL) ServersMemory configuration, restrictions and considerations

• http://panacea/twiki/pub/Main/StefOPL/OPL-Memory.pdf
• Mx000 Seies Resources – Excellent Internal Site! (includes “Deep Dive” Presentation)
• http://esp.west/~grc/opl/
• OPL - Service Password Generator (includes link to Escalation Password Generator)

http://oplpass.sfbay/
• Sun TSC Internal Home Page for Mx000 Servers (great startting point for any Technical Information)
• http://panacea.central.sun.com/twiki/bin/view/Products/ProdInfoSunSPARCEnterpriseMx000
• Mx000 server fan/fantray redundancy and thermal environments

• http://sunsolve.sun.com/search/document.do?assetkey=1-61-235941-1


• Sun SPARC® External I/O Expansion Unit Device Paths

• Snapshot / Log Archives Analysis Tools

• http://panacea.central.sun.com/twiki/bin/view/Main/SnapshotAnalysisTools


Eat That Frog

Загружено:

Сведения о документе

Авторское право

Доступные форматы

Поделиться этим документом

Поделиться или встроить документ

Параметры публикации

Этот документ был вам полезен?

Это неприемлемый материал?

Авторское право:

Доступные форматы

Eat That Frog

Загружено:

Авторское право:

Доступные форматы

Mx000 Server User Guide

Version Number: 2.0

Author: Kevin Hickman

Document Date: 11th August 2009

© Sun Microsystems, 2009

Mx000 Server User Guide

Project : 30024250 Page 2 of 162 Author : Kevin Hickman

Sun Confidential: Internal Only

Mx000 Server User Guide

Project : 30024250 Page 3 of 162 Author : Kevin Hickman

Sun Confidential: Internal Only

Mx000 Server User Guide

Project : 30024250 Page 4 of 162 Author : Kevin Hickman

Sun Confidential: Internal Only

Mx000 Server User Guide

Project : 30024250 Page 5 of 162 Author : Kevin Hickman

Sun Confidential: Internal Only

Mx000 Server User Guide

0.2. Document References

Project : 30024250 Page 6 of 162 Author : Kevin Hickman

Sun Confidential: Internal Only

Mx000 Server User Guide

• Provide administration support for the Solaris OS

Project : 30024250 Page 7 of 162 Author : Kevin Hickman

Sun Confidential: Internal Only

Mx000 Server User Guide

Advanced Product Line

Niagara Product Line OPL (Olympus Product Line)

LE ("Low End") FF ("Form Factor") DC ("Data Center")

LE1 ( SPARC Enterprise FF1 ( SPARC Enterprise

• FF1 (M4000) replaces the V490

Project : 30024250 Page 8 of 162 Author : Kevin Hickman

Sun Confidential: Internal Only

Mx000 Server User Guide

2.2. Meet the processors

• The dual-core SPARC64 VI processor (Olympus)

• 2.1Ghz on Form Factor Servers

• The quad-core SPARC64 VII processor (Jupiter)

• 2.4Ghz on Form Factor Servers

• The M3000 only supports the SPARC64 VII (Jupiter CPU).

2.2.1. Olympus Processor

2.2.2. Jupiter Processor

Project : 30024250 Page 9 of 162 Author : Kevin Hickman

Sun Confidential: Internal Only

Mx000 Server User Guide

2.3. Meet the Servers

• Sun SPARC Enterprise M3000 Server (Ikkaku)

2.4. Solaris Support

• OS Support for SPARC64 VI: Solaris 10_u3 (11/06) or higher

Project : 30024250 Page 10 of 162 Author : Kevin Hickman

Sun Confidential: Internal Only

Mx000 Server User Guide

3.1.2. M3000 Broad Specs

• Solaris 10_u4 (8/07) or higher

• No IO Expansion Unit support

Project : 30024250 Page 11 of 162 Author : Kevin Hickman

Sun Confidential: Internal Only

Mx000 Server User Guide

3.1.3. M3000 Front View

3.1.4. M3000 Rear View

Project : 30024250 Page 12 of 162 Author : Kevin Hickman

Sun Confidential: Internal Only

Mx000 Server User Guide

3.1.5. M3000 Top View