Core Voice O&M Guide

AllIPCDMA2000|AllIPGSM
SonataCoreVoiceNetwork
IS41R4.5.3|GSMR4.5.4
Operations,Maintenanceand
ProvisioningGuide

PartNumberD02651GSRevA0
AllIPCDMA2000|AllIPGSM
IS41R4.5.3|GSMR4.5.4
Operations,Maintenanceand
ProvisioningGuide

STA R S O LUT I O N S CO N F I D E N TI A L
TheinformationcontainedhereinisthepropertyofStarSolutionsInternationalInc.StarSolutionsandisstrictlyconfidential.
ExceptasexpresslyauthorizedinwritingbyStarSolutions,theholdershallkeepallinformationcontainedhereinconfidential,in
wholeorinpart,fromdisclosureanddisseminationtothirdparties.
COP YR I G H T N OT I CE
Copyright2012,StarSolutionsInternationalInc.Allrightsreserved.Nopartofthisdocumentationmaybereproducedinany
formorbyanymeansorusedtomakeanyderivativework(suchastranslation,transformation,oradaptation)withoutprior
writtenpermissionfromStarSolutions.
StarSolutions,theStarSolutionslogo,iCellandSonataareregisteredtrademarksortrademarksofStarSolutionsInternational
Inc.anditssubsidiaries.Allotherbrandandproductnamesmayberegisteredtrademarksandarethepropertyoftheir
respectiveowners.
StarSolutionsreservestherighttorevisethisdocumentationandtomakechangesincontentfromtimetotimewithout
obligationonthepartofStarSolutionstoprovidenotificationofsuchrevisionorchange.
StarSolutionsprovidesthisdocumentationwithoutwarrantyofanykind,eitherimpliedorexpressed,including,butnotlimited
to,theimpliedwarrantiesofmerchantabilityandfitnessforaparticularpurpose.StarSolutionsmaymakeimprovementsor
changesintheproduct(s)and/ortheprogram(s)describedinthisdocumentationatanytime.
U N I T E D S TAT ES G OVE R N M E N T LEG E N D S
IfyouareaUnitedStatesgovernmentagency,thenthisdocumentationandthesoftwaredescribedhereinareprovidedtoyou
subjecttothefollowing:
UnitedStatesGovernmentLegend:Alltechnicaldataandcomputersoftwareiscommercialinnatureanddevelopedsolelyat
privateexpense.SoftwareisdeliveredasCommercialComputerSoftwareasdefinedinDFARS252.2277014(June1995)orasa
commercialitemasdefinedinFAR2.101(a)andassuchisprovidedwithonlysuchrightsasareprovidedinStarSolutions's
standardcommerciallicensefortheSoftware.TechnicaldataisprovidedwithlimitedrightsonlyasprovidedinDFAR
252.2277015(Nov1995)orFAR52.22714(Dec2007),whicheverisapplicable.Youagreenottoremoveordefaceanyportion
ofanylegendprovidedonanylicensedprogramordocumentationcontainedin,ordeliveredtoyouinconjunctionwith,this
documentation.

CONTENTS
ABOUTTHISGUIDE
Conventions 24
Notices 24
FiguresandScreenCaptures 24
Text 24
RelatedDocumentation 25
TechnicalSupport 28
ObtainingTechnicalAssistance 28
ServiceCentersOperationalHours 28
WarrantySupport 29
ContactingTechnicalDocumentation 29
PARTI OVERVIEW
1 OVERVIEW

Overview:Provisioning,Operating,andMaintainingtheCoreVoiceNetwork 33
ElementandNetworkManagement 34
DomainsandRoles 35
OperationsDomain 35
AdministrationDomain 36
OperatorRole 37
AdministratorRole 37
RolePrerequisites 38
GeneralKnowledge 38
UNIXKnowledge 38
ProductKnowledge 38
DocumentationFamiliarityandTraining 38
PARTII PROVISIONING
2 PROVISIONING THECOREVOICENETWORKELEMENTS
ProvisioningOverview 41
UsingtheOMCConsole 41
AccessingtheOMCConsole 42
NetworkElementsontheOMC 43
UsingtheCommandLineInterface 43
CLIOverview 43
AccessingtheCLI 44
STA R SO LU TIONS CO N FI DE N TI A L SonataCoreVoiceNetworkIS41R4.5.3|GSMR4.5.4Operations,MaintenanceandProvisioningGuide

PartNumberD02651GSRevA0|July2012
6 Contents
CLICommandSyntax 44
CancelingaCLICommand 45
NavigatingDisplayedInformation 46
PARTIII OPERATIONS ANDADMINISTRATION
3 BASICOPERATIONS
OperationsandMaintenanceCenter 49
OMCServer 49
OMCConsole 49
OperatingTasks 50
TaskandShiftOverview 50
OperatorTasks 51
ShiftDocumentationandHandover 51
RecordingFaults 52
MonitorAlarms 52
CheckCPUUtilization 53
CheckMemoryUtilization 53
CheckDiskSpace 54
CheckLogUsage 54
CheckPerformanceStatistics 54
EndofShift 55

4 BASICADMINISTRATION
MonitoringLogsandEvents 57
DisplayingtheNEApplicationStatus 58
DisplayingtheSoftwareVersion 58
StartingandStoppingtheNEApplication 58
StoppingtheNEApplication 59
StartingtheNEApplication 60
AbortingtheNEApplication 60
StoppingtheOperatingSystem 61
RebootingtheNE 61
PoweringDowntheNE 61
PoweringUptheNE 63
TroubleLocatingandClearing 66
SystemAdministrationTools 66
5 USERACCOUNTS
NEUserAccounts 67
SystemAdministrationAccount 68
MultiuserMode 68
SingleUserMode 69
NEApplicationAccount 69
UserManagement 70

Contents 7
ConnectingtotheNE 71
IPAddressing 71
ConnectingUsingtelnet 71
ConnectingUsingaTerminalEmulator 71
ConnectingUsingaTerminalServer(Remote) 72
ConnectingUsingaTerminalServer(Local) 74
ConnectingUsingaDirectSerialConnection 75
SerialCableforDirectConnection 75
ConnectingUsingReachthrough 76
LoggingInandOut 77
LoggingIntoNEApplicationUserAccount 77
LoggingInasroot 77
MultiUserMode 77
SingleUserMode 77
LoggingintorootfromtheNEApplicationAccount 78
LoggingOut 78
ChangingUserPasswords 79
PasswordGuidelines 79
6 FILESYSTEMS
AboutDisksandFilesystems 81
UNIXFilesystem 82

NetworkElementApplicationFilesystem 83
rootAccountFilesystem 83
HomeDirectory 83
SignalingServerApplicationFilesystemStructure 83
7 ACTIVESTANDBYOPERATIONS
HighAvailability 85
HighAvailabilityTerminology 86
Active/StandbyModel 86
Active/ActiveModel 86
FailoverStates 86
HighAvailabilityImplementation 87
PhysicalView 87
ManagementView 88
ServiceView 88
upLinkandHWDOperation 89
upLink 89
HWD 90
HighAvailabilityandRedundancyVerification 91
VerifyingupLinkFunctionality 91
CheckinguplinkStatus 91
TestingupLink 92
Testingmanualfailover 92
VerifyingRAFTHWDFunctionality 92

8 Contents
TestingHWDfailover 92
TestingHWDrestart 93
CheckingRedundantConfiguration 94
DisplayingActive/StandbyIPAddress 94
DisplayingtheVirtualMACAddress 95
CheckingtheActiveStatus 96
OperatorInitiatedFailovertoStandby 96
FaultInducedFailover 99
InvestigatingFailoverCause 99
CheckingEventLogRecords 100
DetailsforCustomerSupport 100
sys_mirrorOperation 101
Verifyingsys_mirroroperations 101
ActiveStandbyProvisioning 102
CheckingwhichnodeisActive 102
ResynchronizingtheNodes 103
8 PERFORMANCEMONITORING ANDMANAGEMENT
OverviewofPerformanceManagement 105
WhatisPerformanceManagement 105
QualityofService(QoS) 106
QoSintheIPNetwork 106

PerformanceMonitoringandDataCollection 107
ContinuousChecking 107
RoutineorPeriodicChecking 107
CheckingwithLiveTraffic 107
CheckingwithNoLiveTraffic 107
PerformanceAnalysis 107
PerformanceManagementActivities 108
PerformanceGraphsandReports 108
AboutGraphsandReports 109
Statistics 111
GeneratingandPrintingPerformanceGraphs 111
GeneratingandExportingPerformanceReports 112
CheckingCPUUtilization 116
PerformanceCharacteristics 116
ViewingCPUProcessLists 118
ViewingProcessorUtilization 119
CheckingMemoryUtilization 119
MemoryUtilizationGraphsandReports 119
SwapUtilizationGraphsandReports 120
CheckingSwapUtilization 121
DiskI/OGraphsandReports 122

Contents 9
CheckingDiskI/O 123
NetworkI/OGraphsandReports 123
CheckingNetworkI/O 124
CheckingDiskSpace 125
CheckingLogUsage 126
CheckingTCP/IPLinkStatus 127
PerformanceManagementActivities 127
9 SYSTEMT IMEMANAGEMENT
SystemTimeOverview 129
SystemTime 130
ConfiguringtheNTPClient 130
NTPServer/ClientConfiguration 131
SingleAvailabilityNTPConfiguration 131
HighAvailabilityNTPConfiguration 132
CheckingtheNTPServer 133
SynchronizingNTPServersandClients 133
CheckingtheSystemTime 134
ConfiguringTimeZonesandDaylightSavingsTime 134
ConfiguringTimeZones 134

ConfiguringUTCTimeDisplay 135
CompilingTimeZoneFiles 135
10 DATABASE ANDSYSTEMBACKUP ANDRESTORE

DatabaseBackupOverview 139
DatabaseOverview 139
DatabaseSchema 140
TableVersioning 140
AboutDatabaseBackups 140
BackupDataLocation 141
BackupNamingConvention 141
BackinguptheConfigurationDatabase 141
AutomaticDatabaseBackups 141
Understandingcrontab 142
VerifyingandChangingtheAutomaticDatabaseBackupSchedule 142
ManualDatabaseBackups 143
ManagingDatabaseBackupFiles 144
RestoringtheConfigurationDatabase 145
RestoringDatabaseinSingleAvailabilityConfiguration 145
RestoringDatabaseinHighAvailabilityConfiguration 146
SystemBackup 147
LocalandRemoteTapeDrives 148
DiskMirroring 148
HighAvailabilityConsiderations 149

10 Contents
SystemImpactofTapeBackup 149
SystemBackupUsingLocalTapeDrive 150
CheckingDiskMirrorStatus 150
ShuttingDowntheTargetNE 151
DetachingLogicalDisks 152
RestartingtheNE 152
BackUpLogicalDrives 152
AttachingLogicalDisks 153
SystemBackupUsingRemoteTapeDrive 154
RemoteTapeSystemSetup 154
ClientSystemSetup 157
RecordingEthernetInterfaceConfiguration 160
PerformingTapeBackup 160
VerifyingDiskHealth 163
ConfiguringInterfaceandRouting 163
UsingtheTapeDeviceforBackup 164
SettingupTapeSystem 164
BackingUpDisk 165
HardwareRAIDNetra440HLRSystemBackup 166
Prerequisites 166
SoftwareMirroredDiskStrategy 166
BackingUptheHLRSystem 167
SystemRestore 168

SystemRestoreUsingLocalTapeDrive 168
SystemRestoreDuetoSimultaneousHardDriveFailure 168
ShutdownTargetNE 169
EnterSingleUserModefromCDROM 169
FormatHardDrives 170
LoadRestoreTape 171
CreatingaNewFilesystemandCheckingTargetDisk 171
RestoretheDisk 171
RestoringDisk 171
RewindandEjectTape 172
ConfigureSystemDisk 172
InstallBootSectorandCheckDisk 173
RestartTargetNEandVerifySystem 174
EnableDiskMirror 174
SetupReplicas 174
EnableOneWayDiskMirror 175
ConfigureDisk 175
EnableTwoWayMirror 176
SystemRestoreUsingRemoteTapeDrive 178
OverviewofSystemRestoring 178
PerformingSystemRestore 179
SingleUserMode 179
FormatDisk1 179
FormatDisk2 182

Contents 11
ClientSystemSetup 182
PerformingtheRestore 182
RebootingandSystemVerification 185
HardwareRAIDNetra440HLRSystemRestore 186
ShutdownTargetNE 186
EnterSingleUserModefromCDROM 186
FormatHardDrives 187
LoadRestoreTape 189
CreatingaNewFilesystemandCheckingTargetDisk 190
RestoretheDisk 190
RestoringtheDisk 190
RewindandEjectTape 191
ConfigureSystemDisk 191
InstallBootSectorandCheckDisk 192
RestartHLRandVerifySystem 192
RecovertheOracleDatabase 192
RestoreDiskSuiteMirroring 194
Restoresystemservice: 196
PARTIV MAINTENANCE
11 MAINTENANCEOVERVIEW

CoreVoiceNetworkMaintenanceOverview 201
MaintenanceGuidelines 202
FaultManagement 202
12 ALARMMANAGEMENT
AlarmsOverview 205
AlarmSeverity 205
Critical 205
Major 205
Minor 205
Warning 206
AlarmTypes 206
ProcessingError 206
Equipment 206
Communications 206
AutomaticandManualAlarmClearing 206
AutomaticAlarmClearing 206
ManualAlarmClearing 207
MonitoringAlarms 207
AlarmRootCauses 208
CommunicationsProtocolError 208
ConfigurationorCustomizationError 208
Congestion 208
CorruptData 208
12 Contents
FileError 208
OutofMemory 208
PowerProblem 209
ResourceatorNearingCapacity 209
ResponseTimeExcessive 209
SoftwareProgramAbnormallyTerminated 209
SoftwareError 209
TransmitFailure 209
UnderlyingResourceUnavailable 209
UnspecifiedReason 209
FaultLocalization 210
CommonAlarms 210
CriticalCommonAlarms 210
NERestart 211
cpuCMSoftFail 211
cpuCmVitalDeath 211
cpuPMCMDDeath 212
cpuPMHWDDeath 212
DblDbRecErr 212
ipcTcpHostDown 213
tpmTmrDrop 213
DiskAlarm 214
RootDiskAlarm 214

MajorCommonAlarms 215
PowerProblem 215
MinorCommonAlarms 215
WarningCommonAlarms 215
cpuCMShutDown 215
IpcTcpHostUp 216
13 V IEWINGLOGS ANDEVENTS
ThevlogUtility 217
vlogMessages 217
vlogMessageFormats 217
vlogSeverityLevels 218
ViewingEventsandRecords 218
EventLogRecordTimeframe 218
SearchingforEventsbyNameorSeverity 220
14 TROUBLELOCATING ANDCLEARING
PacketCaptureandAnalysis 224
Aboutsnoop 224
AboutWireshark 224
Usingsnoop 225
PacketCapturePoint 225
ProtocolsandPorts 225

Contents 13
GeneratingaNamedsnoopFile 225
GeneratingSpecificProtocolsnoopFiles 226
GeneratingsnoopFilesforaSpecificNetworkElement 226
UsingWireshark 227
DownloadingandInstallingWireshark 227
RetrievingsnoopFiles 228
AnalyzingsnoopFiles 228
CorrectingforCaptureandAnalysisTimeDifference 229
ApplyingFilters 230
LoggingIssues 231
IntelligentMediaGatewayLogsnotCapturedbysyslog 231
RecoveryfromOutageorDisaster 233
RecoveringfromServiceOutage 233
RecoveringfromNetworkOutage 233
RecoveringfromDisaster 233
CreatingaDisasterRecoveryPlan 233
TerminalServerRecovery 233
TraceManager 234
TraceTypes 234
ModuleTracing 234
ListingTraceableModules 235
ProcessTracing 236
ListingTraceableProcesses 237

ResourceTracing 238
ListingTraceableResources 238
TraceFiles 238
CheckingActiveTraceStatus 239
TraceLevels 239
StartingandStoppingTraces 240
StartingandStoppingModuleTraces 240
StartingandStoppingProcessTraces 241
AddingaResourcetoaTrace 241
TracingTipsforTroubleshooting 242
WritingTraceFilesFromBuffer 243
FlushingModuleandProcessTraceBuffers 243
FlushingResourceTraceBuffers 244
ViewingTraceFiles 244
ConvertingTraceFiles 245
ArchivingTraceFiles 245
RemovingTraceFiles 246
CONXTracingfromtheOMC 246
TroubleshootingHWD 247
15 SOFTWAREMAINTENANCE
OverviewofSoftwareMaintenance 249
SystemandComponentSoftware 249
ComponentSoftware 249

14 Contents
SystemSoftware 250
SoftwareReleaseTypes 250
EmergencyPatchRelease 250
PatchLevelRelease 251
PointLevelRelease 251
SoftwareReleasesandVersioning 252
SoftwareVersioning 252
EmergencyPatchReleaseVersioning 253
PatchLevelReleaseVersioning 253
PointLevelReleaseVersioning 254
InstallingEmergencyPatches 254
OverviewofEmergencyPatchInstallation 254
EmergencyPatchInstallationOverview 255
SystemImpact 255
EmergencyPatchPackage 255
UsingEmergencyPatchReleaseNotes 256
BeforeInstallingEmergencyPatches 257
GettingEmergencyPatchSoftware 257
GettingRequiredDocumentation 257
SettingUpInstallerClient 258
ConnectingtoTargetNE 258
CheckingEmergencyPatchUpgradePath 259
BackingUptheDatabase 259

RemovingEmergencyPatches 261
BeforeRemovingEmergencyPatches 261
RemovingEmergencyPatches 261
SoftwareandSystemVerification 262
InstallingPatchLevelReleases 263
OverviewofPatchLevelReleaseInstallation 263
PatchLevelReleaseInstallationOverview 263
PatchLevelReleaseMaintenanceWindow 263
SystemImpact 263
GettingPatchLevelReleaseSoftware 263
PatchLevelReleaseContents 264
UsingPatchLevelReleaseNotes 264
BeforeInstallingNewPatchLevel 264
CheckingPatchLevelUpgradePath 265
InstallingaNewPatchLevelRelease 265
ListingtheInstalledPatchVersions 265
SoftwareandSystemVerification 265
RemovingPatchLevelReleases 266
PointLevelVersionUpgrades 266
OverviewofSoftwareVersionUpgrades 266
PointLevelVersionUpgradePaths 266
GettingReleaseSoftwareandDocumentation 266
SoftwareReleasePackageDescription 267

Contents 15
BeforeUpgradingSoftwareVersion 268
UpgradePrerequisites 268
UpgradeMaintenanceWindow 268
SystemImpact 268
UsingReleaseNotesforSoftwareVersionUpgrade 268
PointLevelReleaseInstallation 269
PointLevelReleaseInstallationInstructions 269
SoftwareUpgradeVerification 269
PostUpgradeProcedures 269
PostUpgradeConfiguration 269
16 HARDWAREMAINTENANCE
ChangingAirFilters 271
SUNNetra240FilterReplacement 271
RemovingtheAirFilter 272
InstallingtheAirFilter 272
SUNNetra440FilterReplacement 272
RemovingtheAirFilter 272
InstallingtheAirFilter 272
PARTV APPENDICES

A PERFORMANCECOUNTERS ANDSTATISTICS
AboutThisAppendix 275
TypesofCounters 275
CumulativeCounters 275
Gauges 275
Gauge_Max 275
Gauge_Min 276
DiscreteEventRegistration 276
Usage 276
CounterNaming 276
SystemHardwarePerformanceStatistics 276
CPUStatistics 276
MemoryUtilizationStatistics 277
I/OStatistics 277
B SUBSYSTEMEVENTS
UnderstandingEventDescriptions 279
CPUSubsystemEvents 279
ASPSubsystemEvents 281
CSISubsystemEvents 296
VSMSubsystemEvents 301
SSFSubsystemEvents 310

16 Contents
C DIAGNOSTICTOOLS
DiagnosticTools 315
SystemDiagnostics 316
ALMDiagnostics 316
CDRDiagnostics 317
COMDiagnostics 318
EVLDiagnostics 318
GSSDiagnostics 319
STADiagnostics 319
TPMDiagnostics 320
CallProcessingDiagnostics 321
ASPDiagnostics 321
AUADiagnostics 322
AURDiagnostics 322
AUSDiagnostics 323
LFMDiagnostics 324
PPXDiagnostics 325
325
SSFDiagnostics 325
326
SSPDiagnostics 326
VSMDiagnostics 327

MediaControlDiagnostics 329
MEGDiagnostics 329
MEGRESDiagnostics 330
MEGTSADiagnostics 330
AUEDiagnostics 331
RadioResourceDiagnostics 332
BSMDiagnostics 332
332
CSIDiagnostics 332
RCMDiagnostics 334
RCIDiagnostics 335
WJTDiagnostics 336
GAPDiagnostics 337
IS41MSCTrunkManagement 337
IMPDiagnostics 337
MCIDiagnostics 338
ISIDiagnostics 339
IS41MobilityManagement 341
VLRDiagnostics 341
MIPDiagnostics 342
GSMMobilityManagement 343
MCGDiagnostics 343
MAUDiagnostics 344

Contents 17
D ACRONYMS
INDEX

18 ListofTables
LISTOFTABLES
Table 1 NoticeIconDescriptions 24
Table 2 TextConventionDescriptions 24
Table 3 GenericCLISyntax 45
Table 4 CLINavigationKeys 46
Table 5 OperatorShiftChecklist 51
Table 6 SystemAdministratorAccountandTaskMapping 68
Table 7 NEApplicationUserAccountandTaskMapping 69
Table 8 UserIDsandPasswords 70
Table 9 UserIDsandTasks 70
Table 10 DB9PinoutsonSerialCable 76
Table 11 RJ45PinoutsonSerialCable 76
Table 12 DiskTerminology 81
Table 13 PartitionTerminology 82
Table 14 FilesystemDescription 82
Table 15 FailureTraceFiles 101
Table 16 Active/StandbyStateFields 102
Table 17 PerformanceDataforGraphs 109
Table 18 PerformanceDataforReports 110
Table 19 SystemResourceControlMechanisms 127
Table 20 DefaultCompiledTimeZones 135

Table 21 ContinentalAreas 136
Table 22 crontabCommandOptions 142
Table 23 crontabSyntaxDescription 142
Table 24 DiskSliceFilesystemCheck 151
Table 25 useraddOptions 156
Table 26 DiskPartitionRecord 158
Table 27 EthernetConfigurationRecord 160
Table 28 netstatOptions 160
Table 29 DiskSliceFilesystemCheck 163
Table 30 mtCommandOptions 164
Table 31 TapeLabelingGuidelines 166
Table 32 36GbHardDiskPartitionsizes 170
Table 33 72GBHardDiskPartitionsizes 170
Table 34 ufsrestoreCommandOptions 183
Table 35 72GBHardDiskPartitionSizesforDisks1&2ontheHLR 188
Table 36 146GBHardDiskPartitionSizesforDisks1&2ontheHLR 188
Table 37 72GBHardDiskPartitionSizesforDisks3&4 189
Table 38 146GBHardDiskPartitionSizesforDisks3&4 189
Table 39 NERestartClearingInformation 211
Table 40 cpuCmSoftFailClearingInformation 211
Table 41 cpuCMVitalDeathClearingInformation 211
Table 42 cpuPMCMDDeathClearingInformation 212
Table 43 cpuPMHWDDeathClearingInformation 212
Table 44 DblDbRecErrClearingInformation 212
Table 45 ipcTcpHostDownClearingInformation 213
Table 46 tpmTmrDropClearingInformation 213
Table 47 DiskAlarmClearingInformation 214
Table 48 RootDiskAlarmClearingInformation 214
Table 49 PowerProblemClearingInformation 215
ListofTables 19
Table 50 cpuCMShutDownClearingInformation 215

Table 51 IpcTcpHostUpClearingInformation 216
Table 52 vlogMessageFormat 218
Table 53 SystemConsoleSeverityLevels 218
Table 54 ProtocolsandPortMappings 225
Table 55 FileFormatsandTimestamps 230
Table 56 TraceableProcesses 236
Table 57 UsefulTraceLevels 239
Table 58 TroubleshootingScenarios 242
Table 59 Example:HLRSoftwareComponents 250
Table 60 EPFilenameSyntax 253
Table 61 PatchLevelFilenameSyntax 254
Table 62 PointLevelFilenameSyntax 254
Table 63 EmergencyPatchReleaseNotesHeadings 256
Table 64 PatchlevelReleaseContents 264
Table 65 PatchLevelReleaseNotesDescription 264
Table 66 PointLevelSoftwareReleaseContentsforanHLR 267
Table 67 SystemHardwarePerformanceStatistics 277
Table 68 CPUSubsystemEvents 279
Table 69 ASPSubsystemEvents 281
Table 70 CSISubsystemEvents 296
Table 71 VSMSubsystemEvents 301
Table 72 SSFSubsystemEvents 310
Table 73 CommandOptionsforalm_dump 317
Table 74 CommandOptionsforcdr_dump 317
Table 75 CommandOptionsforcom_dump 318
Table 76 CommandOptionsforevl_dump 318

Table 77 CommandOptionsforgss_dump 319
Table 78 CommandOptionsforsta_dump 319
Table 79 CommandOptionsfortpm_dump 320
Table 80 CommandOptionsforasp_dump 321
Table 81 CommandOptionsforaua_dump 322
Table 82 CommandOptionsforaur_dump 322
Table 83 CommandOptionsforaus_dump 323
Table 84 CommandOptionsforlfm_dump 324
Table 85 CommandOptionsforppx_dump 325
Table 86 CommandOptionsforssf_dump 325
Table 87 CommandOptionsforssp_dump 326
Table 88 CommandOptionsforvsm_dump 327
Table 89 CommandOptionsformeg_dump 329
Table 90 CommandOptionsformeg_res_dump 330
Table 91 CommandOptionsformeg_tsa_dump 330
Table 92 CommandOptionsforaue_dump 331
Table 93 CommandOptionsforbsm_dump 332
Table 94 CommandOptionsforcsi_dump 333
Table 95 CommandOptionsforrcm_dump 334
Table 96 CommandOptionsforrci_dump 335
Table 97 CommandOptionsforwjt_dump 336
Table 98 CommandOptionsforgap_dump 337
Table 99 CommandOptionsforimp_dump 337
Table 100 CommandOptionsformci_dump 338
Table 101 CommandOptionsforisi_dump 339
Table 102 CommandOptionsforvlr_dump 341
Table 103 CommandOptionsformip_dump 342
Table 104 CommandOptionsformau_dump 343

20 ListofTables
Table 105 CommandOptionsformau_dump 345

Table 106 ListofAcronyms 347

ListofFigures 21
LISTOFFIGURES
Figure 1 CoreVoiceNetworkDocumentMap 26
Figure 2 ElementandNetworkManagementStructure 35
Figure 3 NEOperationsandAdministrationDomains 37
Figure 4 OMCConsole 42
Figure 5 OMCManaginganHLRandAccessServer/MRF 43
Figure 6 OperatorTaskFlowchart 50
Figure 7 ActiveAlarmsDisplay 52
Figure 8 cpu_stopprompts 60
Figure 9 FrontPanel 62
Figure 10 ShutdownOutput 62
Figure 11 LOMStartupOutput 64
Figure 12 NEUserAccounts 68
Figure 13 TerminalEmulator 72
Figure 14 ArchitectureforSerialConnection 73
Figure 15 DirectSerialConnection 75
Figure 16 SerialCableConnectors 76
Figure 17 HighAvailabilityPhysicalView 88
Figure 18 UpLinkImplementationforHighAvailabilityConfiguration 90
Figure 19 NodeFailoverDecisionTree 100
Figure 20 GraphsPrintedWYSIWYGFormat 111

Figure 21 GeneratedPerformanceReport 113
Figure 22 TextImportWizard 115
Figure 23 CPUUtlizationGraph 116
Figure 24 MemoryUtilizationGraph 119
Figure 25 SwapUtilizationGraph 121
Figure 26 DiskI/OGraph 122
Figure 27 NetworkI/OGraph 124
Figure 28 SANTPDeploymentScenario 131
Figure 29 HANTPDeploymentScenario 132
Figure 30 PartitionsinMirroredDiskConfiguration 149
Figure 31 FlowchartforRemoteTapeSystemSetup 155
Figure 32 FlowchartforClientSystemSetup 157
Figure 33 FlowchartforTapeBackup 161
Figure 34 SoftwareMirroredDisksandPartitionsforOSandApplication 167
Figure 35 SoftwareMirroredDisksforOracleandHRLSubscriberDatabases 167
Figure 36 FlowchartforRestoringTwoDiskBackupfromTape 178
Figure 37 ActiveAlarmsDisplay 207
Figure 38 HistoricalEventLogRecords 220
Figure 39 ListingTraceableModulesonanSSVR 236
Figure 40 ListingTraceableProcesses 237
Figure 41 ListingTraceableResources 238
Figure 42 CheckingActiveTraceStatus 239
Figure 43 SoftwareMaintenanceReleaseTypesandFlow 252
Figure 44 FrontofSUNNetra240ShowingBezelFingerHolds 271
Figure 45 FrontofSUNNetra440ShowingtheSystemDoorandAirFilter 272
Figure 46 CounterNamingSyntax 276


ABOUTTHISGUIDE
Thisguidecoversthecommonproceduresthatapplytooperating,maintaining,
andprovisioningthefollowingCoreVoiceNetworkelements:
SonataSoftExchangeMSCServer
SonataFeatureServerHLR/AuC
SonataFeatureServerSignalingServer(SSVR)
SonataSoftExchangeAccessServer(MRF:MediaResourceFunctionandLIR:
LawfulInterceptMediaRouter)
SonataSoftExchangeLawfulInterceptServer(LIS)
TheCallConferenceServerandIntelligentMediaGateway(IMG)donotshare
commonprocedureswiththeaboveNEs.
TheNetworkElement(NE)guidescovertheNEspecificprocedures.Referto
RelatedDocumentationonpagepage 25.

Thisguideisintendedfornetworkarchitects,designers,andoperationsand
supportengineersresponsibleforthedesign,development,operations,and
maintenanceofwirelessvoiceanddatacommunicationsnetworks.Prior
experiencewithStarSolutionsproductsishelpfulbutnotrequired.Theguide
assumesitsreadershaveathoroughunderstandingoftelecommunications,
InternetProtocol(IP)telephony,andtheSS7andSIGTRANprotocols.Thisguide
alsoassumesitsreadershaveadequateknowledgeoftheexistingnetworkand
systemadministratorprivileges.
TheAboutThisGuidesectionincludes:
Conventions
RelatedDocumentation
TechnicalSupport
WarrantySupport
ContactingTechnicalDocumentation
Releasenotesareissuedwithsomeproducts.Iftheinformationintherelease
notesdiffersfromtheinformationinthisguide,followtheinstructionsinthe
releasenotes.

24 AboutThisGuide
Conventions Thisguidemaycontainnotices,figures,screencaptures,andcertaintext
conventions.
Notices Table 1listsnoticeiconsusedinthisguide.
Table 1NoticeIconDescriptions
Icon NoticeType Description

InformationNote Informationthatcontainsimportantfeaturesor
instructionsbutisnothazardrelated.
Cautionor CautionsareprecededwiththewordCaution.This
Warning typeofcautionindicatesapotentiallyhazardous
situationwhich,ifnotavoided,mayresultinminor
Cautionor ormoderateinjury.Itmayalsoalertagainstunsafe
Warningdueto practicesandpotentialprogram,data,system,or
potential devicedamage.
electricalhazard WarningsareprecededwiththewordWarning.This
typeofwarningindicatesapotentiallyhazardous
situationwhich,ifnotavoided,couldresultindeath
orseriousinjury.
Warningsduetopotentialelectricalhazardsare
precededwiththewordWarning.Thistypeof
warningindicatesapotentialelectricalhazard.This
hazard,ifnotavoided,couldresultindeathor
seriousinjury.
ESD Informationthatindicatespropergrounding
precautionsarerequiredbeforehandlingaproduct.

FiguresandScreenCaptures Thisguideprovidesfiguresandscreencapturesasexamples.Theseexamples
containsampledata.Thisdatamayvaryfromtheactualdataonaninstalled
system.
Text Table 2liststextconventionsinthisguide.
Table 2TextConventionDescriptions
Convention Description
Textrepresentedasscreen Thistypefacerepresentstextthatappearsona
display terminalscreen,forexample:login:.
Textrepresentedasuser Thistypefacerepresentscommandsenteredbythe
entry. user,forexample,cd$HOME.
Textrepresentedasmenus, Thistypefacerepresentsallmenus,submenus,buttons,
submenus,buttons,tabs, tabs,directories,andfieldnameswithinprocedures,for
directories,andfieldnames example:
OntheFilemenu,clickNew.
Textrepresentedby Thistypefacerepresentsarequiredvariable,for
<variable> example:<filename>

RelatedDocumentation Figure 1mapstherelationsamongthedocumentsintheStarSolutionsSonata

CoreVoiceNetworkdocumentationsetandsuggestsanorderinwhichtoread
them.Forexample,tooperateandmaintaintheSignalingServer(SSVR),the
documentmapsuggestsreadingthedocumentsinthefollowingorder:
1 IS41orGSMSystemOverviewGuide
2 IS41orGSM,andOMCReleaseNotes(BaseLevelandanyPatchLevel)
3 CoreVoiceNetworkInstallationandInitialConfigurationGuideorcPCISystem
InstallationandInitialConfigurationGuide(initialinstallonly)
4 OMCServerGuide(initialinstallonly)
5 OMCUserInterfaceGuide
6 CoreVoiceNetworkOperations,Maintenance,andProvisioningGuide
7 MSCServerProvisioningGuideandMSCServerOperationsandMaintenance
Guide,orHLR/AuthenticationCenterGuide
8 SignalingServerGuide

26 AboutThisGuide
Figure 1CoreVoiceNetworkDocumentMap
IS41R4.5.4SystemOverviewGuide
GSMR4.5.4SystemOverviewGuide
SonataCoreVoiceNetwork SonataCoreVoiceNetwork
IS41R4.5.4BaseLevelReleaseNotes
SonataCoreVoiceNetwork IS41R4.5.4PatchLevelReleaseNotes
GSMR4.5.4BaseLevelReleaseNotes GSMR4.5.4PatchLevelReleaseNotes
SonataOperationsandMaintenanceCenter SonataOperationsandMaintenanceCenter
OMCR4.5.4BaseLevelReleaseNotes OMCR4.5.4PatchLevelReleaseNotes
InstallationandInitialConfigurationGuide
cPCISystemInstallationandInitCfgGuide
SonataOperationsandMaintenanceCenter
OMCR4.5.4ServerGuide
SonataOperationsandMaintenanceCenter
OMCR4.5.4UserInterfaceGuide
IS41/GSMR4.5.4Ops,Maint&ProvisioningGuide

SonataSoftExchange
IS41R4.5.4MSCServerProvisioningGuide
SonataSoftExchange
GSMR4.5.4MSCServerProvisioningGuide SonataFeatureServer
IS41R4.5.4HLR/ACGuide
SonataFeatureServer
GSMR4.5.4HLR/AuCGuide
SonataSoftExchange
IS41R4.5.4MSCServerOps&MaintGuide
SonataSoftExchange
GSMR4.5.4MSCServerOps&MaintGuide
SonataSoftExchange
IS41/GSMR4.5.4AccessServer(MRF)Guide SonataFeatureServer
IS41/GSMR4.5.4SignalingServerGuide
SonataSoftExchange
IS41/GSMR4.5.4LawfulInterceptGuide
Thisguide
SonataSoftExchange
IS41/GSMR4.5.4CallConferenceServGuide GuidesforIS41andGSMsystems
GuidesforGSMsystems
SonataSoftExchange
IS41/GSMR4.5.4Media/SignalingGwGuide GuidesforIS41systems

TheCoreVoiceNetworkOperations,Maintenance,andProvisioningGuide(this
guide)andtheOMCUserInterfaceGuidecontaincommonproceduresand
informationforallCoreVoiceNetworknodesexcepttheCallConferenceServer
andMedia/SignalingGateway.ReadtheCoreOMPGuidepriortoreadingthe
nodespecificguides.
Theothernodespecificguidesare:
MSCServerProvisioningGuide,whichcontainstheMSCspecifictablesand
howtoprovisionthem.
MSCServerOperationsandMaintenanceGuide,whichcontainsthe
MSCspecificproceduresforoperatingandmaintainingtheMSCServer,such
asproceduresrelatedtoupperlayercallprocessingprotocols,CallDetail
Records(CDR),andMSCspecificalarms.
HLRandAuthenticationCenterGuide,whichcontainsallHLRspecific
provisioning,operationsandmaintenanceprocedures,andHLRspecific
alarmsetc.
MRFGuide,whichcontainsadescriptionofthetonesandannouncements
availabletotheMSC,adescriptionoftheMediaResourceFunction(MRF),
andoperations,maintenance,andprovisioningproceduresspecifictothe
AccessServer.
LawfulInterceptGuide,whichcontainsadescriptionoftheLawfulIntercept
service,thecomponentsthatareinvolvedtheservice,andtheoperations,

maintenance,andprovisioningspecifictotheLawfulInterceptServerand
service.
CallConferenceServerGuide,whichcontainsadescriptionoftheCall
ConferenceServer,adescriptionofthethreewaycallfunctionality,and
operations,maintenance,andprovisioningproceduresspecifictotheCall
ConferenceServer.
Media/SignalingGatewayServerOperations,Maintenance,andProvisioning
Guide,whichcontainsthespecificproceduresforoperatingandmaintaining
theMGW/SGW.
TheMedia/SignalingGatewayServerandCallConferenceServerdonotusethe
OMCorthecommonoperationsandproceduresthattheothercorenodesuse.

28 AboutThisGuide
TechnicalSupport TheStarSolutionsProductSupportTeamdeliversthesupportservicesrequired
forbusinessandprofessionalneeds.OurproductexpertsdeliverTier1,2and3
technicalsupportdirectlytonewandcontractentitledcustomersincludingthe
followingservices:
BasicSupportPackage:Nonemergencytechnicalsupport
PremiumSupportPackage:24hoursaday,7daysaweek,and365daysayear
Emergencytechnicalsupport
TheStarSolutionsServiceGuideoutlinesthespecificdetailsforobtaining
technicalsupport.Theguideisavailablefromasalesaccountmanager.Referto
theServiceGuideforservicesandoptionsspecifictoindividualsupportplans,
includingguidelinesforproblemseverityandthetechnicalresolutionescalation
process.
ObtainingTechnical StarSolutionsmaintainsaglobalpresencethroughitsTechnicalResponseand
Assistance ServiceCenters.Thesecentersareavailablefortechnicaltelephonesupportto
entitledcustomersduringnormalbusinesshours.
Beforecontactingtechnicalsupport,havethisinformationavailable:
Productinformation
Softwareandhardwarerevisions
Serialnumbers

Problemdescription
Symptoms
Knowncauses
Troublelocatingandclearingattempts.
Forinformationaboutcustomerservice,includingsupport,training,code
releasesandupdates,contracts,anddocumentation,contactusat:
http://www.starsolutions.com/support
ServiceCentersOperationalHours
NorthAmerica/CALARegion:
09:0018:00PacificTime(UTC8:00)
EMEA/AsiaPacificRegion:
09:3018:30IndianStandardTime(UTC+5:30)

WarrantySupport 29
WarrantySupport StarSolutionsprovidesitscustomerswarrantysupportperthetermsoftheStar
SolutionsWarrantyStatementfortheirequipment.Customerswhorequire
warrantysupportshouldcontacttheStarSolutionsCustomerServiceCenteras
specifiedinthecustomerserviceguideorat:
http://www.starsolutions.com/servicesupport
ContactingTechnical Toprovidecommentsonthisdocumentation,sendanemailto:
Documentation techcom@starsolutions.com
Pleaseincludethenameandpartnumberoftheguidebeingreferenced.If
applicable,providethechapterandpagenumber.


OVERVIEW
I
Chapter 1 Overview

OVERVIEW
1
Thischaptercontainsanoverviewofthecommonoperations,maintenance,and
provisioningtopicsfortheCoreVoiceNetworkelementsoftheStarSolutions
Sonatasystem.
Thischapterincludes:
Overview:Provisioning,Operating,andMaintainingtheCoreVoiceNetwork
ElementandNetworkManagement
DomainsandRoles
Overview:Provisioning, Therearecommonproceduresthatapplytoprovisioning,operating,and
Operating,andMaintaining maintainingtheSonataCoreVoiceNetworkelements.
theCoreVoiceNetwork
TheoperatorusestheSonataOperationsandMaintenanceCenter(OMC)
ConsoletoprovisiontheCoreVoiceNetworkelementtables(exceptfortheCall
ConferenceServerandMedia/SignalingGateway).TheCommandLineInterface
(CLI)isavailableasabackupmethodintheeventthattheOMCConsoleisnot
available.
TheoperatoralsousestheOMCConsoletoperformmanyoftheoperationsand
maintenancetasksontheCoreVoiceNetworkelements(exceptfortheCall
ConferenceServerandMedia/SignalingGateway).Otherproceduresinvolve
commandlinetasksusingeithertheUNIXcommandlineortheCLI.
Thisguidecontainsfoursections:
Overview
Provisioning,whichincludes:
Chapter 2,ProvisioningtheCoreVoiceNetworkElements
OperationsandAdministration,whichincludes:
Chapter 3,BasicOperations
Chapter 4,BasicAdministration
Chapter 5,UserAccounts
Chapter 6,Filesystems
Chapter 7,ActiveStandbyOperations
Chapter 8,PerformanceMonitoringandManagement
Chapter 9,SystemTimeManagement

34 Chapter1:Overview
Chapter 10,DatabaseandSystemBackupandRestore
Maintenance,whichincludes:
Chapter 11,MaintenanceOverview
Chapter 12,AlarmManagement
Chapter 13,ViewingLogsandEvents
Chapter 14,TroubleLocatingandClearing
Chapter 15,SoftwareMaintenance
Chapter 16,HardwareMaintenance
ElementandNetwork TheMSC,HLR,SignalingServer(SSVR),AccessServer(MRF),LawfulIntercept
Management Server,CallConferenceServer,andMedia/SignalingGateway(MG/SG)are
NetworkElements(NEs)intheCoreVoiceNetwork.Thisguidecoversthe
followingNetworkElementssincetheyarebuiltonacommonplatformandare
managedbytheOperationandMaintenanceCenter(OMC)andtheCommand
LineInterface(CLI):
MSC
HLR
SSVR
AccessServer(MRF)
LawfulInterceptServer
BoththeOMCandtheCLIareinterfacestoFault,Configuration,Accounting,
Performance,andSecurity(FCAPS)functionalityintheNEs,andareboth
NetworkElementManagers.TheCLIisstrictlyforElementManagement.
TheOMCalsomanagesNEsintheAccessNetwork,soitalsoactsasa
SubNetworkManagerfortheCoreandAccessNetworks.
SincetheOMCmanagesmorethanoneSubNetwork,theOMCalsoactsasa
NetworkManagementSystem.WhendiscussedinthisguidetheOMCisa
NetworkElementManager.
Figure 2showsthestructureofElementManagementandNetwork
ManagementusingtheOMC.

DomainsandRoles 35
Figure 2ElementandNetworkManagementStructure
ThegraphicalOMCConsoleisintendedforusebyOperators.AnAdministrator
canuseboththeOMCandtheCLItomanagetheNEs.
DomainsandRoles Adomainisanarea(hardwareandsoftware)whereauthorityorcontrolis
exercised.Aroleisthesumoftheactionsandactivitiesassignedto,requiredof,
andexpectedof,aperson.
ThissectiondescribestheNEdomainsandtheroles,toolsandskillsetrequiredto
operateandadministertheNEs.
TheNEscontainanoperationsdomainandanadministrationdomain.The
OperatormanagestheoperationsdomainandtheAdministratormanagesthe
administrationdomain.TheroleofOperatorisdescribedinOperatorRoleon
page 37,andtheroleofAdministratorisdescribedinAdministratorRoleon
page 37.
MaintenanceoftheNEcrossesintoboththeoperationsandadministration
domains;however,itisusuallyperformedbyaseparatemaintenancerole,
workinginconcertwiththeAdministrator.Maintenanceiscoveredinthe
Maintenancesectionofthisguide.
OperationsDomain Operationsconsistsofroutineactivitiestomonitorandcontrolthesystem.This
includes:
routinefault,alarm,event,performance,andusagemonitoring
routineconfigurationchanges
initialfaultmanagement.

Forexample,performanceandusagemonitoringincludes:
checkingCPUandmemoryutilization
checkingfreediskspace
checkinglogfilesize
ManyofthesetasksareperformedbytheOperatoraspartofthedailyshift(see
OperatorTasksonpage 51).
AdministrationDomain TheAdministrationDomainincludestheentiresystem(OperatingSystemandNE
application)whereastheOperationsDomainincludestheNEapplicationonly.
Thedomainsdifferinthatwhileboththeoperationsandadministrationdomains
canmanagetheNEremotely,onlytheadministrationdomainshouldmanagethe
NElocally.TheNetworkOperatingCenter(NOC)usuallymanagestheNE
remotely.
Thedomainsalsodifferinthetypesofmanagementtasksthatareperformed.
Themostcommonmanagementtasksintheoperationsdomainare:
Monitoringalarms
Monitoringevents
Monitoringsystemperformance
Themostcommonmanagementtasksintheadministrationdomainare:
Faultmanagement
Performancemanagement
Configurationmanagement
Figure 3showstheremoteandlocalaspectsofeachdomaintotheNE,and
showswheretheOperatorandAdministratortypicallywork.

DomainsandRoles 37
Figure 3NEOperationsandAdministrationDomains
OperatorRole TheOperatorrole,orfunction,includesallofthetasksoutlinedintheoperations
sectionofthisguideplusanyotherfunctionsrequiredbyanemployer.
AdministratorRole TheAdministratorperformsmaintenancetasksasoutlinedinthemaintenance
sectionofthisguide,toensure:
Servicesareperformed
Thenetworkisusedefficiently
Prescribedservicequalityobjectivesaremet

RolePrerequisites ThissectiondescribestheknowledgeandskillsthatanOperatorand
AdministratorneedtohaveinordertoproperlyoperatetheNEandrelated
components.
GeneralKnowledge
TheOperatorandAdministratorshouldhaveagoodknowledgeofgeneral
telecommunicationsfundamentals.KnowledgeoftheNEarchitectureis
beneficial.
TheAdministratorshouldalsohaveaverygoodknowledgeoftroubleshooting
techniques.
UNIXKnowledge
TheOperatorrequiresbasicUNIXknowledge.Thisincludes:
SomeunderstandingoftheUNIXfilestructure
Loggingin/out
Navigatingthefilestructure
IssuingUNIXcommands.
TheAdministratorrequiresanintermediateknowledgelevelofUNIX.
ProductKnowledge
TheMSCServerisdesignedtodeployinnumerousconfigurationsandwith
differentcolocatedNetworkElements,dependingonthecustomer
requirements.
TheOperatoronlyneedstohaveMSCproductknowledgeandbasicknowledge
abouttheotheroperatormanagedcomponents(Ethernetswitchandterminal
server).
TheAdministratorneedstohaveadvancedproductknowlegeandintermediate
knowledgeabouttheotheroperatormanagedcomponents(Ethernetswitchand
terminalserver).
DocumentationFamiliarityandTraining
TheOperatorshouldbefamiliarwiththeNEdocumentationsetbeforeoperating
theNE.SeeRelatedDocumentationonpage 25foradditionalinformation.
TheOperatorandAdministratorshouldattendanyStarSolutionstrainingcourses
directedatNEOperators.TrainingcoursesusedocumentsintheNE
documentationsetastoolsandreferences.

PROVISIONING
II
Chapter 2 ProvisioningtheCoreVoiceNetworkElements

PROVISIONINGTHECOREVOICENETWORK
2 ELEMENTS
ThischapterprovidesanoverviewofprovisiongCoreVoiceNetworkElements.
ProvisioningisusuallyperformedusingtheOMCconsole.Thischapteralso
coverstheCommandLineInterface(CLI)asabackupprovisioningmethod.
ProvisioningOverview
UsingtheOMCConsole
NetworkElementsontheOMC
UsingtheCommandLineInterface
ProvisioningOverview DatarequiredbytheCoreVoiceNEs(exceptfortheIMGandCallConference
Server)isviewedandprovisionedusingtablesdisplayedontheOMC.
TheOMCUserInterfaceGuidecontainsallofthecoreproceduresfor
provisioningtheCoreVoiceNEs.
ForNEspecificdatarequirementsandprovisioning,refertotheappropriate
guidefromthefollowinglist:
MSCServerProvisioningGuide
HLRandAuthenticationServerGuide
SignalingServerApplicationGuide
AccessServer(MRF)Guide
LawfulInterceptServerGuide
CallConferenceServerGuide
UsingtheOMCConsole TheOperationsandMaintenanceCenter(OMC)runsonaseparateserver.
TheOMC:
Managesconfiguration
Gathersandstoresperformanceinformation
Gathersandstoresalarmrecords
Gathersandstoresaccountrecords
ThissectiondescribesbasicaccesstotheOMCConsole.RefertotheOMCUser
InterfaceGuideforadditionalinformationonhowtousetheOMCConsole.

42 Chapter2:ProvisioningtheCoreVoiceNetworkElements
AccessingtheOMCConsole ToaccesstheOMCConsoleuserinterface:
1 StarttheOMCConsoleusingawebbrowser.TheURLshouldbeofthefollowing
pattern:
http://omchostname:6688/omc/omc.html
TheOMCUserInterfaceGuidedescribesinstallingandoperatingtheOMC
Console.
2 ClickLaunchOMCConsoleandthenlogintoopentheOMCConsoleuser
interface.
ThefirsttimetheOMCConsoleislaunched,JavaWebStartisinstalled.Referto
theOMCUserInterfaceGuideforadditionalinformation.
Figure 4showsanexampleoftheOMCConsole.
Figure 4OMCConsole

NetworkElementsontheOMC 43
NetworkElementsonthe OntheOMC,theCoreVoiceNetworkelements(orobjects)arenamed
OMC appropriately,withdifferenticonsforeachtypeofnode.Figure 5showsanOMC
ConsolemanaginganHLR(GSMHLRCONX4_5=orange)andanAccessServer/
MRF(GSMMRFCONX4_5=lithium).
Figure 5OMCManaginganHLRandAccessServer/MRF
Expandingthenodeicon,severalsoftwareobjectswithfoldericonsappear.An
exampleforanexpandedAccessServer/MRFisshowninFigure 5.Refertothe
NEspecificguidesforadditionalinformationonthesoftwareobjectsspecificto
eachNE.
UsingtheCommandLine Usethecommandlineinterface(CLI)to:
Interface
QuerytableinformationontheNE
AddtableinformationontheNE
DeletetableinformationontheNE
ModifytableinformationontheNE
Warning:
UsetheCLIonlywhentheOMCisnotavailableandonlyunderthesupervisionof
StarSolutionspersonnel.
CLIOverview ToaccesstheCLI,logintotheNEapplicationaccount.Togettotheloginprompt,
dooneofthefollowing:
ConnectamonitorandkeyboardtotheNE
ConnectanullmodemcablebetweenanNEserialportandanother
computer,anduseaterminalprogramonthecomputer
TelnettotheNE

RefertoConnectingtotheNEinChapter 5,UserAccountsforadditional
information.
UsetheCLItoaccesstheMSCdatabasetables(locatedinthe/telos/msc/DATA
folder).TheNEsusethedatabasetablesto,forexample:
ListalltheannouncementsandtonesthattheMSCcanplay
Definethemediaconfiguration,
DefinetheNE'soperationalparameters.
Beforerunningtheseprocedures,makesuretheNEsoftwareisinstalledand
running.
AccessingtheCLI ToaccesstheCLI:
1 LogontotheNEastheNEapplicationuser.
RefertoLoggingInandOutinChapter 5,UserAccounts.
2 Atthecommandprompt,enterthefollowingcommand:
cli
TheCLI>promptappears.
TheCLIiscasesensitive.Alwaysuselowercasewhenenteringcommands.
3 TostoptheCLI,enterthefollowingcommand:
exit
CLICommandSyntax CLIcommandsconsistofthefollowingelements:
commandtheactiontoperform
tablenamethenameofthetablebeingaccessed
keyauniqueidentifierforeachrowinthetable
keyvaluethevalueoftheuniqueidentifier
parameterafieldinthetable
parametervaluethevalueoftheparameter

UsingtheCommandLineInterface 45
Thecommandsyntaxvariesdependingonthetypeofcommandissued.Table 3
liststhecommandsandtherelatedsyntax:
Table 3GenericCLISyntax
Command Description Syntax Example

add Createsnewrecords add<tablename> CLI>addsnaprocess
inthetable. Thesystempromptsforthetable
data.
change Replacesthe change<tablename><key> CLI>changesnaprocessprocess_nm=
contentsofthe =<keyvalue>to MRPsap_id=0tosap_id=15
fieldsinonerecord. <parameter>=<parameter
value>
copy Createsanew copy<tablename><key>= CLI>copytransclass=1toclass=2
recordbycopying <keyvalue>to<key>=
thecontentsofan <keyvalue>
exitingrecord. Notalltablessupportthecopy
command.
delete Removesrecords delete<tablename><key> CLI>deletesnaprocessprocess_nm=
fromthetable. =<keyvalue> MRP
query Retrievesthe query<tablename>% CLI>querysnaprocess%
contentsofthe query<tablename> (returnsalltablerows)
specifiedtableand <parameter>=<parameter CLI>querysnaprocessprocess_nm=MRP
returnseitherthe value> (returnsallrowswhereprocess_nmisMRP)
fulltablecontentsor
aspecificrecord.
report Retrievesthesame report<tablename>% CLI>reportsnaprocess%
informationasthe report<tablename> (returnsalltablerows)
querycommand, <parameter>=<parameter CLI>reportsnaprocessprocess_nm=
plussummary value> MRP
information. (returnsallrowswhereprocess_nmisMRP)
verify Reportsanyinvalid verify<tablename> CLI>verifysnaprocess
entriesinthetable.
cancel Abortsthecurrent cancel CLI>querysnaprocesscancel
CLIcommandand
returnstotheCLI
prompt.
exit ExitstheCLIand exit CLI>exit
returnstothemsc quit CLI>quit
commandprompt.
Ifapartialcommandisentered(forexample,acommandwithoutparameters),
theCLIliststheparametersneededtocompletethecommand.Partial
commandsareusefultodisplaythecorrectdatabasetablenameorfieldname.If
thecommandisinvalid(forexample,misspelled),thesystemreturnsanerror.
CancelingaCLICommand TocancelaCLIcommand,entercancel.ThesystemreturnstotheCLIprompt.

NavigatingDisplayed Ifthecommandresultdisplaysmorelinesofdatathancanbedisplayedonthe
Information screenatonetime,thefollowingpromptappears:
Line1/?Presshforhelp,qtoquit
where:
Line#/indicateswhichlineofthequeryresultsisatthetopofthescreen
?Indicatestherearemorelinestobeviewed
Table 4liststhenavigationoptions:
Table 4CLINavigationKeys
Pressthiskey Todothis
Enter Scrolltheinformationforwardbyoneline
Spacebar Scrolltheinformationforwardbyonefullscreen
Homeorg Displaythefirstscreenofinformation
EndorG Displaythelastscreenofinformation
PageUporb Displaythepreviousscreen
PageDownorSpacebar Displaythenextscreenofinformation
Ctrlg Displaythefilenameandlinenumbers
jorEnter Scrolltheinformationforwardbyoneline
k Scrolltheinformationbackbyoneline
r Refreshthescreen
/textstring Searchtheresultsforwardforamatchingline
?textstring Searchtheresultsbackwardsforthematchingline
n Repeattheprevioussearch
N Repeattheprevioussearchinthereversedirection
Ctrlc Stoporinterruptthedisplayofresults
q ExittheCLIcommand
h DisplaytheCLIhelpscreen

OPERATIONS ANDADMINISTRATION
III
Chapter 3 BasicOperations
Chapter 4 BasicAdministration
Chapter 5 UserAccounts
Chapter 6 Filesystems
Chapter 7 ActiveStandbyOperations
Chapter 8 PerformanceMonitoringandManagement
Chapter 9 SystemTimeManagement
Chapter 10 DatabaseandSystemBackupandRestore

BASICOPERATIONS
3
ThischapteroutlinesthebasicCoreVoiceNetworkElementoperations.This
chapterisdesignedfortheOperatorandAdministrator.TheAdministrator'sextra
tasksarecoveredinChapter 4,BasicAdministration.
OperationsandMaintenanceCenter
OperatingTasks
OperatorTasks
Operationsand TheOperationsandMaintenanceCenter(OMC)runseitherstandaloneorona
MaintenanceCenter FileServer.Thesoftwareismadeupoftwocomponents:OMCServerandOMC
Console.TheOMCServerrunsthesameoperatingsystemastheCoreVoice
NetworkElements.TheOMCConsoleisaJavabasedapplicationthatcanrunon
bothSolarisandWindowsplatforms.
TheOMC:
Managesconfiguration(refertoChapter 2,ProvisioningtheCoreVoice
NetworkElements)
Gathersandstoresperformanceinformation
Gathersandstoresalarmrecords
Gathersandstoresaccountrecords
OMCServer TheOMCServerconsistsofthefollowingcomponents:
WebServersoftwareuploadstheOMCConsoleeachtimeitislaunched
fromtheclient'swebbrowser.
OMCServersoftwarecollects,forwards,andvalidatesdatabeingexchanged
betweennetworkelementsandtheOMCClient.
LightweightDirectoryApplicationProtocol(LDAP)Serversoftwarestores
configurationdata.
OMCConsole TheOMCConsoleistheuserinterfacetotheOMC.Nomanualinstallationof
softwareisrequiredfortheOMCConsole.TheOMCConsoleisdownloadedfrom
theOMCServerthroughawebbrowser.Itisinstalledthefirsttimeyouaccess
thesoftware.Theconnectionsbetweennetworkentitiesusestandard
communicationsprotocols.

50 Chapter3:BasicOperations
RefertoUsingtheOMCConsoleinChapter 2,ProvisioningtheCoreVoice
NetworkElements)forhowtostarttheOMCConsole.
RefertotheOMCUserInterfaceGuideforadditionalinformationonhowtouse
theOMCConsole.
OperatingTasks TaskandShiftOverview
TheflowchartinFigure 6showsthetasksthatanOperatortypicallyperforms
duringeachshiftforeveryNE.Thefollowingsectionsdescribeeachtaskshownin
theflowchart.
Figure 6OperatorTaskFlowchart
Shift Documentation
and Handover
Monitor Alarms
Check CPU Utilization
Check Memory
Utilization
Check Disk Space
Check Log Usage
Check Performance
Statistics
Shift Documentation
and Handover
D00506-122.gif

OperatorTasks 51
OperatorTasks ThissectiondescribestheoperatortasksoutlinedinFigure 6,OperatorTask

Flowchart,includesabriefprocedureforeachprocedure,andrefersto
subsequentchaptersforadditionalinformation.
Table 5providesatabletouseasanoperator'sshiftchecklist.Copythistable,or
createonetouseasashiftchecklistforoperators.Additionaltasksspecificto
eachNEaredescribedineachNEspecificguide.
Table 5OperatorShiftChecklist
Step Task Done Record/Comment

1 ShiftDocumentation
andHandover
2 MonitorAlarms PreviousUnresolvedActiveAlarms:
NewActiveAlarms:
3 CheckCPUUtilization
4 CheckMemory
Utilization
5 CheckDiskSpace Capacityrootslice(%):
Capacityapplicationslice(%):
6 Example1
8 CheckPerformance
Statistics
17 EndofShift
ShiftDocumentationand CollectandreviewtheshiftdocumentationfromthepreviousshiftOperator,and
Handover alsodothefollowing:
Reviewpreviousshiftdocumentationwiththesupervisor.
Getfamiliarwithanyknownoractiveissueinthenetwork.
Getfamiliarwithanyconfigurationchangesoradditionstothenetwork.
Monitoranyactivealarms.
Reviewanyothershiftloginformation.
PreparenewOperatorShiftChecklistforcurrentshift.

RecordingFaults
TheOperatormaintainsashiftrecordofallfaultsreportedbythesystem.The
OperatorinformsNetworkOperatingCenter(NOC)personnelofthefaults
reportedduringtheshift.Theshiftrecordshouldincludeinformationasfollows:
Dateandtimeafaultwasreportedoractuallyoccurred
Thenatureofthereportedfault
Thereportedlocation
Thelocationofthefault,whenfound
Thefaultconditionfoundandthecorrectiveactiontaken
MonitorAlarms MonitortheactivealarmsontheActiveAlarmsDisplayontheOMCConsole.The
ActiveAlarmsDisplayshowsallactivealarmsforallmanagedNEs.
ToMonitorAlarms:
1 StarttheOMCConsole.
2 Fromthebrowsermenu,selectFaultandthenclickShowalarms.
ThewindowshowninFigure 7appears.
Figure 7ActiveAlarmsDisplay
3 ClicktheManagedObjectClasscolumnheadertosortthealarms.
4 UsethescrollbarstolocateallalarmsraisedonthetargetNE.
5 Reviewanyunresolvedalarms.
6 Reportanynewalarms(allseverities)toFaultManagementpersonnelfor
resolution.
7 RecordanyunresolvedalarmsinOperatorshiftChecklist.
8 KeeptheActiveAlarmsDisplaywindowopen.

OperatorTasks 53
CheckCPUUtilization ChecktheCPUprocessorutilizationatthebeginningofeachshift.Generatea
CPUUtilizationreportattheendoftheshiftandsaveit.
ToCheckCPUUtilization:
2 RightclicktheNEandselectPerformance,24HRealTimeGraph,andthenclick
CPUUtilization.
TheCPUUtilizationgraphisgenerated(SeeCheckingCPUUtilizationonpage 116
foradditionalinformation).
3 Scanfor,andreportanyprolongedperiodsofhighCPUutilization,andthenclose
thegraph.
ShorttermCPUutilizationspikesareexpected.
4 GeneratetheCPUUtilizationgraphperiodicallythroughouttheshiftandcheck
forhighCPUutilization.
5 GenerateaCPUUtilizationreportfortheentireperiodoftheshift.
SeeGeneratingandExportingPerformanceReportsonpage 112.
6 Savethereportintheperformancereportsdirectory.
7 RecordtheresultofthisactivityintheOperatorShiftChecklist.
CheckMemoryUtilization Checkthememoryutilizationatthebeginningofeachshift.GenerateaMemory
Utilizationreportattheendoftheshiftandsaveit.
ToCheckMemoryUtilization:
2 RightclicktheNEandselectPerformance,24HRealTimeGraph,andthenclick
MemoryUtilization.
TheMemoryUtilizationgraphisgenerated(SeeMemoryUtilizationGraphsand
Reportsonpage 119foradditionalinformation).
3 Scanfor,andreportanyprolongedperiodsofhighmemoryutilization,andthen
closethegraph.
4 GeneratetheMemoryUtilizationgraphperiodicallythroughouttheshiftand
checkforhighmemoryutilization.
5 GenerateaMemoryUtilizationreportfortheentireperiodoftheshift.
6 Savethereportintheperformancereportsdirectory.

CheckDiskSpace Checktheavailablediskspaceatthebeginningofeachshift.Recordtheavailable
diskspacefortherootandapplicationslicesintheshiftchecklist.
ToCheckDiskSpace:
1 LogontotheNEastheappropriateuser.
SeeConnectingUsingReachthroughonpage 76.
2 Checkavailablediskspacebyexecuting:
dfk
Therootslice(/dev/dsk/c1t0d0s0)containstheOperatingSystem,mountedon
/,andshouldnotexceed80%usedcapacity.
Theapplicationslice(/dev/dsk/c1t1d0s7)containstheNEapplication,
mountedon/export/home.Thereisasoftlinkfrom/export/hometo/telos.
Theusedcapacityshouldnotexceed80%.
Record%agecapacityontherootslice.
Record%agecapacityontheapplicationslice.
SeeExample1.
Example1CheckDiskSpace
pro>dfk
Filesystem kbytesused availcapacityMountedon
/dev/dsk/c1t0d0s0 81682373950049 413650649%/
/proc 0 0 00%/proc
fd 00 00%/dev/fd
mnttab 00 00%/etc/mnttab
swap112518328112518241%/var/run
swap112540242200112518241%/tmp
/dev/dsk/c1t1d0s7705925051989513678970673%/export/home
CheckLogUsage Checkalllogfiles(LOG*)in~/RUN.
Ausersessionshouldalreadybeestablished.Ifnot,seeConnectingUsing
Reachthroughonpage 76.
ToCheckLogFileSize:
1 Changetothelogfiledirectorybyexecuting:
cd~/RUN
2 Listlogfilesbyexecuting:
lsLOG*
SeeExample2.
Example2CheckLogUsage
pro>cd~/RUN
pro>lslLOG*
CheckPerformanceStatistics CheckingperformancestatisticsontheNEinvolvesgeneratingbothperformance
graphsandperformancereports.

OperatorTasks 55
Generateperformancegraphsandexaminethemforpeformanceanomaliesand
bringthemtotheattentionoftheAdministrator,ifnecessary.Exportthe
performancegraphstothereportstoragedirectorytohelpbuildaperformance
history.
Fordetailsonhowtogenerateandprintperformancegraphs,seeGenerating
andPrintingPerformanceGraphsonpage 111.Fordetailsonhowtogenerate
andexportperformancereports,seeGeneratingandExportingPerformance
Reportsonpage 112.
EndofShift PreparetheshiftdocumentationforthenextshiftOperator.Inparticular:
Makenoteofanyknownoractiveissueinthenetwork.
Makenoteofanyconfigurationornetworkchangeduringtheshift.
EnsuretheOperatorShiftChecklistiscompleted.

BASICADMINISTRATION
4
ThisChapterexplainshowtoperformbasicadministrationoftheCoreVoice
NetworkElements(NE).ThischapterisdesignedfortheAdministrator.The
AdministratorshouldalsobefamiliarwithallofthematerialinChapter 3,Basic
Operations.
BackgroundinformationonNEuseraccountsandfilesystemsisprovidedin
Chapter 5,UserAccountsandChapter 6,Filesystems.
Morecomplexadministrationtopicsarecoveredin:
Chapter 7,ActiveStandbyOperations
Chapter 8,PerformanceMonitoringandManagement
Chapter 9,SystemTimeManagement
Chapter 10,DatabaseandSystemBackupandRestore
MonitoringLogsandEvents
DisplayingtheNEApplicationStatus
StartingandStoppingtheNEApplication
StoppingtheOperatingSystem
RebootingtheNE
PoweringDowntheNE
PoweringUptheNE
TroubleLocatingandClearing
SystemAdministrationTools
MonitoringLogsandEvents TheNEsusevlog,aLogViewerutility,toviewrecordsofeventsinthesystem.
Eventlogshelpmonitorandtroubleshootthesystem,especiallyasitrelatesto
configurationchangesorerrorsinnetworkconfiguration.
TheNEsgenerateeventlogrecordsasaresultofbothnormalsystemactivities
andunusualsystembehavior.Theeventlogrecordsdisplayeventsforthe
historicalpast(upto20daysago),thecurrentday,andinrealtime.Inrealtime
format,vlogprintseventstostandardoutput(thescreen)oryoucanusevlogto
selectfromalistofhistoricaleventfiles.

58 Chapter4:BasicAdministration
Eventlogsareclassifiedbylevelofseverity.Severitylevelsrangefrom
informational(INFO),forwhichnooperatorinterventionisrequired,tosevere
serviceaffectingmalfunctions(FATALERROR).
RefertoChapter 13,ViewingLogsandEventsforinformationonrunningvlog
andviewinglogsandevents.
DisplayingtheNE Usethecpu_statuscommandtoverifythestatusoftheNEapplicationandnode.
ApplicationStatus Thiscommandisusefulafterstarting,stopping,orabortingtheapplication.
Tocheckapplicationstatus:
1 Loginasmsc,pro,lis,orfsv,asappropriatetotheNE.
2 Atthecommandprompt,typecpu_statusandpressEnter.
Thecpu_statuscommanddisplays:
Systemstatus(eitherUP,DOWN,orGOINGUP)
Systemmode(Masterforasimplexsystem,MasterorSlaveforan
activestandbysystem)
Timeanddatewhentheapplicationwasstarted
HowlongtheUNIXoperatingsystemhasbeenup
Numberofusersonthesystem
CPUloadaverage
AnexampleofoutputfromaSSVR,"sigserv1",isshownbelow:
pro.sigserv1.429>sigserv1:UPasMastersinceApr1406:29Unixup:15
min(s),1user,loadaverage:0.23,0.14,0.16hours
DisplayingtheSoftware Usethecpu_versioncommandtodisplaythecurrentapplicationversionrunning
Version ontheNE.
TodisplaytheNEapplicationsoftwareversion:
2 Atthecommandprompt,typecpu_versionandpresstheEnterkey.
StartingandStoppingthe TheNEapplicationstartsautomaticallywhenthehardwarecomponentspower
NEApplication onorwhentheoperatingsystemstartsorrestarts.
TheNEapplicationsrunontopoftheOperatingSystem.TheNEapplicationsmay
stopduetoanoperatorcommand(cpu_stop),orduetoahardwareorsoftware
fault.Thecpu_startcommandisusedtorestarttheNEapplication.
Asystemrestartisdistinctfromasystemreboot.Asystemrebootistheshutting
downandstartingbackuptheentireUNIXsystem.Asystemrebootisrarely
performed.Asystemrestartcanbefairlycommon.

StartingandStoppingtheNEApplication 59
Asystemrestarttakesapproximatelyaminute,afterwhichtheNEmustloadthe
networkconfigurationinformation.Thetimeittakestoloadthenetwork
configurationinformationvaries,dependingonthenumberandtypeofNEsand
thespecificnetworkdeploymentandconfiguration.
AnNEentersarestartcyclewheneverittriestostartup.TheNEwilltrytostart
upandreachastableoperatingstateamaximumofthreetimes.Astable
operatingstateisover120secondsofcontinuousruntime.IftheNEdoesnot
runfor120consecutivesecondsinthethreeattempts,thennorestartis
attempted.
IftheNEdoesreachthestableoperatingstate(120seconds),thentherestart
counterisreset.
ViewthestartupmessagesusingtheConsoleconnectedthroughaserialport
serverwhileitisstartingup,thenloginwhenyouseetheloginprompt
displayed.
Duringnormaloperation,theNEapplicationisalwaysrunning;however,the
adminstratormayneedtotostopandstarttheNEapplicationindependentlyto
investigatesystemproblems.ThissectiondescribeshowtostartandstoptheNE
application.
StoppingtheNEApplication TostoptheNEapplication:
2 Executethecpu_stopcommandtostoptheNEapplication.
Onaproductionsystemthecpu_stopcommandwillpromptformore
informationtologthereasonforstoppingtheapplicationandtomakesurethat
itwasnotaccidental.RefertoFigure 8foranexample.

Figure 8cpu_stopprompts
StartingtheNEApplication TostarttheNEapplication:
2 Typecpu_start,andpressEntertostarttheNEapplication.
3 WatchthestartupmessagesonthescreentoensurethattheNEapplication
startsup.
AbortingtheNEApplication Usethecpu_abortcommandtobypassthenormalshutdownproceduresand
stoptheNEapplicationsimmediately.
Warning:
Executingthecpu_abortcommandtostopanapplicationwillinterruptcall
traffic.Thiscommandisnottypicallyusedinproductnetworkenvironments.
ConsultwiththeStarSolutionscustomerserviceorganizationbeforeexecuting
thiscommand.
ToaborttheNEapplications:
2 Executethecpu_abortcommand.
Onaproductionsystemthecpu_abortcommandwillpromptformore
informationtologthereasonforabortingtheapplicationandtomakesurethat

StoppingtheOperatingSystem 61
itwasnotaccidental.Thepromptsarethesameasforthecpu_stopcommand.
RefertoFigure 8foranexample.
TheNEbeginsterminatingallsoftwareprocesses.Whentheabortiscomplete,
thecommandpromptdisplays.
StoppingtheOperating Tostoptheoperatingsystem,firststoptheNEapplications,thenuseoneofthe
System followingcommandstostoptheoperatingsystem:
ToexittheoperatingsystemandentertheOpenBootPROM,usethe
commandshutdowniOorinit0.
Toexittheoperatingsystemandpoweroffthesystem,usethecommand
shutdowni5orinit5.
Toimmediatelyrebootfromthedefaultkernel/UNIXandreturntointeractive
mode,usethecommandshutdowni6orinit6.
Toexittheoperatingsystemandentermaintenancemode,usethecommand
initsorinitS.
RebootingtheNE WhentheNEreboots,theoperatingsystemandNEapplicationsstopandstart
automatically.
TheadministratormayneedtoreboottheNEduringsomeadministrativetasks.
Theapplicationpromptstorebootattheappropriatetime.
ToreboottheNE:
1 Loginas,orswitchuserto,root.
2 Atthecommandprompt,enterinit6.
Afterabriefpause,asystemconsolemessageadvisesthatthesystemisdown
andthattheautomaticrestartprocessisbeginning.UNIXrestartstheoperating
system.NEapplicationsrestartautomatically.
PoweringDowntheNE AnNEmayrequirepoweringdownforsomemaintenanceactivities.The
followingprocedureperformsagracefulshutdownclosesfilesandstops
applicationsandtheoperatingsysteminthecorrectordertoavoidanydata
corruptionorloss.
ThisprocedurerequiresconsoleaccesstotheNEwhichisusuallyperformed
throughtheterminalserver.
ToPowerDowntheNE:
1 StoptheNEapplicationsusingtheprocedureStoppingtheNEApplicationon
page 59.
2 MakesurethebuttononthefrontpanelissettoNormal.RefertoFigure 9.

Figure 9FrontPanel
3 Accessthesystemconsole,forexample:
#telnetucits1903
4 Switchusertoroot:
suroot
5 Stoptheoperatingsystemfromthecommandlineprompt.Enter:
init5
6 RefertoFigure 10foranexampleoftheoutputfromtheinit5command.
Figure 10ShutdownOutput
INIT:Newrunlevel:5
Thesystemiscomingdown.Pleasewait.
Systemservicesarenowbeingstopped.
Printservicesstopped.
Mar1516:51:49sandiegosgwbsyslogd:goingdownonsignal15
Terminated
umount:/export/homebusy
Thesystemisdown.
syncingfilesystems...done
SCAlert:CRITICALALARMisset
SCAlert:Hostsystemhasshutdown.
SCAlert:CRITICALALARMisset
7 Waituntiltheserveriscompletelyshutdown.
8 Switchoffthebreakersanddisconnectthepower.
PoweringUptheNE 63
PoweringUptheNE PerformthisproceduretopoweronanNEthatwaspoweredoffusingthe
procedureinPoweringDowntheNE.
ToPowerUptheNE:
1 Accessthesystemconsole,forexample:
#telnetucits1903
2 MakesurethebuttononthefrontpanelissettoNormal.RefertoFigure 9.
3 Connectthepowerandswitchonthebreakers.TheLightsOffManagement
(LOM)systemontheNEwillstartautomatically.RefertoFigure 11foran
exampleoftheoutputfromtheLOM.

Figure 11LOMStartupOutput
ALOM BOOTMON v1.5.4
ALOM Build Release: 008
Reset register: e0000000 EHRS ESRS LLRS
ALOM POST 1.0
Dual Port Memory Test, PASSED.
TTY External - Internal Loopback Test
TTY External - Internal Loopback Test, PASSED.
TTYC - Internal Loopback Test
TTYC - Internal Loopback Test, PASSED.
TTYD - Internal Loopback Test
TTYD - Internal Loopback Test, PASSED.
Memory Data Lines Test
Memory Data Lines Test, PASSED.

Memory Address Lines Test
Slide address bits to test open address lines
Test for shorted address lines
Memory Address Lines Test, PASSED.
Memory Parity Test
Memory Parity Test, PASSED.
Boot Sector FLASH CRC Test
Boot Sector FLASH CRC Test, PASSED.
Return to Boot Monitor for Handshake
ALOM POST 1.0
Status = 00007fff
Returned from Boot Monitor and Handshake
Clearing Memory Cells
Memory Clean Complete
Loading the runtime image...
SC Alert: CRITICAL ALARM is set
SC Alert: SC System booted.
Sun(tm) Advanced Lights Out Manager 1.5.4 (sandiego-sgw-b)

PoweringUptheNE 65
Full VxDiag Tests

BASIC TOD TEST
Read the TOD Clock: THU MAR 15 22:40:12 2007
Wait, 1 - 3 seconds
Read the TOD Clock: THU MAR 15 22:40:14 2007
BASIC TOD TEST, PASSED
ETHERNET CPU LOOPBACK TEST
50 BYTE PACKET - a 0 in field of 1
's.
50 BYTE PACKET - a 1 in field of 0
's.
900 BYTE PACKET - pseudo-random data.
ETHERNET CPU LOOPBACK TEST, PASSED
Full VxDiag Tests - PASSED
Status summary - Status = 7FFF
VxDiag - - PASSED
POST - - PASSED
LOOPBACK - - PASSED
I2C - - PASSED
EPROM - - PASSED
FRU PROM - - PASSED
ETHERNET - - PASSED
MAIN CRC - - PASSED
BOOT CRC - - PASSED
TTYD - - PASSED
TTYC - - PASSED
MEMORY - - PASSED
MPC850 - - PASSED
Please login:
4 LogintotheLOMasadmin.
5 Pressthepowerbuttonorexecutethepoweroncommand.
6 Switchtoconsoletomonitorthestartupmessages:
sc>console
7 Enter#.toreturntotheLOM.
8 Monitorthestartupmessages.Waituntilthesystemhasstartedupcompletely.
9 Logintothesystemandverifythatallapplicationsarerunning.

TroubleLocatingand Thefirststepsintroublelocatingandclearingare:
Clearing
1 Reviewtheresultsoftheshift'sOperatorTasks(refertopage 51).
2 FollowtheproceduresinMonitoringLogsandEvents(refertopage 57).
3 FollowtheproceduresinDisplayingtheNEApplicationStatus(refertoprocedure
inthischapter)
Anumberofothertoolsareavailablefortroublelocatingandclearingafterthese
basicstepshavebeenperformed.Chapter 14,TroubleLocatingandClearing
discussesthesetools:
PacketCaptureandAnalysis
LoggingIssues
RecoveryfromOutageorDisaster
TraceManager
TraceManager
TroubleshootingHWD
SystemAdministration SometoolsthatarenotpartoftheOMCorCLIthatmaybeusefulforsystem
Tools administrationincludeUNIXcommandsthatareavailableintheSolarisUNIX
distribution.Thetoolsinclude:
Texteditor(suchasvioremacs)
Telnet
FTP
snoop

USERACCOUNTS
5
ThischapterexplainstheuseraccountsoneachNetworkElement(NE),howto
connectto,loginandoutoftheNE,andhowtochangepasswords.
NEUserAccounts
ConnectingtotheNE
LoggingInandOut
ChangingUserPasswords
NEUserAccounts UseraccountsaresetupontheNEduringinitialinstallation.Theseaccountsare
setupwithspecificenvironmentstoprovideuseraccesstosystemtools.The
accountsarealsousedbyinternalNEprocessestoperformautomaticand
ongoingactivitiesontheNE.
Warning:
Changingaccountnames,ordeletingaccountsontheNEwillseverelyimpactthe
operationoftheNE.Passwordsfortherootandproaccountscanbechanged
(seeChangingUserPasswords).
ThefollowinguseraccountsaresupportedonallNEs:
root:usedforsystemadministration,suchasbackup,reboot,andpoweroff
commands.
drum:usedformaintainingdataaccessedbytheOMC.DRUMistheOMC
proxythatrunsontheNEandisusedbytheOMCtosendandreceive
managementdata,includingoperatingsystemandapplicationperformance
files.
Inaddition,eachNEhasanapplicationuseraccountasfollows:
msc:OntheMSCandAccessServer,thisaccountmaintainstheapplications.
TheSignalingServerapplicationalsorunsunderthemscaccountwhenthe
MSCisoperatinginaSimplexconfigurationwiththefullSignalingstack.
lis:OntheLIS,thisaccountmaintainstheapplications.
pro:OntheMSCandSSVR,theSignalingServerapplicationrunsunderthe
proaccount.OntheSSVR,thisaccountmaintainsallapplications.
fsv:OntheHLR,theapplicationrunsunderthefsvaccount.

68 Chapter5:UserAccounts
Figure 12showstherelationshipbetweenusers,useraccounts,andNE
managementresponsibility.
Figure 12NEUserAccounts
SystemAdministration OnaUNIXsystem,superuserreferstoaprivilegedaccountthathasunrestricted
Account accesstoallfilesandcommands.TheuserIDofthesuperuseraccountisroot.A
personusingtherootaccountisasuperuser,andtheuseofthisaccountshould
berestrictedtosomeoneactingintheroleofSystemAdministrator.
Usetherootaccountonlywhenperformingtasksthatrequiretheprivileges
giventotheaccount.UsetheNEapplicationuseraccount(msc,pro,orfsv)for
allothertasks.
Table 6liststhegeneraltasksareasrequiringtherootaccount.
Table 6SystemAdministratorAccountandTaskMapping
Account Role GeneralTaskArea

root SystemAdministrator Mostsoftwareupgrades.
root SystemAdministrator Restartthesystem.
root SystemAdministrator Powerdownthesystem.
root SystemAdministrator Backupandrestoresystem.
root SystemAdministrator Setsystemtime.
Warning:
AsuperusercanaccidentallycrashorotherwisenegativelyimpactanyUnix
system.Usethesuperuserrootaccountwithcare,andonlywhenrequired.
MultiuserMode DuringnormalUNIXsysteminitialization,thesystembootsintomultiusermode.
Mostofthetime,thesystemwillbeinmultiusermode.

NEUserAccounts 69
SingleUserMode Occasionally,thesystemneedstobebootedintosingleusermodeforlowlevel
maintenancetasks.Singleusermodeisgenerallyusedtoperformaspecifictask
andthenthesystemisreturnedtomultiusermode.Singleusermodeisfortasks
wherecompleteandunsharedcontrolofthesystemisrequired,andis
sometimescalledmaintenancemode.
Asthenamesuggests,whenthesystemisinsingleusermode,otherusers
cannotlogin.Networkservicesareshutdownwhengoingintosingleusermode
soaconsoleconnectionisneededtooperateinthismode.
Warning:
OnlyentersingleusermodewhenconnectedtotheNEbyaconsoleconnection.
Warning:
TheNEapplicationmustbeshutdownbeforegoingtosingleusermode.Ifthisis
theactiveNE,ashutdownwillcauseafailovertothestandbyNE.
NEApplicationAccount TheNEapplicationuseraccount(msc,pro,orfsv)managestheSonata
applicationontheNE.BoththeOperatorandtheSystemAdministratorcanuse
theNEapplicationuseraccount,butonlytheSystemAdministratorshoulduse
therootaccount.
AsingleaccountmanagestheNEapplication,thereforeOperatorsusingtheNE
applicationuseraccountshouldrestricttheirmanagementactivitiestothe
generaltasksareasoutlinedforOperators.Thereis,however,nowaytopreventa
useroftheNEapplicationuseraccountfromperformingallactivitiespermitted
bytheNEapplicationuseraccountprivileges.
Table 7liststhegeneraltasksareasandrolesforNEapplicationaccountusers.
Table 7NEApplicationUserAccountandTaskMapping
Role GeneralTaskArea
Operator GeneralOperatortasks.
Operator FaultManagementactivities.
Operator Somesoftwareupgrades,mostlyatEmergency
SystemAdministrator Patchlevel.
Operator ExecutingCLIcommands.
SystemAdministrator
SystemAdministrator Backupandrestoredatabase.
SystemAdministrator ConfigurationManagementactivities.
SystemAdministrator StartandstoptheNEapplication.

UserManagement TherootandNEapplicationuserIDsanddefaultpasswordsaredefined
automaticallyduringinstallation.Table 8showstheuserIDsanddefault
passwords.
Warning:
Donotchangethedefaultaccountnames.
Table 8UserIDsandPasswords
UserID DefaultPassword Purpose

root telos Usedforsystemadministration,suchasbackup,
reboot,andpoweroffcommands.
msc msc UsedforconfiguringandmanagingtheMSC
application.
lis lis Usedforconfiguringandmanagingthelawful
interceptserverapplication.
pro pro UsedforconfiguringandmanagingtheSignaling
ServerApplicationProtocolService(CONX).
fsv fsv UsedforconfiguringandmanagingtheHLR
application.
drum drum UsedforOMCproxy.Notusedtologin.
Table 9UserIDsandTasks
UserID Tasks
root UNIXsystemadministration
Rebootthesystem
Powerdownthesystem
Backupthesystemimage
Setthedateandtime
Checkprocessorutilization
Upgradethesoftware
msc/pro/fsv StartandstoptheNEapplication
ChecktheNEstatus
ViewtheNElog

ConnectingtotheNE Thissectionincludes:
ConnectingUsingtelnet
ConnectingUsingaTerminalEmulator
ConnectingUsingaTerminalServer(Remote)
ConnectingUsingaTerminalServer(Local)
ConnectingUsingaDirectSerialConnection
ConnectingUsingReachthrough
IPAddressing UsetheappropriatemanagementIPaddresstoconnecttotheactiveorstandby
NE.UsethesharedIPaddresstoconnecttotheactiveNE,butnotethatifthe
systemfailsover,theconnectionisdropped.
ConnectingUsingtelnet TherearemanywaystoconnecttoanNEbuttheeasiestwayistosimplytelnet
totheNE.
IdentifytheNEhostnameorIPaddressandtheNEuseraccountpasswordbefore
attemptingtologintotheNE.
ToConnectUsingtelnet
1 Openaterminalwindoworcommandpromptandusetelnettoconnecttothe
NE.
>telnet<hostname>
2 LoginusingtheNEapplicationuserID(msc,pro,orfsv).
SeeExample3foranexampleloginsession.
Example3TelnetLoginSession
SunOS5.8
WelcometotheTelosTechnologySONATASESystem
login:pro
Password:
Lastlogin:FriAug514:57:13
WelcometotheSONATASESystem
from
TelosTechnology
Terminalsettoansi
pro>
ConnectingUsingaTerminal AterminalemulatorcanalsobeusedtotelnettotheNE.Manyterminal
Emulator emulatorsareavailableforWindows;checkthecomputertoseeifanemulatoris
alreadyinstalled.
ToConnectUsingaTerminalEmulator:
1 Launchtheterminalemulator.
SeeFigure 13.
2 EntertheconnectiondetailsfortheNEandthenconnect.
ThesessiondetailsarethesameasshowninExample3.

Figure 13TerminalEmulator
ConnectingUsingaTerminal OperatorsandAdministratorsintheNOCcanusearemoteserialconnectionto
Server(Remote) connecttotheNE.
Figure 14showsthearchitectureforaremoteserialconnectionfromthe
terminalintheNOCtotheNEintheswitchroom.Thisfigureshowsthe
connectiontotheserialmanagementportontheNetraV440.Thephysicalport
locationisdifferentforeachsupportedhardwareplatformbuttheterminal
serverconnectionalwaysterminatesattheserialmanagementport.
TheIPaddressandportassignmentforthespecificNEisrequiredtoconnectto
theNE.

Figure 14ArchitectureforSerialConnection
ToConnectUsingaTerminalServer(Remote):
1 Openacommandpromptorterminalwindow,oruseaterminalemulator.
2 Telnettotheterminalserver.
#>telnet<IPaddress><port>
ThesessiondetailsarethesameasshowninExample3.

ConnectingUsingaTerminal TheAdministratorintheswitchroomcanusealocalserialconnectiontoconnect
Server(Local) totheNE.Donotmakeadirectserialconnectionbetweentheterminalandthe
serialmanagementportontheNEunlessnoconnectioncanbemadethrough
theIPnetworkbecauseofswitchproblemsorterminalserverproblems.
Thearchitectureforalocalterminalserverconnectionisthesamefortheremote
connection(seeFigure 14)exceptthattheterminaliscolocatedwiththeNEin
theswitchroom.
TheterminalserverhasasingleportthatconnectstotheNE.Anexistingserial
sessionbetweentheNEandremoteterminalmayhavetobeterminatedbefore
alocalserialconnectioncanbemade.Ifthisisthecase,thenseeToTerminate
ExistingRemoteSerialSession:.
ToConnectUsingtheTerminalServer:
1 TelnettotheNEportontheterminalserver.
#>telnet<IPaddress><NEport>
2 Entertheterminalserverusernameandpassword.
Administrativecommandscannowbeexecuted.
Theterminalserveradministrationusernameandpasswordarerequiredtolog
in.
SeeExample4foraterminalserverloginsession.
Example4TerminalServerLogin
#telnet<IPaddress>
TelnetconnecttoJETSTREAM8500mir
PerleJETSTREAM8500RS232
Version2.6.1.G
login:admin
Password:
JS_8500#
ToTerminateExistingRemoteSerialSession:
Onlyaterminalserveruserwith'admin'privilegescanterminateprocessesona
line.
1 Logintotheterminalserver.
2 TerminatetheprocessesontheportcorrespondingtotheNE.
JS_8500#killline<port>
3 Logoutfromtheterminalserver.
JS_8500#logout
Thelineisnowfree.LogintotheNEusingtheterminalserverIPaddressandNE
portnumber.

ConnectingUsingaDirect Donotmakeadirectserialconnectionbetweentheterminalandtheserial
SerialConnection managementportontheNEunlessnoconnectioncanbemadethroughtheIP
networkbecauseofswitchproblemsorterminalserverproblems.
Figure 15showsthearchitectureforadirectserialconnectionbetweenthelocal
terminalandtheserialmanagementportontheNE.
Figure 15DirectSerialConnection
ToConnectUsingaDirectSerialConnection:
1 Unplugtheterminalserverserialcablefromtheserialmanagementportonthe
NE.
2 ConnecttheDB9endofthespeciallypreparedserialcabletotheterminal.
SeeSerialCableforDirectConnection.
3 Connecttheotherendofthespeciallypreparedserialcabletotheserial
managementportontheNE.
4 PressReturnseveraltimes.
Theserialconnectionisestablished.
Ifaserialconnectioncannotbemade,checktheserialcableforthecorrectpin
outs.
SerialCableforDirectConnection
TheserialmanagementportontheNEisadataterminalequipment(DTE)port.
TheserialcableshouldhaveaDB9connectorend(terminalside)andanRJ45
connectorend(NEside).Theserialcableforadirectconnectionbetweenthe
terminalandtheNEmusttakethepinoutsontheNEintoaccount.Therequired
serialcableisdepictedinFigure 16andthepinoutsforbothconnectorendsare
showninTable 10andTable 11.

Figure 16SerialCableConnectors
Table 10DB9PinoutsonSerialCable
Pin SignalDescription
1 DataCarrierDetect
2 ReceiveData
3 TransmitData
4 DataTerminalReady
5 Ground
6 DataSetReady
7 RequesttoSend
8 CleartoSend
9 RingIndicate
Table 11RJ45PinoutsonSerialCable
Pin SignalDescription
1 RequesttoSend
2 DataTerminalReady
3 TransmitData
4 Ground
5 Ground
6 ReceiveData
7 DataSetReady
8 CleartoSend
ConnectingUsing ReachthroughfromtheOMCconsolewilllaunchthedefaultTelnetclientonthe
Reachthrough machineandstartatelnetsessiononthetargetmachine.Ifaterminalemulator
isthedefaulttelnetclient(seeConnectingUsingaTerminalEmulatoronpage 71)
thenthesessionstartsinthatprogram,otherwise,atelnetsessionopensina
commandwindow(Windows).

LoggingInandOut 77
ToConnectUsingReachthrough:
1 ConnecttotheOMC.
RefertotheOMCUserInterfaceGuideforinformationonhowtousetheOMC.
2 RightclicktheNEinthetreeontheleft,selectLaunchandthenclick
Reachthroughto<NE>.
AnewtelnetsessionstartsontheNEinthedefaultTelnetclient.
3 EntertheusernameandpasswordfortheNEuserapplicationaccount.
LoggingInandOut Toperformoperationsandmaintenancetasks,logintotheNEaseitherthemsc/
pro/lis/fsvorrootuser.
ToperformoperationstasksfromtheOMC,refertotheOMCUserInterface
Guide.
LoggingIntoNEApplication TologintotheNEapplicationuseraccount:
UserAccount
1 Attheloginprompt,typemsc,pro,orfsv,asappropriatetotheNE,andpress
Enter.
2 Atthepasswordprompt,typeyourpasswordandpresstheEnterkey.Formore
informationaboutpasswords,seeChangingUserPasswordsinthischapter.
Whenthecommandpromptdisplays,theloginfortheNEapplicationuseris
complete.
LoggingInasroot Tworootusermodesareavailable:
Multiusermodeallowsmorethanonerootusertologin.
Singleusermodeallowsonlyonerootuseratatimetologinfromthe
systemconsole.
MultiUserMode
Tologinastherootuserinmultiusermode:
1 Attheloginprompt,typerootandpressEnter.
2 Atthepasswordprompt,typethepassword,andpressEnter.Formore
informationaboutpasswords,refertoChangingUserPasswordsinthischapter.
SingleUserMode
Logintothesystemasrootuserinsingleusermodeto:
Upgradethesoftware
BackuporrestoretheNEsystemimage,includingthefilesystemand
operatingsystem
Performmaintenanceonormodifythefilesystem

Tologinforsingleusermode,youmustbeloggedinviathesystemconsole.The
systemisthenrebootedintosingleusermode:
1 Loginasroot
2 Enterinits
3 Rebootbacktomultiusermodewhenfinishedbyenteringreboot
LoggingintorootfromtheNE Usethesucommandtologinastherootsuperuserfromtheusercommand
ApplicationAccount prompt.
Tologinastherootsuperuser:
1 AttheNEapplicationusercommandprompt,typesu,andpressEnter.
2 EntertherootpasswordandpressEnter.
3 Whenthecommandmenuappears,therootloginiscomplete.
LoggingOut Whenmaintenanceactivitiesarecomplete,logout.Loggingoutensuresthat
nonauthorizedpersonneldonothaveaccesstotheNE.
TologoutofrootortheNEapplicationuseraccount:
1 Atthecommandprompt,typeexitandpresstheEnterkey.
2 Thesystemlogsoutanddisplaystheloginprompt.
Ifyouloggedinoriginallyasroot,thesystemlogsoutanddisplaysthelogin
prompt.IftherootwasaccessedfromtheNEapplicationuser,thesystem
reinstatestheNEapplicationuseranddisplaysthecommandprompt.
Atleastonesessionmustalwaysberunning.Thesystempreventsyoufrom
loggingofffromtheconsoleifyouaretheonlyuserloggedon.

ChangingUserPasswords 79
ChangingUserPasswords PasswordsauthenticateusersloggingintotheNE.Theyalsoensurethatonly
authorizedusershaveaccesstothesoftware.
TheNEshipswithdefaultpasswordsalreadyconfigured.Whenlogginginforthe
firsttime,usethedefaultpasswords,thenusethepasswdcommandtodefine
newpasswords.
Toensureconfidentiality,passwordsareneverdisplayedontheconsole,andthe
cursordoesnotmoveasyoutypethepassword.
PasswordGuidelines Passwordactivitieshavethefollowingcharacteristics:
Anewpasswordmusthaveatleastthreenewcharacters.
CancelthepasswdcommandanytimebytypingquitandpressingEnter.
Enterandconfirmthenewpasswordcorrectlyorthesystemwillnotaccept
thepassword.
Thesystemallowsthreeattemptstochangeapassword.Ifthepasswordis
notenteredandconfirmedcorrectlyonthefourthattempt,thesystem
cancelsthepasswdcommand.
Examplesare:meta456,foxtrot3,okay.
Tochangeapassword:
1 Loginastheuserrequiringthepasswordchange,usingthedefaultorcurrent
password.
2 Atthecommandprompt,typepasswdandpresstheEnterkey.
3 EntertheoldpasswordandpresstheEnterkey.
4 EnteranewpasswordandpresstheEnterkey.Thepasswordmustbeatleast
threecharacterslong.
5 ReenterthenewpasswordandpresstheEnterkeytoconfirmit.
Thesystemchangesthepassword,anddisplaysthecommandprompt.

FILESYSTEMS
6
Thischapterprovidesabriefoverviewofdiskterminology,theUNIXfilesystem,
andtheNetworkElementapplicationfilesystems.
AboutDisksandFilesystems
UNIXFilesystem
NetworkElementApplicationFilesystem
AboutDisksand Filesstoredonadiskarecontainedinfilesystems.Eachfilesystemonadiskis
Filesystems assignedtoaslice,whichisagroupofsectorssetasideforusebythatfile
system.Slicesaresometimesreferredtoaspartitions.Eachdisksliceappearsto
theoperatingsystemasthoughitwereaseparatediskdrive.
Table 12listssometerminologyrelatedtodisks.
Table 12DiskTerminology
DiskTerm Description
Track Aconcentricringonadiskthatpassesunderasinglestationary
diskheadasthediskrotates
Cylinder Thesetoftrackswiththesamenominaldistancefromtheaxis
aboutwhichthediskrotates.
Sector Sectionofeachdiskplatter.Asectorholds512bytes
Block Adatastorageareaonadisk.Adiskblockis512bytes.
Diskcontroller Achipanditsassociatedcircuitrythatcontrolsthediskdrive.
Disklabel Thefirstsectorofadiskthatcontainsdiskgeometryandpartition
information.

82 Chapter6:Filesystems
Table 13listssometerminologyrelatedtodiskpartitions.
Table 13PartitionTerminology
PartitionTerm Value Description

Number 07 Partitionsorslices,numbered07.
Tag 0=UNASSIGNED Anumericvaluethatusuallydescribesthefile
1=BOOT systemmountedonthispartition.
2=ROOT
3=SWAP
4=USR
5=BACKUP
7=VAR
8=HOME
Flag wm Thepartitioniswritableandmountable.
Flag wu Thepartitioniswritableandunmountable.
Thisisthedefaultstateofpartitionsthatare
dedicatedforswapareas.
UNIXFilesystem TheUNIXfilesystemholdfilesrelatedtotheOperatingSystem(inthiscase,SUN
Solaris)andapplications.Ithasastandardstructure.Table 14mapsthediskslices
andthebasicUNIXfilesystem.
Table 14FilesystemDescription
Slice Filesystem Description

0 root(/) Therootorbaseofthefilesystem.Theoperatingsystem
andapplicationdirectoriesandfilesallfallbelowthe
root.
1 swap Providesvirtualmemory,orswapspace.
2 Referstotheentiredisk,byconvention.Thesizeofthis
sliceshouldnotbechanged.
3 /export Optionalslice.Definedatinstallationtime.
4 Optionalslice.Definedatinstallationtime.
5 Optionalslice.Definedatinstallationtime.
Canbeusedtoholdapplicationsoftwareaddedtoa
system.Ifasliceisnotallocatedforthe/optfilesystem
duringinstallation,the/optdirectoryisputinslice0.
6 /usr Holdsoperatingsystemcommands(alsoknownas
executables).Thisslicealsoholdsdocumentation,
systemprogramsandlibraryroutines.
7 /export/home Holdsfilesthatarecreatedbyusers.

NetworkElementApplicationFilesystem 83
NetworkElement TheNEapplicationssitontopoftheUNIXoperatingsystemanduseUNIX
ApplicationFilesystem operatingsystemservices.TheapplicationfilesystemsfitintotheUNIXfilesystem
andhaveadefinedstructureunderaspecifichomedirectory.
rootAccountFilesystem Thesuperusercanreadandwritetoanyfileinthefilesystem,andexecute
programsthatotherscannot.
HomeDirectory ThehomedirectorydiffersdependingontheNE:
MSCServer:mschomedirectory
HLR:fsvhomedirectory
SSVR:prohomedirectory
AccessServer:mscormrfhomedirectory
LawfulInterceptServer:lishomedirectory
OMCServer:omchomedirectory
IntheUNIXoperatingsystem,thetildesymbol(~)usuallyrepresentsahome
directory.Thehomedirectorybecomestheworkingdirectorywhenyoufirstlog
intoUNIX.Theworkingdirectoryisthedirectoryyouarecurrentlyworkingin
andthereforechangesasyounavigatearoundthefilesystem,whereasthehome
directoryisalwaysconstant.
Thehomedirectory'sactualpathis/telos/<homeaccount>.Forexample,
themschomedirectory'sactualpathis/telos/msc
ThehomedirectoryforthesuperuseristherootoftheUNIXfilesystem(/).
SignalingServerApplication ThedirectoriesunderthehomedirectorythatmakeuptheNetworkElement
FilesystemStructure (NE)applicationare:
CONF
ThisruntimedirectoryholdsalltheconfigurationfilesrequiredbytheNE
application.
DATA
Thisruntimedirectoryholdstheconfigurationdatabase,CDRfiles,and
applicationperformancefiles.
EXEC
ThisruntimedirectoryholdstheprogramsthatexecutewhentheNE
applicationstarts.
LIBR
ThisruntimedirectoryholdssharedcodelibrariesusedbytheNEapplication.
RJE
Thisruntimedirectoryholdsfilesformaintenanceand/orupgradepurposes.
Boththeprouserandsuperusercancreatefilesinthisdirectory.
RUN
Thisruntimedirectoryistheworkingdirectoryforalldaemonprocesses
startedbytheNEapplication.ItholdsEventLogfilesandanyotherruntime
84 Chapter6:Filesystems
diagnosticfilescreatedbydaemonprocesses.Italsoholdsanycoredumpfiles
producedwhenadaemonprocessabortsexecution.
TMP
ThisruntimedirectoryholdstemporaryfilesusedbytheNEapplication.
TemporaryfilesaredeletedaftertheNEapplicationnolongerrequiresthem.
TOOL
ThisruntimedirectoryholdsexecutablesusedbytheNEapplicationforuser
commandsanddiagnostictools.

ACTIVESTANDBYOPERATIONS
7
Thischapterdescribesthehighavailability(activestandby)architectureofthe
MSCandSSVR,andtheproceduresforoperatingandtestingtheactivestandby
functionalityoftheseCoreVoiceNetworkElements.
HighAvailability
HighAvailabilityImplementation
HighAvailabilityandRedundancyVerification
OperatorInitiatedFailovertoStandby
FaultInducedFailover
sys_mirrorOperation
ActiveStandbyProvisioning
HighAvailability HighAvailability(HA)referstotheavailabilityofresourcesinanetworkorsystem
whenacomponentinthenetworkorsystemfails.Thegoalistoensurethe
systemdoesnotloseservice,andisalways'highlyavailable'.Onemethodfor
creatingahighlyavailablesystemistoprovideredundancy.Abackupsystemcan
takeoverforthefailedsystem(failover)andavoidaservicedisruptiontothe
useroftheapplication.
HighAvailabilitycanbeimplementedwithhardware,software,oracombination
ofboth.HAaimstoremovesinglepointsoffailure,eitherinsoftwareor
hardware,usuallybyprovidingaredundantcomponent.
Failovercanbeplanned,forhardwareorsoftwaremaintenanceactivities,or
unplanned.Unplannedfailovercouldbedueto:
Powerloss(powersupplyfailure)
LossofheartbeatbetweenActive/Standbycomponent
Diskfailure
UNIXoperatingsystemfailure
Application(MSC)softwarefailure
Operatorerror
Configurationerror

86 Chapter7:ActiveStandbyOperations
HighAvailabilityTerminology SomerelatedtermswillhelpinunderstandingHighAvailability:
ContinuousAvailability
Thisimpliesnonstopservice,withnolapseinservice.Thisrepresentsanideal
state,andgenerallyindicatesahighlevelofavailabilityinwhichonlyavery
smallquantityofdowntimeisallowed.Highavailabilitydoesnotimply
continuousavailibility.Generally,thepenultimateavailabilityinTelecom
networksis5nines'(inservice99.999%ofthetime).
FaultTolerance
Thisisameanstoachieveveryhighlevelsofavailability.Afaulttolerant
systemhastheabilitytocontinueservicedespiteahardwareorasoftware
failure,andischaracterizedbyredundancyinhardware,includingCPU,
memory,andI/0subsystems.Highavailabilitydoesnotimplyfaulttolerance.
SinglePointofFailure
Ahardwareorsoftwarecomponentwhoselossresultsinthelossofservice;
suchcomponentsarenotbackedupbyredundantcomponents.
Failover
WhenacomponentinanHAsystemfails,resultinginalossofservice,the
serviceisstartedbytheHAsystemonanothercomponentinthesystem.This
transferofaservicefollowingafailureinthesystemistermedfailover.
Active/StandbyModel ToprovideHighAvailability,anactivecomponentcanhaveastandby
component.Thestandbycomponentwilltakeoverserviceiftheactive
componentfails.ThisisknownasanActive/Standbymodel.TheMSCandSSVR
usetheActive/Standbymodeltoprovidehighlyavailableservice.
Active/ActiveModel IntheActive/Activemodel,bothcomponentsareactiveandeachisactingasa
standbyfortheotheractivecomponent.Ifeitheractivecomponentfailsthen
onlytheremainingcomponentisactive.Thereisnostandbycomponentuntilthe
failedcomponentisrestored.TheNEsdonotusetheActive/Activemodel.
FailoverStates Failovertoastandbycomponentcanbeeitherstatelessorstatefulfailover.Ina
statelessfailover,anapplicationorprocessissimplyrestartednodataor
informationismaintainedbythestandbycomponent.Inastatefulfailover,the
integrityofthedataandinformationismaintained.TheMSCandSSVR
implementastatefulfailover.

HighAvailability Thissectionprovidesinformationon:
Implementation
PhysicalView
ManagementView
ServiceView
upLinkandHWDOperation
PhysicalView Twonodes(twoMSCsortwoSSVRs)andtwoEthernetswitches,deployedintwo
separateracks,implementtheHighAvailabilityarchitecture.
EachnodeandEthernetswitchisaredundantpair.Eachnode(activeand
standby)isequivalentintermsofhardwareconfiguration,OperatingSystem,
SignalingServerapplicationconfiguration,andnetworkconfiguration.
Configurationchangesarepropagatedfromtheactivenodetothestandbynode
every5minutes.Ifafailoveroccursduringanetworkconfigurationchange,the
systemwillbeoutofsynchronizationforamaximumperiodof5minutes.Inthis
case,reapplyanyconfigurationchangestothenewlyactivenodeifrequired
beforethenextsynchonization.
Eachnode(activeandstandby)hastwophysicalethernetportsinorderto
provideconnectionstobothEthernetswitches.TheupLinkapplicationmapsthe
twophysicalinterfacesoneachnodetoasinglelogicalinterface.Toimplement
this,theupLinkapplicationcreatesavirtualMACaddressforthislogical
interface.SeeDisplayingtheVirtualMACAddressonpage 95andupLinkand
HWDOperationonpage 89foradditionalinformation.

Figure 17showsthephysicalviewoftheHighAvailabilityarchitecture.
Figure 17HighAvailabilityPhysicalView
ManagementView Eachnode(activeandstandby)hasitsownpermanentmanagementIPaddress
andatemporaryactive(shared)IPaddress.Thesharedinterfaceisonlyenabled
ontheactivenodeifthesystemisrunningasMaster.ThesharedIPaddresson
thestandbynode(Slave)isdisabled.
ThepermanentIPaddressoneachnodemanagesthenode.ThisistheIPaddress
forremoteconnections,suchasTelnetandFTP.
Shuttingdowntheactivenodeviathecpu_stoporcpu_abortcommands,
disablestheactiveinterfaceandcausesafailovertothestandbynode.Any
failoveractivatesthesharedaddressonthenewMasternodeanddisablesthe
sharedaddressonthenewSlavenode.Theswappingofthesharedaddress
typicallytakesaboutonesecond.Theoperatormustrestartallconnectionbased
services,likeTelnetandFTP,onthenewlyactivenode.
ServiceView Afailovertothestandbynodeisessentiallyequivalenttoasystemstart.Alllinks
toexternaldevicesdowntotheMTP3layeraretorndownwhentheactivenode
failsandarethenreestablishedwhenthestandbynodeisbroughtintoservice.

Thenewlyactivenodeloadsanyrecentconfigurationchanges,establishes
connectionstotheSignalingGateway,MSC,andHLR,ifnecessary,andstarts
processingagain.
upLinkandHWDOperation upLinkisalogicalsoftwarecomponentthatprovidesthehighavailability
ethernetservices.RAFTHWDmonitorsthestateoftheredundantnodeand
managesfailoverbetweentheactiveandstandbynode.
upLink
upLinkcreatesasinglevirtualEthernetinterface(IPaddressandMACaddress)to
representthetwophysicalEthernetinterfaces(hme0andhme1)oneachnode.
upLink0mapstothemanagementIPaddressandupLink0:1mapstotheshared
virtualIPaddresswhichisonlyactiveontheMasternode.
WithintheupLinkinstanceoneachnode,oneofthephysicalEthernetinterfaces
isintheactivestate,whiletheotherisinthestandbystate.Thephysical
interfacesarereferredtoastheactivelinkandthestandbylink.Theactivelinkis
theonecarryingthenetworktraffic.
ThevirtualEthernetinterfaceaddressremainsthesamenomatterwhich
physicalEthernetinterfaceistheactivelink.
upLinkdetectsheartbeatsoverthetwophysicalinterfacesandprovidestraffic
failoverfromoneinterfacetotheotherincaseoflinkfailureorheartbeat
timeout.
Ifnoheartbeatornetworktrafficisreceivedononeofthephysicalinterfacesfor
agivenperiodoftime,upLinktriessendingaheartbeatoutthroughtheother
physicalinterface.Ifthefirstinterfacedoesnotreceivethatheartbeat,itishighly
likelythattheinterfaceisdown.Toconfirm,theprocesssendsabroadcastping
throughthesuspectinterface.Ifthereisnoresponse,andtheinterfaceisstillnot
receivingheartbeats,thentheinterfaceisdown.
Figure 18showstwonodes,eachhavingtwophysicalEthernetinterfacesthatare
connectedtotwoEthernetswitchfabrics.TheEthernetswitchesarealso
interconnected.

Figure 18UpLinkImplementationforHighAvailabilityConfiguration
HWD
Ahardwaredaemon(HWD)runsonboththeactiveandstandbynode.This
processcontinuallymonitorsthestateoftheothernodeandmanagesfailover
betweentheactiveandstandbynodes.
HWDmonitorsthenode'sapplication,theoperatingstatus,thestatusofthe
peerprocess,andtheOperatingSystem.Theysendinfomationbackandforth
severaltimesasecondcommunicatingtheirstate,andthestateofotherdevices
(orsensepoints)theyhavebeenconfiguredtowatch.
HWDcontrolsFailover.IfHWDonthestandbynodefailstodetecttheheartbeat
oftheactivenode,itwillassumetheactive(Master)state.Iftheactivenodehas
notactuallyfailed,thenthisisanundesirablecondition,asboththeformer
activeMSCandformerstandbynodebothassumetheyaretheactiveMSC.

HighAvailabilityand Thissectionincludesprocedurestocheckthefunctionalityofthehighavailability
RedundancyVerification configuration:
VerifyingupLinkFunctionality
TestingupLink
Testingmanualfailover
VerifyingRAFTHWDFunctionality
TestingHWDfailover
TestingHWDrestart
CheckingRedundantConfiguration
DisplayingActive/StandbyIPAddress
CheckinguplinkStatus
DisplayingtheVirtualMACAddress
CheckingtheActiveStatus
VerifyingupLinkFunctionality ToverifythefunctionalityofupLink,use:
uplinkconfigguplink0
Theoutputlookssimilarto:
uplink0:ActiveLink:BLinkA:1LinkB:1
ThisindicatesthatLinkBistheactivelink,andbothlinksAandBareup(their
statusis1).
CheckinguplinkStatus ChecktheuplinkstatusontheActivenodetodeterminethatuplinkisUP.The
nodeactingasMasterhastheuplinkinanUPstate.AnodeinaSingleAvailability
configurationalsohasanuplinkintheUPstate.
ToCheckuplinkStatus:
1 LogintotheActivenodeasmsc,proorfsv,asappropriatetothenode.
SeeCheckingRedundantConfigurationtodeterminetheActivenode.
2 Checkthestatusoftheuplink.
pro>ifconfiga
Example5showstheuplinkstatusonanActivenodeandExample6showsthe
uplinkstatusonaStandbynode.OntheStandbynode,theuplink0:1interface
doesnotshowasbeingUP.
Example5uplinkStatusonActiveSSVR
pro>ifconfiga
uplink0:flags=1000843<UP,BROADCAST,RUNNING,MULTICAST,IPv4>mtu1500index1
inet172.25.140.187netmaskfffffc00broadcast172.25.143.255
uplink0:1:flags=1000843<UP,BROADCAST,RUNNING,MULTICAST,IPv4>mtu1500index1
lo0:flags=1000849<UP,LOOPBACK,RUNNING,MULTICAST,IPv4>mtu8232index2
inet127.0.0.1netmaskff000000
msca>

Example6uplinkStatusonStandbySSVR
mscb>ifconfiga
uplink0:1:flags=1000842<BROADCAST,RUNNING,MULTICAST,IPv4>mtu1500index1
mscb>
TestingupLink TotestupLink:
1 Verifythatthevirtualinterfaceisworkingbypinginganexternaladdress.
2 Disconnectthecablefromtheactivephysicalinterface.
3 Pinganexternaladdress.
Testingmanualfailover Totestthemanualfailover:
1 CheckwhichlinkiscurrentlybeingusedbytestingupLink(refertoTestingupLink
onpage 92).
2 Initiatethefailoverusing:
uplinkconfigfuplink0
3 Checkthattheotherlinkisnowactivewhichwillindicatethatthefailoverwas
successful.Forexample,ifLinkBwasactiveinstep1,thenLinkAshouldnowbe
active.
VerifyingRAFTHWD TocheckthecurrentstatusofHWD:
Functionality
1 CheckthecurrentstatusofHWDusing:
hwd_clid
TotestHWD:
2 Runhwd_cliintestmodesuchthatitappearstobeamessagefromthenodeby
usingthefollowingcommandoneachnode.(TheexampleisforanSSVR.Use
mscfortheMSC.)
hwd_cliassvrf&
3 Runhwd_clidagaintogetthecurrentstatususing:
hwd_clid
Theoutputshouldbesimilartothefollowing:
Agents LocalStatusPidDevPeerStatus

HWD(Manager) Master14884Slave
NET(Agent) Master14899253Slave
ssvr(Friend) Master15628251Slave
OneofthehostsshouldbeintheMasterstate,ifnotitmeansthattheNETagent
couldnotchangestate.Thisislikelyduetoaconfigurationissue.
TestingHWDfailover Thehwd_clifailovertoolinterrogatesandmanipulatesHWDfortestingbasic
HWDfunctionality,anddisplayingHWDstatus.
TotestHWDfailover:
1 KillanyofthemasterHWDprocesses:
kill15628(fromthepreviousstatuscheck)
2 Checkthestatusagaintoverifyfailover:
hwd_clid
Theoutputissimilartothefollowing:

HWD(Manager) Slave 14884Master
NET(Agent) Slave 14899253Master
ssvr(unregistered) Unknown 0 0 Master
3 Restartthehwd_clitoolintestmode:
hwd_cliassvrf&
Note:Duetothesimplenatureofhwd_cliwhenrunintestmode,youmayfind
theoddcasewhereyoucankillaprocessontheActiveside,itwillfailtoSlave
(Standby),butthenrecovertoActive(Master)beforetheothersidehasachance
togoActive.
4 Repeatstep3,butthistimedoakill9onthenode'sprocessthisisequivalent
totheapplicationfailingand'cpu_abort'beingrun.Remembertologontothe
Activehosteachtimeafterafailover.
TestingHWDrestart TotestHWDprocessrestart:
1 killtheHWDprocess:
kill14884
2 Checkthestatusagaintoconfirmtherestart.Theprocessids(pid)aredifferent
andthetesttoolhasexitedthisisbydesignifHWDfails,theapplicationsmust
alsostop,sincethepeerwouldbecometheActivehost.
hwd_clid

HWD(Manager) Slave 16417 Master
NET(Agent) Slave 16432 253Master
ssvr(unregistered) Unknown 0 0 Master
3 OntheStandbyhost,checkthatthesharedinterfaceisnotUP:
ifconfiga|grepuplink0:1
uplink0:1:flags=1000842<BROADCAST,RUNNING,MULTICAST,IPv4>mtu
1500index1
4 OntheActivehost,checkthatthesharedinterfaceisUP:
ifconfiga|grepuplink0:1

uplink0:1:flags=1000843<UP,BROADCAST,RUNNING,MULTICAST,IPv4>mtu
1500index1
CheckingRedundant Checkcomponentstoconfirmthatthedesignatedprimarycomponentisthe
Configuration mainprocessorandthedesignatedsecondarycomponentistheredundant
backup.
ThereisnoredundantnodeintheSingleAvailabilityconfiguration.
ToCheckRedundantConfiguration:
1 Logintothetargetnodeasmsc,pro,orfsv.Thetargetnodecanbeeitherthe
ActiveorStandbynode.
2 Checkthestatusoftheredundantcomponents.
pro>hwd_clid
3 Examinetheoutput.
TheActivenodedisplaysMasterintheLocalStatuscolumn.TheStandbynode
displaysSlaveintheLocalStatuscolumn.ThePeerStatuscolumnindicates
thestatusoftheredundantcomponent(Slave/Master).
Example7showssampleoutputfromthehwd_clicommandontheActivenode.
Example7ActiveSSVROutput
pro>hwd_clid
AgentsLocalStatusPidDevPeerStatus

HWD(Manager)Master420Slave
NET(Agent)Master573252Slave
ssvr(Friend)Master986251Slave
Example8showssampleoutputfromthehwd_clicommandontheStandby
SSVR(pro).
Example8StandbySSVROutput
pro>hwd_clid
AgentsLocalStatusPidDevPeerStatus

HWD(Manager)Slave384Master
NET(Agent)Slave634252Master
ssvr(Friend)Slave7383251Master
SSVRAdoesnotalwaysrepresenttheActive(Master)SSVR.Usethehwd_clid
commandtodeterminethestatusofthetargetnode.
DisplayingActive/StandbyIP Thehwd_clicommanddoesnotdisplaytheIPaddress(orhostname)ofthe
Address redundantcomponent.Displaythehwd.conffiletodeterminetheIPaddressof
eachnode.

Thehwd.confhardwareconfigurationfilecanuseIPaddressorhostname.IP
addressesaremorereliable.HostnamesandIPaddressesmustbeaddedto
/etc/hostsoneachnodeiftheyareused.
Thecommandpromptoneachnodecanbeconfiguredtodisplayanytext,and
thereisnotnecessarilyanycorrelationbetweentheprompt,hostname,
host_ip_aorhost_ip_b,orwhichnodeiscurrentlyActiveorStandby.
ToDisplaytheActive/StandbyIPAddress:
1 Logintothetargetnodeasmsc,pro,orfsv.
ThetargetnodecanbeeithertheActiveorStandbynode.
2 Pagethehardwaredaemonconfigurationfile.
pro>more/etc/hwd.conf
3 Examinetheoutput.
ThemanagementIPaddressesareparametershost_ip_aandhost_ip_b.The
virtualIPaddress(common/shared)isdefinedbythenet_ext_address
parameter.
Example9showsanexampleofthehardwaredaemonconfigurationfile.
Example9hwd.confHardwareDaemonConfigurationFile
...
host_ip_a="<IPaddress>"
host_port_a=9500
#host_ip_a_0="10.1.1.2"
#host_port_a_0=9501
host_ip_b="<IPaddress>"
host_port_b=9500
#host_ip_b_0="10.2.2.2"
#host_port_b_0=9501
...
#
net_ext_device=uplink0:1
net_ext_address=<IPaddress>
#net_ext_mask=255.255.248.0
DisplayingtheVirtualMAC EachnodehastwoNICcards,andeachNIChasitsownMACaddress.Theuplink
Address softwarecombinesthephysicaladdressesintoalogicalor'virtual'physicalMAC
address.ThisisdonebecauseanIPaddressismappedtoaMACaddress,andthe
functionalIPcannotbemappedtoaspecificNICbecausefailovercouldnotbe
supported.
The'virtual'MACaddresscanbedisplayed.TheMACaddressismadeupoftwo
parts:thefirstpartisaOrganizationallyUniqueIdentifier(OUI)thatuniquely
definesthemachineinthenetwork(andintheworld)andthesecondpartis
derivedfromtherealphysicalMACaddressesoftheNIC.
ToDisplaytheVirtualMACAddress:
1 Logintothetargetnodeasmsc,pro,orfsv.

2 Changetosuperuser.
pro>suroot
3 SelectAdministrationcommands.
1.Administrationcommands
4 PrinttheMACaddressesforthetargetnode.
root>ifconfiga
5 Exitsuperuser.
root>exit
The'virtual'MACaddressdisplay,asshowninboldinExample10.Theother
nodeintheActive/Standbyconfigurationhasthesameprefix(forexample
02:02:00),buttheremainingMACaddressisdistinct.
Example10DisplayMACAddresses
root>ifconfiga
ether02:02:00:7d:ab:51
uplink0:1:flags=1000843<UP,BROADCAST,RUNNING,MULTICAST,IPv4>mtu1500index1
CheckingtheActiveStatus CheckthecurrentstateofthenodetodetermineifitisactingasMaster(Active)
orSlave(Standby).
ToCheckActiveStatus:
1 Logintothetargetnodeaspro.
2 ChecktheMaster/Slavestatusofthetargetnode.
pro>cpu_status
SeeExample11foroutputfromanodeinActive(Master)stateandExample12
foroutputfromanodeinStandby(Slave)state.
Example11ActiveSSVRinMasterState
pro>cpu_status
<hostname>:UPasMasterfor27minutesUnixup:3day(s),21:10hours
Example12StandbySSVRinSlaveState
pro>cpu_status
<hostname>:UPasSlave(insynccold)for19minutesUnixup:3day(s),
20:47hours
OperatorInitiatedFailover TheOperatorinitiatesafailovertotheStandbynode.Thereisanimpacttotraffic
toStandby sowaituntiltrafficisloworpostponeuntilaplannedmaintenancewindow.
Operatorinitiatedfailovercanbeperformedto:
SimulateanalarmforFaultManagementortroubleshootingpurposes
Conductplannedmaintenanceonthecomponent

OperatorInitiatedFailovertoStandby 97
Operatorinitiatedfailovercanalsobeperformedto:
ResettotheoriginalActive/Standbyredundantconfigurationafterafault
inducedchangeover
ResettotheoriginalActive/Standbyredundantconfigurationafterplanned
maintenance
ThereisnocheckingorverificationrequiredontheStandbynodebefore
initiatingfailover.
Checkalarmandeventlogstodetermineifacomponentisnotoperatingat
capacity,impactingsystemperformanceorcallprocessing,orshowingsignsof
possiblefailure.Usethisinformationtodetermineiftheredundantcomponent
shouldberemovedfromserviceandreplaced.
Caution:
Acomponentchangingfromactivetostandbyimpactstraffic.
Allcallsinconversationstateandinsetupphasearedropped.Newcallsare
setuponthenewactivecomponent.
Caution:
Donotperformanyconfigurationchangesonthenodewhilethenodeisina
nonredundantconfiguration.
ToInitiateFailovertoStandby:
1 LogintotheActivenodeasmsc,pro,orfsv.
2 Stopthenode'sapplications.
pro>cpu_stop
itwasnotaccidental.RefertoFigure 8onpage 60foranexample.
Thehardwaredaemon(seeupLinkandHWDOperation)detectsthe'stopped'
stateofthenode'sapplicationontheActivenode,disablesthefunctionalIP
interface,enablesthefunctionalIPinterfaceontheStandbynode,andsignalsto
thenewActivenode(oldStandby)tobegincallprocessing.Thenodeapplication
ontheStandbynodehasalwaysbeenrunning,butallmessagesrelatedtoCall
Processingwerediscarded.
ThecpuCMShutDownalarmisgeneratedonthenodethatwasshutdown(see
cpuCMShutDownonpage 215).
3 Performanymaintenanceonthestoppednode,andreturnittoserviceas
StandbytotheActivenodeassoonaspossible.

Caution:
TheSystemisnotinaredundantconfigurationuntiltheActive/Standby
configurationisrestored.

FaultInducedFailover 99
FaultInducedFailover AfaultintheActivenode,eitherhardwareorsoftware,couldcauseittofail.If
theActivenodefailsthentheStandbynodewilltakeovercallprocessing.
Caution:
Failoverimpactstraffic.
Alltraffichandledbythefailedcomponentwillbedropped.Thisincludescallsin
conversationstateandinsetupphase.Newcallswillbesetuponthenewactive
component.
WithonlyanActivenodeandnoStandbynode,theMSCorSSVRisnotina
redundantconfiguration.Thefailedcomponentshouldbereplacedasoonas
possible.
ThecpuCMShutDownalarmwillbegeneratedonthenodethatwasshutdown
(seecpuCMShutDownonpage 215).
InvestigatingFailoverCause Thefailureofanodecanbecategorizedintothreebroadcategorieshardware
failure,softwarefailure,orconfigurationerror.
MSCorSSVRhardwaremalfunctionoroutrightfailureisestimatedtobe
responsibleforaverysmallpercentageoffailures.BecausetheMSCandSSVR
aresosoftwareintensive(bothattheOSandapplicationlevel),slightlymore
failurescanbeattributedtothebroadcategoryofsoftwarefailure.However,
configurationerroristhemostlikelycauseofafailure.
ContactCustomerSupportforallfailures,dueeithertohardware,software,or
configurationerrors.IftheapplicationontheActivenodefails,theStandbynode
willtakeoverallnewcallprocessing.Thenode'sapplicationwillretstartand
enterStandbymode.
Amoreserioussituationariseswhenthefailednodedoesnotrestartandassume
theStandbymode.Duringthisperiod,theMSCorSSVRisnotinaredundant
configuration.ContactCustomerSupportassoonaspossibleinthissituation.
Figure 19showsthedecisiontreeforinvestigatingthereasonforafailover.A
configurationerrormaynotcauseafailover,butservicecouldstillbeimpacted.
ConfigurationchangesarepropagatedbetweentheActiveandStandbynodes,so
failovertotheStandbynodeisnotanoptioniftheActivenodehasbeen
misconfigured.Networkmisconfigurationwillaffectserviceinallconditions.

Figure 19NodeFailoverDecisionTree
CheckingEventLogRecords CheckEventLogRecordsfornetworkeventsthatmaygivesomeindicationofany
networkconfigurationissues.
ToCheckEventLogRecords:
1 Startvlog.
2 ExaminetheEventLogRecordsforthedaythefailureoccurred.
3 SearchtheEventLogRecordandsearchforevents(USERERROR,ERROR,FATAL
ERROR).
4 Usetheeventdescriptionstotroubleshootanynetworkconfigurationissue.
SeeChapter 14,TroubleLocatingandClearing.
DetailsforCustomerSupport Numeroustracefilesaregeneratedwhenanodefails.TheMSCandSSVRcreate
adirectoryin/RJEandwriteallofthetracefileshere.Thedirectoryisnamed
ABORT<timestamp>,where<timestamp>isthedateandtimeofthefailure(for
example,ABORTOct20200510:50:39).
ThesefilesareusefulonlytoCustomerSupportasadvancedtroubleshooting
tools,inparticularwhennetworkconfigurationerorsarenotresponsibleforthe
failure.CustomerSupportwillconnecttothemachineinquestionandretrieve
thetracefiles.

sys_mirrorOperation 101
Table 15liststhefilesgeneratedin/RJE/ABORT<timestamp>afterSSVRfailure.
Notethatthesefilesaregeneratedonthenodethatfailed,andthefilesarelisted
forreferencepurposesonly.
Table 15FailureTraceFiles
TraceFiles
alm.trace cpu.trace LFMginasn.trace orq.trace
almd.trace csi.trace LFMnauasn.trace prot.trace
ASPginasn.trace IMPginasn.trace LFMnscasn.trace rtd_bak.trace
ASPnauasn.trace IMPnauasn.trace LFMnsoasn.trace ssfasn.trace
ASPnscasn.trace IMPnscasn.trace lfm.trace sta.trace
asp.trace IMPsmiasn.trace mipd.trace stat.trace
aua.trace IMPwinasn.trace NPIginasn.trace status.trace
cdr.trace imp.trace NPInauasn.trace sys_log.trace
cdr_transfer.trace ipc_remoted.trace NPInpiasn.trace sys_mirror.trace
comd.trace ipcd.trace NPInscasn.trace vlr.trace
CONX_DELAY.trace lfmasn.trace npi.trace vsm.trace
sys_mirrorOperation Oneachredundanthost,sys_mirrorisusedtomirrorthedatabaseandother
configurationanddatafilesbetweeneachnode.
Sys_mirrorwillrunasoftenasisconfiguredincrontab.Ifitisrunningonthe
Activenode,itwillchecktoseeifanyfileshavechanged.Ifitisstandby,itwill
exitimmediately.
TheOMChostisusedasaintermediaterepositoryforallfilestransferred
betweennodes.Thisalsoprovidesaseparatebackupofcriticalsystemdata.
sys_mirrorqueriesHWDtodetermineifthecurrentnodeisintheActiveor
Standbystate.IfitisActive,sys_mirrorpushesitsdatatotheOMC,otherwiseit
pullsthelatestfilesdownfromtheOMC.
Verifyingsys_mirror Crontabentriesshouldexistintheproandrootaccounts,andmodifiedfiles,
operations directories,anddatabaseconfigurationshouldbemirroredfromtheActiveto
Standbymachinewithinthetimeconfiguredinthecrontabentries.
1 Verifythecrontabentriesinthemsc,pro,orfsv,androotaccounts:
a Forthemsc,pro,andfsvaccounts,thefollowingcrontabentryshould
exist:
0,5,10,15,20,25,30,35,40,45,50****/usr/local/bin/sys_mirror>/dev/null
2>&1
b Fortherootaccount,thefollowingcrontabentryshouldexist:
0,5,10,15,20,25,30,35,40,45,50****/usr/local/bin/sys_mirror>/dev/null
2>&1

2 Toverifythesys_mirrorconfiguration,runthefollowingcommandastheroot
andmsc/pro/fsvusers:
sys_mirrors
3 ToverifyfiletransfertotheStandbynocd:
a Touchafileinthe/telos/pro/CONXdirectoryontheActivenocd.
Within10minutesitshouldbemirroredtotheStandbynode.
4 ToverifydirectorytransferstotheStandbynode:
a Addafiletothe/tftpbootdirectoryontheActivenode.
Within10minutesyoushouldseethefileontheStandbynodeunderthe
/tftpbootdirectory.
5 Toverifyconfigurationdatabasemirroring
a Modifyanydatabasetableontheactivenode.
Within10minutesyoushouldseethemodifiedfileonthestandbynode.
ActiveStandby TheMSCServer,HLR,andSignaingServer(SSVR)canoperateinActiveStandby
Provisioning configurations.PerformallprovisioningontheActivehostinanActiveStandby
configuration.IfchangesaremadeontheStandby,theywillbeoverwrittenwhen
thedatabasefilesaremirrored.
CheckingwhichnodeisActive TheOMCdoesnotcurrentlypreventprovisioningontheStandbyhost.Tocheck
whichSSVRiscurrentlyActive,viewtheStandbyStatefieldintheprocessstate
table.
TocheckwhichnodeisActive:
1 StarttheOMCConsole(refertoUsingtheOMCConsoleinChapter 2,
ProvisioningtheCoreVoiceNetworkElements).
2 Rightclickanyelementintherequiredgroup.
3 SelectConfigurationfromthedropdownmenu.
4 SelectModifyfromthenextdropdownmenu.
5 ScrollrighttofindtheoperationalStateandstandbyStatefields.Table 16shows
howtointerpretthefieldvalues.NotethatavalueoffalseinthestandbyState
fieldmeans"notintheStandbystate",or,inotherwords,active.
Table 16Active/StandbyStateFields
StateoftheNode operationalStateField standbyStateField

Active Enabled false
Standby Enabled true
Down Disabled (anyvalue)
6 ClickAdd.
7 ClickClose.

ActiveStandbyProvisioning 103
ResynchronizingtheNodes Thedatabasefilescanberesynchronizedmanuallyifthereareconcernsabout
datainconsistency.
ToResynchronizetheMSC,HLR,orSSVR:
1 StarttheOMCbrowserandrightclicktheappropriatenode.
2 SelectConfiguration,thenResynchronize.
3 ClickYesinresponsetotheprompt"Areyousureyouwanttoresynchronizethis
NE?".
Duringresynchronization,abluesquaredisplaysaroundthenodeicon.
Resynchronizationcompleteswithadialogboxthatstates:"Upload/Resync
ActionSuccessfullyCompletedforNE".
4 ClickOKinthedialogbox.

PERFORMANCEMONITORINGAND
8 MANAGEMENT
Thischapterprovidesanoverviewofperformancemonitoringandmanagement,
definesQualityofServiceguidelines,describeshowtomonitortheCoreVoice
NetworkElements(NE)forperformancedata,howtoanalyzethedatato
determineperformancelevels,andwhatpreventativeandcorrectiveactionscan
bedonetomaintainandtunetheperformanceoftheNEs.
OverviewofPerformanceManagement
PerformanceMonitoringandDataCollection
PerformanceGraphsandReports
CheckingCPUUtilization
ViewingCPUProcessLists
ViewingProcessorUtilization
CheckingMemoryUtilization
CheckingDiskSpace
CheckingLogUsage
CheckingTCP/IPLinkStatus
PerformanceManagementActivities
OverviewofPerformance Thissectionprovidesangeneraloverviewofperformancemanagement
Management philosophy,andthengivesanoverviewofperformancemanagementonthe
CoreVoiceNetworkElements(NE).
WhatisPerformance Performancemanagementisthecontinuousprocessofreportingperformance
Management statisticsandevaluatinganNEtodeterminewhetherornotitcandeliverthe
levelofperformancethat'srequired.
Performancemanagementisrelatedtocapacityplanningandnetwork
dimensioning,butperformancemanagementinvolvestuningthecurrentsystem
sothatitcanperformbetterandsupporttherequirednumberofsubscribers.
Capacityplanninganddimensioningfocusesonhowmanysubscribersthe
systemcansupport,andatwhatservicelevel.
Thegoalofperformancemanagementistodetectandcorrectanyimpactsto
servicelevelsbeforetheyaredetectedbysubscribers,andbeforetheyescalate
intofaultsandalarmsonthesystem.Thisisachievedbygatheringandanalyzing

106 Chapter8:PerformanceMonitoringandManagement
statisticaldataandpresentingitsothattheOperatorcanmonitorthesystemand
takeanycorrectiveactionwherenecessary.
Performancemanagementincludes:
QualityofService(QoS)
PerformanceMonitoringandDataCollection
PerformanceAnalysis
PerformanceManagementActivities
Undernormalconditionsinthenetwork,performanceinformationanddata
shouldbegatheredfromboththeNEandthenetworkonacontinuousand
periodicbasis.Thisperformancedatacanbeusedtodetectimpendingfault
conditionswhichcouldgeneratealarmsandrequireincreasedefforttoresolve.
Aftertheoccurrenceofafailureinthenetwork,maintenanceactionsare
requiredtocorrectthefaultandtoprotect,whenpossible,anyimpactedtraffic.
Furtherperformanceanalysiscanbeusedtodetectanydegradationinservice
levelsandoverallnetworkperformance.
QualityofService(QoS) QualityofService(QoS)istheefforttomaintainandimprovetheservicelevels
experiencedbythesubscribers,andtheperformancelevelofthenetwork.
Generally,thenetworkisdimensionedaccordingtoanticipatedsubscriber
requirementsandfeaturelist.Actuallevelsofservicecanonlybedetermined
afternetworkdeploymentbutaservicelevelbenchmarkcanbeestablishedand
theneffortsmadetooptimizethatservicelevel.
QoSincludesanydataorinformationgeneratedabouttheservicelevelsoffered
tosubscribersandanynetworkperformanceissues.Thisdataisusedtohelp
optimizethesystemtoprovidebetterservice,bothatthesubscriberleveland
thenetworklevel.
ThereisnoformaltoolorprocessforestablishingQoSontheNE(andthe
network),butbyusingtheperformancetoolsdescribedinthissection(reports,
graphs,shortandlongtermhistories)aservicelevelcanbebenchmarked,
maintained,andevenoptimized.
QoSintheIPNetwork QualityofService(QoS)referstothecapabilityofanetworktoprovidebetter
servicetoselectednetworktrafficovervarioustechnologies,includingtheair
interfaceandIProutednetworks.
Fundamentally,QoSprovidesbetterservicetocertainflows.Thisisdoneby
eitherraisingthepriorityofafloworlimitingthepriorityofanotherflow.The
goalofQoSforVoiceoverIP(VOIP)istocontrollatency,jitter,anddropped
packets.Theseserviceimpactingaspectscanbemanagedby:
Classification
IdentifyingandsortingallIPtrafficintoclassesofservice.
Marking
Makingiteasyfornetworkequipmenttorecognizeserviceclasses.
PerformanceMonitoringandDataCollection 107
Layer2SwitchOptimization
Usingqueuemanagementandcongestionavoidancetechniques.
CongestionPointManagement
Prioritizingcriticaltrafficonsloworexpensivelinks.
QoSintheIPnetworkcanbeimplementedaspartoftheIPinfrastructure(if
providedaspartoftheoffering)orcanbeimplementedafternetwork
deployment.FurtherinformationonQoSintheIPnetworkcanbemadeavailable
fromStarSolutionsCustomerSupport.
PerformanceMonitoring Differenttypesofperformancemonitoringcanbeused:continuouschecking,
andDataCollection routineorperiodicchecking,checkingofbehaviorinlivetraffic,checkingof
behaviorintheabsenceoflivetraffic.Thetypeofperformancemonitoring
dependsontheQoSrequiredbysubscribersandbythenetwork
ContinuousChecking Continuouscheckingisperformedtheentiretimethenetworkand/orfeatures
areup.PerformancethatdoesnotmeetQoScanbeconsideredtobe'faulty',and
correctiveactioncanbetaken.
PerformancegraphsareawaytocontinuouslychecktheperformanceoftheNE.
RoutineorPeriodicChecking RoutineorperiodiccheckingcanbeperformedatthedirectionofNOCstaff,and
wouldusuallybedonebeforeandaftermajorconfigurationchanges(including
softwareupgrades,featureactivations,networkexpansions,etc)andbeforeand
afterexpectedheavytrafficsloads.
Periodiccheckingcanbedoneatanytimetoconfirmtheresultsofcontinuous
checking.Thefrequencyofcheckingdependsontheimportanceofthe
functionality,expectedfailurerates,andthelevelofredundancyandHigh
Availability(HA)inthenetwork.
PerformancegraphsandreportsareawaytochecktheperformanceoftheNE
onaroutineorperiodicbasis.
CheckingwithLiveTraffic Checkingnetworkbehaviorduringlivetrafficcanbeperformeddirectlyor
statistically,andisdoneiftheNEorthenetworkisexhibitingcontinuous
performanceimpactsorfaults.
Performancegraphsandreportscanbeusedtocheckthetrafficperformanceof
theNE,eithercontinuouslyorperiodically.
CheckingwithNoLiveTraffic Checkingsystembehaviorwithnolivetrafficcanbeperformedbeforethesystem
(orsystemcomponent)isbroughtbackintoserviceafterafaultornetwork
outage.
PerformanceAnalysis Performanceanalysistypicallyrequiresfurtherprocessingoftheperformance
datacollectedduringperformancemonitoring.Theresultsoftheanalysiscanbe
used:todetermineanyareasofthenetworkwhichneedperformance
improvements,toevaluatenetworkcapacityimprovements,todocumenttraffic

historyandforecastfuturetrafficloads,toevaluatesubscribertraffic
performance,andtojustgenerallyevaluatetheperformanceofthenetwork.
PerformanceManagement Performancemanagementactivitiestakeintoaccounttheresultsofperformance
Activities analysisandaretheactionsthatcanbeperformedonthetrafficnetwork,the
transportnetwork,andNEs.Activitiescanbepreventative,corrective,orrelated
tosystemorcapacityimprovements.
Performancemanagementactivitiesarepartoftheperformancefeedbackloop.
Changestothesystem,whetherpreventative,corrective,orimprovements,
shouldbeconstantlymonitoredandevaluatedfortheirimpactonperformance.
Furtherperformancemanagementactivitiesmayberequireduntilthesystemis
optimized.
PerformanceGraphsand PerformancedataisgeneratedontheNEandsenttotheOMCusingFTP.Acron
Reports jobsendstheperformancedatatotheOMCin5minuteintervals,andthe
reportingperiod(levelofgranularity)forreportsandgraphsis5minutes.
ThisguidefocusesonusingtheOMCtomanageperformanceontheNE.Referto
theOMCUserInterfaceGuideforinformationonhowtousetheOMCtomonitor
theNE.
PerformancestatisticsaregeneratedandcollectedontheNE.Detailed
descriptionsoftheperformancestatisticsarelistedinTable 17andTable 18.
Eachperformancegraphhasitsownsetofstatisticscollectionpoints,andthese
aredetailedintherelevantgraphsections.
ThissectiondescribesperformancemonitoringanddatacollectionontheNE.
TheNEgeneratesperformancestatisticsanduploadsthemtotheOMCatregular
intervals.TheOMCisusedtomonitorperformanceontheNEwithperformance
graphs,andtogeneratereports.

Thissectionincludes:
AboutGraphsandReports
GeneratingandPrintingPerformanceGraphs
GeneratingandExportingPerformanceReports
CheckingCPUUtilization
MemoryUtilizationGraphsandReports
SwapUtilizationGraphsandReports
DiskI/OGraphsandReports
NetworkI/OGraphsandReports
AboutGraphsandReports GraphsproduceagraphicviewoftheperformanceoftheNE.Performance
graphscanbegeneratedontheNEforthepreceding24hours(considered
'realtime')andforthepreceding60days.Theperformancedataforbothgraphs
isthesame,exceptthereportingperiod.
Ageneratedgraphthatisleftopenwillbecontinuouslyupdatedwithdata
accordingtoanintervalspecifictoeachgraphthereisnoneedtoregenerate
graphstomonitorperformance.
Reportsaregeneratedusingthesamestatisticaldatathatgraphsuse,butthe
reportintervalcanbespecifiedandthedataispresentedinASCIItabularformat
forprinting,analysis,orarchiving.
TheusercanexportreportstoafilefromtheOMC.
Graphscanbeusedtovisuallyinterprettheperformanceofvariousaspectsof
thesystem.TheseaspectsarelistedinTable 17.
Table 17PerformanceDataforGraphs
SystemAspect CollectedData Description

CPUUtilization CPUUSR SeeSystemHardwarePerformance
CPUSYS Statisticsonpage 276forfurther
CPUWIO informationonperformance
CPUIDLE countersandcollecteddata.
MemoryUtilization MEMTOTAL SeeSystemHardwarePerformance
MEMFREE Statisticsonpage 276forfurther
informationonperformance
countersandcollecteddata.
SwapUtilization TOTALSWAP SeeSystemHardwarePerformance
FREESWAP Statisticsonpage 276forfurther
DiskI/O DISKI/O SeeSystemHardwarePerformance
Statisticsonpage 276forfurther

Table 17PerformanceDataforGraphs

NetworkI/O NETWORKI/O SeeSystemHardwarePerformance
Statisticsonpage 276forfurther
ReportsproduceatabularviewoftheperformanceoftheNE.Performance
reportscanbegeneratedontheNEforadaterangespecifiedatthetimeof
reportgeneration.
Reportscanbeusedtointerprettheperformanceofvariousaspectsofthe
system,andcouldbeexportedtoaspreadsheetforfurtherprocessingand
analysis.TheseaspectsarelistedinTable 18.
Table 18PerformanceDataforReports

CPUUtilization SameasshowninTable 17. SeeSystemHardware
PerformanceStatisticson
page 276forfurther
informationon
performancecountersand
collecteddata.
MemoryUtilization SameasshowninTable 17. SeeSystemHardware
page 276forfurther
informationon
collecteddata.
SwapUtilization SameasshowninTable 17. SeeSystemHardware
page 276forfurther
informationon
collecteddata.
DiskI/O SameasshowninTable 17. SeeSystemHardware
page 276forfurther
informationon
collecteddata.
NetworkI/O SameasshowninTable 17. SeeSystemHardware
page 276forfurther
informationon
collecteddata.
TheusercanexportreportstoafilefromtheOMC.

Statistics TheOMCprovides24houror60dayhistoricgraphsfrom5minutetomultiday
intervals,andhourlyreportsfor:
CPUUtilization(%):
CPUuserutilization
CPUsystemutilization
CPUIOutilization
CPUidleutilization
MemoryUtilization(KB):
Totalmemory
Freememory
SwapUtilization(KB):
Totalswapdisk
Freeswapdisk
DiskI/O(%)
NetworkI/O(numofpackages)
GeneratingandPrinting TheOMCgeneratesperformancegraphswhichcanthenbeprinted.Graphsare
PerformanceGraphs printedin"WhatYouSeeIsWhatYouGet"(WYSIWYG)formatinlandscape
orientation(seeFigure 20).Performancegraphsplotthedifferentdatasources
withdifferentcolors.
Figure 20GraphsPrintedWYSIWYGFormat

ToGenerateaGraph:
1 StarttheOMCConsole.(RefertotheOMCUserInterfaceGuideforhowtostart
theOMCConsole.)
2 RightclicktheNEicon.
3 SelectPerformanceandthenagraphperiodandthenagraphtype.
Theselectedgraphisgenerated.
ToPrintaGraph:
1 Generateagraph.
2 Inthegraphwindow,clickFileandthenPrint.
APrintdialogboxappears.
3 SelectaprinterandclickOK.
ThegraphisprintedinWYSIWYGformat(SeeFigure 20).
GeneratingandExporting TheOMCgeneratesperformancereportswhichcanthenbeexportedtoanASCII
PerformanceReports (text)fileforarchiving,viewing,printing,orimportingintoaspreadsheet
program.PerformancereportsarenotprintedinWYSIWYGformat.Reportscan
beprinteddirectlyfromthemenuorexportedtoafileandthenprinted.
TheOMCexportsperformancereportstotheclientmachine(theonehostingthe
OMCapplet)ortoanothernetworklocation,andarenotstoredontheNEitself.
ToGenerateaPerformanceReport:
1 StarttheOMCConsole.(RefertotheOMCUserInterfaceGuideforhowtostart
theOMCConsole.)
2 RightclicktheNEicon.
3 SelectPerformanceandthenReportsandthentheclickthereporttype.
4 SelectaToandFromdateandthenclickFetch.
Theselectedreportforthespecifiedperiodisgenerated.
AnexampleofageneratedperformancereportisshowninFigure 21.

Figure 21GeneratedPerformanceReport
ToCreatePerformanceStorageDirectories:
1 Ontheclientmachine,createanew\perf_reportsdirectory.
Alternatively,adirectoryforreportscanbecreatedonanetworkdriveandwith
anyname.
2 Createanewdirectoryin\perf_reportsforeachreportingperiod.
Forexample,\day,\week,\month,\special.
TheOperatorcanstoredailyreports(likeCPUUtilization)in\daywheretheycan
beusedtooptimizeortroubleshootthesystem.
ToExportaPerformanceReport:
1 GenerateareportusingtheOMC.
2 ClickFileinthewindowofthegeneratedreportandthenclickExporttoFile.
ASavedialogboxappears.
3 Navigatetoareportstoragelocationandenterafilename.
Namereportswithadatestampsothattheycanbearchivedcorrectlyand
processedfurtherwithaspreadsheet.
4 ClickSave.
Aperformancereportcanalsobeprintedafteritisexportedtoafile.Openthe
exportedfileinatexteditorandprintfromthere.Textfilescanalsobeimported
intospreadsheetapplications.

ToPrintaPerformanceReport:
1 Generateareport.
2 ClickFileinthewindowofthegeneratedreportandclickPrint.
APrintdialogboxappears.
3 SelectaprinterandclickOK.
Thereportisprinted.
Aperformancereportcanalsobeprintedafteritisexportedtoafile.Openthe
exportedfileinatexteditorandprintfromthere.
TheprintableformatofaperformancereportisshowninPrintoutofCPU
UtilizationReport.
Example13PrintoutofCPUUtilizationReport
REPORT:CPUUtilization(%)
OBJECT:SAR4_5:NEID=leopard,telos:TelosID=config
FROM:2011060112:25(PacificStandardTime)
TO:2011062112:25(PacificStandardTime)
CPUUSRCPUSYSCPUWIOCPUIDLE
========================================================================
13H/Jun0111098
14H/Jun0111098
15H/Jun0111098
16H/Jun0111098
17H/Jun0111098
18H/Jun0111098
19H/Jun0111098
20H/Jun0111098
21H/Jun0111098
22H/Jun0111098
23H/Jun0121097
00H/Jun0211098
01H/Jun0211098
PerformancereportsexportedtoASCIIformatcanbeimportedtoaspreadsheet,
forexample,toExcel.
ToImportaReporttoExcel:
1 OpenanewemptyspreadsheetinExcel.
2 SelectData,GetExternalData,andclickImportTextFile.
3 Navigatetotheperformancereportsdirectory,selectthefile,andclickImport.
Forexample,\perf_reports\day\pro01_20050630.txt.
ThewindowshowninFigure 22appears.

Figure 22TextImportWizard
4 ClickNext,thenclickNextagain.
5 ClickFinishandthenclickOKintheImportDatadialogbox.
Datacanbeaddedtoanewworksheetoranexistingworksheet.
6 Savethefile.
UsethesameNetworkElementnameandtimestampastheoriginalreport.
Furtherspreadsheetanalysisofperformancedataisnotdiscussedinthisguide.

CheckingCPUUtilization AhighlevelofCPUutilizationisnotinherentlybad(itmeansthereisalotof
processinggoingon),buthighlevelsofCPUuseoverasignificantperiodoftime
mayindicateaprobleminthesystem.Itmaynotbethecauseoftheproblem,
butmaybesymptomaticofanotherproblem,orevencontributetotheproblem.
ShorttermCPUutilizationspikesarenormal.
Figure 23showsanexampleofaCPUutilizationgraph.
Figure 23CPUUtlizationGraph
SeeTable 17forwhatismeasuredinCPUutilizationgraphsandreports.
PerformanceCharacteristics Thesystemisdimensionedsothatafullsystemload(trafficatpeak)willonly
utilizetheCPUat75%.
MostCPUuseshouldbespentrunninguserprocesses(notsystemprocesses).
SustainedperiodsofhighsystemCPUusagecanindicatethattoomanyprocesses
areexecuting.HighsystemCPUusagemayalsoindicateamemoryshortage.
AgeneralrangeforsystemuseoftheCPUis<10%,andnomorethan30%,and
totalprocessorusage(CPUUSER+CPUSYS)shouldnotexceed80%90%.
Averyhighwiopercentagecouldbeexperiencedduringatapebackup(waiting
forI/O),andisfurtherreasontobackupthesystemduringoffpeak(lowtraffic)
hours.
Ahighsyspercentagecouldindicatethataprocessismakingexcessivedemands
onthesystem,thatthereisabottleneckinthesystem,oralargenumberofNFS
operationsarebeingprocessed.
Seealsosarandvmstatcommand.

7 Alsoexecutesarcommand.
SeeExample18onpage 123.
rootpermissionisrequiredtorunsar.
CheckingCPUUtilization CheckCPUutilizationduringperiodsofhightraffic,orifotherperformancedata
seemstoindicateaprobleminthesystem.CPUutilizationisalsocheckeddailyas
partoftheOperatortasks.UseanyCPUutilizationreportsgeneratedbythe
Operatortohelpdiagnosetheproblem.
FormoredetailedinformationonCPUutilizationgraphsandreports,see
CheckingCPUUtilizationonpage 116.
ToCheckCPUUtilization
1 GenerateandexamineCPUUtilizationgraphfortheNE.
2 ExamineandcomparedailyorweeklyCPUUtilizationreports.
3 Ifmoresystemperformancedataisneeded,tryrunningOSperformance
commands:
a LogintotheNEastheNEApplicationuser(msc,pro,lis,orfsv).
b Runaperformancecommand.
RelevantOSperformancecommandsincludetopandvmstat.
c Examinethecommandoutputforfurtherperformancedata.
ExamplesofthesecommandsareshowninExample14andExample15.
Example14topCommand
pro>top
lastpid:5515;loadaverages:0.13,0.14,0.16
11:34:44
76processes:74sleeping,2oncpu
CPUstates:96.8%idle,2.8%user,0.4%kernel,0.0%iowait,0.0%swap
Memory:2048Mreal,754Mfree,2137Mswapinuse,7626Mswapfree
PIDUSERNAMETHRPRINICESIZERESSTATETIMECPUCOMMAND
16141root248013M3248Kcpu/017.1H2.87%hwd
5327pro15802808K1704Kcpu/10:000.07%top
5215pro14001984K1344Ksleep0:000.04%ksh
1root1580872K168Ksleep15:270.02%init
11148drum1258047M21Msleep13:090.02%java
224root202003272K2216Ksleep3:320.00%nscd
183root15802504K320Ksleep1:210.00%inetd
290root15603624K544Ksleep4:250.00%sshd
68root44203808K1984Ksleep2:350.00%picld
210root15802016K992Ksleep1:280.00%cron
203root135803504K1544Ksleep0:390.00%syslogd
18594pro14351836M7584Ksleep0:170.00%peer_mon
18591pro14351836M12Msleep0:140.00%evl_backend
18592pro14351836M7608Ksleep0:140.00%evl_spooler
18593pro15321836M7592Ksleep0:140.00%tpm_timerd
Example15vmstatCommand
pro>vmstat
procsmemorypagediskfaultscpu
rbwswapfreeremfpipofrdesrs0s1s3insycsus
syid

002078628485390321165959219011100033825865425
293
ViewingCPUProcessLists TheCPUprocesslist(~/CONF/cpu_processlist)identifies,describes,and
configuresallthecoreprocessesrunningontheNE.
WhentheNEstartsup,eitherduringanNErestartorfromthecpu_start
command,theCPUmonitorprocess,child_mon,starts.Thechild_monprocess
performsthefollowing:
Startstheapplicationprocessesintheorderlistedinthecpu_processlist
file.
Monitorsthehealthofindividualprocesses,andtakesrecoveryactionifit
detectsfailure.
Whenchild_monreachesthepointwherethecoreprocessesarerunning,it
detachesfromtheterminalandrunsinthebackground.Thecpu_stopcommand
stopschild_monwhichinturnstopstheapplicationprocesses.
RefertothespecificNEguideforadditionalinformationontheprocessesrunning
ontheNE.

ViewingProcessorUtilization 119
ViewingProcessor UsethetopcommandtocheckprocessorutilizationontheNE.Checkprocessor
Utilization utilizationwhenthereisaheavyvolumeoftrafficonthenetwork.CPUutilization
thresholdsaresitespecificanddependontheconfigurationoftheNE.Contact
CustomerServiceforinformationonoptimalandabnormalprocessorutilization
foryoursite.
Tocheckprocessorutilization:
1 LogintotheNEasmsc,pro,lis,orfsv,asappropriate.
2 Atthecommandprompt,enterthefollowingcommand:
top
3 Toexitthecommand,enterq.
CheckingMemory Bothuserandsystemprocessesuseavailablememory.Certainprocesses(suchas
Utilization cronjobs)mayusememoryinadifferentwaythannormalNEapplication
processing.Whenmemoryrequirementsforcurrentprocessesexceedsthe
availablememorythenthesystemwillusepaging(movingsectionsofprocess
memorytodisk)andswapping(writinganentireprocesstodiskandfreeingup
physicalmemory).
MemoryUtilizationGraphs Figure 24showsanexampleofamemoryutilizationgraph.

andReports
Figure 24MemoryUtilizationGraph
SeeTable 17forwhatismeasuredinmemoryutilizationgraphsandreports.

PerformanceCharacteristics
Shorttermspikesinmemoryutilizationarenormal,butregularorsustained
memoryissuescanbeindicatedbythefollowing:
Availablefreememoryshouldgenerallyalwaysbebetween5%15%.
TotalmemoryisshownbyMEMTOTALandcalculationsneedtobeperformed
tocalculate%agefreememory.
Heavypagingactivity.
CheckingMemoryUtilization Checkmemoryutilizationduringperiodsofhightraffic,orifotherperformance
dataseemstoindicateaprobleminthesystem(likehighCPUutilization).High
memoryutilizationcouldmeanthatsomejobsorprocessesarenotrunningas
theyshould.
MemoryutilizationisalsocheckeddailyaspartoftheOperatortasks.Useany
memoryutilizationreportsgeneratedbytheOperatortohelpdiagnosethe
problem.
Formoredetailedinformationonmemoryutilizationgraphsandreports,see
MemoryUtilizationGraphsandReportsonpage 119.
ToCheckMemoryUtilization:
1 GenerateandexamineMemoryUtilizationgraphfortheNE.
2 ExamineandcomparedailyorweeklyMemoryUtilizationreports.
3 Ifmoresystemperformancedataisneeded,tryrunningvmstat.
4 Ifpagingactivityishigh(po)thencheckaSwapUtilizationgraphorchart.
Pagingactivityandfreememoryarerelated.
SwapUtilizationGraphsand TheOSdistributesavailablememoryamongcompetingprocessesbypagingand
Reports swapping.Whentotalmemoryexceedsphysicalmemorythenmemorycanbe
swapped(writinganentireprocesstodiskandfreeingupphysicalmemory)and
paged(movingsectionsofprocessmemorytodisk).
Figure 25showsanexampleofaswaputilizationgraph.

Figure 25SwapUtilizationGraph
SeeTable 17forwhatismeasuredinswaputilizationgraphsandreports.
Highuseofswapspace(increasingpagingactivity)indicatesashortageof
availablefreememory.
CheckingSwapUtilization Checkswaputilizationduringperiodsofhightraffic,orfirstifotherperformance
dataseemstoindicateaprobleminthesystem.
ToCheckSwapUtilization
1 GenerateandexamineSwapUtilizationgraphfortheNE.
2 ExamineandcomparedailyorweeklySwapUtilizationreports.
SwaputilizationshowsonlySWAPFREEandnotswapused.
3 Ifmoresystemperformancedataisneeded,tryrunningtop.
SWAPTOTAL=SWAPFREE+swapused.
4 Executetheswapcommandforadditionalinformation.
Listspagingareas(swapblocks)andhowmuchspaceiscurrentlyused(see
Example16).
Example16swapCommand
pro>swapl
swapfiledevswaploblocksfree
/dev/dsk/c1t0d0s132,1161678020816463664

DiskI/OGraphsandReports DiskI/Oisalsoanareawhereperformancecanbeimpacted.Therearetwotypes
ofdiskI/O:sequentialaccess(wheredataisreadindiskblockorder,oneafter
another),andrandomaccess(dataisreadinnoparticularorder).TheNEsuse
sequentialaccess,andthespeedofthediskI/Oislimitedbydisktransferrates
(tps=transactionspersecond).
ThediskI/Otypeisconfiguredatinstallationandperformancetuninghasa
limitedimpactondiskI/O.ThefactorsthatdoimpactdiskI/Oarediskhardware,
datadistributiononsystemdisks,anddataplacementonthephysicaldisk.
Figure 26showsanexampleofadiskI/Ograph.
Figure 26DiskI/OGraph
SeeTable 17forwhatismeasuredindiskI/Ographsandreports.
OfthefactorsthatdoimpactdiskI/Operformance(diskhardware,data
distributiononsystemdisks,anddataplacementonthephysicaldisk),data
placementisreallytheonlyaspectthatcanbeoptimized.AdegradingdiskI/O
performancemaybetheresultoffragmentation(noncontiguousfiles),anddisk
accesstimewillincrease.Fragmentationbuildsovertimeanditmaybe
necessarytouseadefragmentingutilitytocleanupthedisk.
DiskI/OmeasuresthepercentageoftimethediskisinI/Oactivities(%busyinthe
sarcommand).
DiskI/OontheNEwillbeheavierforsometasks(likebackingup),andinsome
configurations(HighAvailability).

CheckingDiskI/O CheckdiskI/Oduringperiodsofhightraffic,orifotherperformancedataseems
toindicateaprobleminthesystem.DiskI/Oisalsoanareawhereperformance
canbeimpacted.
ToCheckDiskI/O
1 GenerateandexamineDiskI/OgraphfortheNE.
2 ExamineandcomparedailyorweeklyDiskI/Oreports.
3 Ifmoresystemperformancedataisneeded,tryrunningiostat.
SeeExample17.
Example17iostatCommand
pro>iostat
ttysd0sd1sd30nfs1cpu
tintoutkpstpsservkpstpsservkpstpsservkpstpsservussy
wtid
233385111400000000052192
Diskusageisshownintheiostatcommandoutputastps(transferspersecond)
andkps(kilobytepersecond).
4 Alsoexecutethesarcommand.
SeeExample18.
Example18sarCommand
pro>sard105
SunOSparis5.8Generic_11735011sun4u07/05/05
10:04:23device%busyavquer+w/sblks/savwaitavserv
10:04:33nfs100.0000.00.0
sd010.0150.04.8
sd0,a00.0140.03.7
sd0,b00.0000.00.0
sd0,c00.0000.00.0
sd0,d00.0000.00.0
sd0,e00.0000.00.0
sd0,h00.0010.08.5
sd100.0000.00.0
sd1,a00.0000.00.0
sd1,b00.0000.00.0
sd1,c00.0000.00.0
sd1,d00.0000.00.0
sd1,e00.0000.00.0
sd1,h00.0000.00.0
sd3000.0000.00.0
ohci0,bu00.0000.00.0
ohci0,ct00.0000.00.0
ohci0,in00.0000.00.0
ohci0,is00.0000.00.0
ohci0,to00.0000.00.0
Needrootpermissiontorunsar.
NetworkI/OGraphsand NetworkI/OrecordsthenumberofIPpackets(TCPandUDP)inandout.The
Reports graphandreportareequivalenttonetstati,andisthesumofIpktsand
Opkts.
Figure 27showsanexampleofanetworkI/Ograph.
Figure 27NetworkI/OGraph
SeeTable 17forwhatismeasuredinnetworkI/Ographsandreports.
NetworkI/OcanbeheavyduetoFTP(fromcronjob)orothertemporary
networkintensiveactivity.SustainedheavynetworkI/Oindicatesaproblem.
VerylownetworkI/Ocanalsoindicateaproblem.
NetworkI/Operformanceisaffectedby:
Networkinterfaceproblems(hardwaremisconfigurationorfailure)
Networkdevices(Ethernetswitch)usuallyshowsdegradedperformanceprior
tooutrightfailure.
CheckingNetworkI/O ChecknetworkI/Oduringperiodsofhightraffic,orfirstifotherperformance
dataseemstoindicateaprobleminthesystem.
ToCheckNetworkI/O
1 GenerateandexamineNetworkI/OgraphfortheNE.
2 ExamineandcomparedailyorweeklyNetworkI/Oreports.
3 Ifmoresystemperformancedataisneeded,tryrunningnetstat.
SeeExample19.
Example19netstatiCommand
pro>netstati
NameMtuNet/DestAddressIpktsIerrsOpktsOerrsCollisQueue
lo08232loopbacklocalhost754102407541024000
bge01500londonlondon36110708023297469000

CheckingDiskSpace 125
Statisticsaresincelastboot.
4 Ifnetstatoutputshowserrors,usePtcpoptiontoisolateproblems.
SeeExample20.
Example20netstatPtcpCommand
pro>netstatsPtcp
TCPtcpRtoAlgorithm=4tcpRtoMin=400
tcpRtoMax=60000tcpMaxConn=1
tcpActiveOpens=236361tcpPassiveOpens=392373
tcpAttemptFails=35654tcpEstabResets=17
tcpCurrEstab=14tcpOutSegs=28754582
tcpOutDataSegs=19836686tcpOutDataBytes=2404942730
tcpRetransSegs=284tcpRetransBytes=95478
tcpOutAck=8917361tcpOutAckDelayed=6899267
tcpOutUrg=0tcpOutWinUpdate=1
tcpOutWinProbe=4tcpOutControl=1310531
tcpOutRsts=88600tcpOutFastRetrans=39
tcpInSegs=30390607
tcpInAckSegs=19740913tcpInAckBytes=2405516481
tcpInDupAck=717941tcpInAckUnsent=0
tcpInInorderSegs=18892215tcpInInorderBytes=2077346869
tcpInUnorderSegs=152tcpInUnorderBytes=201186
tcpInDupSegs=181tcpInDupBytes=190
tcpInPartDupSegs=0tcpInPartDupBytes=0
tcpInPastWinSegs=0tcpInPastWinBytes=0
tcpInWinProbe=0tcpInWinUpdate=4
tcpInClosed=10tcpRttNoUpdate=128
tcpRttUpdate=19303300tcpTimRetrans=393
tcpTimRetransDrop=19tcpTimKeepalive=3185
tcpTimKeepaliveProbe=1401tcpTimKeepaliveDrop=0
tcpListenDrop=0tcpListenDropQ0=0
tcpHalfOpenDrop=0tcpOutSackRetrans=58
5 Checkanyhardwareforconfigurationorpossiblefailure.
EspeciallyEthernetswitch(oranyothernetworkdevices).
CheckingDiskSpace Checktheavailablediskspaceatthebeginningofeachshift.Recordtheavailable
diskspacefortherootandapplicationslicesintheshiftchecklist.
ToCheckDiskSpace
SeeLoggingInandOutinChapter 5,UserAccounts.
2 Checkavailablediskspace.
pro>dfk
Therootslice(/dev/dsk/c1t0d0s0)containstheOperatingSystem,mountedon
/,andshouldnotexceed80%usedcapacity.
Theapplicationslice(/dev/dsk/c1t1d0s7)containstheNEapplication,
mountedon/export/home.Thereisasoftlinkfrom/export/hometo/telos.
Theusedcapacityshouldnotexceed80%.
Record%agecapacityonrootslice.
Record%agecapacityonapplicationslice.
SeeExample21.
Example21CheckDiskSpace
pro>dfk
FilesystemkbytesusedavailcapacityMountedon
/dev/dsk/c1t0d0s081682373950049413650649%/
/proc0000%/proc
fd0000%/dev/fd
mnttab0000%/etc/mnttab
swap112518328112518241%/var/run
swap112540242200112518241%/tmp
/dev/dsk/c1t1d0s7705925051989513678970673%/export/home
/dev/dsk/c1t0d0s75425599953817536596231%/export/home2
CheckingLogUsage Checkalllogfiles(LOG*)in/RUN.
Ausersessionshouldalreadybeestablished.Ifnot,seeLoggingInandOutin
Chapter 5,UserAccounts.
ToCheckLogFileSize
1 Changetothelogfiledirectory.
pro>cd~/RUN
2 Listlogfiles
pro>lslLOG*
SeeExample22.
Example22CheckLogUsage
pro>cd~/RUN
pro>lslLOG*

CheckingTCP/IPLinkStatus 127
CheckingTCP/IPLinkStatus Usethelink_statuscommandtocheckthestatusoftheTCP/IPlinksbetween
NEsandotherdevices.
TodisplaytheTCP/IPlinkstatus:
1 Loginasmsc,pro,lis,orfsv,asappropriate.
2 Atthecommandprompt,typelink_statusandpresstheEnterkey.
3 Thesystemdisplaysthedatalinktable,containingthestatusofTCPlinkstothe
SignalingServer.
PerformanceManagement Performancemanagementactivitiesaretasksthatareperformedtomonitorthe
Activities performanceoftheNEandthesystem.Sometasksareperformedforother
reasons(likenormalOperatorshifttasksortroubleshootingtasks)butthese
taskscanalsobeusedinapreventativewaytoflagpotentialproblemsbefore
theybecomemoretroublesome.
Executetheproceduresinthefollowingsectionstomanageperformanceonthe
NEandinthesystem.
Thecontrolofsystemresources(OS)isgenerallylimitedtothemechanismslisted
inTable 19butnotallofthemechanismscanbeusedtooptimizeperformance
ontheNE.
Table 19SystemResourceControlMechanisms
Resource ControlMechanism
Nicenumbers
Processpriorities
Batchqueues
CPU Schedulerparameters
Processresourcelimits
Memorymanagementrelatedparameters
Memory Paging(swap)space
Filesystemorganization(acrossdisks)
Fileplacementondisk
DiskI/O I/Orelatedparameters
Networkmemorybuffers
Networkrelatedparameters
NetworkI/O Networkinfrastructure

SYSTEMT IMEMANAGEMENT
9
Checkingandmanagingthesystemtimeisanadministrationtaskthatneedsto
beperformedoccasionally.
SystemTimeOverview
ConfiguringtheNTPClient
NTPServer/ClientConfiguration
CheckingtheNTPServer
SynchronizingNTPServersandClients
CheckingtheSystemTime
ConfiguringTimeZonesandDaylightSavingsTime
SystemTimeOverview SystemdateandtimearethedateandtimethatisdisplayedbytheNE(the
system).TheNEcontainsanoscillatorandsomehardwaretointerfacethe
oscillatortotheCentralProcessingUnit(CPU)intheNE.Instabilityinthe
oscillator,andlatencyinhardware(wander)andsoftware(jitter)cancauseerrors
inthesystemclock.
Overtime,thesystemclockwilldivergefromthe'real'time,andthesystem
clocksofothersystemsinthenetwork.Asolutiontotheproblemofreal,or
authoritative,timeinthenetworkisaddressedbytheNetworkTimeProtocol
(NTP).
NTPworksinahierarchical,client/serverfashion,whereauthoritativetime
valuesmovedownfromtoplevelservers,tolowerlevelservers,downtothe
clientlevel.Areferenceclockgeneratestheauthoritativetime,andcanbe
sourcedfromanInternetconnectiontoNTPservers,orfromaGlobalPositioning
System(GPS)(amongothers).
TheNEactsasaclientwhenconnectedtoanNTPserver.NTP(runningontheNE)
willmakeadjustmentstothesystemclockbasedonauthoritativetime.Ifsystem
timediffersbymorethan128milliseconds,NTPwillresetthesystemclock.The
NTPdaemonwillcorrectdrift(offsetbetweentwoclocks)andupdatethesystem
clockonanongoingbasis.
NTPalsoaddressestheproblemofmachinesinthenetworkhavingdifferent
systemtimes.AllmachinesconnectedtoanNTPclocksourcewillhavethesame
systemtime.NTPserversusetheUniversalTimeCoordinated(UTC)format.

130 Chapter9:SystemTimeManagement
SolarisalwaysusesUniversalTimeCoordinated(UTC)formatforinternaltiming.
Forexample:ThuNov1015:08:58GMT2005.
AdditionalinformationontheNetworkTimeProtocolisavailableat
http://www.ntp.org.
SystemTime TheNEactsasanNTPClienttotheprimaryNTPsource.TheprimaryNTPsource
canbeahigherlevelNTPServer,oraGPSclocksource.
TheMSCactsasanNTPServertoalloftheNEsintheconfiguration.Theother
NEs(suchastheAccessServer,SSVR,orOMC)areNTPClientstotheNTPServer
(MSC).IftheprimaryNTPServersourcefails,theNEwilluseitsownclock.
ConfiguringtheNTPClient ATemplatefileexistsontheNEas/etc/inet/ntp.client.SeeExample23on
page 131foranexampleofthentp.clienttemplatefile.
AnNTPclientsynchronizesautomaticallywithanNTPserverwhenitboots.Ifthe
clientbecomesunsynchronized,theclientresynchronizesagainwhentheclient
contactstheNTPServer.
Thexntpddaemonreadsthe/etc/inet/ntp.conffileatsystemstartup.
ToconfiguretheNTPclient:
1 Connecttothetargetclient.
AnNTPClientisanNetworkElement(forexample,AccessServerorCall
ConferencingServer)thatusestheNTPServer.
2 BecomesuperuseronthetargetNTPClient.
3 Changetotheconfigurationdirectory.
#cd/etc/inet
4 Copythetemplatefile.
#cpntp.clientntp.conf
5 Openntp.confforeditingandaddtheNTPClientdetails.
SeeSingleAvailabilityNTPConfigurationandHighAvailabilityNTPConfiguration
foradditional.
6 Addthedriftfilelocationtothefile.
driftfile/etc/ntp.drift
7 Saveandclosethefile.
8 Startthexntpddaemon.
#/etc/init.d/xntpdstart
9 Exitthetargetclient.
SeeSingleAvailabilityNTPConfigurationandHighAvailabilityNTPConfiguration
fordetailsonhowtoeditthentp.clientfile.

NTPServer/ClientConfiguration 131
Example23ntp.clientTemplateFile
#@(#)ntp.client1.296/11/06SMI
#
#/etc/inet/ntp.client
#
#Anexamplefilethatcouldbecopiedoverto/etc/inet/ntp.conf;it
#providesaconfigforahostthatpassivelywaitsforaserver
#toprovideNTPpacketsonthentpmulticastnet.
NTPServer/Client TheNTPServerandClientconfigurationvariesdependingonwhethertheyarein
Configuration aSingleAvailability(SA)orHighAvailability(HA)configuration.
SingleAvailabilityNTP Figure 28showsasampleconfigurationfortheNTPServerandNTPClientsinthe

Configuration SingleAvailabilityconfiguration.
Figure 28SANTPDeploymentScenario
GiventheIPaddressingshowninFigure 28,addthefollowingentriestothe
/etc/inet/ntp.confontheNE:
server172.16.1.1prefer
server127.127.1.0
TheNEclockselfreferenceIPaddressis127.127.1.0,andnotthelocalhostIPof
127.0.0.1.

/etc/inet/ntp.confoneachNetworkElement:
HighAvailabilityNTP Figure 29showsasampleconfigurationfortheNTPServersandNTPClientsin

Configuration theHighAvailabilityconfiguration.
Figure 29HANTPDeploymentScenario
/etc/inet/ntp.confonMSCA:
server127.127.1.0
TheNEclockselfreferenceIPaddressis127.127.1.0,andnotthelocalhostIPof
127.0.0.1.

CheckingtheNTPServer 133
/etc/inet/ntp.confoneachNetworkElementonsideA:
server10.10.1.3
/etc/inet/ntp.confonMSCB:
server10.10.1.2
/etc/inet/ntp.confoneachNetworkElementonsideA:
server10.10.1.2
CheckingtheNTPServer CheckthestateoftheNTPServerandNTPClients,andconfirmthatthentpd
daemonisstillrunning.InaHighAvailabilityconfiguration,thentpqp
commandexecutedontheNEwillprintalistoftheNTPClients(MSCAand
MSCB)knowntotheNTPreferenceclock,andasummaryoftheirstates.
ExecutethecommandoneachNetworkElementtochecktheNTPServerand
Clientstate.
SynchronizingNTPServers NTPServersandNTPClientsneedtobesynchronized.NTPwillmanagesmall
andClients discrepanciesbetweenNTPServersandNTPClientsbutcannotmanagebig
differences.NTPServersandNTPClientsshouldbesynchronizedafterinitial
configuration,orifforsomereason,thedifferenceissogreatthatthexntpd
daemonterminates.
Thexntpddaemonchecksthesynchronizationatstartup,eitheratsystemrestart
orwhenthexntpddaemonismanuallyrestarted.ConfigureallNTPServersand
NTPClientsbeforesynchronizing.
ToSynchronizeNTPServerandClients:
1 ConnecttothetargetNEandloginasmsc,pro,orfsv,asappropriate.
2 Changetosuperuser:
pro>suroot
3 Stopthexntpddaemon.
root>sh/etc/rc2.d/S74xntpdstop
4 Startthexntpddaemon.
root>sh/etc/rc2.d/S74xntpdstart
5 Changebadktothemsc/pro/lis/fsvuserandthenexit,ifrequired.
#exit
pro>exit

CheckingtheSystemTime ToChecktheSystemTime:
1 ConnecttotheNEandloginasmsc,pro,orfsv,asappropriate.
2 Checkthesystemdateandtime(adjustedfortimezone).
pro>date
ThuNov1015:08:58PST2005
3 CheckthesystemdateandtimeinUTCformat.
pro>dateu
ThuNov1023:11:02GMT2005
ConfiguringTimeZones ThesystemtimefromtheNTPServerisinUniversalTimeCoordinated(UTC)
andDaylightSavingsTime format.SolarisalwaysusesUniversalTimeCoordinated(UTC)formatforinternal
timing,andisnotaffectedbychangesintimezoneanddaylightsavings.Solaris
usuallydisplayssystemtimeinnonUTCformat(adjustedfortimezones).Some
applicationsandprocesses,likeCallDetailRecords(CDR)useUTCtimebut
displaysystemtime(nonUTCtime).
IntheHighAvailabilityconfiguration,bothMSCAandMSCBneedtobe
configuredforthesametimezoneandsystemtimeformat(UTC,nonUTC).
ConfiguringTimeZones ToConfigureTimeZones
3 Changedirectory.
#cd/etc/default
4 Openinitforediting.
#viinit
5 Addalinethatpointstothedesiredtimezonefilelocation.
Forexample,TZ=Canada/Pacific
7 Changebacktothemsc/pro/lis/fsvuserandstoptheNEapplications.
#exit
pro>cpu_stop
8 Restartthesystem.
9 LogintotheNEasmsc,pro,lis,orfsv,asappropriateandrestarttheNE.
pro>cpu_start

ConfiguringUTCTimeDisplay SolarisalwaysusesUniversalTimeCoordinated(UTC)formatforinternaltiming.
Forexample,ThuNov1015:08:58GMT2005.Systemtimecanbeconfiguredto
displaytimeinUTCformat.TheCallDetailRecordapplicationcanusesystem
timeinUTCornonUTCformat.
ToConfigureUTCTimeDisplay
3 Changedirectory.
#cd/etc/default
4 Openinitforediting.
#viinit
5 ChangethetimezonetoUTCformat.
TZ=utc
7 Changebacktothemsc/pro/fsvuserandstoptheNEapplication.
#exit
pro>cpu_stop
8 Restartthesystem.
9 LogintotheNEasmsc,pro,orfsv,asappropriateandrestarttheNEapplication.
pro>cpu_start
CompilingTimeZoneFiles
Solarisprovidestimezonefilesformostcommontimezones.Timezonefilescan
begeneratedforsystemsintimezonesnotprovidedbydefault.Table 20liststhe
defaultcompiledtimezonessuppliedbySolaris.
Table 20DefaultCompiledTimeZones
DefaultCompiledTimeZones
Asia Australia Brazil Canada CET Chile
CST6CDT Cuba EET Egypt Eire EST
EST5EDT Factory GB GBEire GMT GMT+0
GMT+1 GMT+10 GMT+11 GMT+12 GMT+13 GMT+2
GMT+3 GMT+4 GMT+5 GMT+6 GMT+7 GMT+8
GMT+9 GMT0 GMT1 GMT10 GMT11 GMT12
GMT2 GMT3 GMT4 GMT5 GMT6 GMT7
GMT8 GMT9 Greenwich Hongkong HST Iceland
Iran Israel Jamaica Japan Kwajalein Libya
MET Mexico Mideast MST MST7MDT Navajo
NZ NZCHAT Poland Portugal PRC PST8PDT
ROC ROK Singapore Turkey UCT Universal
US UTC WSU WET Zulu

Solarisprovidesautility,zic(timezonecompiler),tocompiletimezonefiles.If
thetimezoneforthesystemisnotrepresentedinTable 20,thenuseTable 21to
locatethecontinentalregionandcompiletherequiredtimezonefiles.
Table 21liststhecontinentalareasdefinedinSolaris.
Table 21ContinentalAreas
ContinentalAreas
africa asia europe pacificnew
antarctica australasia northamerica southamerica
ToCompileTimeZoneFiles
3 Changetothetimezonesourcefiledirectory.
#cd/usr/share/lib/zoneinfo/src
4 Compilethetimezonefile.
#/usr/sbin/zic<continentalarea>
5 Changetothetimezonedirectoryandconfirmthatthenewtimezoneiscreated.
#cd..
#ls
Thenewtimezonedirectory(ifcreated)andthenewtimezonelocation
informationarecreated.Solarisdoesnotprovidetimezoneinformationfor
Antarctica.
Example24showshowtocompileanewtimezonefileforAntarctica.
Afterthetimezonefilesarecreatedtheycanbeusedin/etc/default/init.
Forexample,tousethesouthPoletimezoneasthesystemtime,add
TZ=Antarctica/South_Poleto/etc/default/init.
Example24CompileNewTimeZoneInformation
pro>suroot
Password:
root>cd/usr/share/lib/zoneinfo/src
root>ls
READMEaustralasiafactorysolar88
africabackwardnorthamericasolar89
antarcticaetceterapacificnewsouthamerica
asiaeuropesolar87systemv
root>/usr/sbin/zicantarctica
root>cd..
root>ls
AntarcticaEtcGMT+5GMT6LibyaROK
AsiaFactoryGMT+6GMT7METSingapore
AustraliaGBGMT+7GMT8MSTTurkey
BrazilGBEireGMT+8GMT9MST7MDTUCT
CETGMTGMT+9GreenwichMexicoUS
CST6CDTGMT+0GMT0HSTMideastUTC
CanadaGMT+1GMT1HongkongNZUniversal
ChileGMT+10GMT10IcelandNZCHATWSU
CubaGMT+11GMT11IndianNavajoWET
EETGMT+12GMT12IranPRCZulu
ESTGMT+13GMT2IsraelPST8PDTposixrules
EST5EDTGMT+2GMT3JamaicaPolandsrc
EgyptGMT+3GMT4JapanPortugal
EireGMT+4GMT5KwajaleinROC
root>cdAntarctica
root>ls
CaseyDumontDUrvilleMcMurdoSouth_Pole
DavisMawsonPalmerSyowa

DATABASEANDSYSTEMBACKUPANDRESTORE
10
Thischapterdescribeshowtobackupandrestoretheconfigurationdatabase
andtheCoreNetworkElementsystems.
DatabaseBackupOverview
BackinguptheConfigurationDatabase
RestoringtheConfigurationDatabase
SystemBackup
SystemBackupUsingLocalTapeDrive
SystemBackupUsingRemoteTapeDrive
HardwareRAIDNetra440HLRSystemBackup
SystemRestore
SystemRestoreUsingLocalTapeDrive
SystemRestoreUsingRemoteTapeDrive
HardwareRAIDNetra440HLRSystemRestore
DatabaseBackupOverview Thissectionincludes:
DatabaseOverview
DatabaseSchema
TableVersioning
AboutDatabaseBackups
DatabaseOverview Theconfigurationdatabaseisasetoftablesthatcontainsetsofparametersand
values.TheseparametersandvaluesdefinetheconfigurationoftheNetwork
Element(NE),theinterfacesforattachedNEs,andtheconfigurationforthe
networkasawhole.
Tableparametersandvaluesaredefinedduringtheinitialinstallationand
provisioningphases.Furtherchangestonetworkconfigurationmayberequired
byconfiguration,performance,andfaultmanagement.
RefertotheProvisioningsectionofthisguideforinformationontheparameters
andvaluesinthesetables.

140 Chapter10:DatabaseandSystemBackupandRestore
DatabaseSchema Adatabaseschemaisarepositorythatcontainsinformationaboutthestructure
andcontentofagivendatabase.Itisthemodelforwhatthedatacontainsand
howitisstored.
Thedatabaseschemaisdefinedbythestructureofeachtableinthedatabase.
ThestructureofeachtableisdefinedinthePROTOfileforthattable.For
example,vsm_globals.db.protoisthetemplateformofthedatabasetable
vsm_globals.db.
Templatedatabasefilesarein/telos/<account>/CONF(~/CONF)anddatabase
tableswithactualdataarestoredin/telos/<account>/DATA(~/DATA).(Where
<account>willbemsc,fsv,pro,orlisasappropriatefortheNE.)
EmergencyPatches,PatchLevelreleases,andPointlevelupgradescanaffectthe
structureofoneormoretablesintheconfigurationdatabase.Databasebackups
cannotbeusedtorestoreadatabasethathashadtheschemaaltered,duetothe
applicationofanEmergencyPatch,PatchLevelrelease,orPointVersionupgrade.
Anupgradetoolisavailabletoupgradethedatabasetablesandkeepexisting
dataintheeventofaschemachange.
TableVersioning Somesoftwareupgradesrequireachangetoadatabasetable,andanincrement
ofthedatabasetableversionnumber.TheReleaseNotesthataccompanyeach
softwareupgradespecifywhichtablesneedtohavetheversionnumber
incremented.
Thetemplatedatabasefile(*.protoin~/CONF)musthavethesameversion
numberasthedatafile(*.db).
Backupthedatabasebeforeperformingasoftwareupgrade.Ifthesoftware
upgradeneedstoberolledback,andtheupgraderequiredadatabasetable
versionchange,thentheversionnumberinthetemplatefilemustbechangedto
matchthepreviousversion.Thedb_restorescriptchecksthetableversion
number,andwillnotpermittherestoreofadatabasewhereanyoneofthe
tableshasaversionmismatch.
Example25showsthatthetableversioninbold(31)forthe
vsm_switch_msg_timers.dbdatabasefileisthesameasforthetemplatefile
vsm_switch_msg_timers.db.proto.Anymismatchintableversionnumbers
willpreventadatabaserestore.
Example25CompatibleTableVersions
pro>more~/DATA/vsm_switch_msg_timers.db
#31FORMATVUi,i;
pro>more~/CONF/vsm_switch_msg_timers.db.proto
#31FORMATVUi,i;
AboutDatabaseBackups Backingupandrestoringthedatabaseisanimportantmaintenancetaskthat
ensuresthedatabasecanbereadilyrestored,ifnecessary.

Databasetablescanbebackedupmanually(seeManualDatabaseBackups)or
automaticallyaspartofacronjob(seeAutomaticDatabaseBackups).
Thebackupprocedureverifiestheharddiskiswritableandcreatesavolume
labeltoidentifythedataformatanddate.
Duringtherestoreprocess,thevolumelabelisverified,andiftheversion
numbersdonotmatch,anerrormessagepromptstoaborttherestore
procedure.Thispreventsdatabasecorruptionbyrestoringadatabasecreated
fromanearlierincompatibleversionofthesoftware.
BackupDataLocation
Databackeduptoharddiskisstoredinthe/telos/<account>/DATA(~/DATA)
directory.(Where<account>ismsc,fsv,pro,orlisasappropriatefortheNE.)
BackupNamingConvention
Alldatabasebackups,whethercreatedmanuallyorautomatically,havethesame
filenamesyntax.Thedatabasebackupfilenamefollowstheformat:
DBD<year><month><day><hour:minute>.Forexample,
DBD2005112703:14.
Backingupthe Databasebackupscanbeautomatedviaacrontabjoborperformedmanually.
ConfigurationDatabase Referto:
AutomaticDatabaseBackups
ManualDatabaseBackups
ManagingDatabaseBackupFiles
AutomaticDatabaseBackups Thetaskofbackingupthedatabasecanbeautomatedwiththehelpofacrontab
file.
Thecrontabfileisusedbythecrondaemontorunspecifiedjobsatscheduled
times.Adatabasebackupcronjobisinsertedinthecrontabfileinthemsc,fsv,
pro,andlisaccounts(nottherootaccount)inordertoscheduleadatabase
backupautomatically.
ThecronjobontheNErunsthedb_autocommand.db_autocreatesabackup
fileandremovesDBDfilesolderthan10days.RefertoManagingDatabase
BackupFilestokeepfilesformorethan10days.
Example26showsanexampleofthecrontabfilefortheprouseronanSSVR.
Theautomaticdatabasebackupjobisshowninbold.
Bydefault,thecronjobrunsthedb_autocommandat3:14amdaily.

Example26SSVRcrontabFile
#minhrdaymonwkdycommand
#
142***[x/telos/msc/TOOL/clean_dirs]&&
/telos/pro/TOOL/clean_dirs>/dev/null2>&1
143***[x/telos/msc/TOOL/db_auto]&&
/telos/pro/TOOL/db_auto>/dev/null2>&1
1****[x/telos/msc/TOOL/check_disk]&&
/telos/pro/TOOL/check_disk>/dev/null2>&1
0,5,10,15,20,25,30,35,40,45,50,55****/usr/local/bin/sys_mirror>
/dev/null2>&1
*****/telos/pro/mysleep.sh
Understandingcrontab
Thetablesthatfollowprovideaquickreviewofthecrontabcommand.Table 22
describesthecommandoptionsandTable 23describesthecommandsyntax.
Table 22crontabCommandOptions
Option Description
e Editthecurrentcrontabfile.EditorisbasedontheEDITORenvironment
variable.
l Listallthejobentriesinthecrontabfile.
r Removesallofthejobentriesinthecrontabfile.
Table 23crontabSyntaxDescription
Parameter ValidValues
DateandTimeFormats [minute][hour][dayofmonth][month][day
ofweek]
[minute] 0to59
[hour] 0to23
[dayofmonth] 1to31
[month] 1to12
[dayofweek] 0to6(Sundayis0)
* Anasterisk(*)inanydateortimefieldmeans'all'.
, Acomma(,)canbeusedtoseparatetimesanddatesinthe
samejobdefinition.
VerifyingandChangingtheAutomaticDatabaseBackupSchedule
ToVerifytheAutomaticBackupSchedule:
1 LogintotheNEasmsc,pro,orfsv,orlisasappropriate.
2 Listthejobsinthecrontabfile:
msc>crontabl
3 Examinetheoutputtoverifythebackuptime.Thelineofinterestcontains
db_auto.

ToChangetheAutomaticBackupSchedule
1 LogintotheNEasmsc,pro,fsv,orlis,asappropriate.
2 Openthecrontabfileforediting:
pro>crontabe
3 Makethedesiredchanges.
Saveandclosethefile:
:wq
ManualDatabaseBackups Daytodaychangestothedatabasecanbebackedupbytheautomaticdatabase
backupfacility(seeAutomaticDatabaseBackups).
Thedatabasecanalsobebackedupmanuallyifrequiredinoneofthefollowing
situations:
Aftermakinganysignificantnetworkconfigurationchange.
Beforeperformingsoftwaremaintenance.
Iftheautomaticdatabasebackupfacilityfails.
Backupthedatabasemanuallywiththedb_backupcommand.Thefollowing
procedureassumesthedatabaseisbackeduptodisk(recommended).
ToManuallyBackUptheDatabase:
2 Runthedatabasebackupcommand:
pro>db_backup
3 Pressy,andthenpressEntertocontinue.
4 Savethedatabasebackuptodisk,byselecting:
3.Diskfile
Thebackupprocesscreatesthebackupfile,displaysthefilename,andverifies
themedia.Theterminalsessiondisplaysthemessage"DatabaseBackup
successfull."andreturnstheusertothecommandpromptwhentheprocessis
complete.
SeeExample27forasamplesessionthatmanuallybacksupthedatabase.
Example27ManualDatabaseBackup
pro>db_backup
DatabaseBackupProcedure
WhilethebackupisbeingperformednochangestoANYdatabaserecords
areallowed.Ifarecordischangedwhilebackupisproceedingthen
databasecorruptioncanoccur.
Doyouwishtoproceed?(y/n)y
Choosethelocationofthebackuponeof:
1.SCSITapeDevice
2.FloppyDisk
3.Diskfile
Type1,2or3thenpressENTER(1/2/3):3

WritingDatabasefilestobackupmedia
/telos/pro/DATA/DBD2005041915:17...
VerifyingDatabaseBackupmedia...
DatabaseBackupsuccessfull.
ManagingDatabaseBackup Thedb_autoscriptinthecronjobthatbacksupthedatabaseautomaticallyalso
Files removesalldatabasebackupsolderthan10days.Tokeepabackupformore
than10days,changethefilenamesothatthebackupfileisnotautomatically
removed.Itisrecommendedtousea.savextensionforconsistency.
ToKeepaDatabaseBackup:
2 Changetothedatabasebackupdirectory.
pro>cdDATA
3 Renamethebackupfiletobesaved.Thefollowingshowsanexampleof
renamingabackupfile:
pro>mvDBD2005112703:14DBD2005112703:14.sav
ThedatabasebackupisnowcalledDBD2005112703:14.savandwillnotbe
removedbythebackupcronjob.Thefilecanbemanuallyremovedorrenamed
atalaterdate.
Therestoreprocedurerequiresthatthebackupfiletoberestoredhasthe
standarddatabasebackupfilenamesyntax,otherwiseitwillnotappearinthelist
offilestorestorefrom.Anybackupthatistobeusedtorestorethedatabase
mustbenamedusingthestandardsyntaxsothattherestoreprocesswill
recognizethefile.Usethemvcommandtorenamethefile.

RestoringtheConfigurationDatabase 145
RestoringtheConfiguration Theconfigurationdatabasecanberestoredfromadatabasebackupfile.The
Database db_restorecommandrestoresaselecteddatabasebackup.Thecurrentactive
databasewillbeoverwrittenbythedatafromthebackup.
Thedb_restorecommandlistsallavailabledatabasebackupfiles(fromboth
manualandautomaticbackups).Thedb_autocronjobremovesalldatabase
backupsolderthan10daysfromthe~/DATAdirectoryontheNE,sounless
databasebackupswerekeptbyrenamingthebackupfile(seeManaging
DatabaseBackupFilesonpage 144),theoldestbackupis10daysold.The
db_restorecommandonlylistsbackupfilesthathavethebackupfileformat.
Duringthedatabaserestoreprocess,thevolumelabelisverified,andtherestore
processwillbehaltedifthetableversionnumbersdonotmatch.Thisprevents
databasecorruptionbyrestoringadatabasecreatedfromanearlierincompatible
versionofthesoftware.
Warning:
StoptheNEapplicationbeforerestoringthedatabase.
RestoringDatabaseinSingleAvailabilityConfiguration
Thereisanimpacttoservice(traffic)whiletheNEapplicationisstopped.
ToRestoreaDatabase(SingleAvailability)
2 StoptheNEapplication.
pro>cpu_stop
3 Runthedatabaserestorecommand.
pro>db_restore
4 Pressy,andpressEntertocontinue.
5 Restorethedatabasebackupfromdisk.
3.Diskfile
TheNEdisplaysalistofdatabasebackupfiles.
6 Selectthebackupfiletorestore.
OR,selectQuittoexitthecommandwithoutrestoringthedatabase.
7 Pressy,andpressEntertocontinue.
8 StarttheNEapplication.
pro>cpu_start
TheNEreloadsthenewconfiguration.
Ifthedatabaseisrestoredafterasoftwareupgradethatrequiredachangetothe
templatedatabasetablesthenthetemplatefilemustbemaunallyeditedto
restoreit.TheNEwillnotrestartofthereisamismatchbetweentableversions.
Example28showsanexamplesessionthatrestoresadatabase.

itwasnotaccidental.RefertoFigure 8onpage 60foranexample.
Example28DatabaseRestore
pro>cpu_stop
Sourcing/telos/msc/EXEC/hab_stop_hook
/telos/pro/EXEC/cpu_stop:running
pro>db_restore
DatabaseRestoreProcedure
WARNING:Thisprocedurewilldestroythepresentcontentsofyour
Database.YoushouldonlyrestoreaDatabaseifyouaresurethatthe
currentDatabaseisnotusable.
DoyouwishtoProceed?(y/n)y
Choosethelocationofthebackuponeof:
1.SCSITapeDevice
2.FloppyDisk
3.Diskfile
Type1,2or3thenpressENTER(1/2/3):3
1)DBD2005041916:58
2)DBD2005041915:41
3)DBD2005041915:17
4)DBD2005041903:14
5)QuitdoNOTrestorediskfile
Pickafiletorestorefromthelist:1
RestoringtheDatabaseBackupdoneonTueApr1916:59:00PDT2005
Doyouwishtocontinue?(y/n)y
DestroyingexistingDatabase...
RestoringDatabasefilesfrombackupmedia...
DatabaseRestoresuccessful.Youmaynowrestarttheapplication.
pro>cpu_start
RestoringDatabaseinHighAvailabilityConfiguration
Thereisnoimpacttoservice(traffic)inaHighAvailabilityconfiguration.TheNE
failsovertothestandbyNEwhentheNEapplicationisstopped.
ThesystemmirrorutilitymustalsobestoppedintheHighAvailability
configurationtopreventdatabasemirroringduringtherestore.
Warning:
StoptheNEapplicationbeforerestoringthedatabase.

SystemBackup 147
ToRestoreaDatabase(HighAvailability):
pro>cpu_stop
TheActiveNEfailsovertotheStandbyNE.
3 DisablethesystemmirrorutlilityontheformerActiveNE.
pro>sys_mirrorm[commentrequired]
Disablingthesystemmirrorrequiresacomment.Addadescriptivesentenceasto
whythedatabaseisbeingrestored.
4 RestorethedatabaseontheformerActiveNE.
Seesteps37inToRestoreaDatabase(SingleAvailability).
5 StarttheNEapplication.
pro>cpu_start
6 Enablethesystemmirror.
pro>sys_mirrorm
Donotaddacommentwhenenablingthesystemmirror.
TheformerActiveNEisnowactingastheStandbyNE.ThenewActiveNE(former
StandbyNE)willusethesystemmirrortopropagatethelastknownworking
configurationtothedatabaseonthenewStandbyNE(formerActiveNE).
Ifthedatabaseisrestoredafterasoftwareupgradethatrequiredachangetothe
templatedatabasetablesthenthetemplatefilemustbemaunallyeditedto
restoreit.TheMSCwillnotrestartofthereisamismatchbetweentable
versions.
SystemBackup AsystembackupisafulldumpoftheUNIXfilesystem,andincludestheNE
filesystem.ThesystembackupusestheUNIXufsdumputility.Thischapter
describeshowtobackupthesystemtotape,butotherstoragemediacanbe
used.
ThissystembackupprocedureappliestoSolaris5.8installationsonly.
ForasystembackupofamultidiskHLR(hardwareRAIDsystem),referto
HardwareRAIDNetra440HLRSystemBackuponpage 166.
PerformanentiresystembackupwhentheentireUNIXandapplication
filesystemshouldbebackedup.TheUNIXandapplicationfilesystemsshouldbe
backedupafterthesystemisinstalled,initiallyconfigured,andhaspassed
AcceptanceTestProcedures(ATP).Thissystembackupshouldbeusedinthecase
ofanymalfunctionwhichresultsinthelossofthefilesystem.

Routineconfigurationandprovisioningchangesarecapturedwithbackupsofthe
database(seeDatabaseBackupOverviewonpage 139).Ifasystemrestoreis
performed,anyconfigurationchangesapplied(andnotbackedup)afterthe
systembackupwillhavetobeappliedagainafterthesystemisrestored.
LocalandRemoteTapeDrives TheNEintherackcanbecolocatedwithatapedriveforsystembackups.When
thetapedriveiscolocatedwiththeNE,thetapedriveislocal.Ifthetapedrive
usedforsystembackupsisnotcolocatedwiththeNE,thenthetapedriveis
remote.
Whiletheprinciplesfordoingasystembackuparethesamewhenusingboththe
localandremotetapedrives,theactualproceduresdiffersomewhat.First
determinethelocationofthetapedriveusedforsystembackupandthenfollow
therelevantprocedures.Forperformingasystembackupusingacolocatedtape
drive,seeSystemBackupUsingLocalTapeDriveonpage 150.Forperforminga
systembackupusingaremotetapedrive,seeSystemBackupUsingRemoteTape
Driveonpage 154.
DiskMirroring EachNEhas2harddisks.ThelogicaldrivesontheNE(D0,D1,andD2)are
mappedtologicaldisks(D10,D11,D12andD20,D21,D22).Thelogicaldisksare
mappedtopartitionsonthephysicalharddisk.
Diskmirroringsoftware(DiskSuite)managesthereplicationfromtheprimary
disk(c0t0d0)tothesecondarydisk(c0t1d0)ontheNE.Thedisksarec0t0d0(c0)
andc0t1d0(c1).Eitherdiskcanbetheprimaryorcontroldiskforreplication.The
diskmirroringsoftwarerunsonboththeActiveandStandbyNE.Diskmirroringis
distinctfromsystemmirroring;diskmirroringreplicatesdataacrossdisksinthe
sameNE,andsystemmirroringreplicatesdataacrossprimarydisksintheNEsin
aHighAvailabilityconfiguration.
Figure 30showsthelogicaldrives,logicaldisks,andpartitionsinthemirrored
physicaldisksontheNE.DetachingthelogicaldisksD20andD22willbreakthe
diskmirror.ReattachingthelogicaldisksD20andD22willreestablishthedisk
mirror.

SystemBackup 149
Figure 30PartitionsinMirroredDiskConfiguration
HighAvailability InaHighAvailabilityconfiguration,performasystembackuponthetargetNE
Considerations afteritchangesfromActivetoStandby.ThetargetNEwillnotbeoperatingasa
standbyNEwhilebeingbackedup.
Whenrestoringasystem,restoreittotheStandbyNE,whichisthenmadethe
ActiveNEaftertherestore.Changesmadeinthenetworksincethetapebackup
arenotreflectedintherestoredNE.
Foradditionalinformationonthehighavailabilityconfigurationand
activestandbyoperations,refertoChapter 7,ActiveStandbyOperations.
SystemImpactofTape InaSingleAvailabilityconfiguration,thesystemwillbeoutofserviceduringthe
Backup backup.InaHighAvailabilityconfiguration,theNEtobebackedupshouldbe
madetheStandbyNE,andthereshouldbenoadditionalimpacttoserviceaside
fromthesystemimpactofgoingtoStandby.

Anysystembackupshouldbedoneinalowtrafficperiodtominmizetheimpact
toserviceandchangesmadeinthenetworksincethetapebackupwillnotbe
reflectedintherestoredNE.
Asystembackupshouldnotbeattemptedifanyofthemirroreddrivesarenotin
theOkaystate.
SystemBackupUsingLocal Thissectionincludes:
TapeDrive CheckingDiskMirrorStatus
ShuttingDowntheTargetNE
DetachingLogicalDisks
RestartingtheNE
BackUpLogicalDrives
AttachingLogicalDisks
CheckingDiskMirrorStatus Checkthestatusofthediskmirroringbeforebackingupthesystem.Thestateof
allmirroreddrives(D0,D1,andD2)shouldbeOkay.
ToCheckDiskMirrorStatus:
1 LogintothetargetNEasroot.
2 Checkthestatusofthediskmirror.
root>metastat
3 ChecktheoutputandconfirmthatallmirrorsandsubmirrorsdisplaytheOkay
state.
SeeExample29.
Donotproceedwiththebackupifthediskmirroringisnotoperational.Usethe
DiskSuiteguidestoresolveanydiskmirroringissues.
4 Stayloggedinasroot.
Example29metastatCommandOutput
#metastat
d0:Mirror
Submirror0:d10
State:Okay
Submirror1:d20
State:Okay
Pass:1
Readoption:roundrobin(default)
Writeoption:parallel(default)
Size:8392072blocks
d10:Submirrorofd0
State:Okay
Size:8392072blocks
Stripe0:
DeviceStartBlockDbaseStateHot
Spare
c0t0d0s00NoOkay

d20:Submirrorofd0
State:Okay
Size:8392072blocks
Stripe0:
DeviceStartBlockDbaseStateHot
Spare
c0t1d0s00NoOkay
ShuttingDowntheTargetNE ShutdowntheNEandentersingleusermode,thenverifythediskhealth.
ToShutDowntheNEandEnterSingleUserMode:
Caution:
ItisimportantthattheNEbeinsingleusermodewhenufsdumpisrunning.Do
nottrytorunufsdumponanactivefilesystem.
root>cpu_stop
3 Rebootintosingleusermode.
root>inits
SeeExample30forexampleoutput.
Example30initCommandOutput
root>inits
#
Typecontroldtoproceedwithnormalstartup,
(orgiverootpasswordforsystemmaintenance):
INIT:SINGLEUSERMODE
#
Thefsckutilitywillperformconsistencychecksonthefilesystemandtryto
repairanyproblems.Runfsckbeforebackingupafilesystemsothatthebackup
isnotcorrupt.Thetwologicaldrives(D0andD2)shouldbechecked.
ToVerifyDiskHealth:
1 CheckthefilesystemforthefirstlogicaldrivelistedinTable 24.
root>fsck/dev/rdsk/d0
2 CheckthefilesystemfortheremaininglogicaldrivelistedinTable 24.
Donotproceedwiththebackupifthereareerrorsonanyslice.
Table 24DiskSliceFilesystemCheck
PhysicalDisk LogicalDrive
c0t0d0s0 D0
c0t0d0s0 D2

Example31fsckCommandOutput
#@(#)ntp.client1.296/11/06SMI
#
#/etc/inet/ntp.client
#
#Anexamplefilethatcouldbecopiedoverto/etc/inet/ntp.conf;it
#providesaconfigforahostthatpassivelywaitsforaserver
#toprovideNTPpacketsonthentpmulticastnet.
DetachingLogicalDisks Thelogicaldisksshouldbedetachedfromthelogicaldrives,whichwilldisable
thediskmirror.Thelogicaldisk(D20)shouldbedetachedfromtherootlogical
drive(D0)andthelogicaldisk(D22)shouldbedetachedfromthe/export/home
logicaldrive(D2).
ToDetachLogicalDisks
1 Synchronizethedisks.
root>synch
2 Detachthefirstlogicaldisk.
root>metadetachd0d20
3 Detachthesecondlogicaldisk.
root>metadetachd2d22
RestartingtheNE ThetargetNEshouldberestarted.
ToRestarttheTargetNE:
1 RestartthetargetNE.
root>reboot
BackUpLogicalDrives First,loadtheblankDigitalDataStorage3(DDS3)tapethatistobeusedforthe
backup.
ToLoadtheBackupTape:
1 InsertthetapeintotheDATdrive.
2 Checkthestatusofthetape.
root>mtf/dev/rmt/0nstatus
3 Examinetheoutputandconfirmthatfileno=0.
SeeExample32.
4 Ifthevalueisnot0,thenrewindthetape.
root>f/dev/rmt/0rewind
Example32TapeStatusCheckResult
ArchivePython4mmHelicalScantapedrive:
sensekey(0x6)=UnitAttentionresidual=0retries=0
fileno=0blockno=0

Thelogicaldrivescannowbebackeduptotape.Thelogicaldisksfromthe
mirroredphysicaldisk(c0t1d0Disk2)arebackedup.Thatis,backuptheD20
(c0t1d0s0)andD22(c0t1d0s7)fromdisk2.SeeFigure 30forfurtherreference.
ToBackUptheLogicalDrives:
1 Dumpthefirstlogicaldrive(D20)tothetape.
root>ufsdump0f/dev/rmt/0n/dev/rdsk/c0t1d0s0
2 Dumpthesecondlogicaldrive(D22)tothetape.
root>ufsdump0f/dev/rmt/0n/dev/rdsk/c0t1d0s7
3 Rewindthetape.
root>mtf/dev/rmt/0rewind
4 Ejectthetapeandlabelitbeforestorage.
Labelthetapesasiftheywerefromtheprimarydisk(c0t0d0disk1)insteadof
fromthesecondarydisk(c0t1d0disk2).Thereasonisthatthebackupwillbe
restoredtotheprimary(boot)diskandnotthesecondarydisk.
AttachingLogicalDisks Attachthelogicaldiskstoenablethediskmirror.
ToAttachtheLogicalDisks:
1 Attachthefirstlogicaldisk.
root>metattachd0d20
2 Attachthesecondlogicaldisk.
root>metattachd2d22

SystemBackupUsing Thissectionincludes:
RemoteTapeDrive
RemoteTapeSystemSetup
ClientSystemSetup
RecordingEthernetInterfaceConfiguration
PerformingTapeBackup
VerifyingDiskHealth
ConfiguringInterfaceandRouting
UsingtheTapeDeviceforBackup
SettingupTapeSystem
BackingUpDisk
RemoteTapeSystemSetup Theremotetapesystemneedstobeproperlyconfiguredforinternetservices
beforeitcanconnecttothetapedrive.
TheInternetdaemon(inetd)controlsInternetserviceshostedbythesystem.
inetdisstartedwhenthesystemisbootedininanynetworkedrunstate.The
daemonreadstheconfigurationfileat/etc/inetd.confatstartupbutit
cannotbekilledandthenmanuallystarted.Theinetdprocessisrestartedby
sendingaHUPsignalwiththekillcommand.
Auser(administrative)accountshouldbecreatedexpresslyforthepurposeof
doingbackups.Donotusetherootaccounttodobackups.Alsocreatea
/backupshomedirectoryandpassword.
TheNE(actingasaclient)mustaccesstheremotetapesystem(actingasa
server).TheNEhostname(orIPaddress)anduseraccountmustbedefinedinthe
.rhostsfileontheremotetapesystem.The.rhostsfileisplacedintheusers
homedirectory(forexample,export/home/backups/.rhosts)andliststhe
hostsandusersthatcanlogintotheaccount.
Also,definethehostnameinthehostsfileontheremotetapesystem.
TheflowforsettinguptheremotetapesystemisshowninFigure 31.

Figure 31FlowchartforRemoteTapeSystemSetup
Theremoteshellserver(shell),theremoteexecutionserver(exec),andlogin
daemonsmustbestarted.
1 ChecktheInternetServiceConfiguration:
a Logintotheremotetapesystem.
b Pagethrough/etc/inetd.conf.
>more/etc/inetd.conf
c ExaminetheoutputforInternetservicedaemons.
SeeExample33.
Example33inetd.conf
..
loginstreamtcp6nowaitroot/usr/sbin/in.rlogindin.rlogind
execstreamtcp6nowaitroot/usr/sbin/in.rexecdin.rexecd
shellstreamtcp6nowaitroot/usr/sbin/in.rshdin.rshd
...
d Iftheinetddaemonsarepresentthengotostep 3.
e ConsulttheSolarisdocumentationtoseehowtoconfigureinetddaemonsif
thedaemonsarenotlisted.
f Toeditthefile,stopandrestarttheinetddaemon.
SeeRestarttheinetdDaemon:onpage 156.
2 RestarttheinetdDaemon:
a Changetoroot.
pro>suroot
b Findthepidfortheinetdaemon.
root>psef|grepinet
SeeExample34.
Example34inetdProcess
root1791011:53:04?0:00/usr/sbin/inetds
c Restarttheinetddaemon.
root>killHUP<pid>
Theinetdaemonwillrestartusingtheeditedinetd.conffile.
d Changetoanadministrativeacount.
root>su<adminaccount>
3 CreateaBackupAccount:
a Addanewbackupuseraccount.
admin>useraddd/export/home/backupsms/bin/kshbackups
SeeTable 25foradescriptionoftheoptions.
Table 25useraddOptions
Option Description
d Setsthehomedirectory(/export/home/backups).
m Createsanewhomedirectoryfortheuserbackups.
s Setsthedefaultshellfortheuserbackups(/bin/ksh).
b Addpasswordtobackupaccount.
admin>passwdbackups

4 EdittheRemotehostsFile:
a Createremote.rhostsfile.
admin>ed/export/home/backups/.rhosts
b Addthehostnameandusernameto.rhosts.
Theformatofthe.rhostsfileis:hostnameusername
c Saveandclosethe.rhosts.
d Addthehostnametothehostsfile.
admin>ed/etc/hosts
e Saveandclosehosts.
ClientSystemSetup Recordtheharddiskpartitioninformationandstorewiththebackuptapesothat
itcanbeusedintherestoreprocess.Usenetstattoshowallsystemnetwork
connections,routinginformation,andnetworkinterfaceinformation.
TheflowforsettinguptheclientsystemisshowninFigure 32.
Figure 32FlowchartforClientSystemSetup

1 RecordPartitionInformation:
a Haveasuitabletableavailabletorecorddiskpartitioninformation.
CreateanewtableorusetheoneshowninTable 26.
Table 26DiskPartitionRecord
Slice Tag Flag Cylinders Size

c0t0d0
0
1
2
3
4
5
6
7
c0t1d0
0
1
2
3
4
5
6
7
b LogintotheNEasusermsc,pro,fsv,orlis,asappropriateand
thenchangetoroot:
pro>su
c Formatthedisk.
root>format
Example35formatCommandOutput
Searchingfordisks...done
AVAILABLEDISKSELECTIONS:
0.c0t0d0<ST39140Acyl17660alt2hd16sec63>
/pci@1f,0/pci@1,1/ide@3/dad@0,0
Specifydisk(enteritsnumber):
d Selectthedisk.
Specifydisk(enteritsnumber):0

Example36FormatMenu
selectingc0t0d0
[diskformatted,nodefectlistfound]
Warning:CurrentDiskhasmountedpartitions.
FORMATMENU:
diskselectadisk
typeselect(define)adisktype
partitionselect(define)apartitiontable
currentdescribethecurrentdisk
formatformatandanalyzethedisk
repairrepairadefectivesector
showtranslateadiskaddress
labelwritelabeltothedisk
analyzesurfaceanalysis
defectdefectlistmanagement
backupsearchforbackuplabels
verifyreadanddisplaylabels
savesavenewdisk/partitiondefinitions
volnameset8charactervolumename
!<cmd>execute<cmd>,thenreturn
quit
e Selectpartition.
format>partition
Example37partitionCommandOutput
PARTITIONMENU:
0change`0'partition
1change`1'partition
2change`2'partition
3change`3'partition
4change`4'partition
5change`5'partition
6change`6'partition
7change`7'partition
selectselectapredefinedtable
modifymodifyapredefinedpartitiontable
namenamethecurrenttable
printdisplaythecurrenttable
labelwritepartitionmapandlabeltothedisk
quit
f Selectprint.
partition>print
Example38printCommandOutput
Currentpartitiontable(original):
Totaldiskcylindersavailable:17660+2(reservedcylinders)
PartTagFlagCylindersSizeBlocks
0rootwm062413.00GB(6242/0/0)6291936
1swapwu6242104032.00GB(4162/0/0)4195296
2unassignedwm00(0/0/0)0
3unassignedwm104041042410.34MB(21/0/0)21168
4unassignedwm104251044510.34MB(21/0/0)21168
7homewm10446176583.47GB(7213/0/0)7270704
partition>

g RecordthepartitioninformationinTable 26.
RecordingEthernetInterface ToRecordEthernetInterfaceConfiguration:
Configuration
1 CheckEthernetinterfaceconfiguration.
root>ifconfighme0
Example39ipconfigCommandOutput
hme0:flags=863<UP,BROADCAST,NOTRAILERS,RUNNING,MULTICAST>mtu1500
inet10.2.1.73netmaskff000000broadcast10.255.255.255
ether8:0:20:a8:96:93
2 Recordthehme0configurationinformationinTable 27.
Table 27EthernetConfigurationRecord
Interface IPAddress NetworkMask BroadcastMask DefaultGateway

hme0
3 Checkdefaultgatewayconfiguration.
root>netstatnr
SeeTable 28foradescriptionoftheoptions.
Table 28netstatOptions
Option Description
n ShowsIPaddressesandnothostnames.
r Showsroutingtables.
Example40netstatCommandOutput
RoutingTable:
Destination Gateway Flags Ref Use Interface

127.0.0.1 127.0.0.1 UH01291lo0
10.0.0.0 10.2.1.73 U3632hme0
224.0.0.0 10.2.1.73 U30hme0
default 10.2.1.1 UG00
4 RecordthedefaultgatewayIPaddressinTable 27.
5 Exitallaccountsbacktouserpro.
PerformingTapeBackup TheflowforperformingthetapebackupisshowninFigure 33.
Warning:
Performingatapebackupwillcauseasystemoutage.

Figure 33FlowchartforTapeBackup

1 ToGotoSingleUserMode:
Caution:
nottrytorunufsdumponanactivefilesystem.
a LogintotheNEasuserroot.
b StoptheNEapplication.
root>sh/etc/rc2.d/s98telosstop
c Rebootintonewrunstate.
root>init0
Runninginitstate0willstopallsystemservices,unmountfilesystems,and
bootintosingleusermode.
root>init0
INIT:Newrunlevel:0
Thesystemiscomingdown.Pleasewait
Printservicesstopped.
Thesystemisdown.
Syncingfilesystems...done
programterminated
{0}ok
d InserttheSolarisbootCD.
e BootintosingleusermodefromtheCD.
{ok}bootcdroms
Example42bootCommandOutput
Resetting....
Screennotfound.
Can'topeninputdevice.
Keyboardnotpresent.Usingttyaforinputandoutput.
....
INIT:SINGLEUSERMODE
>

VerifyingDiskHealth Thefsckutilitywillperformconsistencychecksonthefilesystemandtryto
repairanyproblems.Runfsckbeforebackingupafilesystemsothatthebackup
isnotcorrupt.
2 ToVerifyDiskHealth:
a CheckthefilesystemforthefirstslicelistedinTable 29.
root>fsck/dev/rdsk/c0t0d0s0
Table 29DiskSliceFilesystemCheck
Disk Slice
Disk1 c0t0d0s0
Disk1 c0t0d0s3
Disk2 c0t1d0s0
Disk2 c0t1d0s3
Disk2 c0t1d0s4
**/dev/rdsk/c0t0d0s0
**Lastmountedon/mnt
**Phase1CheckBlocksandSizes
**Phase2CheckPathnames
**Phase3CheckConnectivity
**Phase4CheckReferenceCounts
**Phase5CheckCylGroups
42892files,823358used,12422081free
(4577frags,1552188blocks,0.0%fragmentation)
b CheckthefilesystemfortheremainingslicesinTable 29.
Donotproceedwiththebackupifthereareerrorsonanyslice.
'Plumb'theinterfacebeforeconfiguringit.
ConfiguringInterfaceand ToConfiguretheInterfaceandRouting:
Routing
1 RetrievetheEthernetinterfaceanddefaultgatewayconfigurationinformation.
Thisinformationshouldhavebeenstoredwiththebackup.SeeTable 27.
2 PlumbtheEthernetinterface.
root>ifconfighme0plumb
3 Configuretheinterfacewiththevaluesstoredwiththebackup.
root>ifconfighme0<IPaddress>netmask<netmask>broadcast
<broadcastaddress>up
4 Verifythattheinterfaceisconfiguredandup.
root>ifconfiga
Example44ifconfigCommandOutput
hme0:flags=863<UP,BROADCAST,NOTRAILERS,RUNNING,MULTICAST>mtu1500
inet10.2.1.73netmaskff000000broadcast10.255.255.255
ether8:0:20:a8:96:93
5 Configureandverifydefaultgateway.
root>addroute<IPaddress>
6 Verifydefaultgatewayconfiguration.
root>netstatnr
Example45netstatCommandOutput
RoutingTable:
Destination Gateway Flags Ref Use Interface

default 10.2.1.1 UG 0 0
7 CheckEthernetconnectionwithsystemtoseethatitcanbereached.
root>ping<IPaddress>
UsingtheTapeDevicefor Theremotetapedeviceiscontrolledwiththemagnetictapecontrolcommand
Backup (mt).Thesyntaxformtismt[ftapename]command...[count].
Table 30mtCommandOptions
Option/Command Description
ftapename Thenameofthetapedevice.Bydefualtmtusesthedevice
/dev/rmt/0n.
rewind Rewindthetape
offline Rewindthetapeand,ifappropriate,takethedriveunitofflineby
unloadingthetape.
status Printstatusinformationaboutthetapeunit.
Usethe"norewind"(n)optiontoputmorethanonefileontothetape.Ifthe
backuprunsoutofspace,thetapedoesnotrewindbeforeufsdumpasksfora
newtape.
Ifthedeviceisspecifiedasuser@host:device,ufsdumptriestoaccessthe
deviceontheremotesystemasthespecifieduser.Inthiscase,thespecifieduser
mustbeincludedinthe/.rhostsfileontheremotesystem.
Theufsdumputilityisusedtobackupfilesystemstoabackupdevice.Usetheo
optiontoejectthetapeafterthebackupiscompleteandthenoptiontonotify
usersinthesysgroupofanyproblemsduringthebackup.Whenufsdumpwrites
toanoutputdevice,itcreatesasinglebackupfilethatmightspanmultipletapes.
Thesyntaxforufsdumpis:
ufsdump[options][arguments]filestobackup.
SettingupTapeSystem ToSetuptheTapeSystem:
1 Insertablanktapeintotheremotetapesystem.

2 LogintotheNEandcheckthestatusoftheremotetapedrive.
pro>rshlguest<IPaddress>mtf/dev/rmt/0nstatus
Example46StatusofTapeDrive
fileno=0blockno=0
3 Iffileno=0thenthetapeisrewoundandreadyforthebackup.
4 If"fileno=0"isnotintheoutputthenrewindthetape.
pro>rshlguest<IPaddress>mtf/dev/rmt/0rewind
BackingUpDisk Thediskisbackedup,startingwiththefirstslice.
Caution:
nottrytorunufsdumponanactivefilesystem
1 BackUpFirstSlice:
pro>ufsdump0cfguest@<IPaddress>:/dev/rmt/0n
/dev/rdsk/c0t0d0s0
Wherec0t0d0s0isthefirstsliceondisk1.
2 BackUpSecondSlice:
pro>ufsdump0cfguest@<IPaddress>:/dev/rmt/0n
/dev/rdsk/c0t0d0s3
Wherec0t0d0s3isthesecondsliceondisk1.
3 Rewind,EjectandLabelTape:
a Rewindthetape:
pro>rshlguest10.3.1.51mtf/dev/rmt/0rewind
b Manuallyejectthetapeandremoveit.

c Labelthetape.SeeTable 31forguidelinesonlabelingbackuptapes.
Table 31TapeLabelingGuidelines
FilenameItem Description Example

Product/Name NEname SSVR_04
FileIndex Locationontape File0,File1,...
DiskPartition Physicallocation c0t0d0s0
Filesystem Pathintheufs /,/root,/usr,...
Flag Diskiswritableandmountable/unmountable wm,wu
Size Sizeofpartition 800MB
Date Dateofbackup 20050630
User Usernameofonewhodoesthebackup.thisis N/A
companyspecific.
4 RebootSystem:
a EjecttheSolarisbootCD.
b RestarttheNEapplication.
pro>cpu_start
HardwareRAIDNetra440 ThisbackupprocedureappliestoanHLRthatisbasedonaSUNNetra440server
HLRSystemBackup platformutilizingbothsoftwaremirroring(system/applicationdisk)and
hardwaremirroringwithabuiltinCDROMdrive.TheHLRhasdirectaccesstoa
tapedrive.
AsystembackupofahardwareRAIDbasedHLRincludestheoperatingsystem
andalldataoneachessentialdiskpartitiontoatapesystem.
AsystembackupoftheHLRshouldbeperformed:
Priortoanysignificantapplicatoinsoftwareupgradesofpatches
Afteranysignificantapplicationupgradesorpatches
PriortoanysignificantOSupgradesorpatches
AfteranysignificantOSupgradesorpatches
Atleastonceaweek
Prerequisites AbackuprequirestworormoreDigitalDataStorage4(DDS4)tapes.
SoftwareMirroredDisk TwodisksontheHLRaresoftwaremirroredfortheOSandapplicationbinaries
Strategy andconfigurationfiles.AnothertwodisksaremirroredfortheOraclebinaries
andHLRsubscriberdatabase.ThedisksandpartitionsareshowninFigure 34and
Figure 35.

HardwareRAIDNetra440HLRSystemBackup 167
Figure 34SoftwareMirroredDisksandPartitionsforOSandApplication
D2
/export/home
D1
swap
D0
D12 D11 D10 D20 D21 D22
C1T0D0
C1T1D0
C1T0D0S0 C1T1D0S0
Archive and Backup Archive and Backup
C1T0D0S1 C1T1D0S1
C1T0D0S7 C1T1D0S7
Figure 35SoftwareMirroredDisksforOracleandHRLSubscriberDatabases
/export/home/oracle
C1T2D0
C1T2D0S7
C1T2D0
C1T3D0
BackingUptheHLRSystem Tobackupthesystemwithaminimumofoutagetime:
CopythelatestOracleOnlineDatabaseBackupto/telos/ora_db_backup
Backupthesystem,followingthestepsinSystemBackupUsingLocalTape
Drive.
RefertoHardwareRAIDNetra440HLRSystemRestoretorestoretheHLRif
necessary.

SystemRestore Asystemrestoreperformsacompletesystemrestore,includingtheoperating
systemandalldataoneveryessentialdiskpartition,fromalocalorremote
networktapesystem.
Thesystemshouldberestoredfromtapewhenoneoftwoconditionsissatisfied:
BothharddrivesintheNEhavefailedandtherestoreisperformedontwo
newphysicaldisks.
Theintegrityofthesystemdataonthedisksiscompromised.
Torestorefromabackuptape,thefollowingitemsarerequired:
Thebackuptapes
Remoteorlocaltapedrive.
Solarissoftwaredisc1.
AttachedCDROMdrive
IftherestoreisrequiredbecauseoffailureofbothNEharddrives,thentwo
newharddrivesofthesamedimensionsneedtohavebeeninstalled.
SystemRestoreUsingLocal Thissectionincludes:
TapeDrive SystemRestoreDuetoSimultaneousHardDriveFailure
ShutdownTargetNE
EnterSingleUserModefromCDROM
FormatHardDrives
LoadRestoreTape
CreatingaNewFilesystemandCheckingTargetDisk
RestoretheDisk
RestartTargetNEandVerifySystem
EnableDiskMirror
ThissystemrestoreprocedureappliestoSolaris5.8installationsonly.
StarSolutionsmustbenotifiedbeforeasystemrestoreisattemptedandthe
stepsshouldbecarriedoutunderStarSolutionssupportpersonnelsupervision.
SystemRestoreDueto Ifasystemrestoreisrequiredduetothesimultaneousfailureofthetwohard
SimultaneousHardDrive drivesintheNEthenbegintherestoreprocedureatEnterSingleUserModefrom
Failure CDROM.SimultaneousharddrivefailureintheActiveNEwillcausefailoverto
theStandbyNE.Thenewdrivecanberestoredwiththebackuptape.
IftheharddrivesfailonaStandbyNE,thenmakeabackupoftheActiveNEand
restoreittothenewdrivesonthestandby.

SeeSystemBackupUsingLocalTapeDriveonpage 150foradditionalinformation
aboutwhatisrequiredbeforerestoringabackup.
ShutdownTargetNE ShutdownthetargetNEbeforerestoringabackup.
ToShutdownTargetNE:
root>sh/etc/rc2.d/S98telosstop
3 BootintoNVRAM.
root>init0
SeeExample47.
root>init0
#
INIT:Newrunlevel:0
Printservicesstopped.Thesystemisdown.
programterminated
{0}ok
EnterSingleUserModefrom TheSolarisSoftwareDisc1CDROMisrequiredinordertoentersingleuser
CDROM mode.
ToEnterSingleUserModefromCDROM:
1 InserttheSolarisSoftwareDisc1CDROMintotheCDROMdrive.
2 Bootintosingleusermode.
{ok}bootcdroms
SeeExample48.
Example48SingleUserModefromCDROM
{ok}bootcdroms
Resetting...
Screennotfound.
...
INIT:SINGLEUSERMODE
#

FormatHardDrives Theharddrivesmustbeformattedbeforetheycanaccepttherestored
filesystem.Theharddriveformattingmustmatchtheformattingoftheoriginal
sourcedriveofthebackuptape.
ToFormatHardDrives:
1 Usetheformatcommandtopartitionthetargetharddrives:
#format
UsetheinformationinTable 32for36GBdisksorTable 33for72GBdisksto

formatthedisks.Disk1(primary)andDisk2(secondary)partitionsshouldbe
identical.
Table 3236GbHardDiskPartitionsizes
Disk Slice LogicalDrive Size

Disk1(PrimaryDisk) 0 root 4GB
Disk1(PrimaryDisk) 1 swap 4GB
Disk1(PrimaryDisk) 2 backup 36GB(Fulldisk)
Disk1(PrimaryDisk) 3 Reservedformetadevicestate 100MB
database
database
Disk1(PrimaryDisk) 5 N/A N/A
Disk1(PrimaryDisk) 7 /export/home 26GB+
Table 3372GBHardDiskPartitionsizes

database
database

LoadRestoreTape ToLoadtheRestoreTape:
SeeExample49.
fileno=0blockno=0
CreatingaNewFilesystem Createanewfilesystemontheharddrivesandthenchecktheintegrityofeach
andCheckingTargetDisk sliceonthetargetdisk.
1 CreateaNewFilesystem:
a Createthefilesystemonthepartitionsontheprimarydisk.
#newfs/dev/rdsk/c0t0d0s0
b Createthefilesystemonthepartitionsonthesecondarydisk.
2 Checkthetargetdisk:
a Checkthefilesystemontheprimarydisk.
#fsck/dev/rdsk/c0t0d0s0
b Checkthefilesystemonthesecondarydisk.
RestoretheDisk Thissectionincludes:
RestoringDisk
RewindandEjectTape
ConfigureSystemDisk
InstallBootSectorandCheckDisk
RestoringDisk
Thediskcannowberestored.

ToRestoreDisk:
1 Mounttherootlogicaldrive.
#mount/dev/dsk/c0t0d0s0/mnt
2 Changedirectorytothemounteddriveandrestore.
#cd/mnt
#ufsrestorervf/dev/rmt/0n
3 Removetherestoresymtablefile.
#rmrestoresymtable
4 Unmounttherootlogicaldrive.
#cd/
#unmount/mnt
5 Mounttheapplicationlogicaldrive.
#cd/mnt
#rmrestoresymtable
8 Unmounttheapplicationlogicaldrive.
#cd/
#unmount/mnt
RewindandEjectTape
Thetapeshouldberewoundandejected.
ToRewindandEjectTape:
1 Rewindthetape.
#mtf/dev/rmt/0rewind
2 Ejectthetape.
ConfigureSystemDisk
Configurationchangesarerequiredtofilesontherootlogicaldrive.
ToConfigureSystemDisk:
#cd/mnt
2 OpenthesystemfileforeditingandremovethelinesshowninboldinExample
50.
#vi/mnt/etc/system
3 Openthemddb.cffileforeditingandremovethelinesshowninboldinExample
51.
#vi/mnt/etc/lvm/mddb.cf

4 Openthevfstabfileforeditingandremovethelines(andcomments)shownin
boldinExample52.
#vi/mnt/etc/vfstab
Example50systemFileChanges
setshmsys:shminfo_shmmax=2048000000
setsemsys:seminfo_semmap=250
setsemsys:seminfo_semmni=500
setsemsys:seminfo_semmns=500
setmsgsys:msginfo_msgmap=2048
setmsgsys:msginfo_msgmax=32767
setmsgsys:msginfo_msgmnb=65535
setmsgsys:msginfo_msgmni=200
setmsgsys:msginfo_msgssz=32
setmsgsys:msginfo_msgseg=4096
setmsgsys:msginfo_msgtql=16384
*BeginMDDrootinfo(donotedit)
forceload:misc/md_trans
forceload:misc/md_raid
forceload:misc/md_hotspares
forceload:misc/md_stripe
forceload:misc/md_mirror
forceload:drv/pcipsy
forceload:drv/simba
forceload:drv/glm
forceload:drv/sd
rootdev:/pseudo/md@0:0,0,blk
*EndMDDrootinfo(donotedit)
*BeginMDDdatabaseinfo(donotedit)
setmd:mddb_bootlist1="sd:3:16sd:3:1050sd:4:16sd:4:1050sd:11:16"
setmd:mddb_bootlist2="sd:11:1050sd:12:16sd:12:1050"
*EndMDDdatabaseinfo(donotedit)
Example51mddb.cfFileChanges
#metadevicedatabaselocationfiledonothandedit
#driverminor_tdaddr_tchecksum
sd316192
sd416193
sd1116200
sd1216201
Example52vfstabFileChanges
#device device mount FSfsck mountmount
#tomount tofsck point typepass atbootoptions
#
#/dev/dsk/c1d0s2 /dev/rdsk/c1d0s2 /usrufs1 yes
fd /dev/fd fd no
/proc /proc proc no
/dev/dsk/c0t0d0s1 swap no
/dev/dsk/c0t0d0s0/dev/rdsk/c0t0d0s0/ ufs1 no
/dev/dsk/c0t0d0s7/dev/rdsk/c0t0d0s7/export/home ufs2 yes
#/dev/md/dsk/d1 swap no
#/dev/md/dsk/d0 /dev/md/rdsk/d0 / ufs 1 no
#/dev/md/dsk/d2 /dev/md/rdsk/d2 /export/home ufs2 yes
swap /tmp tmpfs yes
Installthebootsectoronbothharddrives,atcylinder0ofthebckupslice.

ToInstallBootSectorandCheckDisk:
1 Installbootsectors.
#installboot/usr/platform/`unamei`/lib/fs/ufs/bootblk
/dev/rdsk/c0t0d0s0
/dev/rdsk/c0t1d0s0
2 Checkthefilesystemontheprimarydisk.
#cd/
#unmount/mnt
RestartTargetNEandVerify RestarttheNEandverifysystemoperation.
System
ToRestartNE:
1 RemovetheSolarisSoftwareDisc1CDROM.
2 RestarttheNE.
#reboot
3 Verifysystemoperation.
EnableDiskMirror Thissectionincludes:
SetupReplicas
EnableOneWayDiskMirror
ConfigureDisk
EnableTwoWayMirror
SetupReplicas
Setupthe4replicas.
ToSetupReplicas:
2 Setup4replicas.
#metadbafc0t0d0s3c0t0d0s4c0t1d0s3c0t1d0s4
3 Verifythediskconfiguration.
#metadbi
SeeExample53.
Example53metadbOutput
flagsfirstblkblockcount
au161034
/dev/dsk/c0t0d0s3
au161034
/dev/dsk/c0t0d0s4
au161034
/dev/dsk/c0t1d0s3

au161034
/dev/dsk/c0t1d0s4

oreplicaactivepriortolastmddbconfigurationchange
ureplicaisuptodate
llocatorforthisreplicawasreadsuccessfully
creplica'slocationwasin/etc/lvm/mddb.cf
preplica'slocationwaspatchedinkernel
mreplicaismaster,thisisreplicaselectedasinput
Wreplicahasdevicewriteerrors
areplicaisactive,commitsareoccurringtothisreplica
Mreplicahadproblemwithmasterblocks
Dreplicahadproblemwithdatablocks
Freplicahadformatproblems
Sreplicaistoosmalltoholdcurrentdatabase
Rreplicahaddevicereaderrors
EnableOneWayDiskMirror
Enabletheonewaydiskmirror.
ToEnableOneWayDiskMirror:
1 Createthemd.tabfileandaddtheentriesshowninExample54.
root>vi/etc/lvm/md.tab
2 Applytheconfigurationchanges.
root>metainitaf
Example54md.tabFile
d1011c0t0d0s0
d0md10
d2011c0t1d0s0
#swapslice
d1111c0t0d0s1
d1md11
d2111c0t1d0s1
#/export/homeslice
d1211c0t0d0s7
d2md12
d2211c0t1d0s7
ConfigureDisk
Configurethedisk.
ToConfigureDisk:
1 Establishrootfilesystem.
root>metarootd0
2 Makethesecondarydiskbootable.
root>installboot/usr/platform/`unamei`/lib/fs/ufs/bootblk
/dev/rdsk/c0t1d0s0
3 OpenthevfstabfileforeditingandmakethechangesshowninboldinExample
55.
root>vi/etc/vfstab

Example55ModifyvsftabFile
au161034
/dev/dsk/c0t0d0s3
au161034
/dev/dsk/c0t0d0s4
au161034
/dev/dsk/c0t1d0s3
4 Haltthesystem.
root>init0
5 Setbootparameters.
oksetenvautoboot?true
oksetenvbootdevicedisk0disk1disk
ok
6 Restartthesystem.
#reset
EnableTwoWayMirror
Enablethetwowaymirror.ThedisksaremirroredafterthestatusshowsOkay.
ToEnableTwoWayMirror:
2 Enabletwowaymirror.
root>metattachd0d20
root>metattachd1d21
root>metattachd2d22
3 Verifymirrorstatus.
root>metastat
Themirrorsshouldberesynched.SeeExample56.
Example56SynchronizedMirroredDisks
#metastat
d0:Mirror
Submirror0:d10
State:Okay
Submirror1:d20
State:Okay
Pass:1
Size:8392072blocks
...
d1:Mirror
Submirror0:d11
State:Okay
Submirror1:d21
State:Okay
Pass:1
Size:8392072blocks
...
d2:Mirror
Submirror0:d12
State:Okay
Submirror1:d22

State:Okay
Pass:1
Size:18169472blocks

SystemRestoreUsing Thissectionincludes:
RemoteTapeDrive
OverviewofSystemRestoring
PerformingtheRestore
OverviewofSystemRestoring Asystemisrestoredfromasystembackup.thedisksareformattedandthendata
isrestoredtothetwodisksontheNE,startingwiththefirstsliceonthefirstdisk.
TheflowforperformingthetapebackupisshowninFigure 36.
Warning:
Performingarestorefromtapebackupwillcauseasystemoutage.
Figure 36FlowchartforRestoringTwoDiskBackupfromTape

PerformingSystemRestore Thissectionincludes:
SingleUserMode
FormatDisk1
FormatDisk2
ClientSystemSetup
RebootingandSystemVerification
SingleUserMode
GotosingleusermodeontheNEtodoarestorefromtape.
ToGotoSingleUserMode:
1 SeeShuttingDowntheTargetNEonpage 151.
FormatDisk1
EachofthetwodisksontheNEneedstobeformattedbeforetheycanacceptthe
restore.Eachdestinationdiskneedstobepartitionedaccordingtothepartition
informationrecordedforeachsourcedisk.Thisinformationshouldhavebeen
recordedandkeptwiththebackup.
ToFormatDisk1:
1 Executeformat.
root>format
Example57formatCommandOutput
Specifydisk(enteritsnumber):
2 Selectthefirstdisk.
Specifydisk(enteritsnumber):0

Example58formatMenu
selectingc0t0d0
[diskformatted,nodefectlistfound]
Warning:CurrentDiskhasmountedpartitions.
FORMATMENU:
diskselectadisk
typeselect(define)adisktype
partitionselect(define)apartitiontable
currentdescribethecurrentdisk
formatformatandanalyzethedisk
repairrepairadefectivesector
showtranslateadiskaddress
labelwritelabeltothedisk
analyzesurfaceanalysis
defectdefectlistmanagement
backupsearchforbackuplabels
verifyreadanddisplaylabels
savesavenewdisk/partitiondefinitions
volnameset8charactervolumename
quit
3 Selectpartition.
format>partition
Example59partitionMenu
PARTITIONMENU:
0change`0'partition
1change`1'partition
2change`2'partition
3change`3'partition
4change`4'partition
5change`5'partition
6change`6'partition
7change`7'partition
selectselectapredefinedtable
modifymodifyapredefinedpartitiontable
namenamethecurrenttable
printdisplaythecurrenttable
labelwritepartitionmapandlabeltothedisk
quit
4 Selectmodify.
partition>modify
Example60modifyCommandOutput
Selectpartitioningbase:
0.Currentpartitiontable(default)
1.AllFreeHog
Choosebase(enternumber)[0]?1
5 Selectoption1.
Choosebase(enternumber)[0]?1

Example61AllFreeHogCommandOutput
PartTagFlagCylindersSizeBlocks
0rootwm00(0/0/0)0
1swapwu00(0/0/0)0
2backupwu0176598.49GB(17660/0/0)17801280
6usrwm00(0/0/0)0
Doyouwishtocontinuecreatinganewpartition
tablebasedonabovetable[yes]?y
FreeHogpartition[6]?3
Entersizeofpartition'0'[0b,0c,0.00mb,0.00gb]:0
Okaytomakethisthecurrentpartitiontable[yes]?y
6 Acceptalldefaultsfortableconstructionanduseitforpartitions.
Doyouwishtocontinuecreatinganewpartitiontablebasedonabove
table[yes]?y
FreeHogpartition[6]?3
Example62TableLabeling
Entertablename(rememberquotes):disk0
Readytolabeldisk,continue?y
7 Labelthetable.
Entertablename(rememberquotes):disk0
Readytolabeldisk,continue?y
ToModifyDefaultPartitionsandLabelDisk1:
1 Modifythepartitionsonthefirstdisk.
partition>0
Example63PartitionTableModification
Enterpartitionidtag[root]:
Enterpartitionpermissionflags[wm]:
Enternewstartingcyl[0]:
Enterpartitionsize[0b,0c,0.00mb,0.00gb]:
2 Configurethefirstpartitiononthedisk.
Usethevaluescapturedduringthebackup.

Caution:
Thestartingcylindershouldbe0.
3 Configureeachremainingpartition.
Caution:
Donotleaveanygapsbetweencylinders.
4 Labelthefirstdisk.
partition>disk1
FormatDisk2
Theseconddiskisformatted,partitioned,andlabeledjustlikethefirstdisk
(exceptthattheseconddiskisc0t1d0andlabeleddisk2).
ToFormat,Modify,andLabelDisk2:
1 Formatdisk2.
SeeToFormatDisk1:onpage 179andusethevaluesfordisk2.
2 Modifythedefaultpartitionsandlabeldisk2.
SeeToModifyDefaultPartitionsandLabelDisk1:onpage 181andusethevalues
fordisk2.
ClientSystemSetup
TheclientsystemshouldbesetupasdescribedinClientSystemSetupon
page 157.
ToConfigureInterfaceandRouting:
SeeConfiguringInterfaceandRoutingonpage 163.
Usethevaluescapturedduringthebackup.
ToSetupTapeSystem:
1 Testconnectivitytoremotetapesystem.
root>ping<IPaddress>
Thetapesystemshouldalreadybesetup(seeSettingupTapeSystemon
page 164).
2 Insertthefirstbackuptapeandmakesurethetapeisrewound.
root>rshlguest<IPaddress>mtf/dev/rmt/0nstatus
mountisusedtomountafilesystemlocatedonaspecificdiskpartition.Mounting
afilesystemattachesittotheexistingfilesystem.Youcancreateatemporary
directoryandmountafilesystemonit.
Thefilesystembackupisrestoredusingtheufsrestoreutility.

Therestoresymtablefilestorestherecordofdumps(whetherthereareany
incrementaldumps,etc).Thereshouldbenoincrementaldumpsonthebackup
tape.Thefilewillberestoredtothemountpointalongwiththerestofthe
filesystem.Thereisarestoresymtablefileforeachbackeduppartition.The
restoresymtablefileshouldberemovedaftertherestoreiscomplete.
ToCreateandVerifyFilesystemonDisk1:
1 Createnewfilesystemonfirstpartition.
root>newfs/dev/rdsk/c0t0d0s0
2 Createnewfilesystemonsecondpartition.
3 Verifyhealthoffirstpartition.
4 Verifyhealthofsecondpartition
ToCreateandVerifyFilesystemonDisk2:
1 Createnewfilesystemonfirstpartition.
2 Createnewfilesystemonsecondpartition.
3 Createnewfilesystemonthirdpartition.
4 Checkthehealthofeachnewpartition.
Table 34describestherelevantcommandoptionsforufsrestore.
Table 34ufsrestoreCommandOptions
Option Description
r Recursivelyrestoresanentirefilesystemintothecurrentdirectory.
v Verbose.
f Theargumentisthenameofthedeviceholdingthedump.
ToRestoreDisk1
1 Mountthefirstdiskslicetothemountpoint.
root>mount/dev/dsk/c0t0d0s0/mnt
2 Changetothemountpointandrestorethepartition.
Usethenoptiontopreventtaperewind.
root>cd/mnt
root>ufsrestorervfguest@<IPaddress>:/dev/rmt/0n

3 Removethedumprecordfileandunmountthefirstslice.
root>rmrestoresymtable
root>cd/
root>unmount/mnt
4 Mounttheseconddiskslicetothemountpoint.
root>mount/dev/dsk/c0t0d0s3/mnt
5 Changetothemountpointandrestorethepartition.
Usethenoptiontopreventtaperewind.
root>cd/mnt
root>ufsrestorervfguest@<IPaddress>:/dev/rmt/0n
6 Removethedumprecordfileandunmountthesecondslice.
root>rmrestoresymtable
root>cd/
root>unmount/mnt
7 Rewindandthenmanuallyejectthetape.
root>rshlguest<IPaddress>mtf/dev/rmt/0rewind
ToRestoreDisk2
1 Insertthebackuptapeintothetapedrive.
SeeToSetupTapeSystem:onpage 182.
2 Repeatsteps13fromToRestoreDisk1onpage 183torestorethefirstsliceon
theseconddisk.
Usethevaluesfortheseconddiskandfirstslice.
3 Repeatsteps47fromToRestoreDisk1onpage 183torestorethesecondslice
ontheseconddisk.
Usethevaluesfortheseconddiskandsecondslice.
4 Storethebackuptapesaccordingtocorporateguidelines.

RebootingandSystemVerification
Thebootsectorisinstalledonthefilesystemfromthefirstsliceofthenewly
restoredfirstdisk.installbootinstallsbootblocksinadiskpartition.
Theufsbootobjectsareplatformdependent,andresideinthe
/usr/platform/platformname/lib/fs/ufsdirectory.
Theufsbootblockwillbeinstalledonslice0oftarget0oncontroller0.
ToInstallBootSectorandCheckDiskHealth:
1 Installthebootsector.
root>installboot
/usr/platform/SUNW,Ultra5_10/lib/fs/ufs/bootblk
/dev/rdsk/c0t0d0s0
2 Checkthehealthofthefirstrestoredpartitiononthefirstdisk.
Caution:
Donotproceedwiththerestoftheprocedureifthereareanyerrorsonthedisk.
**/dev/rdsk/c0t0d0s0
**Lastmountedon/mnt
**Phase1CheckBlocksandSizes
**Phase2CheckPathnames
**Phase3CheckConnectivity
**Phase4CheckReferenceCounts
**Phase5CheckCylGroups
42892files,823358used,12422081free
(4577frags,1552188blocks,0.0%fragmentation)
3 Checkthehealthoftheremainingpartitionsonbothdisks.
Caution:
Donotproceedwiththerestoftheprocedureifthereareanyerrorsonanyofthe
partitions.
ToRebootSystem:
1 RemovetheSolarisCDfromtheCDdriveontheNE.
2 StoretheCDaccordingtocorporateguidelines.
3 ReboottheNE.
root>reboot
Thesystemrestartsusingtherestoredfilesystem.

HardwareRAIDNetra440 ThisrestoreprocedureappliestoanHLRthatisbasedonaSUNNetra440server
HLRSystemRestore platformutilizingbothsoftwaremirroring(system/applicationdisk)and
hardwaremirroringwithabuiltinCDROMdrive.TheHLRhasdirectaccesstoa
tapedrive.
AsystembackupofahardwareRAIDbasedHLRincludestheoperatingsystem
andalldataoneachessentialdiskpartitiontoatapesystem.
Asystemrestoreshouldbedone:
ifdataintegritywascompromised,or
ifbothharddrivesfailedinamirror.
ShutdownTargetNE ShutdowntheHLRbeforerestoringabackup.
ToShutdowntheHLR:
1 LogintotheHLRasroot.
2 StoptheHLRapplication.
root>sh/etc/rc2.d/S98telosstop
3 BootintoNVRAM.
root>init0
SeeExample65.
root>init0
#
INIT:Newrunlevel:0
Printservicesstopped.Thesystemisdown.
programterminated
{0}ok
EnterSingleUserModefrom TheSolarisSoftwareDisc1CDROMisrequiredinordertoentersingleuser
CDROM mode.
ToEnterSingleUserModefromCDROM:
1 InserttheSolarisSoftwareDisc1CDROMintotheCDROMdrive.
2 Bootintosingleusermode.
{ok}bootcdroms
SeeExample66.
Example66SingleUserModefromCDROM
root>init0
{ok}bootcdroms
Resetting...
Screennotfound.

...
INIT:SINGLEUSERMODE
#
FormatHardDrives Theharddrivesmustbeformattedbeforetheycanaccepttherestored
filesystem.Theharddriveformattingmustmatchtheformattingoftheoriginal
sourcedriveofthebackuptape.
TheRAIDdisksmustbehardwaremirroredfortheOracledatabaseandtheother
twodisksmustbesoftwaremirrored.
ToMirrorandFormatthedisks:
1 Usetheformatcommandtoverifythecorrectnumberofdisks(shouldbe4),disk
size,andcorrectnaming.
#format
0.c1t0d0<SUN146Gcyl14087alt2hd24sec848>
/pci@1f,700000/scsi@2/sd@0,0
/pci@1f,700000/scsi@2/sd@1,0
/pci@1f,700000/scsi@2/sd@2,0
/pci@1f,700000/scsi@2/sd@3,0
Specifydisk(enteritsnumber):^D
#
2 Usetheraidctlcommandtocreateahardwaremirroreddiskusingthelastdisks
inthelist.Repeattheraidctlwithnooptionstoverifythatthemirroringwas
successful.
#raidctlcc1t2d0c1t3d0
#
#raidctl
RAIDRAIDRAIDDisk
VolumeStatusDiskStatus

c1t2d0OKc1t2d0OK
c1t3d0OK

3 Usetheformatcommandtoverifythenewnumberofdisks(shouldbe3).c1t2d0
nowrepresentstwodisksthatarehardwaremirrored.
#format
/pci@1f,700000/scsi@2/sd@0,0
/pci@1f,700000/scsi@2/sd@1,0
/pci@1f,700000/scsi@2/sd@2,0
Specifydisk(enteritsnumber):^D
#
4 Usetheformatcommandtopartitionthedisks:
#format
UsetheinformationinTable 32for72GBdisksorTable 33for146GBdisksto

formatthefirsttwodisks.Disk1(primary)andDisk2(secondary)partitions
shouldbeidentical.UsetheinformationinTable 37andTable 38toformatthe
secondtwodisks.
Table 3572GBHardDiskPartitionSizesforDisks1&2ontheHLR

database
database
Table 36146GBHardDiskPartitionSizesforDisks1&2ontheHLR

database
database

Table 3772GBHardDiskPartitionSizesforDisks3&4

Disk3(PrimaryDisk) 0 N/A
Disk3(PrimaryDisk) 7 unassigned 72GB(Fulldisk)
Table 38146GBHardDiskPartitionSizesforDisks3&4

Disk3(PrimaryDisk) 7 unassigned 146GB(Fulldisk)
LoadRestoreTape ToLoadtheRestoreTape:
SeeExample67.
fileno=0blockno=0

CreatingaNewFilesystem Createanewfilesystemontheharddrivesandthenchecktheintegrityofeach
andCheckingTargetDisk sliceonthetargetdisk.
1 CreateaNewFilesystem:
Forexample:
2 Checkthetargetdisk:
Forexample:
RestoretheDisk Thissectionincludes:
RestoringtheDisk
RewindandEjectTape
ConfigureSystemDisk
RestoringtheDisk
Thediskcannowberestored.
ToRestoretheDisk:
#cd/mnt
#rmrestoresymtable
#cd/
#unmount/mnt
5 Mounttheapplicationlogicaldrive.
#cd/mnt
#rmrestoresymtable
8 Unmounttheapplicationlogicaldrive.
#cd/
#unmount/mnt
RewindandEjectTape
Thetapeshouldberewoundandejected.
ToRewindandEjectTape:
1 Rewindthetape.
#mtf/dev/rmt/0rewind
2 Ejectthetape.
ConfigureSystemDisk
Configurationchangesarerequiredtofilesontherootlogicaldrive.
ToConfigureSystemDisk:
#cd/mnt
2 OpenthesystemfileforeditingandremovethelinesshowninboldinExample
68.
#vi/mnt/etc/system
3 Openthemddb.cffileforeditingandremovethelinesshowninboldinExample
69.
#vi/mnt/etc/lvm/mddb.cf
4 Openthevfstabfileforeditingandremovethelines(andcomments)shownin
boldinExample70.
#vi/mnt/etc/vfstab
Example68systemFileChanges
setshmsys:shminfo_shmmax=2048000000
setsemsys:seminfo_semmap=250
setsemsys:seminfo_semmni=500
setsemsys:seminfo_semmns=500
setmsgsys:msginfo_msgmap=2048
setmsgsys:msginfo_msgmax=32767
setmsgsys:msginfo_msgmnb=65535
setmsgsys:msginfo_msgmni=200
setmsgsys:msginfo_msgssz=32
setmsgsys:msginfo_msgseg=4096
setmsgsys:msginfo_msgtql=16384
*BeginMDDrootinfo(donotedit)
forceload:misc/md_trans
forceload:misc/md_raid
forceload:misc/md_hotspares
forceload:misc/md_stripe
forceload:misc/md_mirror
forceload:drv/pcipsy
forceload:drv/simba
forceload:drv/glm
forceload:drv/sd
rootdev:/pseudo/md@0:0,0,blk
*EndMDDrootinfo(donotedit)
*BeginMDDdatabaseinfo(donotedit)
setmd:mddb_bootlist1="sd:3:16sd:3:1050sd:4:16sd:4:1050sd:11:16"
setmd:mddb_bootlist2="sd:11:1050sd:12:16sd:12:1050"
*EndMDDdatabaseinfo(donotedit)

Example69mddb.cfFileChanges
#metadevicedatabaselocationfiledonothandedit
#driverminor_tdaddr_tchecksum
sd316192
sd416193
sd1116200
sd1216201
Example70vfstabFileChanges
#device device mount FSfsck mountmount
#
#/dev/dsk/c1d0s2 /dev/rdsk/c1d0s2 /usr ufs1 yes
fd /dev/fd fd no
/proc /proc proc no
/dev/dsk/c0t0d0s0 /dev/rdsk/c0t0d0s0/ ufs1 no
/dev/dsk/c0t0d0s7 /dev/rdsk/c0t0d0s7/export/homeufs2 yes
#/dev/md/dsk/d1 swap no
#/dev/md/dsk/d0 /dev/md/rdsk/d0 / ufs1 no
#/dev/md/dsk/d2 /dev/md/rdsk/d2 /export/homeufs2 yes
swap /tmp tmpfs yes
Installthebootsectoronbothharddrives,atcylinder0ofthebackupslice.
ToInstallBootSectorandCheckDisk:
1 Installbootsectors.
/dev/rdsk/c0t0d0s0
/dev/rdsk/c0t1d0s0
2 Checkthefilesystemontheprimarydisk.
#cd/
#unmount/mnt
RestartHLRandVerifySystem RestarttheHLRandverifysystemoperation.
ToRestartHLR:
1 RemovetheSolarisSoftwareDisc1CDROM.
2 RestarttheHLR:
#reboot
3 Verifysystemoperation.
RecovertheOracleDatabase ToRecovertheOracleDatabase:
1 LogintotheHLRasroot.
2 Changetotheoracledirectory:
cd/telos/oracle

3 InstallOraclebinariesfromFullSystembackupstagingdirectory:
#zcat/telos/ora_binaries/ORACLE.TGZ|tarxf
4 Executeconfigurationscript(acceptalldefaultresponses):
#./root.sh
5 Switchusertooracle:
#suoracle
<oracle>#
6 Recreatebackupdirectorystructures
<oracle>#mkdir/telos/oracle/db_backup
<oracle>#mkdir/telos/oracle/backup_restore_scripts
7 CopyOraclebackupfilesfromtheFullSystemBackupstagingdirectorytothe
Oraclepartition.
<oracle>#cpRp/telos/ora_db_backup/*/telos/oracle/db_backup
8 Modify/telos/oracle/network/admin/listener.ora.Changemachine
namefrom"quark"tonewmachinename/IPaddress.
Example:
<oracle>#vi/telos/oracle/network/admin/listener.ora
9 Modify/telos/oracle/network/admin/tnsnames.ora.Changemachine
namefrom"quark"tonewmachinename/IPaddress.
Example:
<oracle>#vi/telos/oracle/network/admin/tnsnames.ora
10 CleanUpbaseOraclebinaryinstall:
<oracle>#cd/telos/oracle/oradata/fsv
<oracle>#rm*
11 Copyallbackupscriptsfromrestored
/telos/oracle/db_backup/backup_YYYYMMDD_HHmmdirectorytothescript
directory:
<oracle>#cp/telos/oracle/db_backup/backup_20071101_1349/*.sh\
/telos/oracle/backup_restore_scripts
12 ExecutetheOracledatabaserecoveryscriptandperformlinkages:
<oracle>#cd/telos/oracle/backup_restore_scripts
<oracle>#./dbs_rman_recovery.sh
<oracle>#mvf/telos/oracle/dbs/initfsv.ora\
/telos/oracle/dbs/initfsv.ora.bak2>/dev/null
<oracle>#lns/telos/fsv/CONF/initfsv.ora/telos/oracle/dbs/initfsv.ora
13 StarttheOracle"listener"applicationandverifysuccessfulstartup:
<oracle>#lsnrctlstart
14 LogintoOracledatabaseusing"sqlplus"andverifyaccessbyissuingsomebasic
commands:
<oracle>#sqlplushlr/hlr_devel@fsv
SQL>selecttable_namefromuser_tables;
SQL>selectcount(*)fromsubscriber;
SQL>quit
15 Logoutoforacle.Loginasfsvuser.StartHLRandverifyHLRsystemoperation:
<oracle>#exit
#sufsv
#cpu_start
#cpu_status
hlr:UPasMasterfor1day20:57hoursUnixup:1day(s)20:57hours
RestoreDiskSuiteMirroring RestoringtheDiskSuitemirroringconsistsofsettingup4replicas,
creatingaonewaymirrorandthenestablishingthetwowaymirrors.
ToRestoretheDiskSuiteMirroring:
1 Loginasroot
2 Usethemetadbcommandtosetup4replicas,forexample:
#metadbafc1t0d0s3c1t0d0s4c1t1d0s3c1t1d0s4
3 Usethe"metadbi"commandtoverifytheconfiguration:
#metadbi
au161034/dev/dsk/c1t0d0s3

oreplicaactivepriortolastmddbconfigurationchange
ureplicaisuptodate
llocatorforthisreplicawasreadsuccessfully
creplica'slocationwasin/etc/lvm/mddb.cf
preplica'slocationwaspatchedinkernel
mreplicaismaster,thisisreplicaselectedasinput
Wreplicahasdevicewriteerrors
areplicaisactive,commitsareoccurringtothisreplica
Mreplicahadproblemwithmasterblocks
Dreplicahadproblemwithdatablocks
Freplicahadformatproblems
Sreplicaistoosmalltoholdcurrentdatabase
Rreplicahaddevicereaderrors
4 Createthe/etc/lvm/md.tabfilewiththeonewaymirrorconfiguration,for
example:
#vi/etc/lvm/md.tab
d1011c1t0d0s0
d0md10

d2011c1t1d0s0
#swapslice
d1111c1t0d0s1
d1md11
d2111c1t1d0s1
#/export/homeslice
d1211c1t0d0s7
d2md12
d2211c1t1d0s7
5 Executemetainittoapplythechangesinthe/etc/lvm/md.tabfile:
#metainitaf
6 Establishrootfilesystem:
#metarootd0
7 Makethesecondarydiskbootable(ifnotperformedduringtherestore
procedure):
/dev/rdsk/c1t1d0s0
8 Edit/etc/vfstabtomodifythepartitions.
#vi/etc/vfstab
VFSTABBEFORE
#device device mount FS fsck mountmount

#
#/dev/dsk/c1d0s2 /dev/rdsk/c1d0s2 /usr ufs1yes
fd /dev/fd fd no
/proc /proc proc no
/dev/md/dsk/d0 /dev/md/rdsk/d0 / ufs 1 no
/dev/dsk/c1t0d0s7/dev/rdsk/c1t0d0s7/export/home ufs 2 yes
/dev/dsk/c1t2d0s7 /dev/rdsk/c1t2d0s7/export/home/oracle ufs 1 yes logging
swap /tmp tmpfs yes
VFSTABAFTER
#device device mount FS fsck mountmount

#
#/dev/dsk/c1d0s2 /dev/rdsk/c1d0s2 /usr ufs 1yes
#/dev/dsk/c1d0s2 /dev/rdsk/c1d0s2 /usr ufs 1yes
fd /dev/fd fdno
/proc /proc procno
/dev/md/dsk/d1 swapno
/dev/md/dsk/d0 /dev/md/rdsk/d0 / ufs1no
/dev/md/dsk/d2 /dev/md/rdsk/d2 /export/home ufs 2yes
/dev/dsk/c1t2d0s7 /dev/rdsk/c1t2d0s7 /export/home/oracle ufs 1yes logging
swap /tmp tmpfs yes
9 Haltthesystemtogettotheokprompt:
#init0
ok
10 Settheautoboot?andbootdeviceparameters:
oksetenvautoboot?true
oksetenvbootdevicedisk0disk1disk
ok

Restoresystemservice: ToRestoreSystemService:
1 Bootthesystem:
okboot
2 Whenthesystemisreacy,loginasroot
3 Usemetattachtoestablishtwowaymirrors:
#metattachd0d20
#metattachd1d21
#metattachd2d22
4 Verifymirrorstatususingthemetastatcommand.Thesystemshouldshowthat
itissyncingthemirrordisks:
#metastat
d0:Mirror
Submirror0:d10
State:Okay
Submirror1:d20
State:Resyncing
Resyncinprogress:7%done
Pass:1
Size:8392072blocks
d10:Submirrorofd0
State:Okay
Size:8392072blocks
Stripe0:
DeviceStartBlockDbaseStateHotSpare
c1t0d0s00NoOkay
d20:Submirrorofd0
State:Resyncing
Size:8392072blocks
Stripe0:
c1t1d0s00NoOkay
d1:Mirror
Submirror0:d11
State:Okay
Submirror1:d21
State:Resyncing
Pass:1
Size:8392072blocks
d11:Submirrorofd1
State:Okay
Size:8392072blocks
Stripe0:
c1t0d0s10NoOkay
d21:Submirrorofd1
State:Resyncing
Size:8392072blocks
Stripe0:
c1t1d0s10NoOkay
d2:Mirror
Submirror0:d12
State:Okay
Submirror1:d22
State:Resyncing
Pass:1

Size:18169472blocks
d12:Submirrorofd2
State:Okay
Size:18169472blocks
Stripe0:
c1t0d0s70NoOkay
d22:Submirrorofd2
State:Resyncing
Size:18169472blocks
Stripe0:
c1t1d0s70NoOkay

MAINTENANCE
IV
Chapter 11 MaintenanceOverview
Chapter 12 AlarmManagement
Chapter 13 ViewingLogsandEvents
Chapter 14 TroubleLocatingandClearing
Chapter 15 SoftwareMaintenance
Chapter 16 HardwareMaintenance

MAINTENANCEOVERVIEW
11
MaintainingtheCoreVoiceNetworkelementsincludesallactivitiesnecessaryfor
maintainingtheNetworkElements(NE)withintheirprescribedlimitsand
operatingrange.
CoreVoiceNetworkMaintenanceOverview
CoreVoiceNetwork TherearetwotypesofmaintenancethatcanbeperformedontheNE:
MaintenanceOverview
Preventivemaintenance
Preventivemaintenanceisthesystematicapplicationofmeasurementand
analysistechniquestoreducecorrectivemaintenance.
Anexampleofpreventivemaintenanceischecking,andmaintainingwithin
appropriatelimits,CPUandmemoryutilization,freediskspace,andlogfilesize.
CheckingtheseparametersshouldbeperformedbytheOperatoraspartofthe
dailyshift(seeTaskandShiftOverviewonpage 50)andisvitalfordiscovering
whenthesystemisstartingtoperformoutsideofacceptedperformancelimits.
Correctivemaintenance
Correctivemaintenanceismaintenancecarriedoutafteranalarmorfaulthas
occurredinthesystem.Thiskindofmaintenanceisintendedtorestorethe
NetworkElementornetwork/systemcomponenttoastateinwhichitcan
performitsrequiredfunction.
Examplesofcorrectivemaintenanceareexecutinganalarmcorrective
procedure,replacinghardwarecomponents,installingsoftwarepatches,and
resolvingsystem/networkfaultsusingtroubleshootingtechniques.
MaintenanceontheNEincludes:
Planningandimplementingamaintenanceschedule.
Performinganysystemandnetworkmeasurementsandtestsneededfor
preventivemaintenance.
Carryingoutanysystemandnetworkadjustmentsbasedonmeasurements
andtests.
Locatingandclearingfaultsandalarms.
Troubleshootingandreplacinghardwarecomponents.
Troubleshootingtheentiresystem.
Installingsoftwarepatchesandupgrades.

202 Chapter11:MaintenanceOverview
MaintenanceGuidelines Maintenanceactivityonanypartofacommonchannelsignallingsystemmust
onlybeundertakenwiththeagreementandknowledgeoftheadministrative
control.Suchactivitiesmightrelatetoroutinemaintenancemeasurementofthe
signallinglinkorservicesaffectingreconfigurationsoftransmissionsystemsover
whichsignallinglinksarerouted(forexample,plannedoutages).
Itisrecommendedthatallroutinemaintenanceandscheduledworkactivitiesbe
performedduringhoursofminimumtrafficsothatanyinadvertentfailure
impactstheleastnumberofcustomers.
Ataminimum,highrisk,potentiallyserviceaffectingmaintenanceandgrowth
proceduresshouldbescheduledduringweekendsandoffhours.
Theactivitiesthatmayaffectothernetworkserviceprovidersmustbe
coordinated,whichincludesbothintraandinteradministrationnetworks.
Minimumworkactivityshouldoccurtorestorethesystemtoastable
performancestate.Completesystemrecoveryactivityshouldbescheduledfor
thenextminimumtrafficperiod.
Intheeventoftotalfailureofasignallingsystemduetoamalfunctionofoneof
itsparts,immediatestepsshouldbetakentoremedythefaultcondition.Assoon
aspossible,theadministrativecontrolshouldbeinformedsothattheeventcan
becorrelatedwithotherreportedeventsorknownsignallingfailure.
FaultManagement Inatelecommunicationsnetwork,FaultManagementisthedetection,isolation
andcorrectionofmalfunctionsinthenetwork.FaultManagementactivities
includemaintainingandexaminingeventanderrorlogs,acceptingandactingon
anyerrordetectionnotifications,tracingandidentifyingfaults,carryingout
sequencesofdiagnosticstests,correctingfaults,reportingerrorconditions,and
localizingandtracingfaultsbyexaminingandmanipulatingdatabase
information.
ThereareseverallevelsoffaultmanagementintheNEs:
Alarmhandlingandcorrectiveprocedures
Troubleshootingprocedures
Eventlog,diagnosticfile,andtracefilegeneration
Handlingandclearingalarmsisahighlevelfaultmanagementtask,andalarms
canbemanagedbytheOMCorahigherlevelNetworkManagementSystem
(NMS).AcknowledgingandclearingalarmsintheNEisdescribedinChapter 12,
AlarmManagement.
Logsofeventsandrecordscanbeexaminedtohelpwithdiscoveringthecauses
ofafault.ThisiscoveredinChapter 13,ViewingLogsandEvents.
TroubleshootingtheNEandentiresystemisalsoahighlevelfaultmanagement
activity,butcanuselowerlevelinformationtoresolveissues.Troubleshooting
thesystemisdescribedinChapter 14,TroubleLocatingandClearing.

CoreVoiceNetworkMaintenanceOverview 203
Belowbothalarmhandlingandtroubleshootinginfaultmanagementactivities
areevent,diagnostic,andtracefilegeneration.Becausethisinformationisso
lowlevel,typicallytheOperatorcannotderiveanyvaluefromit.Customer
Supportcanmakemoreuseoflowlevelfaultmanagementinformationto
resolvesystemmalfunctionandhelptroubleshootthesystem.
Faultmanagementactivitiesshouldalwaysbeginatthehighestpossiblelevel,
andonlydrilldownifthemalfunctioncannotberesolvedatahigherlevel.The
NEhasseveraltoolsinthislowerleveloffaultmanagement.
Maintenanceincludesbothsoftwareandhardware.Softwaremaintenance
includesrestoringthesystemand/ordatabasefromabackup,whichiscoveredin
Chapter 10,DatabaseandSystemBackupandRestore.Italsoincludesapplying
softwareversionupgradesandpatcheswhichiscoveredinChapter 15,Software
Maintenance.HardwaremaintenanceiscoveredinChapter 16,Hardware
Maintenance.

ALARMMANAGEMENT
12
ThischapterprovidesanoverviewoftheAlarmsrelatedtotheCoreVoice
Networkelementscommonapplications.
AlarmsOverview
MonitoringAlarms
AlarmRootCauses
CommonAlarms
AlarmsOverview TheOMCConsoleActiveAlarmsDisplaydisplaysalarmsontheSignalingServer.
AlarmmonitoringisperformedonaroutinebasisbytheOperator.
AlarmSeverity AlarmsraisedontheNEarecategorizedbylevelofseverity.Alarmswherethe
severitylevelcannotbedeterminedarelabeledindeterminate.Indeterminate
alarmsareinformationalandarenotcriticalormajoralarms.thereisno
correctiveactionforindeterminatealarms.
Thesealarmseveritylevelsare:
Critical
Criticalalarmsindicatethataserviceaffectingconditionhasoccurred,andthat
immediatecorrectiveactionisrequired.
Forexample,acriticalalarmisreportedwhenaManagedObjectfailsandgoes
outofservice.IftheManagedObjectisnotrestored,thenitmightcauseother
ManagedObjectstogooutofserviceaswell.
Major
Majoralarmsindicatethataserviceaffectingconditionhasdevelopedand
urgentcorrectiveactionisrequired.
Forexample,amajoralarmoccurswhenthereisaseveredegradationinthe
capabilityofamanagedobjecttodeliverservice.However,thedegradationin
servicedoesnotaffectotherManagedObjects.
Minor
Minoralarmsindicatethatanonserviceaffectingfaultconditionhasoccurred,
andthatcorrectiveactionshouldbetakentopreventamoreserious
serviceaffectingfault.

206 Chapter12:AlarmManagement
Forexample,aminoralarmisreportedwhenthedetectedalarmconditionisnot
currentlydegradingthecapacityofthemanagedobject.Noservicedegradation
hasyetbeendetected,butifthesituationpersists,itwillcauseadegradationin
service.
Warning
Warningalarmsindicatethatapotentiallyserviceaffectingfaulthasbeen
detectedbeforeanysignificanteffectshavebeenfelt.Youshouldtakefurther
actiontodiagnoseandcorrecttheproblemtopreventamoreseriousservice
affectingfault.
AlarmTypes AlarmsraisedontheNEaretypedaccordingtothefunctionalareatohelplocate
andisolatethealarms.
Thealarmtypesare:
ProcessingError
Equipment
Communications
ProcessingError
Processingerrorsoccurwheninternalansoftwareprocessfails,shutsdown,or
nearscapacity.Theyarealsocausedbycorruptdatainthedatabase.
Equipment
Equipmentalarmsoccurwhenahardwaredeviceorsubsystemisoutofservice,
suchasspan,port,ortrunk.UsetheOMCConsolereachthroughfeaturetocheck
transmissionequipmentforalarms.
Communications
CommunicationalarmsoccurwhenTCP/IPorSS7communicationlinksaredown
ornotconfiguredproperly.IntheNE,thesealarmsaretypicallyreportedwhen
thesystemrestarts.
AutomaticandManualAlarm Alarmsaretriggeredbyevents.Alarmsmayclearautomaticallyormayrequire
Clearing manualclearing.
AutomaticAlarmClearing
Somealarmsareclearedautomaticallywhenthetriggeringconditionisresolved.
Generally,thesealarmsarerelatedtothresholds,orconnectionstoother
NetworkElements(liketheIntelligentMediaGatewayorBaseStation
Controller).
Thesealarmsareclearedautomaticallywhenthefaultontheresponsible
NetworkElementiscleared,orwhenvaluesdonotexceedthresholdvalues.
Thecorrectiveprocedureforthealarmwillindicateifthealarmiscleared
automatically.

MonitoringAlarms 207
ManualAlarmClearing
OtheralarmsrequireOperatorinterventiontomanuallyclearthealarm.Someof
thesealarmsresultinterminationoftheSignalingServerapplication.Forthese
alarms,theNEapplicationisrestarted,andthealarmissilentlycleared.
Eventbasedalarmsmustbemanuallyclearedbyresolvingtheunderlyingevent
thatistriggeringthealarm.Thealarmisraisedeachtimethetriggeringevent
occurs(thereisnoalarmcorrelationorsuppression).
Thecorrectiveprocedureforthealarmwillindicateifthealarmmustbecleared
manually.
MonitoringAlarms MonitoractivealarmsusingtheActiveAlarmsDisplayontheOMC.TheActive
AlarmsDisplayshowsallactivealarmsforthemanagedNetworkElement.
RefertotheOMCUserInterfaceGuidefordetailedinformationontheActive
AlarmsDisplay.
ToMonitorAlarms:
RefertotheOMCUserInterfaceGuideforhowtostarttheOMCbrowser.
2 SelecttheOMCNavigatorFaultmenu.
3 ClickShowalarms.
ThewindowsimilartothatshowninFigure 37appears.
Figure 37ActiveAlarmsDisplay
4 ClicktheManagedObjectClasscolumnheadertosortthealarms.
5 UsethescrollbarstolocateallalarmsraisedonthetargetNE.
6 Reviewanyunresolvedalarms.
7 Reportanynewalarms(allseverities)tofaultmanagementpersonnelfor
resolution.
8 RecordanyunresolvedalarmsintheOperatorshiftChecklist.
9 KeeptheActiveAlarmsDisplaywindowopen.
AlarmRootCauses Aprobablecausedefinesthelikelyreasonfortheeventthatcausedthealarm.
Oneormorealarmscansharethesameprobablecause.
TheprobablecausesidentifiedintheNEfollowITUX7.33.Thefollowingsections
describetheprobablecausesusedbyalarmsontheAccessServer.
CommunicationsProtocolError
ACommunicationsProtocolErrorindicatesthatacommunicationsprotocolhas
beenviolated.
ConfigurationorCustomizationError
AConfigurationorCustomizationErrorindicatesthatacustomizableparameter
forasystemordeviceisincorrectlyspecified,orisinconsistentwiththeactual
configuration.
AllofthealarmswithConfigurationorCustomizationErrorasaprobablecause
arerelatedtoconfigurationoftheMTP3layer(Lsn)andSCCPlayer(Lsp).
MostofthealarmsareMAJORbecausetheywillimpedeorotherwisenegatively
impactservice.
Congestion
Congestionindicatesthesystem,ornetworkcomponent,hasreacheditscapacity
orisapproachingit.
ThereareseveralMAJORalarmsandoneCRITICALalarmrelatedtotheMTP3
layer(Lsn)intheSS7stack.TheMAJORalarmsindicatecongestiononSS7links
androutes,andtheCRITICALalarmindicatesfailureoftheSS7stack.
CorruptData
CorruptDataindicatesthatanerror(unspecified)hascausedsomedatatobe
incorrectandtherefore,unreliable.
TheCRITICALalarmindicatesaread/writeerrorintheconfigurationdatabase.
TheMINORalarmsareraisedwhencircuitsreceiveimproperblockingmessages.
FileError
FileErrorindicatesthattheformatofafile(orsetoffiles)isincorrectandthus
cannotbeusedreliablyinprocessing.
OutofMemory
OutofMemoryindicatesthatnoprogramaddressablestorageavailable.

AlarmRootCauses 209
PowerProblem
PowerProblemindicatesthatthereisaproblemwiththepowersupplytooneor
moreresources.
ResourceatorNearingCapacity
ResourceatorNearingCapacityindicatesthattheusageofaresourceisator
nearingthemaximumallowablecapacity.
TheMAJORalarmswithResourceatorNearingCapacityasaprobablecause
indicateaproblemattheIntelligentMediaGatewayorwithCallProcessing.
ResponseTimeExcessive
ResponseTimeExcessiveindicatesthattheelapsedtimebetweentheendofan
internalsubsystemmessageandthestartoftheanswertotheinternal
subsystemmessageisoutsideofacceptablelimits.
TheCRITICALalarmisraisedwhenallinterprocesscommunication(ipc)fails.The
WARNINGalarmindicatesthatinterprocesscommunicationisbackonline.
SoftwareProgramAbnormallyTerminated
ASoftwareProgramAbnormallyTerminatedindicatesthatasoftwareprogram
hasterminatedduetosomeunrecoverableerrorcondition.
AllbutoneofthealarmswithSoftwareProgramAbnormallyTerminatedasa
probablecauseareCRITICALalarmsthesealarmsindicatethattheNE
applicationhasstoppedduetoasoftwareerror.TheWARNINGalarmindicates
thattheNEapplicationwasstoppedbyanOperatorcommand.
SoftwareError
ASoftwareErrorindicatesanerrorinsoftwareforwhichnomorespecific
probablecausecanbeassigned.TheCRITICALalarmresultsfromalowerlayer
eventintheSignalingServerapplication,andrequiresarestartoftheapplication.
TransmitFailure
ATransmitFailureindicatesafailuretoreceiveanexpectedmessage.Thereisa
singlealarmofMINORseverityduetoexpiryofinternaltimers.
UnderlyingResourceUnavailable
UnderlyingResourceUnavailableindicatesthatanentityuponwhichthe
reportingobjectdependshasbecomeunavailable.
UnspecifiedReason
UnspecifiedReasonindicatesthattheunderlyingeventcausingthealarmisnot
specified.
ThealarmswithUnspecifiedReasonasaprobablecauseindicatethatusersof
theSCCP(Lsp)connection(forexample,HomeLocationRegister)are
unreachable(MINOR)ornowreachable(WARNING).

FaultLocalization Faultlocalizationisusedtohelplocatethefaultinthenetwork.TheOMCActive
AlarmsDisplaydisplaystheManagedObjectandtheManagedObjectinstanceto
helplocalizethefaultintheNE.ForfaultsinotherNetworkElements,theIP
addressorhostnameisdisplayedtolocatethefault.
UsetheinformationprovidedbyfaultlocalizationtolocatetheNetworkElement
thatgeneratesthefault,andtolocatetheManagedObjectintheconfiguration
databasefortheaffectedNetworkElement(ifapplicable).
Forfaultsrelatedtoconfigurationerrors,refertoSectionIIIProvisioningasa
guidetothetablesintheconfigurationdatabase,andthedefaultand
recommendedparametersettings.
CommonAlarms ThissectiondescribesalarmscommontonodesthatmakeuptheCoreVoice
NEs.ThesecommonalarmscanberaisedonanyoftheNEs.
CommonalarmsraisedontheNEsarecategorizedbylevelofseverity.The
severitylevelsare:
CriticalCommonAlarms
MajorCommonAlarms
MinorCommonAlarms
WarningCommonAlarms
CriticalCommonAlarms CriticalcommonalarmsarealarmsthatarecommonacrossseveralNetwork
Elements.Thecriticalcommonalarmsare:
NERestart
cpuCMSoftFail
cpuCmVitalDeath
cpuPMCMDDeath
cpuPMHWDDeath
DblDbRecErr
ipcTcpHostDown
tpmTmrDrop
DiskAlarm
RootDiskAlarm

CommonAlarms 211
NERestart
Table 39NERestartClearingInformation
AlarmName NERestart
AlarmSeverity Critical
ManagedObject SeeManagedObjectInstancefieldinOMCActiveAlarms
Display
Description EquipmentNErestarted.
SystemImpact NErestarted.MRFservicesunavailablewhileNEwas
restarting.
RootCause(s) UnspecifiedReason
ClearingActions AutomaticAlarmClearing
Nootherinformationcurrentlyavailable
cpuCMSoftFail
Table 40cpuCmSoftFailClearingInformation
AlarmName cpuCmSoftFail
Display
Description ProcessingErrorchild_monwillexitbecauseofasoftware
failure(reasonincludedintext)
SystemImpact AccessServerapplicationfailure
RootCause(s) SoftwareProgramAbnormallyTerminated
ClearingActions ManualAlarmClearing
cpuCmVitalDeath
Table 41cpuCMVitalDeathClearingInformation
AlarmName cpuCMVitalDeath
Display
Description ProcessingErrorchild_monwillexitbecauseavitalprocess
hasdied

cpuPMCMDDeath
Table 42cpuPMCMDDeathClearingInformation
AlarmName cpuPMCMDDeath
Display
Description ProcessingErrorpeermonreportsvitalchild_monprocess
died
cpuPMHWDDeath
Table 43cpuPMHWDDeathClearingInformation
AlarmName cpuPMHWDDeath
Display
Description ProcessingErrorVitalHWDprocesshasdiedorexited
SystemImpact HWDprocessfailure
DblDbRecErr
Table 44DblDbRecErrClearingInformation
AlarmName DblDbRecErr
Display
Description ProcessingErrorDatabaseerror
SystemImpact Notavailable
RootCause(s) CorruptData

CommonAlarms 213
ipcTcpHostDown
Table 45ipcTcpHostDownClearingInformation
AlarmName ipcTcpHostDown
Display
Description CommunicationsIPCTCPhostdownplushostinfo
RootCause(s) ResponseTimeExcessive
tpmTmrDrop
Table 46tpmTmrDropClearingInformation
AlarmName tpmTmrDrop
Display
Description ProcessingErrorTheTMRDROPeventoccursinthelower
layeroftheAccessServerapplication.Messagesusedfor
interprocesscommunicationhaveexceededthemaximum
queuelengthforagivenprocess.Theentiresystemis
overloaded.
SystemImpact Serviceimpact.ThereisnoserviceuntiltheAccessServer
applicationisrestarted.Messageresponseswillbelostand
timeoutwilloccuratfarend.
1 Acknowledgethealarm.
2 StoptheAccessServerapplication.
3 RestarttheAccessServerapplication.

DiskAlarm
Table 47DiskAlarmClearingInformation
AlarmName DiskAlarm
Display
Description EquipmentDiskpartition/telos/prohasexceeded85%.
SystemImpact Ifdiskisnotcleanedup,AccessServerapplicationwillfailif
diskpartitionreaches100%.
RootCause(s) ResourceatorNearingCapacity
ClearingActions CheckUNIXdiskpartitionswithdfkMakesureanytraces
areturnedoff.Cleanuptracefiles.Automaticallycleared
whendiskpartitionfallsbelow85%.
RootDiskAlarm
Table 48RootDiskAlarmClearingInformation
AlarmName RootDiskAlarm
Display
Description EquipmentDiskpartition/hasexceeded85%
SystemImpact Ifdiskisnotcleanedup,AccessServerapplicationwillfailif
diskpartitionreaches100%.
RootCause(s) ResourceatorNearingCapacity
ClearingActions CheckUNIXdiskpartitionswithdfkMakesureanytraces
areturnedoff.Cleanuptracefiles.Automaticallycleared
whendiskpartitionfallbelow85%.

CommonAlarms 215
MajorCommonAlarms MajoralarmscommonacrosstheCoreVoiceNEsare:
PowerProblem
PowerProblem
Table 49PowerProblemClearingInformation
AlarmName PowerProblem
AlarmSeverity Major
Display
Description EquipmentVoltagethresholdcrossed.(Thespecific
equipmentinvolvedisincludedinthedescription.)
SystemImpact Equipmentspecifiedmaynotbeavailable.Checkfansand
otherpoweredequipment.
RootCause(s) PowerProblem
MinorCommonAlarms TherearenoalarmswithMinorseveritycommontoCoreVoiceNEs.
WarningCommonAlarms WarningalarmscommonacrosstheNEsare:
cpuCMShutDown
IpcTcpHostUp
cpuCMShutDown
Table 50cpuCMShutDownClearingInformation
AlarmName cpuCMShutDown
AlarmSeverity Warning
Display
Description ProcessingErrorCPUmonitorbeginningshutdownby
operatorrequest
SystemImpact SignalingServerapplicationwillbeshutdown.

IpcTcpHostUp
Table 51IpcTcpHostUpClearingInformation
AlarmName IpcTcpHostUp
AlarmSeverity Warning
Display
Description CommunicationsIPCTCPhostupplushostinfo

V IEWINGLOGSANDEVENTS
13
Thischapterfocusesonviewinglowleveleventlogs.
ThevlogUtility
vlogMessages
ViewingEventsandRecords
ThevlogUtility TheCoreVoiceNetworkelementsuseaLogViewerutilitycalledvlogtoview
recordsofeventsinthesystem.
Eventlogscanbeusedtomonitorandtroubleshootthesystem,especiallyasit
relatestoconfigurationchangesorerrorsinnetworkconfiguration.
TheNetworkElement(NE)generateseventlogrecordsasaresultofbothnormal
systemactivitiesandunusualsystembehavior.Theeventlogrecordsdisplay
eventsforthehistoricalpast(upto20daysago),thecurrentday,andin
realtime.Inrealtimeformat,vlogprintseventstostandardoutput(thescreen)
oryoucanusevlogtoselectfromalistofhistoricaleventfiles.
Eventlogsareclassifiedbylevelofseverity.Severitylevelsrangefrom
informational(INFO)forwhichnooperatorinterventionisrequired,tosevere
serviceaffectingmalfunctions(FATALERROR).
Eachsubsystem(ormodule)intheNEusesmessagesandeventsforintersystem
communication.Therearenumerouseventsforeachsubsystemandtheevent
logscanbecomequitelarge.Theeventsforsomesubsystemsaremorerelevant
thanothersinunderstandinghowthesystemisworking.
RefertoAppendix B,SubsystemEventsforinformationonspecificsubsystem
events.
vlogMessages vlogMessageFormats
Systemconsolemessagesusethefollowingformat:
date/time<process><severity>:<xxx_message_s><filename>
<linenumber>textinfo
SeeExample71.

218 Chapter13:ViewingLogsandEvents
Example71SystemConsoleMessage
2005/11/1801:30:31VSMINFO:MEG_GATEWAY_SERVICE_STATE_CHANGE_M
gateway_id=13old_service_state=INSERVICE
new_service_state=OUT_OF_SERVICE
Table 52describestheparametersinavlogmessage.
Table 52vlogMessageFormat
Parameter Description
date/time Eventtimestamp
process Softwareprocessthatloggedtheevent
severity Severityleveloftheevent
xxx_message_s Messageusedtoidentifytheevent,where:xxxisthenameof
theinvokingmodulemessageistheactualmessagesisI,W,E,
U,F,ortheseveritylevel
filename Fileassociatedwiththeevent.Onlyappearsifseveritylevelis
WARNING,ERROR,orFATAL.
linenumber Linenumberassociatedwiththeevent.Onlyappearsifseverity
levelisWARNING,ERROR,orFATAL.
textinfo Onaseparateline,thedetailedtextofthemessage
vlogSeverityLevels
TheseveritylevelsforvlogmessagesarelistedinTable 53.
Table 53SystemConsoleSeverityLevels
SeverityLevel Description
INFO Reportsthatthesystemhasautomaticallyrespondedtoaproblem.
WARNING Reportsapossibleproblem.
USERERROR Reportsaproblemthatisoutsideofthesystemcontrol.Operator
interventionisrequired.
ERROR Reportsamajorproblem;TheNEapplicationmayneedtobe
restarted.OperatorandCustomerServiceinterventionisrequired
Reportsasevere,serviceaffectingmalfunction.TheNEattemptsto
FATALERROR restartandrecoverfromtheproblem.OperatorandCustomerService
interventionisrequired.
ViewingEventsand Logsandeventscanbeviewedbytimeframeandsearchedbynameandseverity.
Records
EventLogRecordTimeframe Eventrecordsimplysomechronologicalaspecteventshavealreadyhappened
sometimeinthepast,eventshavehappenedmorerecently,andeventsare
currentlyhappening.EventLogshavesomeaspectsofeachofthesetimeframes.

ToViewEventLogRecords:
1 Startvlogbytypingvlogatthecommandprompt.
ThevlogmenuappearsasshowninExample72.
Example72Startingvlog
pro>vlog
Selectoneoftheoptions:
1.ViewCallDetailRecords
2.ViewCallDetailRecords:LATEST
3.ViewEventLogRecords
4.ViewEventLogRecords:TODAYS
5.ViewEventLogRecords:FUTURE
6.ViewAlarmRecords
7.ViewSMSDetailRecords
8.ViewSMSDetailRecords:LATEST
9.ViewCallandSMSDetailRecords
10.ViewCallandSMSDetailSummary
Qq.Exit
TypethenumberoftheselectionthenpressENTER:
2 SelectanEventLogRecordtimeframe:
OnlytheEventLogandAlarmRecordsselectionsarerelevanttotheNE.
"ViewEventLogRecords"allowsyoutoselectfromallstoredeventlogrecord
files."ViewEventLogRecords:TODAYS"allowsyoutoselectfromonlytheevent
logrecordfilesfortoday'sdate."ViewEventLogRecords:FUTURE"allowsyouto
seecurrenteventlogrecordsastheyaregenerated,inrealtime.
3 ForhistoricalEventLogRecords,selectaday.
SeeFigure 38.
4 Exitvlogwhenfinished:
Qq.Exit

220 Chapter13:ViewingLogsandEvents
Figure 38HistoricalEventLogRecords
Selectoneoftheoptions:
1.ViewCallDetailRecords
2.ViewCallDetailRecords:LATEST
3.ViewEventLogRecords
4.ViewEventLogRecords:TODAYS
5.ViewEventLogRecords:FUTURE
6.ViewAlarmRecords
7.ViewSMSDetailRecords
8.ViewSMSDetailRecords:LATEST
9.ViewCallandSMSDetailRecords
10.ViewCallandSMSDetailSummary
Qq.Exit
TypethenumberoftheselectionthenpressENTER:3
Selectafiletoviewfromthelist:
1)2005Nov04
2)2005Nov05
3)2005Nov06
...
19)2005Nov22
20)2005Nov23
21)2005Nov24
22)2005Nov25
23)Exit
SearchingforEventsbyName EventLogRecordsarequitelargetheycontainalloftheeventsgeneratedon
orSeverity theNE.EventlogRecordscanbesearched,usinganeventnameoreventseverity
level.
vlogmakesuseoftheUnixlessutilitytopagethroughtheoutput.Presshwhen
viewingarecordtodisplaythehelptextforallthecommandsavailablefor
processingEventLogRecordstherearemany.
Themostusefulcommandsforsearchingaretheforwardslash(/)andthe
questionmark(?)symbols.Enterasearchsymbolfollowedbythesearchterm.
Theforwardslashsearchesforwardintheoutputforthesearchterm,andthe
questionmarksearchesbackwardintheoutputforthesearchterm.
Forexample,tosearchforwardforallERRORmessages,enter/ERRORorto
searchbackwardforaneventbyname,enter?<EventName>.
ToViewandSearchforEventLogs:
1 Startvlogbytypingvlogatthecommandprompt.
2 SelectoneoftheEventLogRecordstoviewfromthevlogmenu(all,TODAYS,or
FUTURE).
3 SelectanEventLogRecordfiletoviewfromthelistdisplayed.
4 Enterasearchsymbolfollowedbytheeventseverityleveloreventname.
Enterhtoseeallavailablecommands.
5 Pressanykeytopagethroughtheoutput.
Thesearchtermispresentedinagreytextbox.

Usetheeventtimestampandsearchtermstoisolatetheeventyouarelooking
for.
RefertoAppendix B,SubsystemEventsforinformationonspecificsubsystem
events.

TROUBLELOCATINGANDCLEARING
14
ThischapterdescribeshowtocaptureandanalyzeSS7packets,capturesyslog
fromtheIntelligentMediaGateway,howtoshutdownthesysteminan
emergency,andhowtorecoverfromanunplannedshutdownwhetherdueto
networkoutageordisaster.ThischapteralsoincludessomeHWD
troubleshootinghints.
ThereisalsosomelowleveltroubleshootinginformationavailableinAppendix B,
SubsystemEventsandAppendix C,DiagnosticTools.
PacketCaptureandAnalysis
LoggingIssues
RecoveryfromOutageorDisaster
TraceManager
TroubleshootingHWD

224 Chapter14:TroubleLocatingandClearing
PacketCaptureand SS7separatestheinformationrequiredtosetupandmanagetelephonecallsin
Analysis thePublicSwitchedTelephoneNetwork(PSTN)ontoaseparatepacketswitched
network(SignalingNetwork).ItusesMessageSignalUnits(MSUs),LinkStatus
SignalUnits(LSSUs),andFillInSignalUnits(FISUs)assignalunits.
AnSS7analyzerwouldusuallybeactiveonSS7linksbetweenanMSCandSTP.
TheMSCalsousesaproprietaryprotocolcalledSCCPLitetocommunicatewith
theBaseStationSubsystem(BSS).ToviewSCCPLitemessages,usesnoopand
WiresharkandinstalltheSCCPLiteplugin.
Aboutsnoop InlieuofusinganSS7analyzer,snoopcanbeusedtoanalyzepacketactivityin
thesystem.snoopisapacketmonitoringandcaptureprogramdevelopedbySun
Microsystems.
Thecaptureanddisplayofpacketactivityinthenetworkisveryhelpfulin
troubleshootingnetworkproblems.snoopcanbeusedtocapturepacketsand
thenstoretheminabinaryfile.snoopwritesfilesthatcomplywiththefile
formatdescribedinhttp://www.faqs.org/rfcs/rfc1761.html.
Onlythesuperusercanrunsnoop.
Whenthereissomeindicationofacallornetworkproblem,generaterelevant
snoopfiles(seeGeneratingaNamedsnoopFile,GeneratingSpecificProtocol
snoopFiles,andGeneratingsnoopFilesforaSpecificNetworkElement).
Thebinarysnoopfilesneedtobedisplayedinaviewer,andmanysnoopfile
viewersareavailable.IfSCCPLitemessagesshouldalsobedisplayed,thenuse
Wiresharkasasnoopfileviewer(seeAboutWireshark).
AboutWireshark Wiresharkisanopensourcenetworkpacketanalyzer.Anetworkpacketanalyzer
capturesnetworkpacketsanddisplaysthepacketcontentsindetail.
Wireshark(orotherpacketanalyzer)isinstalledonaWindowsclient.snoopfiles
aretransferredfromtheNEandviewedwithWiresharkontheWindows
machine.
TheAdministratorcanuseWiresharktotroubleshootSS7networkproblems,and
toexamineinternalprocesscommunicationtotroubleshootsystemproblems.
Wiresharksupportsapproximately750protocols.TheSCCPLitepluginfor
WiresharkisrequiredforviewingSCCPLitepacketscapturesfromtheMSCBSS
interface.
WiresharkisanOpenSourcedistributionthatcanbedownloadedandinstalledat
nocost(seeDownloadingandInstallingWiresharkonpage 227).
Usingsnoop Thissectionincludes:
PacketCapturePoint

ProtocolsandPorts
GeneratingaNamedsnoopFile
GeneratingSpecificProtocolsnoopFiles
GeneratingsnoopFilesforaSpecificNetworkElement
PacketCapturePoint
RunningsnoopontheActiveMSCwillcaptureallpacketsthatareroutedthrough
theMSC.snoopcapturespacketsontheuplink0virtualinterface.uplink0is
madeupofthetwophysicalinterfaces(eri0,eri1ORhme0,hme1)ofthetwo
NICcardsinstalledineachMSC.
snoopwillcapturepacketsfromanyspecifieddevice,andnormallysnoopwill
usethefirstnonloopbackinterfaceitfinds.Specifythedeviceifsnoopshould
capturepacketsfrominterfacesotherthanthoserepresentedbyuplink0.
ProtocolsandPorts
Protocolmessages(packets)aredirectedtodefinedportsintheMSC.Table 54
showstheprotocoltoportmappingsforcapturingpacketswithsnoop.Theports
shownaredefaultportsandtheseremainifforceunlessexplicitlychanged.
GeneratingaNamedsnoopFile
Table 54ProtocolsandPortMappings
Protocol Port
SCTP(M2UA/SCTP) 2904
SCTP(M3UA/SCTP) 2905
SCCPLite 5000
MGCP 2427
MEGACO(H.248) 2944
SIP(CallConferencingServer) 5060
SIP(AccessServer) 5061
Theoutputfromthesnoopcommandcanbesenttostandardoutput(the
screen)ortoafile.Wireshark(orotherviewingtool)requiresabinarysnoopfile
asinput.
Usetheooptionfollowedbythefilenametospecifythesnoopfile.snoopruns
inpromiscuousmode,meaningalllevelofpacketdetailiscaptured.
snoopfilesshouldbegeneratedtoatemporarydirectoryandthendeletedafter
theyareexportedtoaWindowsclientforviewing.snoopfilesizedependsonthe
capturetime,butsnoopfilesthatprovideenoughinformationfor
troubleshootingactivitiescaneasilyrangefrom5to30MB.
ManysnoopfileswillbegeneratedonseveralNetworkElementsinthecourseof
troubleshootingthesystemandnetwork.Createasnoopfilenamingconvention
thatreflectsthis.

Accesstotherootaccountisrequiredtorunsnoop.
ToGenerateaNamedsnoopFile:
1 LogontotheMSCasmscuser.
msc>suroot
3 Runsnoop.
root>snoopo<filename>
4 Stopsnoopaftersufficientcapturetime.
Ctrlc
Thefilestreamisclosed,andthegeneratedfilecanbeexportedtoasnoopfile
viewer(seeRetrievingsnoopFilesonpage 228).
SeeExample73foranexamplethatgeneratesanamedsnoopfile.
Example73GeneratingNamedsnoopFile
msc>suroot
root>cd~/msc/TMP
root>snoopotest1.snoop
Usingdevice/dev/eri(promiscuousmode)
^C
GeneratingSpecificProtocolsnoopFiles
Unlessotherwisespecified,snoopcapturespacketssentbyallprotocols.Packets
sentbyaspecifiedprotocolcanbecaptured,packetsfromallotherprotocols
beingrejected.Usetheportcorrespondingtothedesiredprotocoltocapture
onlyprotocolspecificpacketsinthesnoopfile.SeeTable 54fordefaultprotocol
toportmappings.
ToGenerateProtocolSpecificsnoopFiles:
1 Generateanamedsnoopfileandspecifytheport.
root>snoopotest2.snoopport2904
Ctrlc
GeneratingsnoopFilesforaSpecificNetworkElement
ThepacketsreceivedfromaspecifiedNetworkElementcanalsobecaptured
usingsnoop,withpacketsfromallothersourcesbeingrejected.TheNetwork
ElementisspecifiedwitheitherthehostnameortheIPaddress.
Protocolspecific,NetworkElementspecificoptions,andothersnoopoptions
canbecombinedtofurtherisolatepackettraffic.

ToGeneratesnoopFilesforSpecificNetworkElement:
1 GenerateanamedsnoopfileandspecifytheNetworkElement.
root>snoopotest3.snoophost<hostname>
OR
root>snoopotest3.snoop"<IPaddress>"
Ctrlc
SeeExample74foranexamplethatgeneratesanamedsnoopfilefrompackets
capturedfromaspecifiedNE.
Example74GeneratingsnoopFileforaSpecificNE
root>cd~/msc/TMP
root>snoopotest3.snoophostlondon
^C
OR
root>cd~/msc/TMP
root>snoopotest3.snoop"10.10.10.10"
^C
UsingWireshark Thissectionincludes:
DownloadingandInstallingWireshark
RetrievingsnoopFiles
AnalyzingsnoopFiles
CorrectingforCaptureandAnalysisTimeDifference
ApplyingFilters
DownloadingandInstallingWireshark
WiresharkisanOpenSourcedistributionthatcanbedownloadedandinstalledat
nocost.Wiresharkisinstalledonawindowsclient.snoopfilesaretransferred
fromtheMSCandviewedwithWiresharkontheWindowsmachine.
StarSolutionshascreatedpluginsfortheproprietarySCCPLiteprotocolin
Wireshark.YoucaninstalleitherthedefaultWiresharkwithouttheabilityto
analyzeSCCPLitemessages,oryoucanobtaintheWiresharkinstallationpackage
fromStarSolutionswhichincludesthepluginsforSCCPLite.
ThedefaultinstallationforWiresharkisavailableat
http://www.wireshark.org/download.html.
AcceptalldefaultsfortheWiresharkinstall.Theprotocolpluginwillnotinstall
correctlyifthedefaultWiresharkinstalllocationisnotused.

ToDownloadandInstallWireshark:
1 NavigatetotheWiresharkdownloadpage:
http://www.wireshark.org/download.html
orobtaintheWiresharkexecutableinstallationfilefromStarSolutions.
2 SavetheWindowsinstallationfiletoatemporarydirectory.
3 Executetheinstallationfile,forexample:wiresharkwin321.6.0.exe.
Acceptalldefaults.
ThedefaultinstalllocationisC:\ProgramFiles\Wireshark.
RetrievingsnoopFiles
snoopfilesaregeneratedontheNEbuttheyareviewedonaWindowsclient
usingWireshark(orotherpacketviewer).Manysnoopfileswillbegeneratedon
severalNetworkElementsinthecourseoftroubleshootingthesystemand
network.CreateadirectoryhierarchyontheWindowsclientthatreflectsthis.
UseFTPtotransfersnoopfilesfromthetargetMSCtotheWindowsclient.
Preferably,alsouseagraphicalFTPclientforfiletransfer.
ToRetrievesnoopFiles:
1 CreateanewdirectoryontheWindowsclientforsnoopfiles.
Forexample,C:\snoop\<hostname>.
2 ConnecttothetargetMSCusingFTPclient.
3 TransferfilesfromtargetMSCtoWindowsclient.
Forexample,from~/msc/TMPtoC:\snoop\<hostname>.
4 CloseconnectionandexitFTPclient.
Thesnoopfile(s)shouldnowbeinC:\snoop\<hostname>.
AnalyzingsnoopFiles
snoopfilesareinterestingontheirown,buttherealvalueofanalyzingasnoop
fileistouseitalongsideaCDRfilegeneratedforthesameperiod(using
timestamps).Analyzingasnoopfileinthismannerallowsmessages(packets)to
becorrelatedtoactualcallornetworkbehavior.
Whenthereissomeindicationofcallornetworkproblem,generaterelevant
snoopfiles(seeGeneratingaNamedsnoopFile,GeneratingSpecificProtocol
snoopFiles,andGeneratingsnoopFilesforaSpecificNetworkElement).
UsetherelevantsnoopfileandarelevantCDRfileforthesametimeperiod.
ToOpensnoopFiles:
1 LaunchWireshark.
Start>Programs>Wireshark
2 SelectFileandclickOpen.

3 Openthedesiredsnoopfile.
Thefilemaytakesometimetoopen.
snoopfilesizedependsonthecapturetime,butsnoopfilesthatprovideenough
informationfortroubleshootingactivitiescaneasilyrangefrom5to30MB.
ToAnalyzesnoopFiles:
1 OpenasnoopfileinWiresharkandarelevantCDRfileinatexteditor.
SeeCorrectingforCaptureandAnalysisTimeDifference.
2 SettheWiresharkTimeDisplayFormat.
View>TimeDisplayFormat>DateandTimeofDay
3 LocatetheissueintheCDRfile.
4 Usethetimestampinbothfilestolocatethepacketsforaparticularcall.
TakeanyUTCoffsetintoaccount.
OR
5 Applyanyfilterstothesnoopfile,notethetimestamp,andthensearchtheCDR
filetomatchtheparticularcall.
SeeApplyingFiltersonpage 230.
CorrectingforCaptureandAnalysisTimeDifference
Whenpacketsarecapturedbysnoop,eachpacketistimestampedasitcomesin.
snoopusesthetimestampfromtheOSkernelonthemachineitisrunningon.
ThesetimestampswillbesavedtothesnoopcapturefileandWiresharkusesthe
timestampfromthecapturefileBUTitcorrectsitforthesystemtimeonthe
machineWiresharkisrunningonatviewingtime.
snoopconvertsthetimestampfromlocaltimetoUTCtimeatcapturetime.
WiresharkconvertstheUTCtimestamptolocaltime,anddisplaysthattimewhen
viewing.Wiresharkdisplaysthetimestampasdate(dayssince1.1.1970)andthe
timeofday(innanosecondssincemidnight,ontheviewingmachine),knownas
UTCformat.
ACDRfilealsousesthetimestampformtheOSkernelonthemachineitis
runningon.TheCDRisgeneratedasatextfileandthetimestampisnot
'corrected'whenitisviewed.TheresultisthatasnoopfileandaCDRfromthe
exactsametimeperiodmaydisplaydifferenttimestampstotheextentthatthe
systemtime(timezone)ontheviewingmachine(Windowsclient)differsfrom
thesystemtime(timezone)ofthetargetMSC.
Correctanytimezonedifference(byapplyingrelevantUTCoffset)tomake
analyzingsnoopandCDRfileseasier.IfthesnoopfileandCDRfilearegenerated
andviewedinthesametimezone,thenthereisnocorrectionrequired.

Table 55showsthetimestampformatsforsnoopfilesandCDRfiles.
Table 55FileFormatsandTimestamps
FileType Format Example

CDR <Day><Month><Date><HH:MM:SS><Year FriJun1709:59:512005
>
snoop <Year><Month><Day><HH:MM:SS.MS> 2005061621:59:17.400931
IntheUTCformat,Wiresharkdisplaysmicroseconds(0.123456)disregard
microsecondswhenusingsnoopandCDRfiles.
ToCorrectTimeDifference:
1 SettheWindowsclientmachinetimetothatofthetimezonethattheCDRfile
wascollectedin.
Inthisway,Wiresharkwillcorrectforthetimestampinthesnoopfile.theresult
isthatthetimestampsinboththeCDRfileandthesnoopfilearesynchronized.
OR
2 MentallyapplytheknownUTCoffsetwhenviewingsnoopandCDRfilesfrom
differenttimezones.
ApplyingFilters
Applyfilterstosnoopfilestoisolatemessagesbytypeandthensearchfor
messagesthathaveadesiredvalue(forexample,aspecificIMSI).
Filtersareappliedasaprotocolfield>valuepairing.See
http://www.wireshark.org/docs/dfref/foralistofallsupportedprotocols
andthemessagefields.
Seehttp://www.wireshark.org/docs/foradditionalinformationoncreating
andapplyingfilters.

LoggingIssues 231
LoggingIssues Thissectionincludes:
IntelligentMediaGatewayLogsnotCapturedbysyslog
IntelligentMediaGatewayLogsnotCapturedbysyslog
SolarismayneedtobeconfiguredtocapturesyslogfromtheIntelligent
MediaGateway.
SeeToVerifyIntelligentMediaGatewayINIFilesyslogConfiguration:.
TheINIfileontheIntelligentMediaGatewayspecifiesthecollectionoflog
files.
SeeToAddIMGLoggingwithsyslogonMSC:.
Anoptionalscriptcanbeaddedtocleanupsyslog.
SeeToManageMSCsyslogwithCleanupScript:
ToVerifyIntelligentMediaGatewayINIFilesyslogConfiguration:
1 ConnecttotheIntelligentMediaGateway.
2 OpentheWebmanagementinterfaceanddownloadtheINIfiletolocaldisk.
3 OpentheINIfileinatexteditorandconfirmthattheparametervaluesshownin
Example75exist.
Example75IntelligentMediaGatewaysyslogConfiguration
FORCEEXCEPTIONDUMP=1
ENABLESYSLOG=1
SYSLOGSERVERIP=<IPaddress>
ToAddIMGLoggingwithsyslogonMSC:
1 Verifythatthesystemhasaloghostdefinedin/etc/hosts.
127.0.0.1localhost
<IPaddress><hostname>loghost
2 Modify/etc/syslog.conffileandaddthefollowinglinestothetopofthefile.
#IntelligentMediaGatewaysyslogmessages
#
cron.debug;local0.debug/var/log/audiocodes.log
Usetabsbetweentherulesandthefilename.
3 Createthelogfile.
touch/var/log/audiocodes.log
4 Setpermissionsonthelogfile.
chmod666/var/log/audiocodes.log
5 Restartthesyslogdaemon.
killHUP`cat/etc/syslog.pid`
ToManageMSCsyslogwithCleanupScript:
1 Createthecleanupsyslog.cleanupscriptin/etc.
SeeExample76foranexampleofthecleanupscript.

Example76ScriptfosyslogCleanup
#!/bin/sh
filename="/var/log/audiocodes.log"
filesize="`dusk$filename|/usr/bin/awk'{print$1}'`"
maxsize="20000"
cd/var/log
if[$filesizegt$maxsize]
then
if[r${filename}.1.Z];then
mvf${filename}.1.Z${filename}.2.Z
fi
if[r${filename}.0.Z];then
mvf${filename}.0.Z${filename}.1.Z
fi
if[r${filename}];then
compress${filename}
cp${filename}.Z${filename}.0.Z
mvf${filename}.Z${filename}
fi
cp/dev/null${filename}
killHUP`cat/etc/syslog.pid`
fi
2 Changepermissionsonthecleanupscriptforexecution..
chmod555/etc/syslog.cleanup
3 Addthecleanupscripttosuperusercrontabfile.
suroot
crontabe
SeeExample77foranexampleofthesuperusercrontabfileshowing
syslog.cleanupentry.
Example77superuserCrontabFile
#ident"@(#)root1.1998/07/06SMI"/*SVr4.01.1.3.1*/
#
103**0/etc/syslog.cleanup
Example:superuserCrontabFile
#ident"@(#)root1.1998/07/06SMI"/*SVr4.01.1.3.1*/
#
103**0/etc/syslog.cleanup

RecoveryfromOutageorDisaster 233
RecoveryfromOutageor RecoveringfromServiceOutage
Disaster Thisincludesanoutageofservicestomultiplecustomers.Thismayincludethe
typeofservicesaffected,thenumberofcustomersaffected,andthestartand
endtimesoftheoutage.Reportstatisticsconcerningserviceoutagesby
designatedareasanddesignatedperiods.
RecoveringfromNetworkOutage
Thisincludesoutageinthenetwork,oramajorpartofanetwork.Reports
includestatisticsconcerningbothNetworkElementoutagesandnetwork
outagesbydesignatedareasanddesignatedperiods.
RecoveringfromDisaster
Thisincludescompletenetworkoutageduetonaturalormanmadedisaster.
Asidefromadvisingthedevelopmentofadisasterrecoveryplan(orintegrating
theMovingMedia2000systemintoanalreadyexistingdisasterrecoveryplan)
recoveryfromdisasterisbeyondthescopeofthisguide.
CreatingaDisasterRecovery CreateadisasterrecoveryplanforeachNEtoensurethatthesystemcanbe
Plan restoredtoservicequicklyafterithasbeentakenoutofservicebyanaturalor
manmadedisaster.
Akeyelementinthedisasterrecoveryplanshouldbeensuringthatregular
backupsofyoursystem'ssoftwareareperformed.Backupdataforthesystembe
storedinasecurelocation,inasiteseparatefromtheequipment,toensurethat
theyarenotaffectedbythesamedisaster.
Performregularlyscheduledsystemsoftwarebackupsonboththeactiveand
standbyNEtoprotectcriticalsystemdatasuchasconfigurationfiles,whichare
irreplaceableiflost.Ifacatastrophicfailureoccurs,itismucheasiertorestore
systeminformationfrombackupdatathantorecreateit.Furthermore,sucha
failurecouldcausecriticalconfigurationinformationtobelostifithasnotbeen
backedup.Createabackupschedule,ensuringthatsmallorincrementalbackups
areperformeddaily,andalargeorfullbackuponceaweek.
TerminalServerRecovery Theterminalservercanbeunreachableforseveralreasons.Torecoverthe
terminalserverinanemergency,useBOOTPifitisinstalledandconfigured,and
thepasswordisknownandoneofthefollowingconditionsapplies:
IPaddressismisconfigured,lost,orunknown
cannotobtainaloginonanyport(includingtheconsoleport)
ToRecovertheTerminalServerusingBOOTP:
1 SetupahostmachineonthenetworktorunBOOTP.
UsetheEthernetaddressoftheunit(printedonthebaseoftheproduct).BOOTP
willassigntheunitaknownIPaddress.
2 TelnettoterminalserverandchangeIPaddress.

TraceManager TheTraceManagerperformsmodule,process,andresourcetracing.Tracingisa
methodofmonitoringtheinternaloperationofNEsoftware,including:
programflow
programdata
finitestatemachinestates
errorconditions
othersystemelements
Warning:
OnlyusetheTraceManagerundertheguidanceofCustomerService.Trace
ManagershouldnotbeusedforroutinemonitoringoftheNE.
TraceManagerusesprocessingpowertogeneratethetracefilesandharddisk
spacetostorethetracefiles(whichcanbesometimesverylarge).
IftheTraceManagerisusedincorrectlyorforextendedperiodsoftime,trace
filesbecomesolargethatthesystemperformancebecomessignificantly
degraded,orthesystemstopscompletely.IftheTraceManagerisusedwithout
thedirectionandguidanceofCustomerServicethenCustomerServiceassumes
noresponsibilityforsystemimpact.
Warning:
UsingTraceManagercancausealivesystemtostopcompletely.
CustomerSupportmayruntracesontheNEremotely,oraskthattracesbe
executedlocallyandtracefilestransferredtothemforanalysis.IfCustomer
Servicedoesnothavemodemorinternetaccesstoyoursystem,youmaybe
askedtoturnontracingandthenprovidethetracefileforanalysis.
TraceTypes TherearethreetypesoftracingthatcanbeperformedontheNE:
ModuleTracing
Tracelevelsaresetfrom132toidentifydifferentpointsofinterestin
modulesintheNE.Forexample,settracelevel1toidentifyentrytoa
significantorhigherlevelfunction,andsettracelevel2toidentifyexpected
errors.
ProcessTracing
Processtracingissimilartomoduletracinginthattracelevelsaresetto
identifydifferentpointsofinterestinprocessesintheNE.
ResourceTracing
Resourcetracingallowsyoutomonitortheactivityofaspecificresource.
ModuleTracing ModulesaresoftwarecomponentsoftheNEapplication.TraceManagerrecords
messagespassedbetweenmodulesintheNE.Moduletracesprovideavery
lowlevelrecordoftheapplication.

TraceManager 235
ModuletracesareoflitllebenefittoOperators,buthavehighvalueforCustomer
Service.ForthemodulesthatexistontheNEthatyouaremanaging,refertothe
guideforthatNE.
ListingTraceableModules
Tracefilesaregeneratedonapplicationmodules.Listthetraceablemodulesto
confirmwhichmodulesareavailablefortracing.
ToListTraceableModules:
1 LogintotheNEasmsc,pro,fsv,orlisasappropriate.
2 Listthetraceablemodulesusingthefollowingcommand:
nam_modulesl
Usegrepandapipecommand(|)tosearchforpatternsincommandoutput.
Forexample,pro>nam_modulesl|grepp.

Figure 39liststhemodulesavailableonanSSVRasanexample.
Figure 39ListingTraceableModulesonanSSVR
pro>nam_modulesl
nam dna msg xcp rtd tpm
evl utl sys ipc cpu stk
unx req typ trc fsm orq
tlv asn alm cdr oam sta
cli dbl nem ntk adl conx
hab vsp tss mvu zmv tsm
ProcessTracing ProcessesareatahigherlevelintheNEthanmodules.Processesmapmore
directlytoprocessingfunctionswithintheNE,likeMobilityManagementandCall
Processing.Tracelevelsareusedtoprovidereportingdepthtoprocesstraces.
ProcesstracesareoflittlebenefittoOperators,buthavehighvalueforCustomer
Support.
Table 56listssomecommonprocesses.FortheprocessesthatexistontheNE
thatyouaremanaging,refertotheguideforthatNE,orlisttheprocesses
availableontheNE(refertoListingTraceableProcesses).
Table 56TraceableProcesses
ProcessName Description
CPU_CM Nodescriptionavailable.
CPU_PM Nodescriptionavailable.
RTDBAK Sharedmemorymanagement.
IPCD Interprocesscommunicationdistantcommunicationbetween
platformsordifferentsites.
TPMD Transactionprocessmanagementandtimersforprocess
management.
EVL_B Eventlogging.
EVL_S Eventlogging.
ALMD Nodescriptionavailable.
CDR CallDetailRecordmanagement.
CDR_TRANS Nodescriptionavailable.
STA Statisticsmanagement.
ALM Alarmmanagement.
ASP AdvancedSwitchProcessorhighlevelprocesssupport.
MMI Manmachineinterfacemanagement.
MIPD(IS41only) ServicesforusersoftheMobilityInterface.
LFM Logicalfacilitymanagersetsuplogicalconnectionsbetween
parties.
COMD Nodescriptionavailable.
CSI(IS41only) IS634andIOScompliantAinterfaceBSSmanager.

TraceManager 237
Table 56TraceableProcesses
ProcessName Description
CSG(GSMonly) GSMcompliantAinterfaceBSSmanager.
VSM VirtualSwitchMatrixmanagementfortonesandannouncements.
VLR(IS41only) CDMAVisitorLocationRegisterprocess.
GVR(GSMonly) GSMVisitorLocationRegisterprocess.
IMP(IS41only) IS41complianthandofffunctionsandproceduresmanagement.
MCG(GSMonly) GSMcompliantMSCfunctionsandprocedures.
LIA Nodescriptionavailable.
ADL Nodescriptionavailable.
CONX_DELAY Nodescriptionavailable.
ListingTraceableProcesses
Tracefilesaregeneratedonapplicationprocesses.Thelistofapplication
processesisgiveninTable 56,butsoftwarepatchesandupgradesmayresultin
additionalprocessesbeingavailablefortracing.Listthetraceableprocessesto
determinewhichprocessesareavailablefortracing.
ToListTraceableProcesses:
2 Listthetraceableprocesses.
ipc_table
Forexample,pro>ipc_table|grepCDR.
Figure 40showsapartiallistingofprocessesonanSSVR.
Notallprocessesarelistedintheexample.
Figure 40ListingTraceableProcesses
pro>ipc_table
LOCALIDQSIZEQIDPIDTRCQNUMQBYTESTYPE
===================================================
CPU_CM16553212807797400INIT
CPU_PM26553212806797900INIT
RTDBAK46553212811798300INIT
IPCD552428012812798200INIT
TPMD86553212813797800INIT
EVL_B96553212804797600INIT
EVL_S106553212814797700INIT
ALMD116553212822798400INIT
STA146553212810798800INIT
ADL886553227400799800INIT
...

ResourceTracing Addresourcestomoduleorprocesstracesforspecificdetail.Resourcesare
generallydescribedasthedatapassedinthemessages.Themostcommonand
usefulresourcestotraceareMIN,DN,andPORT.
ListingTraceableResources
Tracefilesaregeneratedondefinedresources.Listthetraceableresourcesto
confirmwhichresourcesareavailablefortracing.
ToListTraceableResources:
2 Listthetraceableresourcesbytypingtmrattheprompt.
Forexample:tmr|grepIMSI
Figure 41ListingTraceableResources
pro>tmr
ResourceTypeValue
==========================
CELLInteger
SMSInteger
USAPInteger
LINKInteger
DNString
IMSIString
MSIDString
ESNInteger
DIDInteger
MINString
BSCInteger
CODEInteger
MCBInteger
ALGString
NODEInteger
CREFInteger
SAPInteger
DGTInteger
LIBString
PORTInteger
XNIDInteger
MIDString
TAGInteger
FWIInteger
TraceFiles TraceManagerwritesthetraceoutputtoafilebuffer(asopposedtoamemory
buffer)ortoanamedfile.TraceManagerwritestothebufferuntilthefileis
saved,oruntilthenamedfileisexplicitlyclosed.Bufferscanbewrtittento
namedfilesafterinitiatingthetrace.Thefilebufferanddefaultsavedfilename
havethesyntaxXXX.trace,whereXXXisthemoduleorprocessname.For
example,thefilebufferfortheASPprocessisasp.trace.
Closeallopentracefilesaftersufficientinformationisobtained.Donotallow
tracefilestorunforextendedperiods.
Tracefilesarewrittentothe~/RUNdirectory.Deletetracefilesafteranalysis,or
archivethemofftheNEdiskiftheyneedtobekept.

TraceManager 239
CheckingActiveTraceStatus Checkthestatusofactivetraces.Thecommandoutputindicatesthetracelevel
foreachprocessormodule,andwhetheritiswrittentothedefaultbufferortoa
namedfile.
ToCheckActiveTraceStatus:
2 Checkthestatusofallactivetraces.
tmQ
Figure 42showsanoutputfromtheTraceManagerstatuscheck.
Figure 42CheckingActiveTraceStatus
pro>tmQ
Process ModuleTraceLevel/Resource(tracelevel32)File
======= =====================================================
CPU_CM MSG1/dev/null
CPU_PM MSG1/dev/null
TM_0X5933MSG1/dev/null
RTDBAK MSG1/dev/null
...
TraceLevels
Tracelevelscanbesetfortracesonmodulesorprocesses.Tracelevel32mustbe
usedifresourcesarebeingaddedtothetrace.
Thereare17tracelevelsavailable.Tracelevels14and15arereservedforRCP
tracing,16isreservedforfunctionentryandexit,and32forresourcetracing.
Thehigherthenumber,themoreverbose.Thetracelevelselectedwilltherefore
impactthesizeanddepthofthetracefile.Themostusefultracelevelsareshown
inTable 57.
Table 57UsefulTraceLevels
TraceLevel Description
1 Functiontracingwithkeyinputparameters
2 Unexpectederrortracing
3 Detailedfunctiontracing
4 Functionresulttracing
5 FSMstate/eventtracing
7 Messageflowtracing
16 Functionentryandexit
32 Resourcetracing
Arangeoftracelevelscanbespecifiedforanyprocessormoduletrace.For
example,tospecifytracelevels14,and7foragiventrace,useT14,7in
thetracecommand.
Agoodtraceleveltouseis14.Include13,14forRCPand16forfunctionentry
andexit.Notethough,thattracelevel16isveryverbose.

StartingandStoppingTraces Runatracebyusingthetmcommandandspecifyingthetracefile,theprocess,
module,orresource,andtracelevel(formodulesandprocesses).
Flushmoduleandprocesstracebuffersbeforerunninganewtrace.SeeFlushing
ModuleandProcessTraceBuffersandFlushingResourceTraceBuffers.
StartingandStoppingModuleTraces
ToStartaModuleTrace:
Themodulenamemustbeinallcapitals.
2 Writethemoduletracetobuffer:
tmM<modulename>T<tracelevel>
OR
3 Writethemoduletracetoanamedfile:
tmo<filename>M<modulename>T<tracelevel>
Thesystemmustberunninginmastermodetoopenandclosetracefiles.
4 Closethetracefilewhenfinished:
tms<filename>
SeeListingTraceableModulesonpage 235fortraceablemodules.
ToStopaModuleTrace:
SeeCheckingActiveTraceStatusonpage 239.
2 Closethetracefileifnecessary:
tms<filename>
3 Stopalltracesforaspecificmoduleusing:
tmM<modulename>T0
Forexample:
tmM"VSM,MEG"T116,32
4 Checktracestatusagaintoconfirmtracehasstopped.
tmQM<modulename>

TraceManager 241
StartingandStoppingProcessTraces
ToStartaProcessTrace:
2 Writetheprocesstracetobuffer:
tmP<processname>T<tracelevel>
Forexample:
tmPCSIT14
OR
3 Writetheprocesstracetoanamedfile.
tmo<filename>P<processname>T<tracelevel>
Thesystemmustberunninginmastermodetoopenandclosetracefiles.
4 Closethetracefilewhenfinished:
tms<filename>
SeeListingTraceableProcessesonpage 237fortraceableprocesses.
ToStopaProcessTrace:
2 Closethetracefileifnecessary:
tms<filename>
3 Stopalltracesforaspecificprocessusing:
tmP<processname>T0
4 Checktracestatusagaintoconfirmtracehasstopped.
tmQP<processname>
AddingaResourcetoaTrace
ToAddaResource:
1 Whenstartingamoduleand/orprocesstrace,makesuretoincludetracelevel
32,forexample:
tmPASPM"ASP,SSF,NAU,GIN"T116,32
2 Addaresourcetothetrace:
tmA<resourcename><value>
Forexample:
tmAMIN6191234506
tmADN4508
tmAPORT9023
SeeListingTraceableResourcesonpage 238fortraceableresources.

ToStopaResourceTrace:
Resourcetracesarestoppedbyflushingthebufferforthenamedresource.See
FlushingResourceTraceBuffersonpage 244.
TracingTipsfor Table 58liststracingtipsforspecifcscenarios.Example78showsacomplete

Troubleshooting exampleforsettingupamodule,process,andresourcetrace.
Table 58TroubleshootingScenarios
Scenario TracingTip
Cannotmakecallsfromamobile. Useresourcetracingwiththemobile's
MIN.
Cannotterminatecallstoamobile. UseresourcetracingwiththemobileDN.
Callscannotbemadeviaatrunk. UseresourcetracingwiththetrunkPORT.
Resourceleakageproblem. UsebasictracingtoidentifyMIN,DN,or
PORT.
Useresourrcetracingwiththeidentified
MIN,DN,orPORT.
Example78Module,Process,andResourceTrace
Initialization:
>tm -T 0
>tm -C
CPE Trace Setting:
>tm -P ASP -M "ASP,SSF,NAU,GIN" -T 1-16,32

>tm -P LFM -M "LFM,NSO,NSC,NAU,NAI" -T 1-16,32
>tm -P VSM -M "VSM,MEG" -T 1-16,32
MOB Trace Setting:
>tm -P CSI -M "CSI,CAP" -T 1-16,32

>tm -P IMP -M "RCI,MCI,WIN,SMI,NAU,ISI,IMP,MIP,IMU,IAU" -T 1-16,32
>tm -P VLR -M "VLR,MIP,IMU,IAU" -T 1-16,32
>tm -P MIPD -M "MIP,IAU,IRI,IAR" -T 1-16,32
Add Appropriate Resources:
>tm -A MIN 6191234506

>tm -A DN 4508
>tm -A PORT 9023
Display All Trace Setting:
>tm -Q

TraceManager 243
WritingTraceFilesFrom Thetracemanagerwritestraceoutputtoabufferbydefault.Ifthetraceisnot
Buffer redirectedtoafilewhenthetracewasinitiated,thecontentsofthetracebuffer
canbewrittentoanamedtracefile.Anamedtracefileisatracefilethatdoes
notusethedefaultbufferfilenamesyntaxofxxx.trace.
Bydefault,tracefilesaregeneratedinthe~/RUNdirectoryonthetargetNE(see
TraceFilesonpage 238).
Atracefileremainsopenuntilitisexplicitlyclosed.Tracefilescanbecomequite
largequitequickly,andconsumerequiredprocessingcyclesanddiskspace.Do
notleaveatracefileopenforevenshortperiods.Tracefilescanbearchivedon
anotherstoragemedia,butalwaysremovetracefilesaftertheyaregenerated,
analyzed,andinterpreted.
Warning:
Closeallopentracefilesassoonaspossibletopreventthefilesfrombecoming
verylarge.
Thesavecommandwillrenamealldefaultbufferfilesfromxxx.traceto
whateverisspecifiedasanargument.
Therewillbeonlyonetracewritingtothetracebufferatthesametime,andthe
bufferisclosedwhenthefileissaved.Anynewtracesinitiatedonthesame
processormodulewilloverwritethedefaultbufferfile.Tracefilesshouldbe
archivediftheyneedtobepreserved(seeArchivingTraceFilesonpage 245).
ToWriteaTraceFileFromBuffer:
1 Checkthestatusofactivetraces.
2 Writethetracetoanewnamedfile.
save<filename>
FlushingModuleandProcessTraceBuffers
Flushmoduleandprocesstracebuffersbeforerunninganewtrace.
ToFlushModuleandProcessTraceBuffers:
2 Flushtracebuffers.
zero
Thebufferstakesometimetoflush.

FlushingResourceTraceBuffers
Flushresourcetracebuffersafterthetraceiscomplete,andbeforerunningany
newresourcetraces.
ToFlushResourceTraceBuffers:
2 Flushallbuffersforaspecificresource.
tmC<resourcename>
OR
3 Flushspecificnamedresourcetracebufferbyvalue.
tmC<resourcename><value>
ViewingTraceFiles Tracefilesaregeneratedinbinaryformat.TheEventViewerisusedtoviewtrace
filesinbinaryformat.ConvertbinarytracefilestoASCIIformattoviewthemina
texteditor(seeConvertingTraceFilesonpage 245fordetailsonhowtoconverta
tracefile).
Stoptracefilesbeforeviewing.
ToViewBinaryTraceFiles:
2 Changetothedefaulttracefiledirectory.
pro>cd~/RUN
3 Ifnecessary,listexistingtracefiles,andsearchfordesiredfile.
pro>ls|grep<searchpattern>
4 Stopthetracefile.
SeeStartingandStoppingTracesonpage 240.
5 Displaythetracefile.
pro>ev<filename>|v
ToViewASCIITraceFiles:
cd~/RUN
ls|grep<searchpattern>
4 Stopthetrace.
5 Displaythetracefile.
more<filename>
SeeConvertingTraceFilesonpage 245fordetailsonhowtoconvertabinary
tracefiletoASCIIformat.

TraceManager 245
ConvertingTraceFiles
Tracefilesaregeneratedinbinaryformattheycanonlybeviewedusingthe
EventViewer.ConvertatracefilefrombinarytoASCIIformattoviewitwithan
editor.
Usingsavemeansthatacopyofthefileismade.Thecopyisasnapshotofthe
tracefileatthetimethecommandisissued.Nomoretraceinformationis
writtentothesavedfile,buttheinitialtracefileisstillactive.
Alltracefilesin~/RUNarerenamedandconverted.Theoriginalfilesremain.
Theconversionisperformedintwosteps:
Thedefaultfilenamesareappendedwiththestringspecifiedint
<filename>,andcopiedtothe~/RUNdirectory.Forexample,usingtout,
xxx.tracebecomesxxx.trace.out.
Thenewlyrenamedfileiscopiedagainandappendedwitha.txtextension.
Forexample,vsm.trace.outiscopiedasvsm.trace.out.txt.
ToConvertTraceFiles:
2 Changetodefaulttracefiledirectory.
cd~/RUN
4 Convertthedesiredfile.
savet<filename>
Alltracefilesin~/RUNarerenamedandconverted.Theoriginalfilesremain.
Thenewfilesaregeneratedin~/RUN.Viewthefileswiththe.txtextensionwith
atexteditor.
ArchivingTraceFiles ArchivetracefilesinbinaryorASCIIformat.Tracefilescanbequitelarge,and
shouldbearchivedonstoragemediaseparatefromtheNE.Stoptracefiles
beforearchiving.Removetracefilesfromthe~/RUNdirectoryafterarchiving.
ToArchiveTraceFiles:
3 Stopthetrace.
4 Archivethedesiredtracefile.
5 Removethetracefile.
SeeRemovingTraceFilesonpage 246.

RemovingTraceFiles Tracefilescanbequitelarge,andshouldberemovedfromtheNE.Tracefilescan
bearchivedbeforeremoval(seeArchivingTraceFilesonpage 245).
Bydefault,tracefilesaregeneratedinthe~/RUNdirectoryonthetargetNE(see
TraceFilesonpage 238).Removethebinarytracefile,andtheASCIIversionas
well,ifthetracefilewasconverted.
ToRemoveTraceFiles:
cd~/RUN
4 Removethedesiredbinarytracefile.
rm<filename>
IfthetracefilewasconvertedtoASCIIformat,thenremoveitaswell.
CONXTracingfromtheOMC Performcommonservices(CONX)tracingforSIP,MG,ISUP,Q931,IUA,IS41MAP,
TCAP,SCCP,MTP3,M3UA,M2UA,SCTP,andHIusingthedebugtoolfromthe
OMCConsole.TracesfromCONXaresentto~/RUN/ipc_remoted.trace
ToenabletracingforaCONXapplication:
1 StarttheOMCconsoleandexpandtheelementtreeforthenodewherethe
CONXapplicationresides(SSVR,MSC,orHLR).(RefertotheAccessingtheOMC
ConsoleinChapter 2,ProvisioningtheCoreVoiceNetworkElements)forhowto
starttheOMCconsole.)
2 RightclicktheCONXapplication(eg.ISUP,SCCP,HI).
3 HighlightControls.
4 Highlightdebug.
5 Highlightandclickenable.
TodisabletracingforaCONXapplication:
1 StarttheOMCbrowserandexpandtheelementtreeforthenodewherethe
CONXapplicationresides(SSVR,MSC,orHLR).(RefertotheOMCUserInterface
GuideforhowtostarttheOMCbrowser.)
2 RightclicktheCONXapplication.
3 HighlightControls.
4 Highlightdebug.
5 Highlightandclickdisable.
Warning:
Remembertodisabletracingwhenfinished.

TroubleshootingHWD 247
TroubleshootingHWD AlltracefilesgeneratedbyHWDprocessesarewrittento/tmpintheform:
/tmp/<processname>.log
Themostusefullogfileis:
/tmp/hwd.log
Theabovelogfilegetsrolledoverto/tmp/<processname>.log.oldevery10,000
lines(seehwd.conf).Also,becauseitisstoredin/tmpitwillberemovedwhen
thesystemisbooted.
RAFTalsomakesuseofsyslogforreportingerrorsandothersignificantevents
whichneedtobelogged.
Thedefaultsyslogfileis:
/var/adm/messages
Themostcommonproblemsareatinitialconfigurationandstartup.Herearea
fewthingstocheck:
Ifprocessesorthesystemdonotseemtobestarting,lookinthesyslog
/var/adm/messages.
Ifthereisaconfigurationproblemitwillbenotedthere.
ToenabletracingforHWDoranyRAFTprocesswhileitisrunningyoucanrun:
hwd_clia<processname>t<tracelevel>
IfyouneedtostartuptracingforHWDyouwillneedtomodifythecommandline
entryforHWDin/etc/inittab.Onceyoumakeachangetoinittabyouneed
torun:
initq
toreloadthetableandthenkillHWD:
hwd_clik
soitwillberestartedwiththenewtracinglevels.
Tostartuptracingfortheagents(NET,DMY,etc),youcanmodifythecommand
lineoptionsfortheagentinthe/etc/hwd.conffileundertheseethe
agent_<n>entries.
Also,alwaysmakesureyouhavethesameconfigurationonbothhosts.

SOFTWAREMAINTENANCE
15
ThischapterprovidesanoverviewofsoftwaremaintenancefortheCoreVoice
NetworkElements(NE),describesthetypesofsoftwarereleases,howtoinstall
emergencypatch(EP)releasesandpatchlevel(PL)releases,andhowtoperform
softwareversionupgrades.
UNIXlocalemergencypatchesandemergencypatchesthataffecthardware
daemonsontheNEsarehandleddifferentlyfromapplicationemergency
patches.Thereleasenotesthataccompanyanemergencypatchreleasethat
involveshardwaredaemonorUNIXlocalupgradesdescribeshowtoinstallsucha
patch.
Theproceduresinthissectionapplytoapplicationemergencypatchreleasesand
patchlevelreleases.
OverviewofSoftwareMaintenance
InstallingEmergencyPatches
InstallingPatchLevelReleases
PointLevelVersionUpgrades
OverviewofSoftware Thissectionincludes:
Maintenance SystemandComponentSoftware
SoftwareReleaseTypes
SoftwareReleasesandVersioning
SystemandComponent Thissectionincludes:
Software
ComponentSoftware
SystemSoftware
ComponentSoftware
Componentsoftwareisthelowestlevelunitofsoftwarethatcanbespecified.
Componentsoftwareisnotusedstandalonebutispartofasystemsoftware
release.Componentsoftwarehasitsownproductnameandversioninglifecycle,
andcanbepatchedorupgradedindependentlyofothercomponentsinthe
system.

250 Chapter15:SoftwareMaintenance
UNIXlocalemergencypatchesandemergencypatchesthataffecthardware
daemonsontheNEsarehandleddifferentlyfromapplicationemergency
patches.Theproceduresintheremainderofthissectionapplytoapplication
emergencypatchreleasesandpatchlevelreleases.
ComponentsarebundledtogethertocreateproductfunctionalitysuchasMSCor
HLRfunctionality.Forexample,thesoftwarecomponentsthatimplementHLR
functionalityarelistedinTable 59.TheotherCoreVoiceNEshavesimilar
components.
Table 59Example:HLRSoftwareComponents
Component Example
UNIXSolaris UNIXSolaris5.8(sun4u)kernelid:Generic_11735045
Conx ConxR5.1.0Build6onFriJan417:34:02PST2008
SONATASEIS41 SONATASEIS41R4.5.3PL1
MAISON MAISONV4_5_3PL1onFriJan418:38:22PST2008
PROTOCOLS PROTOCOLSV4_5_3PL1onTueJan814:42:58PST2008
SONATAIS41FSV SONATAIS41FSVR453PatchLevel1onThuOct2322:45:40PDT
2008
UNIXLOCAL UNIXLOCALIS41ULV4_2_6PL1PatchLevel1onTueDec415:34:59
PST2007
SystemSoftware
Systemsoftwareismadeupofacombinationoftwoormoresoftware
components.Thecomponentsaredesignedandtestedasacompletesystem.
Systemsoftwarehasaversioninglifecyclethatisdistinctfromitsunderlying
components.Forexample,a4.5.3releaseisaSystemrelease,consistingof
multiplecomponentsoftware,suchasthoselistedinTable 59.
SoftwareReleaseTypes Thissectionincludes:
EmergencyPatchRelease
PatchLevelRelease
PointLevelRelease
EmergencyPatchRelease
Emergencypatchreleasesareusedtoaddressurgentsoftwareissues.Emergency
patchreleasesarebasedonthereleasedcodeandaddfixesthathavebeen
testedinthelabandincustomerlocations.

Emergencypatchesmayaffectonlyoneofthesoftwarecomponents.The
cpu_versioncommandoutputsthecomponentsandpatchlevelsasinTable 59
andthenthecomponentemergencypatchlevels.Forexample,thefollowing
mightappearattheendofthecpu_versionoutput:
IS41FSVR4.5.3PL1FSVEP2.2(UTIFSV002)createdonMonDec815:54:49
2008
IS41SSVRR4.5.3PL1SSVREP4.4(UTISSV004)createdonMonJul2810:39:37
2008
IS41UXLR4.5.3PL1UXLEP2.2(UTIUXL002)createdonWedJun415:59:50
2008
Thenumberingforemergencypatchesrestartsat1foreachnewpatchlevel(PL)
release.Emergencypatchreleasesaregenerallysmallinsizeandmostcanbe
installedbytheoperator,dependingonthecomplexity.
PatchLevelRelease
Apatchlevel(PL)releasecontainsallfixesandfeaturesthathavebeen
implementedinemergencypatchreleasessincetheprevioussoftwarebaseline
releaseversionaswellasanynewfeatures.Apatchlevelreleaseisafull
recompileofthesystemsoftware.Apatchlevelreleaseisatleastasbigasthe
systemsoftwareloaditreplaces.
Patchlevelreleasesaremorecomplexthanemergencypatchreleases,andare
installedbycustomersupportonly.
PointLevelRelease
Apointlevelreleasedeliversnewplatformandfeatureadditionsandchanges
andenhancementstoexistingplatformfunctionalityandfeatures.
Pointlevelreleasesarethemostcomplex,andinstallationisperformedby
customersupportonly.

Figure 43showsthesoftwarereleasetypesandorderofapplication.
Figure 43SoftwareMaintenanceReleaseTypesandFlow
Initial installation
Existing software baseline 4.5.3
Emergency patch 4.5.3EP1
Patch level 1 (consolidates EPs) 4.5.3PL1
Emergency patch 4.5.3PL1EP1
Patch Level 2 (consolidates EPs&PLs) 4.5.3PL2
New software baseline 4.6.1
SoftwareReleasesand Thissectionincludes:
Versioning
SoftwareVersioning
EmergencyPatchReleaseVersioning
PatchLevelReleaseVersioning
PointLevelReleaseVersioning
SoftwareVersioning
Softwareversionsarelabeledwithauniquenumericalidentifier.Thisistypically
expressedasthreenumbers,separatedbyperiods,suchasversion4.5.3.Version
numbersallowthesoftwaredevelopmentorganizationandcustomersupportto
trackissuesandfixesraisedinthefield(customernetwork).
Inprinciple,themajornumberisincreasedwhentherearemajoradditionsor
improvementstoexistingfunctionality,andtheminornumberisincremented
whenonlyminorfeaturesorsignificantfixeshavebeenadded.Theadditionof

emergencypatchreleasesandpatchlevelreleasesdoesnotchangethemajorof
minorversionnumber.
Itispossibletoleapfrogminorversions(sayfrom4.0to4.5)toindicatethat
significantfeatureshavebeenadded,butnotenoughtowarrantincrementing
themajorversionnumber.
EmergencyPatchReleaseVersioning
EPsareSUNUNIXpackagesandassuchtheEPreleaseversioningissimilartothe
SUNpackagefilenamescheme.TheEPversioningusesthefollowingformat:
<Family><Inventory><Release>EP<Component><ComponentNumber>.
<SequenceNumber>(<Instance>)
Forexample:
IS41MSCR4.5.3HABEP2.5(UTIMSC005)
TheemergencypatchlevelreleasesyntaxisdefinedinTable 60.
Table 60EPFilenameSyntax
FilenamePart MeaningandValues
Family IS41/GSM/SRAN
Inventory Softwarecomponent:MSC/FSV/SSVR/MRF/OMC/DRUM/LIS/
UXL/PRO/MSV
Release Softwarerelease,forexample:R4.5.2PL4/R4.5.3/R4.5.3PL1
Component Softwarecomponent(subsystem)ontheNE.
Component EP<number>Indicatesthenumberofemergencypatches
Number deliveredforthissubsytem.
Sequence SequencenumberindicatestheorderthattheEPshouldbe
Number applied.TheEPexampleshowsthatthisEPmustbeappliedafterEP
4isinstalledandbeforeEP6.
Instance SUNpackagename,eg.UTIMSC023
AsdescribedinTable 59,theNEsoftwareismadeupofunderlyingsoftware
componentsubsytems.Emergencypatchesareappliedtoindividualsubsystems
butbecauseofdependenciesbetweensubsytems,emergencypatchesmustbe
appliedinthecorrectorder.Theorderofapplyingemergencypatchesis
indicatedbytheversioningsyntax.
SeeCheckingEmergencyPatchUpgradePathonpage 259foradditional
informationontheinstallationorderforemergencypatches.
PatchLevelReleaseVersioning
Patchlevel(PL)releaseversioningusesthefollowingformat:
<Family><Inventory><Base_Release><PL_Release>

Forexample:
IS41MSCR4.5.3PL1
ThepatchlevelreleaseversionsyntaxisdefinedinTable 61.
Table 61PatchLevelFilenameSyntax
UXL/PRO/MSV
Base_Release Softwarerelease,forexample:R4.5.2PL4/R4.5.3/R4.5.3PL1
PL_Release Patchlevelrelease,forexamplePL1/PL2
PointLevelReleaseVersioning
Pointlevelreleaseversioningusesthefollowingformat:
<Family><Inventory><Base_Release>
Forexample:
IS41MSCR4.5.3
ThepointlevelreleasesyntaxisdefinedinTable 62.
Table 62PointLevelFilenameSyntax
UXL/PRO/MSV
Base_Release Softwarerelease,forexample:R4.5.2PL4/R4.5.3/R4.5.3PL1
InstallingEmergency Thissectionincludes:
Patches OverviewofEmergencyPatchInstallation
BeforeInstallingEmergencyPatches
InstallingEmergencyPatches
RemovingEmergencyPatches
OverviewofEmergencyPatch TheSonataNEsuseaSunpackagebasedmethodofinstallingEPupgrades.
Installation
EmergencyPatchInstallationOverview
SystemImpact
EmergencyPatchPackage
UsingEmergencyPatchReleaseNotes

EmergencyPatchInstallationOverview
TheEPpackagesareinstalledusingtheep_installtool.
Thetoolmovesthefilestothecorrectlocations,givesthemthecorrect
ownershipandpermissions,backsupfilestobeoverwritten,launchesany
requiredsupportingscripts,andlogsallactionstaken.Thetoolcanalsobeused
touninstallpackages,inwhichcaseitrestorestheoriginalversionsofthefiles
andtheirownershipandpermissions,runsanyrequiredsupportingscripts,and
logsallactionstaken.
TheenduserinstallstheEPandmakessureitgetsinstalledproperly.IftheEP
doesnotinstallproperlyalogfileoftheinstallationprocessisautomatically
generatedwhichwillberequiredbytheStarSolutionssupportteamto
troubleshoottheinstallation.
Emergencypatches(EP)areinstalledoneatatime,andmustbeinstalled
accordingtotheemergencypatchversionnumberingscheme.
Moreimportantly,whilesomeemergencypatchreleasesupgradetheNE
application,someemergencypatchreleasesupgradeUNIXlocalandsystem
daemons.Theseemergencypatchreleasesshouldbeinstalledbycustomer
supportonly.Thereleasenotesfortheemergencypatchreleasestipulate
whetherthereleaseupgradesUNIXlocalorsystemdaemons.
SystemImpact
Thecontentofeachemergencypatchisdifferent.Someemergencypatches
affectliveprocessesandcannotbeinstalledwhiletheNEisrunning.Other
emergencypatchescanbeinstalledwhiletheNEisrunning.Inmostcases
however,theemergencypatchdoesnottakeeffectuntiltheNEisrestarted.
Alwaysinstallanemergencypatchduringamaintenancewindow.
Warning:
TheNEmustusuallybeshutdownandthenrestartedtoimplementan
EmergencyPatch.
NotifyallAllNOCandcustomersupportstaffthatanupgradewillbeperformed.
Noadditionalconfigurationormodificationshouldbeattemptedduringan
upgrade.
EmergencyPatchPackage
EPsareSUNUNIXpackages.Anemergencypatchisdeliveredforeachsubsystem
thatneedscorrection.EachEPaddressesoneormoresoftwareissues.Theissues
correctedbytheEParelistedintheREADMEtextfilethataccompanieseach
emergencypatchrelease,andintheEmergencyPatchReleaseNotes.
Theshortformname,orpackageinstanceusesthefollowingnamingscheme:
<Company><Family><Inventory><SequenceNumber>

Forexample:UTIMSC005
TheEPpackageusesthefollowingnamingscheme.
<Family><Inventory><Release><Component><ComponentNumber>.
<SequenceNumber><Instance>
Forexample:
IS41MSCR4.5.3HABEP2.5UTIMSC005
RefertoTable 60foradescriptionofthepartsofthepackageinstanceandfull
packagenames.
UsingEmergencyPatchReleaseNotes
Eachemergencypatchissupportedbyreleasenotes.Emergencypatchinstallers
shouldrefertothereleasenotesfortheemergencypatchforallinstallation
instructions.
Table 63describestheheadingsandinformationintheemergencypatchrelease
notes.
Table 63EmergencyPatchReleaseNotesHeadings
Heading SubHeading Description

AssociatedEP Listsanydependencieswithother
emergencypatches.Thisincludespatches
thatmustbeinstalledbeforethecurrent
patch,andanypatchesthatshouldbe
installedimmediatelyafterthecurrent
patch.
CorrectedIssues ProblemDescription Descriptionoftheproblem.
RootCauseAnalysis Analysisoftherootcauseoftheproblem.
CorrectiveAction CorrectiveactiontakenbyStarSolutions
R&D.
Verification Proceduresforverifyingthatthecorrective
Procedures actionaddressedtheproblem.
PatchContents Listsallofthefilesthatareincludedinthe
patch.
PatchInstallation Describeshowtoinstallthepatch.
Instructions
PatchRemoval Describeshowtoremovethecurrent
Instructions patch,ifrequired.

BeforeInstallingEmergency Thissectionincludes:
Patches
GettingEmergencyPatchSoftware
GettingRequiredDocumentation
SettingUpInstallerClient
ConnectingtoTargetNE
CheckingEmergencyPatchUpgradePath
BackingUptheDatabase
GettingEmergencyPatchSoftware
DownloademergencypatchesfromthecustomersupportFTPsite.Ausername
andpasswordisrequiredforaccesstotheFTPsite.
Installemergencypatchesontheinstallerclientwellinadvanceofattempting
theemergencypatchupgrade.Someemergencypatcheshavesoftware
dependencieswithotheremergencypatches.
ToGetEmergencyPatchSoftware
1 FTPtothecustomersupportemergencypatchFTPsite.
2 Transfertheemergencypatchtoatemporarydirectoryontheinstallerclient.
3 ExittheFTPsite.
GettingRequiredDocumentation
Supportinginformationfromvariousdocumentsisoftenrequiredtoinstallan
emergencypatch.Thesedocumentsinclude:
READMEfilefromemergencypatchpackage
Emergencypatchreleasenotesfromcustomersupport
Verificationtestplanfromcustomersupport(ifapplicable)
Siteengineeringdocumentfromoperatororcustomersupport
Obtainrequireddocumentationwellinadvanceofattemptingtheemergency
patchupgradeandfamiliarizeyourselfwiththecontents.
ToGetRequiredDocumentation
1 Contactcustomersupportforalldocumentsrequiredtoinstallanemergency
patch.
2 Installthedocumentationontheinstallerclient.
SeeSettingUpInstallerClient).
3 Usethedocumentationtodetermineanysoftwaredependencies.
4 DownloadanyadditionalemergencypatchesfromtheFTPsite.
SeeGettingEmergencyPatchSoftware.
5 Reviewthegeneralcontentsofthedocumentationbeforestartingthe
installation.

6 Havebothasoftandhardcopyofallrequireddocumentation(ifpossible)
availableduringtheinstallationprocess.
SettingUpInstallerClient
TheinstallerclientisalaptopPC.Theemergencypatchsoftwareandrequired
documentationneedtobesetuponthelaptop.Additionally,IPaddressingon
theinstallerclientneedstomatchthesubnetoftheNE,butmustbeuniqueon
thenetwork.UsetheSiteEngineeringdocumenttodeterminethesubnetofthe
targetNE.
Warning:
DonotuseanexistingIPfromthenetwork.AssignauniqueIPaddresstothe
installerclient.
ToSetUpInstallerClient
1 Createadirectoryhierarchyonthelaptopforthecurrentandanyfuture
emergencypatchinstallations.
C:.
+install
|\msc
|\ep
2 Createanewdirectoryforeachemergencypatch.
C:\install\msc\ep>mkdir<EPName>
3 MoveorcopyEPsoftwarefromthetemporarydirectorytothenewEPinstall
directory.
4 CopyanyEPdocumentationtothenewEPinstalldirectory.
Thefollowingstepsrequirephysicalaccesstothetargetmachine.
ConnectingtoTargetNE
ThetargetNEistheNEthatistobeupgraded.
ToConnecttoTargetNE
1 ConnectoneRJ45connectorendoftheEthernetcabletotheinstallerclientand
theotherendtotheEthernetswitchportthatcorrespondstotheNE.
ConfirmtheconnectionwithNOCpersonnel.
2 Boottheinstallerclient.
3 PingthetargetNE.
TroubleshootanyIPconnectivityissuesbeforeproceedingfurther.

CheckingEmergencyPatchUpgradePath
Emergencypatchesshouldbeappliedassoonaspossibleaftertheyarereceived.
Installingemergencypatchespromptlywillhelpensurethatthesoftwareisupto
dateandthatanysubsequentpatchescanbeappliedimmediately.
Thereleasenotesfortheemergencypatchwilllistanyemergencypatchesthat
mustbeinstalledimmediatelybeforeorafterthecurrentpatch.
Havealldependentemergencypatchesandsupportingdocumentationonthe
installerclientbeforestartingtheinstallation.
Thecpu_versioncommandlistsonlythelastinstalledemergencypatch,patch
level,andpointlevelrelease.Forexample:
UNIXSolaris5.8(sun4u)kernelid:Generic_11735045
ConxR5.1.0Build6onFriJan417:34:02PST2008byuid=1017(v5)gid=207(ricdev)
groups=207(ricdev)
SONATASEIS41R4.5.3PL1
MAISONR4.5.3PL1
MAISONV4_5_3bld62_TESTBuild62onFriJan418:38:22PST2008
PROTOCOLSR4.5.3
PROTOCOLSV4_5_3bld48_TESTBuild48onTueJan814:42:58PST2008
SONATAIS41FSVR4.5.3
SONATAIS41FSVR453PatchLevel1onThuOct2322:45:40PDT2008
SONATAIS41FSVV4_5_3bld11_TESTBuild11onMonOct2209:51:38PDT2007
UNIXLOCALR4.5.2
UNIXLOCALIS41ULV4_2_6PL1PatchLevel1onTueDec415:34:59PST2007
UNIXLOCALV4_2_6bld18_TESTBuild18onSunOct2118:57:40PDT2007
ToCheckEmergencyPatchUpgradePath
1 Telnettothetargetmachineasuserfsv,msc,lisorproasappropriateto
theNE.
2 Checkforlastinstalledsoftwarecomponentbyexecuting:
cpu_version
3 ChecktheoutputandconfirmthatthebaselinefortheEPalreadyexistsonthe
targetmachine.
Forexample,toinstallMAISONR4.5.3PL1EP2,confirmthatMAISON
R4.5.3PL1EP1hasalreadybeeninstalledonthemachine.
Warning:
InstallanEmergencyPatchonlyontherequiredsoftwarebaseline.
4 Keepthetelnetsessionopen.
BackingUptheDatabase
TheinstallationtoolmakesadatabasebackupbeforeinstallingtheEPfiles.

InstallingEmergencyPatches Thefollowingprocedureprovidesgenericinstructionsforpatchinstallation.
AlwaysrefertotheReleaseNotesforanyinstallationinstructionspertinentto
thatrelease.
MultipleEPscanbeinstalled(respectinganydependencies)beforetheNEis
restarted.EPscanalsobeinstalledinbulkusingtheboptionoftheep_install
command(refertoxxxx).
ToInstallEmergencyPatch
1 UsetheopentelnetsessionwiththeNE(loggedinastheNEapplicationuser)
tostoptheNEapplicationbyexecuting:
cpu_stop
Note:someEPsdonotrequirethesystemtobeshutdown.RefertotheEP
releasenotes.
2 Switchusertorootandchangedirectoryto/RJE.(Createthe/RJEdirectoryifit
doesnotalreadyexist):
suroot
(mkdir~/RJE)
cd~/RJE
3 FTPtheemergencypatchfromtheinstallerclienttothe~/RJEdirectoryonthe
targetmachine.
4 Unziptheemergencypatchbyexecuting:
gunzip<EPName>.gz
5 Installoneemergencypatchbyexecuting:
ep_install<EPName>
Warning:
ConsultReleaseNotesforspecificinstallationinstructions.
6 IftheReleaseNotesspecifythattheNEFfilesneedtobemergedontotheOMC,
mergetheNEFfilesandthenrunthepostinstallscript.RefertotheInitial
ConfigurationchapteroftheOMCServerGuideforinstructions.
7 RestarttheNEapplication(ifitwasstopped)byexecuting:
cpu_start
8 Checktheemergencypatchinstallationbyexecuting:
cpu_version

Thenewlyinstalledemergencypatchshouldappearinthelist.Forexample:
groups=207(ricdev)
MAISONR4.5.3PL1
PROTOCOLSR4.5.3
SONATAIS41FSVR4.5.3
UNIXLOCALR4.5.2
RemovingEmergencyPatches Thissectionincludes:
BeforeRemovingEmergencyPatches
SoftwareandSystemVerification
BeforeRemovingEmergencyPatches
Theep_installscriptbacksupallofthefilesaffectedbytheinstallationofan
emergencypatch.ItusesanativeSUNpackageutilitytobackupallaffectedfiles
intheEPandalsoperformsadatabasebackup.
Emergencypatchesmustberemovedinthereverseorderthattheywereapplied
whichcanbeperformedinbulkmode.
Obtaintheverificationtestplanfortherollbackreleasetoverifythesystemafter
therollback,ifdesired.
Theemergencypatchinstallscriptbacksupallofthefilesaffectedbythe
softwarechange.Theemergencypatchuninstallscriptwillusethebackupfilesto
overwritethefileschangedbytheinstallationoftheemergencypatch.Theresult
istoreturntheNEtothestatebeforetheemergencypatchwasinstalled.
TheNEapplicationmustbestoppedandrestartedforthesoftwarerollbackto
takeeffect.Emergencypatchesmustalsoberemovedinthereverseorderthat
theywereinstalled.

ToRemoveanEmergencyPatch
1 TelnettothetargetNEasusermsc,pro,lisorfsvasappropriate.
2 StopNEapplicationbyexecuting:
cpu_stop
Note:someEPsdonotrequirethesystemtobeshutdown.RefertotheEP
releasenotes.
3 Switchusertoroot:
suroot
4 Runtheemergencypatchuninstallscriptineithersingleorbulkmode:
ep_installr<EPName>
ThepackageinstancenamecanbeusedastheEPname.
5 Verifyremovaloftheemergencypatchbyexecuting:
cpu_version
Theremovedemergencypatchshouldnotappearinthelist.Forexample:
groups=207(ricdev)
MAISONR4.5.3PL1
PROTOCOLSR4.5.3
SONATAIS41FSVR4.5.3
UNIXLOCALR4.5.2
6 IftheReleaseNotesspecifythattheNEFfilesneedtobemergedontotheOMC,
mergetheNEFfilesandthenrunthepostinstallscript.RefertotheInitial
ConfigurationchapteroftheOMCServerGuideforinstructions.
7 RestarttheNEapplication(ifitwasstopped)byexecuting:
cpu_start
TheNEwillberunningthepreviousbaselinesoftware(preemergencypatch
version).
Verifythesoftwareandsystemaftereveryconfigurationchangeasaresultofan
emergencypatchorpatchlevelsoftwareupgrade.Verifyingthesystemafter
rollingbacktoapreviouslystableversionisdoneattheoperatorsdiscretion.
Softwareandsystemverificationisrecommended,butnotmandatory.
Inthecaseofrollingbacktoaprevioussoftwareconfiguration,theverification
testplanforthatreleaseshouldbeused.

InstallingPatchLevel Thissectionincludes:
Releases
OverviewofPatchLevelReleaseInstallation
BeforeInstallingNewPatchLevel
InstallingaNewPatchLevelRelease
RemovingPatchLevelReleases
OverviewofPatchLevel Thissectionincludes:
ReleaseInstallation
PatchLevelReleaseInstallationOverview
PatchLevelReleaseMaintenanceWindow
SystemImpact
GettingPatchLevelReleaseSoftware
UsingPatchLevelReleaseNotes
PatchLevelReleaseInstallationOverview
Agivenpatchlevel(PL)containsalloftheearlierpatchlevelcodeandfixes.For
example,FSVR4.5.3PL2containsallofthepreviousPLversions(PL1andPL2).All
oftheemergencypatchreleasesforagivenpatchlevelreleasearerecompiled
intothenewpatchlevelrelease.
PatchLevelReleaseMaintenanceWindow
Themaintenancewindowforapatchlevelreleaseislargerthananemergency
patchreleasebecausethepatchlevelreleaseisafullsystemsoftwarerelease.
AllpatchlevelreleasesrequirethattheNEbeoutofserviceduringtheentire
upgradeperiod.
SystemImpact
Thesystemimpactforapatchlevelreleaseismoresubstantialthatforan
emergencypatchrelease.Thepatchlevelreleasecanincludenewfeatures,and
enhancementstoexistingfeatures.
Consultthereleasenotesthataccompanythepatchlevelreleasetoseealistof
newfeaturesandfeatureenhancements.
GettingPatchLevelReleaseSoftware
Thepatchlevelreleasesoftwareisafullrecompilationofthesystemsoftware.
PatchlevelreleasesaredistributedonCDROM.Apatchlevelreleaseisnot
availablefordownloadonanFTPsite.
Contactcustomersupportforallissuesrelatedtoobtainingandinstallingpatch
levelreleases.Thereleasenotesforthepatchlevelreleaseprovideall
informationaboutthecontentsandpurposeoftheupgrade.Theverificationtest
plan(VTP)andreleasenotesforthepatchlevelreleasecontainproceduresto
testandverifyanychangestofeaturesandeachissueaddressedbytheupgrade.

PatchLevelReleaseContents
Apatchlevelreleasecontainsnewbuildsofallsystemsoftwarecomponents.
ThecontentsofapatchlevelreleasearelistedinTable 64.
Table 64PatchlevelReleaseContents
Artifact Description
PCH_fsv_prd_* HLRapplicationupdates.
PCH_lmt_prd_*
PCH_maison_prd_* Runtimeenvironment.
PCH_omc_drum_prd_* OMCproxyapplicationupdates.
PCH_pro_prd_* Signalingprotocolsupdates.
PCH_raft_prd_* RAFTupdates.
PCH_rcp_drum_prd_* DRUMupdates.
PCH_unix_prd_* UNIXupdates.
UsingPatchLevelReleaseNotes
Areleasenotesdocumentaccompanieseachpatchlevel(PL)release.Thefile
namesyntaxforthereleasenotesdocumentis<softwareversion><patch
level>_README.txt.Forexample,FSVR4.5.3PL1_README.txtistherelease
notesfilenameforthepatchlevel3(PL3)release.
Thereleasenotesdocumenthasastandardformat,asdescribedinTable 65.
Table 65PatchLevelReleaseNotesDescription
ReleaseNotesHeading Description
1.Notes Anyadditionalinformation.
2.PatchLocation Locationwherepatchcanbeobtained.
3.Proceduretoapplythispatch Patchinstallationinstructions.
4.Afterinstallationcpu_version Versionlevelsofnewinstalledsoftware
shouldinclude components.
5.Postinstallationinstruction Postinstallationinstructions.
6.Proceduretoremovethe Patchrollbackinstructions.
Emergencypatch
7.Infrastructureversions Lowerlevelsoftwareversions.
8.Newfeatures Listofnewfeaturesincludedinpatch.
9.Issuefixes Listofissuesfixedinrelease.
BeforeInstallingNewPatch Thissectionincludes:
Level
CheckingPatchLevelUpgradePath

CheckingPatchLevelUpgradePath
Thepatchupgradepathshouldbedelineatedbeforeinstallinganypatch.Patches
areincrementalandmustbeappliedinanorderknownasthepatchupgrade
path.
Checkthepatchlevelupgradepathbeforeinstallingthepatch.Allrequired
emergencypatchesandearlierpatchlevelreleasemustalreadybeinstalled.
InstallingaNewPatchLevel Thissectionincludes:
Release
ListingtheInstalledPatchVersions
ListingtheInstalledPatchVersions
Aversionlistofalreadyinstalledpatches,bothemergencypatch(EP)andpatch
level(PL),canbedisplayed.
ToListtheInstalledPatchVersions
1 LogintotheNEasusermsc,pro,lisorfsvasappropriate.
2 Listtheversionofinstalledpatches.
pro>cpu_version
3 Recordthepatchversionsfromtheoutputorgenerateafileandprintitforeasy
reference.
pro>cpu_version>/<location>/<filename>
The/RJEdirectoryisasuitablelocationtowriteto.Useafilenamelike
ver_<current_date>forreadbility.
SeeExample79foranexampleversionlistofinstalledpatches.
Example79ListingInstalledPatchVersions
fsv>cpu_version
groups=207(ricdev)
MAISONR4.5.3PL1
PROTOCOLSR4.5.3
SONATAIS41FSVR4.5.3
UNIXLOCALR4.5.2
Apatchlevelreleaseisinstalledwithproceduresandinstructionsgiveninthe
ReleaseNotesfortherelease.
Allinformationandproceduresforsystemandfeatureverificationintherelease
isgiveninthereleasenotesandverificationtestplanfortherelease.
ToVerifySoftwareandSystem
1 Locatetheverificationtestplanandreleasenotesforthepatchlevelrelease.
2 Executetheverificationproceduresinthereleasenotes.
3 Executetheverificationproceduresintheverificationtestplan.
RemovingPatchLevel Generally,patchlevelreleasesarenotremoved.Afullsystembackupshouldbe
Releases performedtopreservethepreupgradestateofthesoftware.
Ifthereareanysevereissueswiththedeploymentoftheupgradeinthenetwork
thenthesystembackupcanberestored.Thisisnotacommonoccurrence,and
theinstallers,withthehelpofcustomersupport,willhandletheupgradeand
anyunexpectedrollback.
PointLevelVersion Asoftwarepointlevelreleasedeliversnewplatformandfeatureadditions,and
Upgrades changesandenhancementstoexistingplatformfunctionalityandfeatures.
Apointlevelreleaseisinstalledandverifiedbycustomersupportonly.The
installationandverificationofpointlevelupgradereleasesaredescribedin
releasenotesfortherelease.
OverviewofSoftwareVersionUpgrades
SoftwareReleasePackageDescription
BeforeUpgradingSoftwareVersion
PointLevelReleaseInstallation
PostUpgradeProcedures
OverviewofSoftwareVersion Thissectionincludes:
Upgrades
PointLevelVersionUpgradePaths
GettingReleaseSoftwareandDocumentation
PointLevelVersionUpgradePaths
Theupgradepathforasoftwarepointlevelversionupgrademustberespected.
Generally,apointlevelreleaseisinstalledonalowerlevelpointrelease.All
requiredpatchlevelreleasesmustbeinstalledonthebaselinebeforethenew
pointlevelreleaseisinstalled.
Customersupportwillensurethatthebaselinesystemsatisfiestheupgradepath
foragivenpointlevelrelease.
GettingReleaseSoftwareandDocumentation
Apointlevelsoftwarereleaseisquitelarge.SoftwareisdistributedonCDROM.
Releasenotes,releaseinstallationinstructions,andanysoftwareandsystem
verificationproceduresaredistributedseparately.Customersupportwilluse
thesedocumentstoinstallthepointlevelrelease.

SoftwareReleasePackage ApointlevelreleaseincludessoftwareforUNIXlocal,theNEapplication,and
Description more.Thereleasealsoincludessomeconfigurationfiles.
Table 66liststhesoftwareinthepointlevelreleaseforanHLR.
Table 66PointLevelSoftwareReleaseContentsforanHLR
Artifact Description
INV_fsv_prd_* HLRapplicationupdates.
INV_lmt_prd_*
INV_maison_prd_* Runtimeenvironment.
INV_omc_drum_prd_* OMCproxyapplicationupdates.
INV_pro_prd_* Signalingprotocolsupdates.
INV_raft_prd_* RAFTupdates.
INV_rcp_drum_prd_* DRUMupdates.
INV_unix_prd_* UNIXupdates.

BeforeUpgradingSoftware Thissectionincludes:
Version
UpgradePrerequisites
UpgradeMaintenanceWindow
SystemImpact
UsingReleaseNotesforSoftwareVersionUpgrade
UpgradePrerequisites
Thegeneralprerequisitesforapointlevelupgradearethesameasforanyother
typeofupgrade:
Personnelfamiliarwiththeupgradeprocessshouldperformtheupgrade
Allinstallationtoolsshouldbeonsite(hardware,software)
Allrelevantdocumentationshouldbeavailable
Anupgradeplanshouldbemadeforthesite
Thereleasemayalsohavespecificprerequisitesthataredeterminedshortly
beforetheinstallationprocess,andmaydifferforeachspecificnetwork
deployment.Thereleasenotesforthepointlevelreleaseoutlinethe
prerequisitesforupgradingaspecificsitewithapointlevelrelease.
UpgradeMaintenanceWindow
Themaintenancewindowforapointlevelreleasediffersforeachspecific
networkdeployment,butfollowssomegeneralguidelines:
AllsoftwareupgradesrequirestoppingandrestartingtheNE
Theupgradewillresultinlossofservice
SystemImpact
Thesystemimpactforapointlevelreleaseisspecifiedinthereleasenotes.The
biggestimpacttoasystemaretheadditionofnewfeaturesandthe
enhancementofexistingfeatures.Thereleasenotesdescribeanynewfeaturesin
therelease.
UsingReleaseNotesforSoftwareVersionUpgrade
Thisdocumentprovidessomegeneralinformationaboutpointlevelreleases.
Thereleasenotesaretheprimarysourceofinformationandproceduresfor
installingandconfiguringapointlevelrelease.
Thereleasenotesareprovidedseparatelyfromthesoftware.Customerservice
willmakethereleasenotesavailabletotheinstallers.

PointLevelRelease Thissectionincludes:
Installation
PointLevelReleaseInstallationInstructions
SoftwareUpgradeVerification
PointLevelReleaseInstallationInstructions
Thereleasenotesaretheprimarysourceofinformationandproceduresfor
installingandconfiguringapointlevelrelease.
SoftwareUpgradeVerification
Thereleasenotesandotherdocumentationprovidedtotheinstallersby
customerserviceliststheprocedurestotestandverifysystemsoftwareand
features.
PostUpgradeProcedures Thissectionincludes:
PostUpgradeConfiguration
PostUpgradeConfiguration
Somesystemorsoftwareconfigurationmayberequiredaftertheinstallationis
complete.Thisconfigurationisrequiredtointegratethesoftwarereleaseintothe
newtork.Additionalconfigurationmayberequiredtoconfigureanynewfeatures
orenhancedfeatures.
PostupgradeconfigurationisoutlinedintheReleaseNotesfortherelease.

HARDWAREMAINTENANCE
16
Thischapteroutlinesthehardwaremaintenancetasksthatshouldbeperformed
ontheCoreVoiceNetworkElementsonaregularbasis.
ChangingAirFilters
ChangingAirFilters TheSUNserverairfiltersshouldbechangedregularly.Thetimeframeinwhich
theairfiltershouldbereplacedvariesdependingontheenvironmentwherethe
SUNNetraisinstalled;iftheairintheoperatingenvironmentisespeciallydirty,
youmayhavetoreplacethefiltermorefrequently.Intypicallocationswithhigh
levelsofdustanddirtintheair,therecommendedservicetimecouldbeaslow
astwomonths.Incleanairenvironments,thefiltermayonlyneedtobechanged
every18monthsorlonger.
Thesystemmaybecomedamagedifyourunitwithoutanairfilterforan
extendedperiodoftime.Thefiltersarenotdesignedtobecleaned,soyoumust
completelydiscarddirtyfiltersandreplacethemwithnewcleanfilters.
TherearetwoproceduresforreplacingthefiltersonefortheSUNNetra240
andonefortheSUNNetra440.
SUNNetra240Filter Theairfilterislocatedbehindthebezelatthefrontofthesystem.Youshould
Replacement haveareplacementairfilteravailablebeforegoingthroughthisprocedure.
Figure 44FrontofSUNNetra240ShowingBezelFingerHolds

272 Chapter16:HardwareMaintenance
RemovingtheAirFilter
1 Ensurethattheserverisproperlygrounded.
2 Gripthebezelatthetwofingerholds(refertoFigure 44)androtateitdowntoits
openposition.
3 LifttheairfilteroutfromtheinsideofthebezelbygrippingitneartheDVDbay.
InstallingtheAirFilter
1 Snapthenewairfilterintoplaceinthebezel.
2 Closethebezel.
SUNNetra440Filter Theairfilterislocatedbehindthesystemdooratthefrontofthesystem.You
Replacement shouldhaveareplacementairfilteravailablebeforegoingthroughthis
procedure.
Figure 45FrontofSUNNetra440ShowingtheSystemDoorandAirFilter
RemovingtheAirFilter
1 Openthesystemdoor(refertoFigure 45).
2 Locatedtheairfilterbehindthefrontdoorpanel.
InstallingtheAirFilter
1 Installtheairfilterassemblyintothefrontdoorpanel.
2 Closethesystemdoor.

APPENDICES
V
Appendix A PerformanceCountersandStatistics
Appendix B SubsystemEvents
Appendix C DiagnosticTools
Appendix D Acronyms

PERFORMANCECOUNTERSANDSTATISTICS
A
AboutThisAppendix ThisappendixdescribestheperformancecountersintheCoreNEsandmapsthe
counternamestotheperformancestatisticsnamesthataredisplayedinreports.
Thecountersaredescribedinthetablesintherelevantsections.
Thisappendixincludes:
TypesofCounters
SystemHardwarePerformanceStatistics
TypesofCounters Thereareseveraltypesofcountersusedforgeneratingperformancestatisticsin
theCoreNetworkNEs.Thecountersareusedtogeneratestatistics.
CumulativeCounters
Gauges
Gauge_Max
Gauge_Min
DiscreteEventRegistration
Usage
CumulativeCounters Acumulativecountersimplycountsthenumberoftimesaneventoccurswithin
aspecifiedcollectionperiod.Attheendofthecollectionperiod,thecounteris
resettoadefinedvalue(usually0)andthenewcollectionperiodbegins.
CumulativecountersareusedforgeneratingstatisticsontheA1/A2/A5interface,
NEstatistics,andtrunkstatistics(ontheMSC).
Gauges Agaugecounterrepresentsadynamicvariablethatmayincrementor
decrement.Ifagaugespansmultiplecollectionperiods,thenitisreinitializedat
thestartofthenewcollectionperiod.
Gaugecountersareusedforgeneratingsystemperformancestatistics.
Gauge_Max AGauge_Maxcounterisatypeofgaugecounterusedtocalculatethemaximum
valueofthevariableattainedwithinthecollectionperiod.
Gauge_Maxcountersareusedforgeneratingsystemperformancestatistics.

276 AppendixA:PerformanceCountersandStatistics
Gauge_Min AGauge_Mincounterisatypeofgaugecounterusedtocalculatetheminimum
valueofthevariableattainedwithinthecollectionperiod.
Gauge_Mincountersareusedforgeneratingsystemperformancestatistics.
DiscreteEventRegistration ADiscreteEventRegistration(DER)counterrecordstheaveragetimeofthe
intervalbetweenstartingandendingevent,oraverageofanvariable.
DERcountersareuseforgeneratingsomeA1/A2/A5andNEstatistics.
Usage Ausagecountermeasuresthetimeintervalbetweenastartingeventandan
endingevent.
UsagecountersareusedforgeneratingperformancestatisticsontheA1/A2/A5
interfaceandtrunkstatistics(ontheMSC).
CounterNaming PerformanceManagementisaboutusingperformancestatistics,not
performancecounters,butitmayhelptoknowthecounternameandwhereitis
collectedfortroubleshootingpurposes.Thecounternamingsyntaxindicates
wherethecounteriscollected.Thenamingsyntaxappliestoallcountertypes.
Figure 46showsthecounternamingsyntaxandwherethosecountersare
collected.
Figure 46CounterNamingSyntax
SystemHardware Statisticsarealsogeneratedfortheunderlyingsystem.
PerformanceStatistics
CPUStatistics
MemoryUtilizationStatistics
I/OStatistics
CPUStatistics
CPUstatisticsinclude:
cpu_usr
cpu_sys
cpu_wio
cpu_idle

SystemHardwarePerformanceStatistics 277
MemoryUtilizationStatistics
Memoryutilizationstatisticsinclude:
mem_total
mem_free
swap_total
swap_free
I/OStatistics
I/Ostatisticsinclude:
disk_io
net_io
CompletedescriptionsofthesestatisticsarelistedinTable 67.
Table 67SystemHardwarePerformanceStatistics
Statistic Description CounterName Type

CPUUSR Specifiesthepercentoftimethe cpu_usr 32bitGauge
systemisinusermode(runninguser
orapplicationcodeinthiscaseMSC
code).Thetimeisanaverage(%age
instate/numberofsamples).
Thesystemispolledat5minute
intervals.
CPUSYS Specifiesthepercentoftimethe cpusys 32bitGauge
systemisinsystemmode(running
systemprogramsorsystemcalls
fromauserprogram).Thetimeisan
average(%ageinstate/numberof
samples).
intervals.
CPUWIO Specifiesthepercenttimespentin cpu_wio 32bitGauge
idlestatebutwaitingforI/Oactivity.
Thetimeisanaverage(%ageinstate
/numberofsamples).
intervals.
CPUIDLE SpecifiesthepercenttimetheCPUis cpu_idle 32bitGauge
inidlestate.Thetimeisanaverage
(%ageinstate/numberofsamples).
intervals.

278 AppendixA:PerformanceCountersandStatistics
Table 67SystemHardwarePerformanceStatistics
Statistic Description CounterName Type

TOTALMEMORY Specifiesthetotalamountof mem_total 32bitGauge
memory(inKB)usedbyuser
processes(MSCprocesses).
intervals.
FREEMEMORY Specifiestheaveragetotalamount mem_free 32bitGauge
offreememory(inKB)availableto
userprocesses(MSCprocesses).
intervals.
TOTALSWAP Specifiesthetotalamountofdisk swap_total 32bitGauge
space(inKB)availableforpage
swapping.
intervals.
FREESWAP Specifiestheaverageamountoffree swap_free 32bitGauge
diskspace(inKB)availabletouser
processes(MSCprocesses)for
swapping.
intervals.
DISKI/O Specifiestheaveragenumberof disk_io 32bitGauge
blockspersecondtransferred
to/fromdevices.
intervals.
NETWORKI/O Specifiestheaveragenumberof net_io 32bitGauge
blockspersecondtransferred
to/fromthenetwork.
intervals.
Forevenmoredetailedinformationsystemactivityandperformance,including
someoftheoneslistedinTable 67,runmans1msar,mans1mnetstat,
andmans1miostat.

SUBSYSTEMEVENTS
B
AboutThisAppendix Subsystemsregistereventsandexchangemessagestoperformtheirfunctions.
Thisappendixdescribestheeventsforthefollowingsubsystems:
CPUSubsystemEvents
ASPSubsystemEvents
CSISubsystemEvents
VSMSubsystemEvents
SSFSubsystemEvents
NotallNEsubsystemsaredisplayed.
UnderstandingEventDescriptions
Eventdescriptionscontainvariables.Thesevariablesrepresenteitherdigitsor
strings,whicharereplacedatruntimewiththecorrectvalue.Adigitis
representedby%1$dandastringisrepresentedby%1$s.
Theeventdescriptionslistedinthetablesshowthevariables,butrelevant
descriptionswillhavemoremeaningfuldescriptionsthathelpdescribetheevent.
CPUSubsystemEvents Table 68liststheeventsintheCPUsubsystem.
Table 68CPUSubsystemEvents
EventName Severity Description

CPU_CM_VITAL_DEATH_M FATAL PANIC:Vitalchildprocesshasdiedorexited
CPU_CM_BADSTOP_M FATAL PANIC:Cannotstopchildrenforslavetomasterrestart
CPU_CM_SOFTFAIL_M FATAL PANIC:Softwarefailure
CPU_CM_BADLEVSHUT_M FATAL PANIC:Shutdownbylevelfailed
CPU_PM_HWD_DEATH_M FATAL PANIC:VitalprocessHWDhasdiedorexited
CPU_PM_CM_DEATH_M FATAL PANIC:VitalprocessCPU_CMhasdiedorexited
CPU_CM_BADLEVSHUT_M FATAL PANIC:Shutdownbylevelfailed
CPU_PM_RTD_DISCONNECT_M FATAL PANIC:RTDBAKdisconnectedfrommaster
CPU_CM_NO_RESTART_RESP_M ERROR Childprocess%2$s(%3$s)didnothonorGOINGMASTER
within%1$dseconds
CPU_CM_RESTART_INITFAIL_M ERROR Initializationprogram%2$sexitedwithcode%1$Xduring
restart
CPU_CM_NO_EXIT_M ERROR Childprocess%2$s(%3$s)didnothonorEXITcommand
within%1$dseconds

280 AppendixB:SubsystemEvents

CPU_CM_KILLFAIL_M ERROR Attempttosend%2$sto%3$s(%4$s)causeserrno%1$d
CPU_CM_PROCEED_SYNCTIMOUT_M ERROR PROCEEDsynctimeout(%1$dsecs)atlevel%2$d
(Semaphorevalue=%3$dinsteadof0)
CPU_LIC_MGR_ERROR_M ERROR %1$s
CPU_CM_EARLYEXIT_M ERROR Child%1$sexitedatlevel%2$dinsteadoflevel%3$d
CPU_RESTART_M ERROR restartinprogress
CPU_CM_NONVITAL_NO_OK_M WARNING Child%2$s(%3$s)didnotaffirmAREYOUOK?within%1$d
seconds
CPU_ALREADY_ATTACHED_M WARNING cpu_attachalreadycalledcallignored
CPU_CM_IPCFULL_M WARNING IPCqueuefullretryingtimer%1$d
CPU_PM_UNKNOWN_MSG_M WARNING UnknownmessgefromHWD.Id:%1$xRole:%2$xState:%3$x
CPU_ILL_WAIT_M WARNING Illegalcallto%1$s()ignoring
CPU_LIC_MGR_WARNING_M WARNING %1$s
CPU_RCP_WARNING_M WARNING %1$s
CPU_RCP_ERROR_M WARNING %1$s
CPU_CM_WILD_MSG_M INFO Unknownprocess(withIPCid%1$d)sentmebadmessageid
%2$d
CPU_CM_WILD_CHILD_M INFO itwaskilled
CPU_CM_NO_TO_OK_M INFO itwaskilled
CPU_CM_STARTUP_M INFO CPUmonitorstartupin%1$smode
CPU_CM_BEGINEXIT_M INFO %1$dchildrenatlevel%2$dtoldtoexit
CPU_CM_EXTRA_YES_M INFO itwaskilled
CPU_CM_BEGIN_S_TO_M_M INFO Startingslavetomasterroleswitch
CPU_CM_GO_SLAVE_M INFO Switchtoslaveoperatorcommandaccepted
CPU_CM_ALREADY_BUSY_M INFO itwaskilled
CPU_CM_ILLEGAL_BUSY_M INFO Slaveprocess%1$s(%2$s)sentBUSYitwaskilled
CPU_CM_ILLEGAL_WAITING_M INFO itwaskilled
CPU_CM_SIGDEATH_M INFO Childprocess%1$s(%2$s)killedbysignal%3$d(%4$s)
(with%5$sacoredump)
CPU_CM_NO_OK_M INFO Childprocess%2$s(%3$s)didnotaffirmAREYOUOK?within
%1$dsecondsitwaskilled
CPU_CM_PROCEXIT_M INFO Childprocess%1$s(%2$s)exitedwithcode%3$d
CPU_CM_PROCEED_SYNC_M INFO WaitingforPROCEEDsyncwith%1$dchildrenatlevel%2$d
CPU_PM_BECOMING_MASTER_M INFO Becomingmasternow
CPU_PM_DO_IPC_ONLY_M INFO EnteredspecialdoIPConlystate
CPU_PM_GOT_EXIT_M INFO ObeyingEXITcommandfromchild_mon
CPU_CM_PROCRESTART_M INFO Childprocess%2$s(%3$s)restartedwithunixpid=%1$d
CPU_CM_NOSYNCWORK_M INFO NoPROCEEDsyncneededatlevel%1$d
CPU_CM_NOEXITWORK_M INFO NoEXITsyncneededatlevel%1$d
CPU_CM_EVLMODE_M INFO Runningeventloggerin%1$smode


CPU_CM_NOEXGOMWORK_M INFO NoEXIT/GOINGMASTERsyncneededatlevel%1$d
CPU_CM_BEGINGOM_M INFO %1$dchildrenatlevel%2$dsentGOINGMASTER
CPU_CM_PROCEED_OP_M INFO PROCEED/%1$ssenttochild%2$s(PID=%3$d)
CPU_CM_NO_SUCH_PROC_M INFO Unknownprocess%1$d(withIPCid%2$d)wanttodetach
fromme
CPU_CM_NO_SUCH_IC_M INFO Process%1$d(withunknownIPCid%2$d)wanttodetach
fromme
CPU_CM_ATTACH_AGAIN_M INFO Process%1$d(withIPCid%2$d)wanttoattachagain
CPU_CM_PROCTBL_OVERFLOW_M INFO Process%1$d(withIPCid%2$d)wanttoattachme,butthe
processtableisoverflow
CPU_CM_SHUTDOWN_M INFO CPUmonitorbeginningshutdownbyoperatorrequest
CPU_PM_CHANGE_STATE_M INFO Changestatereceived:Current:%1$s%2$sNew:%3$s
%4$s%5$s%6$s
CPU_HWD_INFO_M INFO HardwareDaemon(HWD)%1$s%2$s
CPU_SIGNAL_M INFO CPU_PM:Receivedsignal%1$s%2$s
CPU_LIC_MGR_INFO_M INFO %1$s
CPU_RCP_INFO_M INFO %1$s
ASPSubsystemEvents Table 69liststheeventsintheASPsubsystem.
Table 69ASPSubsystemEvents

ASP_PARTY_NOT_INIT ERROR Attempttoaccesspartybufferbeforeinitialization
ASP_PARTY_PORT_MISMATCH ERROR Attempttoretrieveanexistingpartywithwrongport
number
ASP_PARTY_NOT_FREE ERROR Inusepartyfoundonthefreelist(%1$d)
ASP_PARTY_IN_CALL ERROR Attempttofreeparty%1$dfailed,thepartyisstillinacall
ASP_PARTY_NOT_INUSE ERROR Attempttoaccessapartybuffernotinuse
ASP_INVALID_PARTY_NO ERROR Attempttoaccessinvalidpartybuffer(%1$d)
ASP_PARTY_CALL_CNT ERROR Attempttodecrementparty(%1$d)callcounttolessthan
zero
ASP_CALL_NOT_INIT ERROR Attempttoaccesscallbufferbeforeinitialization
ASP_CALL_NOT_INUSE ERROR Attempttoaccessacallbuffernotinuse
ASP_CALL_NOT_EMPTY ERROR Attempttofreecall%1$dwhilepartiesarestillattached
ASP_CALL_NOT_FREE ERROR Inusecallfoundonthefreelist(%1$d)
ASP_PARTIES_IN_CONV ERROR attempttoaddmorethan%1$dpartiestoaconversation
ASP_DUP_PARTY_IN_CALL ERROR addpartytocallfailed,party%1$dalreadyincall%2$d
ASP_PARTY_NOT_IN_CALL ERROR party%1$dnotfoundincall%2$d
ASP_INVALID_CALL_NO ERROR Attempttoaccessinvalidcallbuffer(%1$d)
ASP_ITW_NOT_ATTACHED ERROR process=%1$s


ASP_PREPAID_PARTY_ALREADY_IN_USE ERROR Attempttoaccesspartyno(%1$d)alreadyinuse
ASP_ADD_PREPAID_PARTY_FAILED ERROR Failedtoaddpartyobject(%1$d)tolinklist
ASP_PREPAID_PARTY_NOT_IN_USE ERROR Attempttoaccesspartyno(%1$d)notinuse
ASP_REMOVE_PREPAID_PARTY_FAILED ERROR Failedtoremovepartyobject(%1$d)fromlinklist
ASP_PREPAID_PARTY_EXPECTED ERROR Expectedtofindpartyobject(%1$d)infunction%2$s
ASP_PREPAID_PARTY_NOT_EXPECTED ERROR Foundprepaidpartyobject(%1$d)inlistwhenitwasnot
expected
ASP_CAN_NOT_OPEN_TABLE ERROR CannotopenRTDtable%1$s
ASP_CAN_NOT_READ_TABLE ERROR FailedtoreadRTDtable%1$s
ASP_INVALID_PREPAIDLINKSTATE_PARAM ERROR Invalidprepaidlinkstateparameterpassedtofunction
ETER (%1$s)
ASP_INVALID_PREPAIDSTATE_PARAMETER ERROR Invalidprepaidstateparameterpassedtofunction(%1$s)
ASP_INVALID_STATS_PARAMETER ERROR Invalidstatsparameterpassedtofunction(%1$s)
ASP_UNEXPECTED_XCP ERROR Unexpectedexceptionincallto%1$s>%2$V%3$s
ASP_BAD_FEATURE ERROR party=%1$dfeature=%2$Mstate=%3$Mfunction=%4$M
step=%5$M
ASP_PARTY_ERR_INFO ERROR party=%1$dxnid=%2$dlist=%3$dparty_type=%4$M
party_function=%5$M
ASP_DIG_RECONSTRUCTION_FAIL ERROR digits=%1$stype=%2$s
ASP_BAD_PATTERN ERROR BadPattern%1$satline%2$d:%3$s
ASP_BAD_PATTERN_INSERT ERROR FailedtoInsertPattern%1$satline%2$d:%3$s
ASP_BAD_DEST_TYPE ERROR badtranslationdestinationtype
ASP_BAD_TYPE_ARG ERROR badtranslationargumentfordestinationtype%1$s
ASP_EMPTY_TRANS_LIST ERROR listfortranslationclass%1$dwasempty
ASP_MISSING_CLASS USERERROR Couldnotfindclassofservicefor:%1$d
ASP_MISSING_SYSFEATURE USERERROR Couldnotfindsystemfeatureforclass:%1$dfeature:%2$d
ASP_MISSING_SUBFEATURE_CLASS USERERROR Couldnotfindsubscriberfeatureforclass:%1$d,feature:
%2$d
ASP_MISSING_TIMER USERERROR Couldnotfindsystemtimersforclass:%1$d
ASP_MISSING_BCAP_CLASS USERERROR Couldnotfindbcapclassfor:%1$d
ASP_CP_WARNING USERERROR File%1$s,line%2$dEventQUEUEING:settingotherparty
classto0
ASP_LOAD_DB_ERR USERERROR database:%1$s:ignoringline%2$d,caughterror:%3$s
ASP_UNKNOWN_CLASS USERERROR translationclass%1$dnotfoundintranslationdatabase
ASP_MIN_OR_ESN_CHANGED USERERROR party=%1$dmin=%2$snew_min=%3$sesn=%4$lx
new_esn=%5$lx
ASP_LOAD_DB_ERROR WARNING databaseerror
ASP_CDR_INVALID_CELL_ID WARNING Invalidbscellid:%1$d
ASP_CDR_FAILED_CONVERT WARNING Nobillingrecordfound:o_party=%1$dt_party=%2$d
ASP_CDR_IHO_NOT_SUPPORTED WARNING NeedtoaddsupportforIHOcalls


ASP_CDR_FAILED_UPDATE_B_PARTY_TRU WARNING Nobillingrecordfound:o_party=%1$dt_party=%2$d
NK
ASP_CDR_FAILED_UPDATE_B_PARTY_TRK WARNING Nobillingrecordfound:o_party=%1$dt_party=%2$d
GRP
ASP_CDR_FAILED_UPDATE_A_PARTY_DIGI WARNING Nobillingrecordfound:o_party=%1$dt_party=%2$d
TS
ASP_CDR_FAILED_UPDATE_A_PARTY_NU WARNING Nobillingrecordfound:o_party=%1$dt_party=%2$d
MBER
ASP_CDR_FAILED_UPDATE_A_PARTY_TYPE WARNING Nobillingrecordfound:o_party=%1$dt_party=%2$d
ASP_CDR_FAILED_UPDATE_B_PARTY_DIGI WARNING Nobillingrecordfound:o_party=%1$dt_party=%2$d
TS
ASP_CDR_FAILED_UPDATE_B_PARTY_TYPE WARNING Nobillingrecordfound:o_party=%1$dt_party=%2$d
ASP_CDR_FEATURE_CALL_INVALID_TYPE WARNING Invalidtype=%1$M
ASP_CDR_FAILED_B_PARTY_UPDATE WARNING Nobillingrecordfound:o_party=%1$dt_party=%2$d
ASP_CDR_FAILED_UPDATE_B_CELL_ID WARNING Nobillingrecordfound:o_party=%1$dt_party=%2$d
ASP_DUP_PATTERN_OVERLAP WARNING Pattern%1$satline%2$disduplicateof%3$satline%4$d,
discarded
ASP_LOAD_DB_CALLPROGRESS WARNING ignoringline%1$d,caughterror:%2$s
ASP_LOAD_DB_SYSFEATURE WARNING ignoringline%1$d,caughtERROR:%2$s
ASP_LOAD_DB_TIMERS WARNING ignoringline%1$d,caughtERROR:%2$s
ASP_ATTEMPT_TO_ACCESS_FREE_PARTY_ WARNING attempttoaccessfreeparty%1$dignored
BUFFER
ASP_TDN_DATA_LOST WARNING tdn=%1$stimer_id=%2$d
ASP_OUT_OF_PARTY_BUFFERS INFO Nofreepartybuffers(%1$dcurrentlyconfigured)
ASP_OUT_OF_CALL_BUFFERS INFO Nofreecallbuffers(%1$d)currentlyconfigured
ASP_RPL_OK INFO Successfulreply
ASP_RPL_ABORTED INFO Abortedreply
ASP_RPL_CAUSE INFO Failedwithcausereply
ASP_RPL_NOT_FOUND INFO notfound
ASP_RPL_RETRY INFO retry
ASP_RPL_FAIL INFO fail
ASP_RPL_CONTINUE INFO continue
ASP_RPL_BAD_CALL INFO badcall
ASP_PARTY_FUNCTION_BUILD_MPTY INFO buildmpty
ASP_RPL_NO_DIGITS INFO nodigits
ASP_DEST_TYPE_SET_CALLED_NAI INFO setcallednai
ASP_DEST_TYPE_SET_CALLING_NAI INFO setcallingnai
ASP_DEST_TYPE_RESET_COS INFO resetCOS
ASP_DEST_TYPE_LAND INFO landdestinationtype
ASP_LAND_TYPE_LAST INFO lastlandtype
ASP_DEST_TYPE_AIN INFO ainquerydestinationtype


ASP_AIN_TYPE_ECC_CALL INFO ainecccalltype
ASP_AIN_TYPE_LNP_CALL INFO ainlnpquerytype
ASP_AIN_TYPE_LAST INFO lastaintype
ASP_DEST_TYPE_SUBSCRIBER INFO subscriberdestinationtype
ASP_SUBSCRIBER_TYPE_MOBILE_DN INFO mobilednsubscribertype
ASP_SUBSCRIBER_TYPE_MOBILE_MIN INFO mobileminsubscribertype
ASP_SUBSCRIBER_TYPE_WIRED INFO wiredsubscribertype
ASP_SUBSCRIBER_TYPE_TDN INFO tdn
ASP_SUBSCRIBER_TYPE_TRIGGER INFO trigger
ASP_SUBSCRIBER_TYPE_ISDN INFO isdn
ASP_SUBSCRIBER_TYPE_CAS_R1 INFO CASR1
ASP_SUBSCRIBER_TYPE_CAS_R2 INFO CASR2
ASP_SUBSCRIBER_TYPE_LAST INFO lastsubscribertype
ASP_DEST_TYPE_FEATURE_UPDATE INFO featureupdatedestinationtype
ASP_FEATURE_UPDATE_RESERVED_1 INFO featureupdatecallforwardalwaysenable
ASP_FEATURE_UPDATE_RESERVED_2 INFO featureupdatecallforwardalwaysdisable
ASP_FEATURE_UPDATE_RESERVED_3 INFO featureupdatecallforwardbusyenable
ASP_FEATURE_UPDATE_RESERVED_4 INFO featureupdatecallforwardbusydisable
ASP_FEATURE_UPDATE_RESERVED_5 INFO featureupdatecallforwardnoanswerenable
ASP_FEATURE_UPDATE_RESERVED_6 INFO featureupdatecallforwardnoanswerdisable
ASP_FEATURE_UPDATE_RESERVED_7 INFO featureupdatecallwaitingenable
ASP_FEATURE_UPDATE_RESERVED_8 INFO featureupdatecallwaitingdisable
ASP_FEATURE_UPDATE_RESERVED_9 INFO featureupdatecallwaitingpercalldisable
ASP_FEATURE_UPDATE_HANDOFF_PC_DIS INFO handoffpcdisable
ABLE
ASP_FEATURE_UPDATE_HANDOFF_PC_EN INFO handoffpcenable
ABLE
ASP_FEATURE_UPDATE_REMOTE_UNCHEC INFO remoteunchecked
KED
ASP_FEATURE_UPDATE_RESERVED_10 INFO remotechecked
ASP_FEATURE_UPDATE_RESERVED_11 INFO featureupdatereserved
ASP_FEATURE_UPDATE_LAST INFO lastfeatureupdate
ASP_DEST_TYPE_FEATURE_ACCESS INFO featureaccessdestinationtype
ASP_FEATURE_ACCESS_ROAMER_GATEWA INFO featureaccessroamergateway
Y


ASP_FEATURE_ACCESS_COLLECT_DTMF INFO featureaccesscollectdtmf
ASP_FEATURE_ACCESS_COLLECT_PIN INFO featureaccesscollectpin
ASP_FEATURE_ACCESS_RESERVED_4 INFO featureaccesscollectaccount
ASP_FEATURE_ACCESS_RESERVED_5 INFO featureaccesscollectsoftMIN
ASP_FEATURE_ACCESS_RESERVED_6 INFO featureaccesscollectfeatureaccesscode
ASP_FEATURE_ACCESS_ANI_LOOKUP INFO featureanilookup
ASP_FEATURE_ACCESS_CARD_LOOKUP INFO featurecardlookup
ASP_FEATURE_ACCESS_COLLECT_CARD INFO featurecollectcard
ASP_FEATURE_ACCESS_VOICE_MAIL INFO featureaccessvoicemail
ASP_FEATURE_ACCESS_DESTINATION_DIGI INFO featureaccessroutedestinationdigits
TS
ASP_FEATURE_ACCESS_ECC_CALL INFO featureaccessreserved
ASP_FEATURE_ACCESS_PPX_CARD_GW_O INFO featureaccessprepaidcardgatewayorigination
RIG
ASP_FEATURE_ACCESS_PPX_ANI_ORIG INFO featureaccessprepaidaniorigination
ASP_FEATURE_ACCESS_PPX_MOBILE_ORI INFO featureaccessprepaidmobileorigination
G
ASP_FEATURE_ACCESS_RESERVED_16 INFO featureaccessreserved
ASP_FEATURE_ACCESS_LAST INFO lastfeatureaccess
ASP_DEST_TYPE_TREATMENT INFO treatmentdestinationtype
ASP_TYPE_TREATMENT_LAST INFO lasttreatmenttype
ASP_DEST_TYPE_RETRANS INFO retranslatedestinationtype
ASP_TYPE_RETRANS_LAST INFO lastretranslatetype
ASP_PARTY_STEP_ECC_SEND_ORIG_IND INFO SendECCOriginationIndication
ASP_PARTY_STEP_ECC_RECV_ORIG_PRCD INFO ReceivedanOriginationProceedfromECC
ASP_PARTY_STEP_ECC_FORCE_RELEASE INFO ReceivedaForceReleasefromECC
ASP_PARTY_STEP_READY_HANDOFF_POR INFO Readyinghandoffport
T
ASP_PARTY_STEP_HANDOFF_PORT INFO Handingoff
ASP_PARTY_STEP_ADD_TO_CONNECTION INFO addtoconnnection
ASP_PARTY_STEP_REMOVE_FROM_CONN INFO removefromconnection
ECTION
ASP_PARTY_STEP_ADD_TO_MONITOR INFO addtomonitor
ASP_PARTY_STEP_REMOVE_FROM_MONI INFO removefrommonitor
TOR
ASP_PARTY_STEP_RELEASE INFO release
ASP_PARTY_STEP_SEND_DTMF INFO senddtmf
ASP_PARTY_STEP_COLLECT_DTMF INFO collect_dtmf
ASP_PARTY_STEP_PPX_SEND_COLLECTED_ INFO SendPPXCollectedINFORequest
INFO


ASP_PARTY_STEP_PPX_SEND_ORIG_ATTE INFO SendPPXOrigAttemptRequest
MPT
ASP_PARTY_STEP_PPX_SEND_ORIG_ATTE INFO SendPPXOrigAttemptRequest
MPT_AUTH
ASP_PARTY_STEP_PPX_SEND_ORIG_TERM INFO SendPPXOrigTermDisconnect
_DISCONNECT
ASP_PARTY_STEP_PPX_SEND_TERM_ATTE INFO SendPPXTermAttemptAuth
MPT_AUTH
ASP_PARTY_STEP_PPX_SEND_TERM_ATTE INFO SendPPXTermAttemptForward
MPT_FORWARD
ASP_PARTY_STEP_PPX_SEND_TERM_ORIG INFO SendPPXTermOrigDisconnect
_DISCONNECT
ASP_PARTY_STEP_PPX_SEND_RELEASE_CA INFO SendPPXReleaseCall
LL
ASP_PARTY_TYPE_LAND INFO landparty
ASP_PARTY_TYPE_SUBSCRIBER INFO mobileparty
ASP_PARTY_TYPE_INTERSYSTEM INFO intersystemparty
ASP_CALL_EVENT_FEATURE_BIT INFO featurebit
ASP_CALL_EVENT_CALL_ANSWERED INFO callanswered
ASP_CALL_EVENT_REMOVED_PARTY INFO removedparty
ASP_CALL_EVENT_CALL_ALERTING INFO alerting
ASP_PARTY_STATE_TWC_COLLECT_DTMF_ INFO twccollectdtmf
DIGITS
ASP_PARTY_STATE_TWC_COLLECT_FLASH_ INFO twccollectdtmf
DIGITS
ASP_PARTY_STATE_ORIGINATING INFO originating
ASP_PARTY_STATE_TERMINATING INFO terminating
ASP_PARTY_STATE_ANSWERED INFO answered
ASP_PARTY_STATE_RELEASING INFO releasing
ASP_PARTY_STATE_HANDOFF INFO handoff
ASP_PARTY_STATE_TO_CONVERSING INFO toconversing
ASP_PARTY_STATE_ALERTING INFO alerting
ASP_PARTY_STATE_WAIT_FOR_ANSWER INFO waitanswer
ASP_PARTY_STATE_TREATMENT INFO treatment
ASP_PARTY_STATE_ROUTING INFO routing
ASP_PARTY_STATE_RESERVED INFO partystatereserved
ASP_PARTY_STATE_WAIT_FOR_ALERTING INFO waitalerting
ASP_PARTY_STATE_FAULT INFO fault
ASP_PARTY_STATE_FEATURE_UPDATE INFO featureupdate
ASP_PARTY_STATE_TRANSLATING INFO translating
ASP_PARTY_STATE_PAGING INFO paging


ASP_PARTY_STATE_ABORT INFO abort
ASP_PARTY_STATE_HANDING_IN INFO handingin
ASP_PARTY_FUNCTION_TERMINATING INFO terminating
ASP_PARTY_FUNCTION_COLLECT_DTMF INFO collectdtmf
ASP_PARTY_FUNCTION_ORIG_SUB_DATA INFO origsubdata
ASP_PARTY_FUNCTION_TRANSLATE_DIGIT INFO translatedialeddigits
S
ASP_PARTY_FUNCTION_ROUTE INFO route
ASP_PARTY_FUNCTION_FEATURE_UPDATE INFO featureupdate
ASP_PARTY_FUNCTION_ANSWER_PARTY INFO answering
ASP_PARTY_FUNCTION_DISCONNECT INFO disconenct
ASP_PARTY_FUNCTION_CALL_PROGRESS INFO callprogress
ASP_PARTY_FUNCTION_HANDOFF_PARTY INFO handoffparty
ASP_PARTY_FUNCTION_WAIT_FOR_ANSW INFO waitanswer
ER
ASP_PARTY_FUNCTION_COLLECT_TWC_DI INFO collecttwcdigits
GITS
ASP_PARTY_FUNCTION_CONVERSATION INFO conversation
ASP_PARTY_FUNCTION_LEAVE_CONVERSA INFO leaveconversation
TION
ASP_PARTY_FUNCTION_TREATMENT INFO treatment
ASP_PARTY_FUNCTION_PPX_PROMPT_CO INFO ppxpromptcollectuserINFOtrigger
LL_USER_INFO
ASP_PARTY_STEP_QUERY_SUB_BY_MIN INFO querysubbymin
ASP_PARTY_STEP_QUERY_SUB_BY_DN INFO querysubbymin
ASP_PARTY_STEP_START_CALL_PROGRESS INFO startcallprogress
ASP_PARTY_STEP_STOP_CALL_PROGRESS INFO stopcallprogress
ASP_PARTY_STEP_SEND_ANSWER INFO sendinganswer
ASP_PARTY_STEP_ALERT_LAND INFO alertland
ASP_PARTY_STEP_ALERT_MOBILE INFO alertmobile
ASP_PARTY_STEP_START_TREATMENT INFO starttreatment
ASP_PARTY_STEP_STOP_TREATMENT INFO stoptreatment
ASP_TIMER_OBJ_PARTY INFO partytimer
ASP_TIMER_TYPE_LONGCALL INFO longcalltimer
ASP_TIMER_TYPE_TREATMENT INFO treatment
ASP_TIMER_TYPE_CALLPROGRESS INFO callprogress
ASP_SUBFEATURE_CFWD_ALWAYS INFO callforwardalways
ASP_SUBFEATURE_CFWD_BUSY INFO callforwardbusy
ASP_SUBFEATURE_CFWD_NO_ANSWER INFO callforwardnoanswer
ASP_SUBFEATURE_CW INFO callwaiting


ASP_SUBFEATURE_RESERVED_1 INFO reserved
ASP_PARTY_STEP_RELEASE_OLD_HANDOF INFO releaseoldhoport
F_PORT
ASP_PARTY_STEP_RELEASE_NEW_HANDO INFO releasenewhoport
FF_PORT
ASP_SUBFEATURE_THREE_WAY_CALLING INFO 3waycalling
ASP_SYSFEATURE_FIRST INFO Notext
ASP_SYSFEATURE_ROAMER_GATEWAY INFO roamergateway
ASP_SYSFEATURE_COLLECT_DTMF INFO collectdtmf
ASP_SYSFEATURE_COLLECT_PIN INFO collectpin
ASP_SYSFEATURE_OPERATOR INFO operatorreroute
ASP_SYSFEATURE_RESERVED_5 INFO collectsoftmin
ASP_SYSFEATURE_RESERVED_6 INFO collectfeatureaccesscode
ASP_SYSFEATURE_TWC INFO 3waycalling
ASP_SYSFEATURE_RESERVED_8 INFO nprdigits
ASP_SYSFEATURE_PCO INFO pco
ASP_SYSFEATURE_VOICE_MAIL INFO voicemail
ASP_SYSFEATURE_DESTINATION_DIGITS INFO routedestinationdigits
ASP_SYSFEATURE_ANI_LOOKUP INFO anilookup
ASP_SYSFEATURE_CARD_LOOKUP INFO cardlookup
ASP_SYSFEATURE_COLLECT_CARD INFO cardlookup
ASP_SYSFEATURE_ECC_CALL INFO ecccall
ASP_SYSFEATURE_PPX_CALL INFO ppxcardgatewayorigination
ASP_SYSFEATURE_LNP_CALL INFO lnpcall
ASP_SYSFEATURE_RESERVED_17 INFO reserved17
ASP_SYSFEATURE_RESERVED_18 INFO reserved18
ASP_SYSFEATURE_LAST INFO reserved
ASP_CALLPROGRESS_EVENT_QUEUING INFO queuing
ASP_CALLPROGRESS_EVENT_ALERTING INFO alerting
ASP_CALLPROGRESS_EVENT_OPERATOR INFO operatorreourte
ASP_CALLPROGRESS_EVENT_ANSWER INFO answer
ASP_CALLPROGRESS_EVENT_CW INFO callwaiting
ASP_CALLPROGRESS_EVENT_HANDOFF INFO handoff


ASP_CALLPROGRESS_EVENT_COLLECT_DT INFO collectdtmf
MF
ASP_CALLPROGRESS_EVENT_FEATURE_FAI INFO featurefail
L
ASP_CALLPROGRESS_EVENT_FEATURE_DE INFO featuredeny
NY
ASP_CALLPROGRESS_EVENT_TWC INFO twcdialtone
ASP_CALLPROGRESS_EVENT_ROAMER_GA INFO roamergateway
TEWAY
ASP_CALLPROGRESS_EVENT_COLLECT_PIN INFO reserved
ASP_CALLPROGRESS_EVENT_CARD_PROM INFO cardprompt
PT
ASP_CALLPROGRESS_EVENT_CARD_SUCCE INFO cardsuccess
SS
ASP_CALLPROGRESS_EVENT_CARD_LOOK INFO cardlookupfail
UP_FAIL
ASP_CALLPROGRESS_EVENT_CARD_COLLE INFO cardcollectfail
CT_FAIL
ASP_CALLPROGRESS_EVENT_ON_HOLD INFO Partywasputonhold
ASP_CALLPROGRESS_ACTION_CONNECT INFO connect
ASP_CALLPROGRESS_ACTION_IGNORE INFO ignore
ASP_CALLPROGRESS_ACTION_NONE INFO none
ASP_CALLPROGRESS_ACTION_RESERVED_ INFO reserved
2
3
4
5
ASP_STATUS_INACTIVE INFO inac
ASP_STATUS_PENDING INFO pend
ASP_STATUS_ACTIVE INFO actv
ASP_STATUS_CLEARING INFO clrg
ASP_CALLPROGRESS_TYPE_TONE INFO tone
ASP_CALLPROGRESS_TYPE_IGNORE INFO ignore
ASP_CALLPROGRESS_TYPE_CONNECT INFO connect
ASP_CALLPROGRESS_TYPE_NONE INFO none
ASP_DIGITS_TYPE_UNANSWERED INFO UNANSWERED
ASP_DIGITS_TYPE_DIALED INFO dialed
ASP_DIGITS_TYPE_FLASH INFO flash
ASP_DIGITS_TYPE_DTMF INFO dtmf


ASP_DIGITS_TYPE_TRANSFER INFO transfer
ASP_DIGITS_TYPE_RESERVED_2 INFO Notext
ASP_TIMER_TYPE_UNANSWERED INFO noanswer
ASP_TIMER_TYPE_PCO_PRETIME_DELAY INFO pcopretime
ASP_TIMER_TYPE_RESERVED_2 INFO Notext
ASP_TIMER_TYPE_RESERVED_3 INFO Notext
ASP_CALL_EVENT_CALL_UNANSWERED INFO UNANSWERED
ASP_PARTY_FUNCTION_GET_NOANSWER_ INFO getnoanswerdigits
DIGITS
ASP_PARTY_FUNCTION_GET_BUSY_DIGITS INFO getnoanswerdigits
ASP_PARTY_FUNCTION_GET_NPR_DIGITS INFO getnoanswerdigits
ASP_CALL_EVENT_CALL_MOVED INFO callmoved
ASP_PARTY_FUNCTION_GET_VOICE_MAIL INFO getvoicemailbox
BOX
ASP_PARTY_FUNCTION_COLLECT_PIN INFO collectpin
ASP_PARTY_STEP_FEATURE_ACCESS INFO featureaccess
ASP_PARTY_STEP_START INFO start
ASP_PARTY_STEP_GENERIC_1 INFO generic1
ASP_PARTY_STEP_CALLPROGRESS_DONE INFO cpdone
ASP_RPL_BAD_STATE INFO badstate
ASP_TDN_DATA_TYPE_CD INFO vlr
ASP_TDN_STATE_FREE INFO free
ASP_TDN_STATE_BUSY INFO busy
ASP_TDN_DATA_TYPE_UNKNOWN INFO unknown
ASP_PARTY_FUNCTION_ROAMER_GATEW INFO roamergateway
AY
ASP_PARTY_FUNCTION_ADD_HANDIN_FA INFO addhandinmoduletocall
CILITY
ASP_PARTY_FUNCTION_ADD_HANDIN_RA INFO addhandinradiotocall
DIO
ASP_TIMER_TYPE_REDIRECT_RELEASE INFO redirectrelease
ASP_REDIRECT_REASON_BUSY INFO busy
ASP_REDIRECT_REASON_NOANSWER INFO UNANSWERED
ASP_REDIRECT_REASON_UNCONDITIONAL INFO unconditional
ASP_REDIRECT_REASON_NPR INFO npr
ASP_RPL_NOT_NETWORKED INFO notnetworked
ASP_PARTY_FUNCTION_REDIRECT INFO redirect


ASP_PARTY_STEP_RUN_REMOTE_FEATUR INFO runremotefeature
E
ASP_RPL_TRANSACTION_FAILED INFO transactionfailed
ASP_RPL_RESERVED_3 INFO reserved3
ASP_RPL_BAD_PARTY INFO badparty
ASP_RPL_REQUEST_FAILED INFO requestfailed
ASP_TIMER_ALREADY_STARTED INFO party=%1$dtimer=%2$salreadystarted
ASP_CALL_EVENT_CALL_NPR INFO npr
ASP_CALL_EVENT_CALL_BUSY INFO busy
ASP_TIMER_TYPE_ABORT_TRANSACTIONS INFO longcalltimer
ASP_PARTY_STEP_PCO_PRETIME_DELAY INFO pcopretimedelay
ASP_CDR_TYPE_STANDARD INFO twopartycall
ASP_CDR_TYPE_FEATURE INFO featureactivation
ASP_CDR_TYPE_FORWARD INFO callhasbeenforwarded
ASP_CALL_EVENT_PARTY_CONNECTED INFO partyconnected
ASP_PARTY_STEP_WAIT_TO_PROCEED INFO waittoproceed
ASP_CDR_TYPE_TWC INFO threewaycall
ASP_CDR_TYPE_CW INFO callwaiting
ASP_CALL_EVENT_SET_FEATURE INFO callwaiting/threeway/etccall
ASP_PARTY_FUNCTION_ANI_LOOKUP INFO cardlookup
ASP_PARTY_FUNCTION_CARD_LOOKUP INFO cardlookup
ASP_PARTY_FUNCTION_COLLECT_CARD INFO cardlookup
ASP_PARTY_STEP_START_CALL_PROGRESS INFO startcallprogress2
2
ASP_PARTY_STEP_START_CALL_PROGRESS INFO startcallprogress3
3
ASP_PARTY_STEP_AIN_QUERY_LNP INFO processlnpqueryreply
ASP_CLEAR_PARTY_TAG_FAILED INFO Notext
ASP_PARTY_FUNCTION_ECC_CALL INFO ecccallvalidation
ASP_CALL_EVENT_REROUTE INFO reroute
ASP_PARTY_FUNCTION_COLLECT_OPERAT INFO collecttwcdigits
OR_DIGITS
ASP_PARTY_FUNCTION_LNP_CALL INFO querylnpdatabase
ASP_PARTY_STATE_OPERATOR_COLLECT_ INFO operatorcollectdtmfdigits
DTMF_DIGITS
ASP_PARTY_STATE_OPERATOR_COLLECT_F INFO operatorcollectflashdigits
LASH_DIGITS
ASP_CDR_TYPE_OPERATOR INFO callhasbeenoperatorextended
ASP_DN_PORTED_IN INFO DNisportedintotheswitch


ASP_DN_PORTED_OUT INFO DNisportedoutoftheswitch
ASP_EVT_HOLD_ACK INFO SendaholdackeventtoLFM
ASP_EVT_RETRIEVE_ACK INFO SendaretrieveackeventtoLFM
ASP_PARTY_FUNCTION_HOLD_CONVERSA INFO hold
TION
ASP_PARTY_FUNCTION_RETRIEVE_CONVE INFO unhold
RSATION
ASP_PARTY_STEP_CALL_WAITING INFO callwaiting
ASP_PARTY_STEP_RELEASE_CALL INFO releasecall
ASP_PARTY_STEP_CLEANUP_TARGET_POR INFO cleanuptargetport
T
ASP_EVT_BUILD_MPTY_ACK INFO SendabuildmptyackeventtoLFM
ASP_LOAD_DB INFO Database(%1$s)loaded
ASP_UNKNOWN_DB INFO Requesttoloadunknowndatabase(%1$s)
ASP_EVT_RELEASE_CALL_ACK INFO SendareleasecallackeventtoLFM
ASP_PARTY_FUNCTION_RELEASE_CALL INFO Releasecall
ASP_PARTY_ORIGINATION INFO OriginationParty
ASP_PARTY_TERMINATION INFO TerminationParty
ASP_PARTY_FORWARD INFO ForwardedParty
ASP_PARTY_UNDEFINED INFO UndefinedParty
ASP_PREPAID_SERVER_LINK_UP INFO Linktoprepaidserver%1$sisup
ASP_PREPAID_SERVER_LINK_DOWN INFO Linktoprepaidserver%1$sisdown
ASP_PREPAID_LINK_DOWN_CLEARING_PA INFO Prepaidserverlinkdown.Clearparty(%1$d)
RTY
ASP_REMOVING_PREPAID_PARTY INFO Attemptingtoremoveprepaidparty(%1$d)fromlinklist
ASP_ADDING_PREPAID_PARTY INFO Attemptingtoaddprepaidparty(%1$d)tolinklist
ASP_INIT_PREPAID_PARTY INFO Initializingtheprepaidandstatslinklist
ASP_REGISTER_EVL_NOTIFICATION INFO RegisteringtoreceiveEVLnotificationtomonitorlinkstatus
ASP_ADD_PREPAID_PARTY INFO Addingprepaidparty%1$dtoprepaidlinklist
ASP_REMOVE_PREPAID_PARTY INFO Removingprepaidparty%1$dfromprepaidlinklist
ASP_REMOVE_ALL_PREPAID_PARTIES INFO Clearingallpartiesandremovingfromtheprepaidlinklist
ASP_STATS_ADJUST INFO Adjustingthenumberofpartiesintheprepaidpartylinklist
ASP_PREPAID_AVAILABILITY_STATE_ADJUS INFO Adjustingprepaidserveravailability
T
ASP_PREPAID_LINK_STATE_ADJUST INFO Adjustingprepaidserverlinkstate
ASP_GET_PREPAID_LINK_STATE INFO Getprepaidserverlinkstate
ASP_GET_PREPAID_AVAILABILITY_STATE INFO Getprepaidserveravailability
ASP_ASSOC_RECOVER INFO function=%1$smodule=%2$dtbl=%3$snode=%4$d
key=%5$d
ASP_ASSOC_TRC INFO function=%1$s
ASP_ASSOC_INFO INFO caughtexceptionduring%1$shandling:\t%2$hV:\t%2$V


ASP_RESET_TRC INFO function=%1$snode=%2$ddelete_associd=%3$d
ASP_NO_BRANCH_ASSOCID INFO Nobranchassociationinthe%1$sprocedure
ASP_RESET_CDR_TRC INFO function=%1$snode=%2$dassocid_flag=%3$d
o_party_no=%4$d
ASP_BASSOC_SET_STABLE INFO Thebranchassociation=%1$disstableparty=%2$d
ASP_BASSOC_SET_DESTROY INFO Branchassociation=%1$dissettodestroyparty=%2$d
ASP_BASSOC_SET_SYNC INFO function=%1$sparty=%2$dstable_flag=%3$d
bassocid=%4$d
ASP_CALL_EVENT_MONITOR INFO Connecttoresource
ASP_CALL_EVENT_PARTY_ON_HOLD INFO Partywasputonhold
ASP_DEST_TYPE_BCAP INFO bcapquerydestinationtype
ASP_BCAP_TYPE_CALL INFO bcapquerydestinationtype
ASP_FUNC_TRACE INFO %1$s:%2$s=%3$d%4$s=%5$d%6$s=%7$s%8$s=%9$d
ASP_FTRACE INFO %1$s()called
ASP_EVT_SUBTRACT_ACK INFO SendasubtractackeventtoLFM
ASP_EVT_MOVE_ACK INFO SendamoveackeventtoLFM
ASP_PARTY_FUNCTION_SUBTRACT INFO subtract
ASP_PARTY_FUNCTION_MOVE INFO move
ASP_TRC_FEATURE_UPDATE INFO party=%1$dxnid=%2$dfeature=%3$Mdata=%4$s
ho_type=%5$T
ASP_PARTY_STEP_SELECT_COMPSEQ INFO selectcompseq
ASP_TRC_GET_FREE_CALL INFO call=%1$d
ASP_TRC_GET_USED_CALL INFO call=%1$d
ASP_TRC_FREE_CALL INFO call=%1$d
ASP_TRC_ADD_PARTY_TO_CALL INFO party=%1$dxnid=%2$dcall=%3$d
ASP_TRC_CALL_SIZE INFO call=%1$dsize=%2$d
ASP_TRC_REMOVE_PARTY_FROM_CALL INFO party=%1$dxnid=%2$dcall=%3$dcause=%4$M
ASP_TRC_CLEAR_CALLS INFO party=%1$d
ASP_CALL_ANSWERED_EVT INFO call_no=%1$dparty=%2$dxnid=%3$dkey=%4$d
ASP_FEATURE_CALL_EVT INFO party=%1$dxnid=%2$dcall_type=%3$M
ASP_GET_XNID2 INFO xnid=%1$d
ASP_CDR_BILLING_REQUIRED INFO o_party=%1$dt_party=%2$d
ASP_CDR_WRITE INFO cause=%1$Mo_party_type=%2$do_party_num=%3$s
t_party=%4$dt_party_type=%5$dt_party_num=%6$s
o_cell_id=%7$dt_cell_id=%8$do_party_trkgrp=%9$d
t_party_trkgrp=%10$d
ASP_CDR_FLUSH_BILLING_RECS INFO o_party=%1$dt_party=%2$dcause=%3$M%4$s
ASP_CDR_FLUSH_BILLING_RECS_A INFO o_party=%1$dt_party=%2$dcause=%3$M%4$s
ASP_CDR_FLUSH_BILLING_LIST INFO o_party=%1$d%2$s
ASP_CDR_UNKNOWN_CAUSE INFO cause=%1$M
ASP_CDR_INITIALIZE_BILLING_REC INFO party=%1$d


ASP_CDR_TYPE_NOT_SUPPORTED INFO Needtoaddsupport
ASP_CDR_CONVERT_TO_FORWARD_REC INFO party=%1$d
ASP_CDR_INITIALIZE_FEATURE_REC INFO party=%1$dt_party=%2$d
ASP_CDR_UPDATE_B_PARTY_TRUNK INFO o_party=%1$dt_party=%2$dt_party_trunk=%3$d
ASP_CDR_UPDATE_B_PARTY_TRKGRP INFO o_party=%1$dt_party=%2$dt_party_trkgrp=%3$d
ASP_CDR_UPDATE_A_PARTY_DIGITS INFO party=%1$ddigits_party=%2$s
ASP_CDR_UPDATE_A_PARTY_NUMBER INFO party=%1$dparty_number=%2$s
ASP_CDR_UPDATE_A_PARTY_TYPE INFO party=%1$do_party_type=%2$M
ASP_CDR_UPDATE_B_PARTY_DIGITS INFO o_party=%1$dt_party=%2$ddigits=%3$s
ASP_CDR_UPDATE_B_PARTY_TYPE INFO party=%1$dt_party_type=%2$M
ASP_CDR_FEATURE_CALL INFO party=%1$dflash_party=%2$dtype=%3$M
ASP_CDR_UPDATE_B_PARTY INFO o_party=%1$dt_party=%2$dparty_type=%3$Mmin=%4$s
digits=%5$s
ASP_CDR_WRITE_ORIG_ERROR INFO type=%1$Mport=%2$dtrkgrp=%3$dcell=%4$dcalled=%5$s
calling=%6$semergency=%7$dcause=%8$M
ASP_PARTY_FUNCTION_START_HELD_TON INFO Startplayingheldtone
E
ASP_PARTY_FUNCTION_STOP_HELD_TONE INFO Stopplayingheldtone
ASP_CDR_UPDATE_B_CELL_ID INFO o_party=%1$dt_party=%2$dt_cell_id=%3$d
ASP_FCI INFO nat_int=%1$dend2end=%2$dinterworking_ind=%3$d
iam_segment_ind=%4$disup_ind=%5$d
isup_pref_ind=%6$disdn_access_ind=%7$d
sccp_method_ind=%8$dtranslated_cd_num_ind=%9$d
ASP_PARTY_FUNCTION_ETC INFO EstablishTemporaryConnection
ASP_PARTY_STEP_TEMP_CON_ORIG INFO TemporaryConnectionOrigination
ASP_PARTY_STEP_TEMP_CON_RELEASE INFO TemporaryConnectionRelease
ASP_PARTY_ANSWER_INFO INFO id=%1$ssystem_id=%2$scall_no=%3$danswering:
esn=%4$lxmin=%5$sdn=%6$sport=%7$d
ASP_PARTY_ORIGINATION_INFO INFO id=%1$ssystem_id=%2$scall_no=%3$d
originating:esn=%4$lxmin=%5$sdn=%6$sport=%7$d
called=%8$slocation=%9$scarrier=%10$s
ASP_PARTY_RELEASE_INFO INFO id=%1$scall_no=%2$dreleasing:location=%3$s
cause=%4$T
ASP_PARTY_SUBJECT_SIGNAL_INFO INFO id=%1$scall_no=%2$dsig_type=%3$Tsig_data=%4$s
ASP_PARTY_CONN_BREAK_INFO INFO id=%1$scall_no=%2$dremoved:esn=%3$lxmin=%4$s
dn=%5$sport=%6$dremaining:esn=%7$lxmin=%8$s
dn=%9$sport=%10$d
ASP_PARTY_CONNECTION_INFO INFO id=%1$scall_no=%2$dconnected:esn=%3$lxmin=%4$s
dn=%5$sport=%6$dconnected:esn=%7$lxmin=%8$s
dn=%9$sport=%10$d


ASP_PARTY_TERM_ATTEMPT_INFO INFO id=%1$ssystem_id=%2$scall_no=%3$d
originating:esn=%4$lxmin=%5$sdn=%6$sport=%7$d
terminating:min=%8$sdn=%9$slast_redirecting:dn=%10$s
orig_called=%11$snum_redir=%12$d
ASP_PARTY_REDIRECTION_INFO INFO id=%1$scall_no=%2$dredirected_to=%3$s
ASP_DUP_PATTERN_DISCARD INFO Inferiorpattern%1$satline%2$ddiscardedforsuperior
%3$satline%4$d
ASP_DUP_PATTERN_UPDATE INFO Superiorpattern%1$satline%2$dreplacesinferior%3$sat
line%4$d
ASP_ITW_ALREADY_ATTACHED DEBUG Notext
ASP_ITW_PREDEFINED_VALUE DEBUG attribute=%1$Tvalue_in=%2$dvalue_out=%3$d
ASP_ITW_LOADED_VALUE DEBUG attribute=%1$Tvalue_in=%2$dvalue_out=%3$d
ASP_ITW_ATTACHED DEBUG Notext
ASP_SSF DEBUG %1$s:party=%2$dxnid=%3$dbassocid=%4$dstate=%5$M
function=%6$Mstep=%7$Mcause=%8$M
ASP_GET_PARTY DEBUG %1$s:call_no=%2$dhalf_call=%3$d
ASP_GET_XNID DEBUG %1$s:call_no=%2$dhalf_call=%3$d
ASP_CALL_EVENT DEBUG event=%1$sid=%2$scall_no=%3$d
ASP_CP_NUMBER DEBUG %1$s=(presence_ind=%2$dnat_addr=%3$d
num_plan=%4$dpres_restrict=%5$dscreen_ind=%6$d
addr_INFO=%7$s)
ASP_EXPAND_PATTERNS DEBUG %1$dextendedpatternsexpandedto%2$dbasicpatterns,
%3$dpatternsignoredordiscarded

CSISubsystemEvents Table 70liststheeventsintheCSIsubsystem.TheCSIsubsystemexistsonIS41

NEsonly.
Table 70CSISubsystemEvents

NOT_FOUND_PARAMS WARNING notfound%1$Mparameter(s)(%2$s)
UNKNOWN_MSGFLOW WARNING UnknownAIFmessageflowdefined(%1$T),defaultingto
IS634
NO_HANDOFF_DATA WARNING Nohandoffdata.min=%1$s
INITIAL_NOTIFY WARNING %1$s=%2$dfoundwithnonempty%3$s=%4$d%5$s
TMSI_NO_INIT ERROR TMSIcontrollogicnotinitialized
TMSI_ALREADY ERROR TMSIalreadyassignedtothisIMSI
IMSI_NOT_ASSIGNED ERROR IMSInotassignedaTMSI
TMSI_RANGE ERROR TMSIoutofrange
KEY_NOT_LINKED ERROR TMSInotboundtoIMSI
KEY_NOT_ASSIGNED ERROR IMSInotassignedinhashtable
DECODE_FAILURE ERROR %1$s
ENCODE_FAILURE ERROR %1$s
CELL_NOT_FOUND_MCC ERROR cell=%1$dnotfoundforBSC=%2$dduetoMCCmismatch,
ourMCC=%3$s!=%4$s
CELL_NOT_FOUND_MNC ERROR cell=%1$dnotfoundforBSC=%2$dduetoMNCmismatch,
ourMNC=%3$s!=%4$s
CELL_NOT_FOUND_LAC ERROR cell=%1$dnotfoundforBSC=%2$dduetoLACmismatch,
ourLAC=%3$d!=%4$d
CELL_NOT_FOUND_BSC ERROR cell=%1$dnotfoundduetoBSCmismatch,ourBSC=%2$d!=
%3$d
CELL_NOT_FOUND ERROR cell=%1$dnotfoundforBSC=%2$d
UNEXPECTED_SCCP_COMP_TYPE ERROR unexpectedSCCPCOMP=%1$T,%2$Mignored
UNEXPECTED_SCCP_TYPE ERROR UnexpectedSCCPtype=%1$dforBSC=%2$d
REQ_DATA_TYPE_UNKNOWN ERROR UnknownREQdatatype.data_type=%1$M
REQ_RPL_STATUS_UNKNOWN ERROR UnknownREQreplystatus.reply_status=%1$T
PARAM_NO_PRESENT ERROR %1$s:Mandatoryparameternotpresent
UNEXPECTED_PARAM ERROR %1$s:=%2$d
UDATA_DECODE_PROBLEM ERROR %1$s:Applicationmessagedecodeproblem
INV_DATA ERROR invalidinput,%1$s=%2$d%3$s
FAILED_USAP_PC_LOOKUP ERROR FailedtofindBSCbyUSAP=%1$dANSIPC=%2$sITU
PC=%3$s%4$s
FAILED_SLR_LOOKUP ERROR FailedtofindBSCbySLR=%1$d%2$s
INVALID_SS7_PC_FORMAT ERROR %1$s:%2$s
BSC_ID_OUT_OF_RANGE ERROR InputBSCID(%1$d)outofrangefrom%2$dto%3$d
DUPLICATE_BSC ERROR DuplicatedBSCID(%1$d)in%2$sERROR
LOAD_DB_ERR ERROR %1$stableloadingERRORatline=%2$d\tERROR=%3$s
NO_SCCP_USAPS ERROR NoSCCPUSAPswereconfigured


MISSING_SAP ERROR CouldnotfindUSAP=%1$dforBSC=%2$d
MISSING_PARAM ERROR %1$s
TBL_EMPTY ERROR %1$stableisempty.PleaseconfiguretheDBbefore
proceeding
TBL_RESOURCE_LIMIT ERROR %1$s:%2$s,Internallinkedlistresourceshortage
KEY_EXIST ERROR couldnotinsert%1$s%2$s=%3$d
UNHANDLED_EXCEPTION ERROR Capturedanunhandledexceptionorassertion:%1$V,%2$hV,
%3$rV,%4$lV,%5$#V
RCV_OBJID_ERR ERROR Receivedaninvalidreceiver_objid=%1$d
UNRECOGNIZED_VALUE ERROR Recognized%1$svalue=%2$d
ASSERT_GUARD ERROR Catchguardedassertionforcondition:(%1$s)
NULL_PTR_RECEIVED ERROR NULLpointerreceived:%1$s
LIB_CONFIG_ERR ERROR %1$lTdynamiclibrarywasnotconfiguredcorrectly
UNEXPECTED_DYN_LIBRARY ERROR Unexpecteddynamiclibraryindex=%1$din%2$s
UNEXPECTED_IS637_MSG_TYPE ERROR UnexpectedIS637messagetype=%1$T
DLG_RETRIEVE_ERR ERROR Unabletoretrieve%1$s,did=%2$d
SMS_DLG_ASSIGN_ERR ERROR UnabletoassignanewSMSdialog
TAG_DLG_ASSIGN_ERR ERROR UnabletoassignanewTAGdialog
TAG_DLG_RACE_COND ERROR Racecondition:%1$s
FLASH_DLG_ASSIGN_ERR ERROR UnabletoassignanewFLASHdialog
AUTH_DLG_ASSIGN_ERR ERROR UnabletoassignanewAUTHENTICATIONdialog
DUPLICATE_USAP_PC ERROR DuplicatedUSAP=(%1$d),PC=(%2$d),in%3$sERROR
DBT_UNSUPPORTED ERROR UnsupportedDataBurstType=%1$T
AUTH_REQ_IN_PROGRESS ERROR AnAUTHENTICATIONREQUESTisalreadyinprogress
conn_ctl=%1$dMIN=%2$sIMSI=%3$sauth_did=%4$d
LAC_UNK ERROR RequestedLACisnotconfigured:%1$d
CFG_PORT_ERROR USERERROR port=%1$dwillnotbeused,%2$s
SND_REG_MSC INFO %6$sIMSI=%1$sESN=0x%2$08lxBSC=%3$dLAC=%4$d
conn_ctl=%5$d
LOAD_DB INFO Database(%1$s)loaded
UNKNOWN_DB INFO Requesttoloadunknowndatabase(%1$s)
SYNC_REPORT INFO SYNC%1$sonBSC=%2$d
BSC_UNAVAILABLE INFO BSC=%1$dunavailable
PORT_INIT INFO BSC=%1$dhas%2$dportsallocated
CELL_DEFAULT_LOADED INFO cell=%1$d,BSC=%2$dloadeddefaultvalueof%3$s=%4$d
ALREADY_PAGING INFO alreadypagingIMSI=%1$sESN=0x%2$x
DUMMY INFO IgnoringreceivedTPMmessageID=%1$d
SUGGESTED_CIC DEBUG SuggestedCIC=0x%1$x,%2$s
BINDING INFO conn_ctl=%1$dMIN=%2$sIMSI=%3$sBSC=%4$d
cell=%5$dport=%6$dCREF=%7$d


HEX_DUMP INFO buflen=%1$d%2$.*1$lS
SSP_STATE_CHANGE INFO OPC=%1$sSSN=%2$dstatechangeto%3$T
RACE_CONDITION INFO %1$s:raceconditiondetected
TBL_NOT_INIT INFO %1$stableisnotinitialized
TBL_ERROR INFO TableLookuperror,key=%1$d,%2$s
SND_ORIGINATION INFO MOBILE_ORIGINATION,min=%1$simsi=%2$sesn=%3$x
called=%4$scalling=%5$s,sender_objid=%6$d
REQ_tag=%7$d,REQ_timer=%8$d
SND_EVT INFO %1$T,port_no=%2$d,tid=%3$s,sender_objid=%4$d,
rcv_objid=%5$d
SND_MOBILE_RELEASE INFO %1$T,port_no=%2$d,min=%3$s,cause=%4$T,
sender_objid=%5$d,rcv_objid=%6$d
SND_PAGE_REQ_RPL INFO PAGE_REQUEST_RPL,min=%1$s,imsi=%2$s,cause=%3$T,
sender_objid=%4$d,req_tag=%5$d
SND_MOBILE_FLASH INFO %5$sMOBILE_FLASH,port_no=%1$d,called=%2$s,
SND_BURST_DTMF INFO %7$sBURST_DTMF,port_no=%1$d,called=%2$s,
inter_digit_time=%3$d,digit_duration_time=%4$d,
EVT_RECV INFO %1$sop_type=%2$s,min=%3$s,imsi=%4$s,
sender_objid=%5$d,receiver_objid=%6$d
DECODE_PDU INFO %1$sDecodeinboundmessage:
ENCODE_PDU INFO %1$sEncodeoutboundmessage:
DECODE DEBUG SCCPCOMP=%1$TPCREF=%2$ddata_len=%3$d
sccp_cause=%4$d
GEN_FUNC DEBUG %1$s:
EVENT_ENTRY DEBUG %1$s
HO_EVENT_ENTRY DEBUG %1$smin=%2$s,imsi=%3$s,port=%4$d,
sender_objid=%5$d,receiver_objid=%6$d
BSC_INACCESS DEBUG BSC=%1$dinaccessibleclearingallconnections
MAP_FUNC DEBUG %1$s:SMScause=%2$d
GENERIC DEBUG %1$s
CONN_LOST DEBUG BSC=%1$d:comp_type=%2$M,conn_ctl=%3$dconnection
hasbeenlost!
RESET_CONN_CTL DEBUG resettingconn_ctl=%1$d
MSG_EMPTY DEBUG ReceivedCAP_OP_LAST_Mmessage,onSCCP%1$M
CORRUPTION DEBUG %1$s=%2$dreferencesinvalid%3$s=%4$d
TI_MISMATCH DEBUG ti=%1$ddoesnotmatchconn_ctl=%2$dti/ti2=%3$d/%4$d
PAGE_TRANSITION DEBUG statetransitionevent%1$sonpage_ctl=%2$dfromstate
%3$sto%4$swithtimer%5$d
CONN_TRANSITION DEBUG statetransitionevent%1$sonconn_ctl=%2$dfromstate
%3$sto%4$swithtimer%5$d
PORT_TRANSITION DEBUG statetransitionevent%1$sfromstate%2$sto%3$s


PAGE_MOBILE DEBUG BSMAPpageIMSI=%1$sinBSC=%2$d
FEATURE_NOTI DEBUG BSMAPfeaturenotificationIMSI=%1$sinBSC=%2$d
MWI=%3$d
ASSIGN_REQ DEBUG BSMAPassignCIC=0x%1$x,CallingDigits=%2$s
HO_REQUIRED DEBUG BSMAPHANDOFF_REQUIRED:BSC=%1$dserving_cell=%2$d
HO_REQUEST DEBUG BSMAPreadyhandinCIC=0x%1$xBSC=%2$d
CLEAR_COMMAND DEBUG BSMAPclear(cause=0x%1$x)
HO_COMMAND DEBUG BSMAPbegin%1$shandoutCELL=%2$dchannel=%3$d
HO_COMMENCED DEBUG BSMAPHANDOFF_COMMENCED
HO_CAND_ENQ DEBUG BSMAPcandidateenquireBSC=%1$d
HO_REQD_REJ DEBUG BSMAPdenyhandout(cause=0x%1$x)
READY_CHANNEL DEBUG serving_cell=%1$d,serving_bsc=%2$d,serving_msc=%3$d,
\ttarget_cell=%4$d,target_bsc=%5$d,target_msc=%6$d
BLOCK_ACK DEBUG BSMAPCIC=0x%1$xBSC=%2$d
UNBLOCK_ACK DEBUG BSMAPCIC=0x%1$xBSC=%2$d
GRP_BLOCK_ACK DEBUG BSMAPcircuitgroupatCIC=0x%1$xBSC=%2$d
GRP_UNBLOCK_ACK DEBUG BSMAPcircuitgroupatCIC=0x%1$xBSC=%2$d
RESET DEBUG BSMAPresetindication(cause=0x%1$x)BSC=%2$d
RESET_ACK DEBUG BSMAPresetindicationacknowledgeBSC=%1$d
RESET_CIRCUIT DEBUG BSMAPCIC=0x%1$xBSC=%2$d(cause=0x%3$x)
RESET_CIRCUIT_ACK DEBUG BSMAPCIC=0x%1$xBSC=%2$d
ADDS_PAGE DEBUG BSMAPADDSPageIMSI=%1$sinBSC=%2$d
DIALED_NUM_ACK DEBUG BSMAPdialednumberACK
CLASSMARK_REQ DEBUG BSMAPsendclassmarkrequest
RESOURCE_REQ DEBUG BSMAPresourcerequestcommandCELL=%1$d
\tperiodicity=%2$dmethod=0x%3$xextendedind=0x%4$x
LOAD_INDICATION DEBUG BSMAPBSC=%1$d
RSSI_REQ DEBUG BSMAPreadchannel=%1$dBSC=%2$d
NULL_RSSI_RESP DEBUG BSMAPnobettercellStrengthMeasurementREF=%1$d
BS_SERVICE_RESP DEBUG BSMAPBSC=%1$dtag=%2$d%3$s=%4$s(cause=0x%5$x)
BSMAP_AUTH_REQ DEBUG BSMAPBSC=%1$dtag=%2$d
SETUP DEBUG DTAPsetupcallingdigits=%1$s
ALERTING DEBUG DTAPalerting
PROCEEDING DEBUG DTAPproceeding
CONNECT DEBUG DTAPconnect(answer)
CONNECT_ACK DEBUG DTAPconnectACK
RELEASE DEBUG DTAPrelease(cause=0x%1$x)
RELEASE_COMPLETE DEBUG DTAPreleasecomplete(cause=0x%1$x)
REG_ACCEPT DEBUG DTAPregistrationaccept
REG_REJECT DEBUG DTAPregistrationreject(cause=0x%1$x)


REG_REJECT_NAK DEBUG DTAPregistrationrejectonSCCPCONNREF(cause=0x%1$x)
AUTH_REJECT DEBUG DTAPauthenticationreject
AUTH_REQ DEBUG DTAPauthenticationrequestcksn=%1$drand=%3$.*2$S
ABORT DEBUG DTAPabort(cause=%1$T0x%2$x)
IDENTITY_REQ DEBUG DTAPidentityrequesttype=%1$T
DISCONNECT DEBUG DTAPdisconnect(cause=%1$T0x%2$x)(ti=%3$d)
HOLD_ACK DEBUG DTAPholdacknowledge(ti=%1$d)
HOLD_REJECT DEBUG DTAPholdreject(cause=%1$T0x%2$x)(ti=%3$d)
RETRIEVE_ACK DEBUG DTAPretrieveack(ti=%1$d)
RETRIEVE_REJECT DEBUG DTAPretrievereject(cause=%1$T0x%2$x)(ti=%3$d)
START_DTMF_ACK DEBUG DTAPstartDTMFacknowledge
START_DTMF_REJECT DEBUG DTAPstartDTMFreject(cause=0x%1$x)
STOP_DTMF_ACK DEBUG DTAPstopDTMFacknowledge
BURST_DTMF_ACK DEBUG DTAPBurstDTMFacknowledge
FLASH_ACK DEBUG DTAPFlashacknowledge
FLASH_WI DEBUG DTAPFlashwithINFOrmation
TMSI_REALLOC_CMD DEBUG DTAPTMSIreallocationcommand(%1$s)
ADDS_DEL DEBUG DTAPADDSDeliver
DTAP_AUTH_REQ DEBUG DTAPAuthenticationRequest
DTAP_SSD_UPD_REQ DEBUG DTAPSSDUpdateRequest
DTAP_BS_CHLNG_RESP DEBUG DTAPBSChallengeResponse
UNEXPECTED_PAGE_EVT DEBUG event=%1$snotexpectedinstate=%2$sforpage
buffer=%3$d
UNEXPECTED_CONN_EVT DEBUG event=%1$snotexpectedinstate=%2$sforconn_ctl=%3$d
NON_IDLE_IN_FLUSH DEBUG nonIDLEstatewhenflush,idlingstate2=%1$s/ti2=%2$d
NO_CREF_ATTACHED DEBUG noSCCPconnectionattachedtoconn_ctl=%1$d
PROCESS_ACK_EVT DEBUG %1$MACKforport=%2$dti=%3$d
ALREADY_BOUND DEBUG %1$s=%2$dalreadyhas%3$s=%4$dbound
BIND_FAILURE DEBUG failedtobindport=%1$dtoconn_ctl=%2$d
BAD_PORT_STATE DEBUG incorrectportstate=%1$dforport=%2$d
RESOLVE_CELL DEBUG resolveMCC=%1$sMNC=%2$sLAC=%3$dCI=%4$d
BAD_PARAMS DEBUG missing%1$Tparameter(s)(%2$s)
PROCESS_BSC DEBUG BSC=%1$d:comp_type=%2$T,op_type=%3$M
PROCESS_DTAP DEBUG BSC=%1$d:comp_type=%2$T,op_type=%3$M
OUT_OF_RESOURCE DEBUG nofree%1$s
DEREF_FAIL DEBUG couldnotdereference%1$sfrom%2$s=%3$d
NO_PORT_ATTACHED DEBUG noportattachedtoconn_ctl=%1$d
AIF_MSG_UNEXPECTED DEBUG unexpectedAinterfacemessageop_type=%1$Mfor
BSC=%2$d(%3$T)


AIF_MSG_WRONG_DIR DEBUG wrongdirectionforAinterfacemessageop_type=%1$Mfor
BSC=%2$d(%3$T)
AIF_MSG_UNSUPPORTED DEBUG unsupportedAinterfacemessageop_type=%1$Mfor
BSC=%2$d(%3$T)
CAUSE_PRESENT DEBUG CAUSE=0x%1$xpresentforop_type=%2$M
CELL_NUM_MISMATCH DEBUG Numberofcellsfor%1$s(%2$d)!=%3$s(%4$d),ignoring
%5$s
CELL_ORDER_MISMATCH DEBUG Orderofcellsfor%1$s!=%2$s,ignoring%3$s
PAGE_MISMATCH DEBUG page_ctl=%1$d%2$smismatch,ours%3$d!=%4$d
REQ_MSG_IGNORED DEBUG %1$s:%2$Mmessageignored
REQ_FUNC DEBUG %1$simsi=%2$s,port=%3$d,REQdatatype=%4$M,
status=%5$T,tag=%6$d,serv_objid=%7$d,clnt_objid=%8$d
EV_FUNC DEBUG %1$s:REQEventdatatype=%2$M,id=%3$T
TIMER DEBUG Timer%1$s(%2$dds)%3$s
SSP_CONNECT_IND DEBUG SCCPConnectRequestPSAP=%1$dPCREF=%2$d
SSP_CONNECT_CFM DEBUG SCCPConnectConfirmPSAP=%1$dPCREF=%2$d
MSG_UNKNOWN DEBUG Unknownmessagereceivedop_type=%1$M
EMPTY_SCCP DEBUG SendingemptySCCPConnection%1$s
SND_MOBILE_ON_CHAN DEBUG %1$Mresult=%2$T
SND_HO_COMMENCED DEBUG %1$smin=%2$s,imsi=%3$s,sender_objid=%4$d,
receiver_objid=%5$d,cause=%6$d
MAP_CAUSE_TO_CDR DEBUG Mappedinputcause=%1$TtoCDRdisconnectcode=%2$d
COLLECT_USAGE DEBUG CollectingusageperformancestatisticforBSC:%1$d
TIMER_WARNING DEBUG Unexpectedtimer%1$d;defaultused(%2$d)
FCAPS_ALARM DEBUG %1$sName:%2$s,Alarmstate:%3$T
CHANGE_ALARM_STATE DEBUG %1$salarm:%2$s
VSMSubsystemEvents Table 71liststheeventsintheVSMsubsystem.
Table 71VSMSubsystemEvents

STA_USAGE_NOT_STARTED WARNING STAUsagenotStartedfor%1$s:Port=%2$d
Trunkgroup=%3$d
LOAD_DB_ERR WARNING database:%1$s:ignoringline%2$d,caughterror:%3$s
ISUP_GROUP_ENTRY_DOES_NOT_EXIST_F WARNING PleaseupdateyourISUPconfigurationforgroup=%1$d..
OR_TRUNKGROUP
INVALID_REQ_FOR_ORIENTATION WARNING port=%1$dorientation=%2$d.
COM_MISSING_PARM WARNING Missing%1$sparameterwhiledecoding%2$smessage
COM_ENC_PARM_NOT_PRESENT WARNING %1$sparameterisnotpresentwhileencoding%2$s
message
COM_FAIL_ENCODE_DIGS WARNING Failedtoencode%1$sdigits(%3$s)in%2$smessage


COM_FAIL_DECODE_DIGS WARNING Failedtodecode%1$sdigitsin%2$smessage
COM_SS7_SEND_FAIL WARNING vsm_com_writefailedtosendout%1$smessage.
COM_IPC_SEND_FAIL WARNING vsm_com_ipc_msg_sendfailedmessage=%1$s
ERROR=%2$s.
COM_INVALID_MSG_TYPE WARNING Invalidmessagecodefoundintheisup_hdr:%1$d.
COM_DEC_PARM_INVALID_VAL WARNING Invalid%1$svalue:%2$dreceivedwhiledecoding%3$s
parameter.
COM_ENC_PARM_INVALID_VAL WARNING Invalid%1$svalue:%2$dreceivedwhileencoding%3$s
parameter.
COM_ENC_FIELD_NOT_PRESENT WARNING %1$sisnotpresent.
ORQ_TYPE_INVALID ERROR Nodescription
RPL_INVALID_PORT ERROR Nodescription
RPL_INVALID_PORT_STATE ERROR Nodescription
RPL_INVALID_SERVICE_STATE ERROR Nodescription
RPL_INVALID_PORT_EVENT ERROR Nodescription
RPL_CORRUPT_PORT_STATE ERROR Nodescription
RPL_INVALID_GROUP ERROR Nodescription
RPL_ABORT ERROR Nodescription
RPL_ABORTED ERROR Nodescription
RPL_RESOURCE_FAULT ERROR Nodescription
RPL_INVALID_TONE ERROR port=%1$dtone=%2$M.
RPL_INVALID_TRUNK_GROUP ERROR Nodescription
RPL_SUCCESS ERROR Nodescription
RPL_SWITCH_FAULT ERROR Nodescription
RPL_SWITCH_TIMEOUT ERROR Nodescription
RPL_PORT_FAULT ERROR port=%1$d
RPL_PORT_TIMEOUT ERROR port=%1$d.
RPL_NOT_STOPPED ERROR Failedtoterminaterequest
RPL_NOT_LISTENING ERROR Portnotconnectedtoannouncement
RPL_ALREADY_ANSWERED ERROR porthasalreadybeenanswered
RPL_PORT_BUSY ERROR outstandingtransactiononportalready
RPL_INVALID_ROUTE ERROR route=%1$d
RPL_INVALID_COMP_SEQ ERROR route=%1$dcompletion_sequence=%2$d.
RPL_ANNOUNCEMENT_NOT_CONFIGURE ERROR VRASannouncement%1$snotconfigured
D
DIG_RECONSTRUCTION_FAIL ERROR digits=%1$stype=%2$s.
TERM_TO_INCOMING_GROUP ERROR group=%1$d.
PORT_NOT_IN_USE ERROR Nodescription
PORT_IN_USE ERROR Nodescription
INVALID_RELEASE_SEQ ERROR sequence=%1$d.


PORT_STATE_REPLY_ERR ERROR Nodescription
SS7_STATUS_OUT_OF_RANGE ERROR Nodescription
ORQ_TYPE_INVALID ERROR Nodescription
ISUP_UNRECOGNIZED_MSG_TYPE ERROR Unrecognizedmessagereceived:code=%1$d.
ISUP_XCP_INVALID_ARGUMENT ERROR Nodescription
ISUP_XCP_MISSING_PARAM ERROR Parameter%1$sismissinginmessage%2$s
ISUP_XCP_UNSUPPORTED_PARAM ERROR Nodescription
RPL_INCORRECT_SYSTEM_STATE ERROR Systemstateisinvalidforrequestedoperation
PORT_DB_ERR ERROR Notrunkgroupdatafoundforport%1$d
INVALID_STATE ERROR port=%1$dstate=%2$T
INVALID_EVENT ERROR port=%1$devent=%2$T
SS7_VARIANT_INVALID ERROR Nodescription
INVALID_REQ ERROR Nodescription
ISP_BIND_FAILED ERROR Bindtosuid=%1$ldfailed
REQ_ERR ERROR %1$s:%2$s=%3$d%4$s=%5$d%6$s=%7$d%8$s=%9$M
INVALID_CP_EVENT ERROR port=%1$dcp_event=%2$M
CPI_CAUSE_BLOCKED ERROR Nodescription
INVALID_PORT ERROR InvalidPortentryforport=%1$d
UNEXPECTED_XCP ERROR UnexpectedException:%1$V
ILLEGAL_ISUP_ASCII_DIGIT ERROR ascii_digit=%1$cindex=%2$d
BAD_IPCD_NAME USERERROR Destinationname%1$sspecifiedingrouptableisinvalid..
CLLI_INVALID_MSCID_PORT USERERROR InvalidMSCID%1$dPORT%2$d
EVT_PORT_ORIGINATION INFO Nodescription
EVT_PORT_ANSWERED INFO Nodescription
EVT_PORT_RELEASING INFO Nodescription
EVT_PORT_RELEASE_FAULT INFO Nodescription
EVT_PORT_TONE_COMPLETE INFO Nodescription
EVT_PORT_PLAY_CUSTOM_ANN_PLAYING INFO Nodescription
EVT_PORT_ANNOUNCEMENT_COMPLETE INFO Nodescription
SIG_FIRST INFO sig_first
SIG_R2_B1 INFO sig_r2_b1


SIG_IDLE INFO sig_idle
SIG_DISCONNECT INFO sig_disconnect
SIG_SENDCAUSE INFO sig_disconnect
SIG_SENDCAUSE_AND_TONE INFO sig_playtone
SIG_LAST INFO sig_disconnect
EVT_MSC_SELECT_TRUNK INFO Nodescription
EVT_MSC_ABORT_SELECT_TRUNK INFO Nodescription
EVT_MSC_SELECT_PORT INFO Nodescription
EVT_MSC_ANSWER_PORT INFO Nodescription
EVT_MSC_IDLE_PORT INFO Nodescription
EVT_MSC_SSC_INS INFO Nodescription
EVT_MSC_SSC_MMB INFO Nodescription
SWITCH_FAULT INFO %1$s
EVT_CONNECT_PORT_PORT INFO Nodescription
EVT_CONNECT_PORT_TONE INFO Nodescription
EVT_CONNECT_PORT_ANN INFO Nodescription
EVT_PLAY_CUSTOM_ANN INFO Nodescription
EVT_DISCONNECT_PORT INFO Nodescription
EVT_DISCONNECT_TONE INFO Nodescription
EVT_DISCONNECT_ANN INFO Nodescription
INVALID_TRUNK_TYPE INFO port=%1$dgroup=%2$d%3$s
COM_DECODE_INFO INFO VSM_COM_DECODE_%1$s_MSG_M:group=%2$d
port=%3$dcic=%4$dinstance_id=%5$d
COM_ENCODE_INFO INFO VSM_COM_ENCODE_%1$s_MSG_M:group=%2$d
port=%3$dcic=%4$dinstance_id=%5$d
COM_SS7_SEND_SUCCEED INFO Successfullysentout%1$smessagetoNMS.
COM_TUP_SS7_SEND_SUCCEED INFO Successfullysentout%1$smessagetoNMS.
COM_IPC_RECV_SUCCEED INFO ReceivedaVSM_COM_IPCmessage.
COM_IPC_SEND_SUCCEED INFO Successfullysentout%1$smessagetoVSM.
COM_IPC_SEND_READY INFO ReadytosendaVSM_COM_IPCmessage.
COM_DECODE_PARM INFO Decodeparameter:%1$s.
COM_ENCODE_PARM INFO Encodeparameter:%1$s.
ISUP_TRACER INFO %1$s
COM_ISUP_INVALID_MSG_TYPE INFO Invalidmessagecode=%1$d.


ISUP_PARAM_FIRST INFO Nodescription
ISUP_PARAM_NAT_OF_CONN INFO NatureofConnection:
ISUP_PARAM_FORWARD_CALL INFO ForwardCall:
ISUP_PARAM_CALLED_PTY INFO CalledParty:
ISUP_PARAM_CALLING_PTY_CAT INFO CallingPartyCategory:
ISUP_PARAM_CALLING_PTY INFO CallingParty:
ISUP_PARAM_CHARGE_NUM INFO ChargeNumber:
ISUP_PARAM_CARRIER_ID INFO CarrierID:
ISUP_PARAM_BACKWARD_CALL INFO BackwardCall:
ISUP_PARAM_BACKWARD_CALL_MORE INFO BackwardCallMore:
ISUP_PARAM_CAUSE INFO Cause:
ISUP_PARAM_GENERIC_ADDR INFO GenericAddress:
ISUP_PARAM_GENERIC_DIGITS INFO GenericDigits:
ISUP_PARAM_RANGE_STATUS INFO RangeandStatus:
ISUP_PARAM_USER_SERV_INFO INFO UserServiceINFO:
ISUP_PARAM_ACCESS_TRANSPORT INFO AccessTransport:
ISUP_PARAM_CARRIER_SELECT INFO CarrierSelect:
ISUP_PARAM_JURISDICTION INFO Jurisdiction:
ISUP_PARAM_ORIG_CALLED_NUM INFO OriginalCalledNum:
ISUP_PARAM_ORIG_LINE_INFO INFO OriginatingLineINFO:
ISUP_PARAM_REDIRECTING_NUM INFO RedirectingNumber:
ISUP_PARAM_REDIRECTION_INFO INFO RedirectionINFO:
ISUP_PARAM_REDIRECTION_NUM INFO RedirectionNumber:
ISUP_PARAM_TRANS_NET_SELECT INFO TransitNetworkSelect:
ISUP_PARAM_OPT_BACKWARD_CALL INFO OptionalBackwardCall:
ISUP_PARAM_AUTO_CONGESTION INFO AutoCongestion:
ISUP_PARAM_CIRCUIT_GRP_SUPV INFO CircuitGroupSupervision:
ISUP_PARAM_CONTINUITY_IND INFO ContinuityIndication:
ISUP_PARAM_OUTG_TRK_GRP_NUM INFO OutgoingTrunkGroupNumber:
ISUP_PARAM_SUS_RES_IND INFO Suspend/ResumeIndication:
ISUP_PARAM_LAST INFO Nodescription
ISUP_DATA_FIRST INFO Nodescription
ISUP_NATURE_OF_CONN_DATA INFO satellite=%1$dcont_check=%2$decho_supp=%3$d.
ISUP_FORWARD_CALL_IND_DATA INFO nat_int=%1$dend2end_meth=%2$dint_work=%3$d
,isup=%4$disup_pref=%5$disdn_access=%6$d
,sccp_method=%7$dtranslated_call=%8$d.
ISUP_CALLED_PTY_NUM_DATA INFO nat_address_ind=%1$dnumbering_plan=%2$d
addr_signal=%3$s.
ISUP_CALLING_PTY_CAT_DATA INFO calling_party_cat=%1$s.


ISUP_CALLING_PTY_NUM_DATA INFO nat_address=%1$dpresentation=%2$d
numbering_plan=%3$dscreen_ind=%4$d
addr_signal=%5$s.
ISUP_CHARGE_NUM_DATA INFO nat_address_ind=%1$dnumbering_plan=%2$d
addr_signal=%3$s.
ISUP_CARRIER_ID_DATA INFO network_id_plan=%1$dtype_of_network=%2$d
ci_digits=%3$s.
ISUP_BACKWARD_CALL_IND_DATA INFO charge_ind=%1$dcalled_status_ind=%2$d
called_pty_cat_ind=%3$dend_2_end_method_ind=%4$d
interworking_ind=%5$d.
ISUP_BACKWARD_CALL_IND_MORE_DATA INFO isup_ind=%1$dholding_ind=%2$disdn_access_ind=%3$d
echo_control_device_ind=%4$dsccp_method_ind=%5$d.
ISUP_CAUSE_DATA INFO location=%1$dcoding=%2$dcause=%3$sdiagnostic=%4$c.
ISUP_GENERIC_ADDRESS_DATA INFO type_of_addr=%1$dnat_address_ind=%2$d
presentation_ind=%3$dnumbering_plan=%4$d
addr_signal=%5$s.
ISUP_GENERIC_DIGITS_DATA INFO type_of_digits=%1$dencoding=%2$ddigits=%3$s.
ISUP_RANGE_STATUS_DATA INFO range=%1$dst[0]=%2$cst[1]=%3$cst[2]=%3$cst[3]=%4$c.
ISUP_USER_SERV_INFO_DATA INFO INFO_tx_cap=%1$dcoding_standard=%2$d
INFO_tx_rate=%3$dtransfer_mode=%4$d
usr_lyr_proto_id=%5$d.
ISUP_ACCESS_TRANSPORT_DATA INFO INFO_element=%1$s.
ISUP_CARRIER_SELECT_DATA INFO INFO_element=%1$d.
ISUP_JURISDICTION_DATA INFO digit1=%1$ddigit2=%2$ddigit3=%3$ddigit4=%4$d
digit5=%5$ddigit6=%6$d.
ISUP_ORIG_CALLED_NUM_DATA INFO nat_of_addr=%1$dpresentation_ind=%2$d
numbering_plan=%3$daddr_signal=%4$s.
ISUP_ORIG_LINE_INFO_DATA INFO line_INFO=%1$d.
ISUP_REDIRECTING_NUM_DATA INFO nat_of_addr=%1$dpresentation_ind=%2$d
numbering_plan=%3$daddr_signal=%4$s.
ISUP_REDIRECTION_INFO_DATA INFO redirect_ind=%1$dorig_redirect_reason=%2$d
redirect_counter=%3$dredirect_reason=%4$d.
ISUP_REDIRECTION_NUM_DATA INFO nat_of_addr=%1$dnumbering_plan=%2$d
addr_signal=%3$s.
ISUP_TRANS_NET_SELECT_DATA INFO network_id_plan=%1$dtype_of_network=%2$d
digits=%3$s.
ISUP_OPT_BACKWARD_CALL_DATA INFO inbnd_INFO_ind=%1$dcfwd_ind=%2$d
usr_net_interact_ind=%3$d.
ISUP_AUTO_CONGESTION_DATA INFO auto_cong_level=%1$d.
ISUP_CIRCUIT_GRP_SUPV INFO indicator=%1$d.
ISUP_CONTINUITY_IND_DATA INFO indicator=%1$d.
ISUP_OUTG_TRK_GRP_NUM_DATA INFO digits=%1$s.
ISUP_SUS_RES_IND_DATA INFO indicator=%1$d.
ISUP_DATA_LAST INFO nodescription


ISUP_CAUSE_CCNOUSRRSP INFO nouserresponse
ISUP_CAUSE_CCUSRBSY INFO userbusy
ISUP_CAUSE_CCCALLRJT INFO callrejected
ISUP_CAUSE_CCNOANSWR INFO noanswer(useralerted)
ISUP_CAUSE_CCUNALLOC INFO unassignednumber
ISUP_CAUSE_CCCALLCLR INFO normalcallclearing
ISUP_CAUSE_CCNORTTOTSFNET INFO noroutetotransitnet
ISUP_CAUSE_CCNORTTODEST INFO noroutetodestination
ISUP_CAUSE_CCNETAOL INFO networkfailure
ISUP_CAUSE_CCTMPFAIL INFO temporaryfailure
ISUP_CAUSE_CCSWTCHCONG INFO switchcongestion
ISUP_CAUSE_CCNOCIRCUIT INFO nocircuits
ISUP_CAUSE_CCRESCUNAVAIL INFO resourceunavailable
ISUP_CAUSE_CCREQUNAVAIL INFO requestedcircuit/channelunconfigured
ISUP_CAUSE_CCMISDIALDTRNK INFO misdialedtrunkprefix
ISUP_CAUSE_CCADDRINCOMP INFO addressincomplete
ISUP_CAUSE_CCINVALCALLREF INFO invalidcallreference
ISUP_CAUSE_CCDESTOUTORD INFO destinationoutoforder
ISUP_CAUSE_CCPROTERR INFO protocolERROR
ISUP_CAUSE_CCINVALPARAMCONT INFO invalidparametercount
ISUP_CAUSE_CCNORMUNPEC INFO normalunspecified
ISUP_CAUSE_CCINTRWRK INFO interworking,unspecified
ISUP_CAUSE_CCMISRTPRTNUM INFO misroutedcalltoportednumber
ISUP_CAUSE_NO_TEXT INFO noisupcausedescription
STATE_LOADING_TABLES INFO loading.
STATE_CONFIGURING_SWITCH INFO Beginingswitchconfiguration.
STATE_ENABLING_PORTS INFO SwitchConfigurationComplete.
STATE_GOING_ACTIVE INFO waitingforportstocomeintoservice.
STATE_ACTIVE INFO SwitchManagerreadyforcallprocessing.
SS7_CND_NOEVENT INFO Portnoeventoninserviceattempt.
SS7_CND_TIMEOUT INFO Porttimeoutduringinserviceattempt.
SS7_CND_INSERVICE INFO Portinservicesuccessful.
SS7_CND_OOS_MM INFO Portoosmm.
SS7_CND_PENDING INFO Portinservicepending.
SS7_LINK_FAIL INFO SS7linkfailed.
SS7_WAIT_FOR_LINK_UP INFO WaitingforSS7linkupforsending.
SS7_GROUP_DATA_ERR INFO Badgroupdataforsendgroupmessages.
MMI_REQ INFO Nodescription
INTERNAL_REQ INFO Nodescription


EVT_PORT_FLASHED INFO Nodescription
INCOMING_CALL INFO Nodescription
OUTGOING_CALL INFO Nodescription
INS_REQ INFO Nodescription
OOS_REQ INFO Nodescription
RSC_REQ INFO Nodescription
OOS_SYS_REQ INFO Nodescription
LOAD_DB INFO Database(%1$s)loaded.
UNKNOWN_DB INFO Requesttoloadunknowndatabase(%1$s).
INVALID_KEY INFO Noentryinframetableforthiskey.
ASSOC_RECOVER INFO Function=%1$smodule=%2$dtbl=%3$snode=%4$d
key=%5$d.
ASSOC_ATTACH INFO function=%1$s.
ASSOC_INFO INFO caughtexceptionduring%1$shandling:.\t%2$hV:.\t%2$V
ASSOC_TRC INFO function=%1$s.
RESET_TRC INFO function=%1$snode=%2$ddelete_associd=%3$d.
NO_BRANCH_ASSOCID INFO Nobranchassociationinthe%1$sprocedure.
BASSOCID_TRC INFO function=%1$sport=%2$dbassocid=%3$d.
BASSOCID_SET_DESTROY_TRC INFO function=%1$sport=%2$dbassocid=%3$d.
BASSOCID_SET_DESTROY INFO Branchassociation=%1$dissettodestroyonport=%2$d.
BASSOCID_CREATED INFO Branchassociation=%1$dcreatedfortheport=%2$d.
BASSOCID_ADDED INFO Branchassociation=%1$daddedtotheport=%2$d.
UNEXPECTED_BASSOCID INFO %1$sport=%2$dbassocid=%3$d,oldvalue=%4$d.
ATTACH_TRCVAR INFO Unexpectedcalltoattachthetracevariable..
INVALID_SIG INFO Nodescription
GET_CFG INFO Function=%1$s,group=%2$d,cic=%3$d.
FUNC_TRC INFO %1$s:%2$s=%3$d,%4$s=%5$d,%6$s=%7$d,%8$s=%9$d.
FUNC_END_TRC INFO %1$s:END%2$s=%3$d,%4$s=%5$d,%6$s=%7$d,
%8$s=%9$d.
ISUP_CAUSE_CCNOPARAMDISC INFO parameternonexistentornotimplementeddiscarded.
ISUP_CAUSE_CCNOMSGTYP INFO messagetypenonexistentornotimplemented.
ISUP_CAUSE_CCNOPARAMPASS INFO parameternonexistentornotimplementedpassedon.
ISUP_CAUSE_CCUNRECPARMDISC INFO messagewithunrecognizedparameterdiscarded.
ISUP_GM_MTPRESUME INFO SendingResumeeventfortotalnumberofports:%1$d.
PMG_COPY_MSG_KEY INFO msg_key=%1$dport=%2$dcp_event=%3$M.
ORQ_ABORT INFO %1$s%2$s=%3$d%4$s=%5$d%6$s=%7$d.
ORQ_ABORTED INFO %1$s%2$s=%3$d%4$s=%5$d%6$s=%7$d.
FUNC_TRC_STR INFO %1$s%2$s=%3$d%4$s=%5$d%6$s=%7$s%8$s=%9$s.
ISUP_STORE_MSG_MULTIPLE_DEL INFO port=%1$diam_key=%2$dacm_key=%3$danm_key=%4$d
con_key=%5$drel_key=%6$d.


PORT_SELECTION_ALGORITHM INFO trunk_group=%1$dselection=%2$s.
FOUND_PORT INFO Foundport=%1$dcallnum=%2$dcontrol_port_num=%3$d
non_control_port_num=%4$d.
TCCS_T5_EXPIRED INFO TCCST5expiredwaitingforRLCport=%1$d.
ISP_SET_OLIP_DONE INFO port=%1$dpres=%2$dvalue=%3$d.
ISP_SET_CHARGE_NUM INFO port=%1$dpres=%2$d.
ISP_SET_CHARGE_NUM_DONE INFO port=%1$dpres=%2$dnai=%3$Tnum_plan=%4$T
odd_even=%5$ddigits=%6$s.
ISP_SET_CHARGE_NUM_OLIP INFO port=%1$dcharge_num_pres=%2$dolip_pres=%3$d
A_buffer=%4$sM_buffer=%5$sI_buffer=%6$s.
ISP_SET_CHARGE_NUM_OLIP_DONE INFO port=%1$dcharge_num_pres=%2$dolip_pres=%3$d.
ISP_SET_TRANSIT_NW_SELECTION_DONE INFO port=%1$dpres=%2$dtype_of_nw_id=%3$T
nw_id_plan=%4$Tdigits=%5$sodd_even=%6$d.
SND_ALM_INITIAL_READY INFO num_ports=%1$dport_index=%2$d.
SND_ALM_INITIAL_READY_PROGRESS INFO Sent%1$dinitialreadys,num_ports=%2$d
port_index=%3$dremaining=%4$d.
SND_ALM_INITIAL_READY_DONE INFO Totalportsconfigured=%1$d.
ALM_THROTTLE_TIMER_STARTED INFO tid=%1$d.
STOP_CP_USAGE INFO Port=%1$dTrunkgroup=%2$dFlags:incoming=%3$d
incoming_effective=%4$doutgoing=%5$d
outgoing_effective=%6$d.
STA_USAGE_START INFO STAUsageStartingfor%1$s:port=%2$dtrunkgroup=%3$d.
STA_USAGE_STOP INFO STAUsageStoppingfor%1$s:port=%2$dtrunkgroup=%3$d.
STA_USAGE_ALREADY_STARTED INFO STAUsageAlreadyStartedfor%1$s:Port=%2$d
Trunkgroup=%3$d.
ALM_REPORT INFO Reportingalarm:level=%1$sport=%2$dtrunkgroup=%3$d
threshold=%4$d.
ISUP_GM_FSM INFO group=%1$devent=%2$Tgm_state=%3$Tcounter=%4$d
mmi_request_count=%5$dRcvd_mmi_sync_end=%6$d
port=%7$dtotal_num_ports=%8$dorientation=%9$T..
ISUP_GM_FSM_DONE INFO group=%1$devent=%2$Tgm_state=%3$Tcounter=%4$d
mmi_request_count=%5$dRcvd_mmi_sync_end=%6$d
orientation=%7$T
ISUP_GM_SEND_INFO INFO group=%1$dg>counter=%2$dg>start_cic=%3$d
g>end_cic=%4$dorientation=%5$T.
UPDATE_FRAME_FIRST_ENTRY INFO Firstentryport=%1$dgroup=%2$dmsg_code=%3$s
cic=%4$dinstance_id=%5$dorientation=%6$T.
UPDATE_FRAME_LOOP INFO Keyalreadyexistsgroup=%1$drequest=%2$Mcic=%3$d
port=%4$dorientation=%5$T.
ISP_SET_GENERIC_DIGITS INFO generic_digits=%1$s%2$s=%3$d%4$s=%5$d.
R2_SS_FSM INFO port=%1$devent=%2$Tstate_in=%3$T.
R2_SS_FSM_DONE INFO port=%1$devent=%2$Tstate_out=%3$T.
R2_CP_FSM INFO port=%1$devent=%2$Tstate=%3$Tsrv_state=%4$T.


R2_CP_FSM_DONE INFO port=%1$devent=%2$Tnew_state=%3$Tsrv_state=%4$T.
MAP_CAS_TO_CPI_CAUSE INFO cas_cause=%1$Tcpi_cause=%2$M.
MAP_CPI_TO_CAS_CAUSE INFO cpi_cause=%1$Mcas_cause=%2$T.
CLEAR_REQ_TAG_ERR INFO tag_count=%1$d
FCAP_PEG DEBUG FCAPpegINFO:TrunkType=%1$TTrunk=%2$dCause=%3$M
Direction=%4$T
SSFSubsystemEvents Table 72liststheeventsintheSSFsubsystem.
Table 72SSFSubsystemEvents

DB_RESOURCE_LIMIT FATAL InternalFSMResourceLimitReached:%1$s
INVALID_SUID ERROR %1$s:InvalidSSFSAPid(%2$d)
INVALID_TYPE ERROR %1$s:item=%2$T
RESOURCE_LIMIT ERROR InternalResourcelimitreached
SRC_OCTS_MISSING ERROR ASNoctscopyfailed:sourceoctsmissing.
UNEXPECTED_XCP ERROR Unexpectedexceptionincallto%1$s>%2$V%3$s
UNKNOWN_EXTENSION ERROR %1$sunknownextensionparametertype:%2$d
NULL_DLG_POINTER ERROR TryingtoaccessadlgwithNULLpointer
EVT_CALL_PROGRESS ERROR evt_call_progress
USR_ERR ERROR %1$s
UNKNOWN_EXTENSION ERROR %1$sunknownextensionparametertype:%2$d
NULL_POINTER ERROR %1$s:unexpectednullpointer.
LOAD_DB_ERR USERERROR Database:%1$s:ignoringline%2$d,caughterror:%3$s
ALREADY_ATTACHED WARNING Attachignored.
NAP_STATUS WARNING ReceivedNAPstatusIndication,USAP=%1$d,status:%2$M:
%3$V
ATTACH INFO Nodescription
ETRACE INFO %3$s%1$s:Messageevent%2$sreceived
FUNC_TRC INFO SSF:function=%1$s
ASSOC_TRC INFO function=%1$s
ASSOC_INFO INFO caughtexceptionduring%1$shandling:\t%2$hV:\t%2$V
ASSOC_RECOVER INFO function=%1$smodule=%2$dtbl=%3$snode=%4$d
key=%5$d
PS_ALL_SEND_NAP_MSG DEBUG Send%1$smessageto%2$s[%3$s]
CLOSE_IND DEBUG %5$s%1$s:ssf_sap=%2$d,su_dlg_id=%3$d,
sp_dlg_id=%4$d
DB_CS DEBUG %1$s:%2$s=%3$d%4$s=%5$d%6$s=%7$d%8$s=%9$d
DB_CSA DEBUG %1$s:%2$s=%3$d%4$s=%5$d%6$s=%7$d%8$s=%9$d
DB_SRF DEBUG %1$s:%2$s=%3$d%4$s=%5$d%6$s=%7$d%8$s=%9$d


DB_EVENT DEBUG %1$s:%2$s=%3$d%4$s=%5$d%6$s=%7$d%8$s=%9$d
DB_GLOBALS DEBUG %1$s:%2$s=%3$d%4$s=%5$d%6$s=%7$d%8$s=%9$d
DB_IH DEBUG %1$s:%2$s=%3$d%4$s=%5$d%6$s=%7$d%8$s=%9$d
DB_FIM DEBUG %1$s:%2$s=%3$d%4$s=%5$d%6$s=%7$d%8$s=%9$d
DB_O_BCSM DEBUG %1$s:%2$s=%3$d%4$s=%5$d%6$s=%7$d%8$s=%9$d
DB_OFFICE DEBUG %1$s:%2$s=%3$d%4$s=%5$d%6$s=%7$d%8$s=%9$d
DB_SSF DEBUG %1$s:%2$s=%3$d%4$s=%5$d%6$s=%7$d%8$s=%9$d
DB_SSME DEBUG %1$s:%2$s=%3$d%4$s=%5$d%6$s=%7$d%8$s=%9$d
DB_T_BCSM DEBUG %1$s:%2$s=%3$d%4$s=%5$d%6$s=%7$d%8$s=%9$d
DB_TDP DEBUG %1$s:%2$s=%3$d%4$s=%5$d%6$s=%7$d%8$s=%9$d
DB_TIMERS DEBUG %1$s:%2$s=%3$d%4$s=%5$d%6$s=%7$d%8$s=%9$d
DB_TRIGGER DEBUG %1$s:%2$s=%3$d%4$s=%5$d%6$s=%7$d%8$s=%9$d
DELIM_IND DEBUG %5$s%1$s:ssf_sap=%2$d,su_dlg_id=%3$d,
sp_dlg_id=%4$d
DEST_ADDR DEBUG %1$s:digits=%2$s,rte=%3$d,ssn=%4$d,transtype=%5$d,
numplan=%6$d,nataddr=%7$d,globaltitlelen=%8$d
DLG_CNF DEBUG %5$s%1$s:ssf_sap=%2$d,su_dlg_id=%3$d,
sp_dlg_id=%4$d
DLG_DID DEBUG %1$s:did=%2$d
EV DEBUG %1$s:FSMid=%2$d,event=%3$T
FN_CS DEBUG %1$s:%2$s=%3$d%4$s=%5$d%6$s=%7$d%8$s=%9$d
FN_CSA DEBUG %1$s:%2$s=%3$d%4$s=%5$d%6$s=%7$d%8$s=%9$d
FN_IH DEBUG %1$s:%2$s=%3$d%4$s=%5$d%6$s=%7$d%8$s=%9$d
FN_FIM DEBUG %1$s:%2$s=%3$d%4$s=%5$d%6$s=%7$d%8$s=%9$d
FN_O_BCSM DEBUG %1$s:%2$s=%3$d%4$s=%5$d%6$s=%7$d%8$s=%9$d
FN_SSF DEBUG %1$s:%2$s=%3$d%4$s=%5$d%6$s=%7$d%8$s=%9$d
FN_SSF_DP_ARMED DEBUG %1$s:%2$s=%3$d%4$s=%5$d%6$s=%7$T%8$s=%9$d
FN_SSF_EVENT_TYPE DEBUG %1$s:%2$s=%3$T%4$s=%5$T%6$s=%7$d%8$s=%9$d
FN_SSME DEBUG %1$s:%2$s=%3$d%4$s=%5$d%6$s=%7$d%8$s=%9$d
FN_T_BCSM DEBUG %1$s:%2$s=%3$d%4$s=%5$d%6$s=%7$d%8$s=%9$d
FTRACE DEBUG %1$s:Functioncalled
IBI DEBUG %1$s:party=%2$dxnid=%3$dcsa_id=%4$dLegId=leg%5$c
IBI_EVT DEBUG %1$s:csa_id=%2$dleg_id=leg%3$ccall_flag=%4$T
MSG DEBUG %1$s:Result=%2$M
NO_SM DEBUG %1$s:StateMachinefordid=%2$disalreadydestroyed,NAP
message:%3$sisignored
NO_SM_USR_CNF DEBUG %1$s:StateMachinefordid=%2$disalreadydestroyed,
MAPmessage:NAP_%3$T_CNFisignored
NO_SM_USR_IND DEBUG %1$s:StateMachinefordid=%2$disalreadydestroyed,NAP
message:NAP_%3$T_INDisignored


NOTICE_IND DEBUG %6$s%1$s:ssf_sap=%2$d,su_dlg_id=%3$d,
sp_dlg_id=%4$d,Cause=%5$T
OPCODE_TRAN DEBUG opcode=%1$Tcsa_id=%2$d
OPEN_CNF DEBUG %8$s%1$s:ssf_sap=%2$d,su_dlg_id=%3$d,
sp_dlg_id=%4$d,AC=%5$T,Result=%6$T,Reason=%7$T
OPEN_REFUSED DEBUG %1$s:ssf_sap=%2$d,su_dlg_id=%3$d,sp_dlg_id=%4$d,
AC=%5$T,Result=%6$T,Reason=%7$T
OPEN_REQ DEBUG %5$s%1$s:ssf_sap=%2$d,sp_dlg_id=%3$d,AC=%4$T
ORIG_ADDR DEBUG %1$s:OriginatingReferenceData:nataddr=%2$2xnum
plan=%3$2xdigits=%4$sorig_ref=%6$.*5$S
P_ABORT DEBUG NAP_P_ABORTProviderReason=%1$lT,Source=%2$lT
PABORT_IND DEBUG %7$s%1$s:ssf_sap=%2$d,su_dlg_id=%3$d,
sp_dlg_id=%4$d,Reason=%5$T,Source=%6$T
RESULT DEBUG %1$s:Result=%2$d(0x%2$x)
SIGCON DEBUG %1$s:party=%2$dxnid=%3$d
SIGCON_EVT DEBUG %1$s:party=%2$dxnid=%3$dcsa_id=%4$d
SM_CS DEBUG %1$s:%2$s=%3$d%4$s=%5$s%6$s=%7$T%8$s=%9$d
SM_CSA DEBUG %1$s:%2$s=%3$d%4$s=%5$s%6$s=%7$T%8$s=%9$d
SM_IH DEBUG %1$s:%2$s=%3$d%4$s=%5$s%6$s=%7$T%8$s=%9$d
SM_FIM DEBUG %1$s:%2$s=%3$d%4$s=%5$s%6$s=%7$T%8$s=%9$d
SM_O_BCSM DEBUG %1$s:%2$s=%3$d%4$s=%5$s%6$s=%7$T%8$s=%9$d
SM_SSF DEBUG %1$s:%2$s=%3$d%4$s=%5$s%6$s=%7$T%8$s=%9$d
SM_SSME DEBUG %1$s:%2$s=%3$d%4$s=%5$s%6$s=%7$T%8$s=%9$d
SM_T_BCSM DEBUG %1$s:%2$s=%3$d%4$s=%5$s%6$s=%7$T%8$s=%9$d
SM_CS_ERROR DEBUG %1$s:%2$s
SRV_CNF DEBUG %9$s%1$s:ssf_sap=%2$d,su_dlg_id=%3$d,
sp_dlg_id=%4$d,Operation=%5$T,ack=%6$#x,uerr=%7$#x,
perr=%8$#x
SRV_IND DEBUG %6$s%1$s:ssf_sap=%2$d,su_dlg_id=%3$d,
sp_dlg_id=%4$d,Operation=%5$T
TDP DEBUG %1$striggerarmedcsa_id=%2$dcs_id=%3$d
dp_specific_criteria=%4$devent_type_bcsm=%5$T
leg_id=%6$dmonitor_mode=%7$dnon_in=%8$d
TEST DEBUG %1$s
TEST_ASP_FUNC DEBUG %1$s:party=%2$dxnid=%3$d
TEST_ASP_GET DEBUG %1$s:call_no=%2$dhalf_call=%3$d
TYP DEBUG %1$s:Result=%2$T
U_ABORT DEBUG NAP_U_ABORTUserReason=%1$lT
UABORT_IND DEBUG %7$s%1$s:ssf_sap=%2$d,su_dlg_id=%3$d,
sp_dlg_id=%4$d,Reason=%5$T,DiagINFO=%6$T
UABORT_IND_ALT DEBUG %6$s%1$s:ssf_sap=%2$d,su_dlg_id=%3$d,
sp_dlg_id=%4$d,Reason=%5$T


DIAGNOSTICTOOLS
C
AboutThisAppendix DiagnostictoolsassociatedwithsoftwaremodulesontheNEsarelistedinthis
appendix.Thisappendixisprovidedforreferencepurposesonly.Thesetools
shouldonlyberunundertheguidanceofStarSolutionstechnicalsupportstaff.
Warning:
ThesediagnosticsshouldonlyberunwhenindepthanalysisisrequiredbyStar
Solutionscustomersupportandshouldnotberunatothertimessincetheymay
haveanimpactonsystemresources.
DiagnosticTools Thereareanumberofdiagnostictoolsassociatedwithsoftwaremodulesonthe
NEs.ThediagnosticsaregroupedintoSystem,CallProcessing,MediaControl,
RadioResource,IS41MSCTrunkManagement,IS41MobilityManagement,and
GSMMobilityManagementcategories.
Thissectiondescribes:
SystemDiagnostics
ALMDiagnostics
CDRDiagnostics
COMDiagnostics
EVLDiagnostics
GSSDiagnostics
STADiagnostics
TPMDiagnostics
CallProcessingDiagnostics
ASPDiagnostics
AUADiagnostics
AURDiagnostics
AUSDiagnostics
LFMDiagnostics
PPXDiagnostics
SSFDiagnostics
SSPDiagnostics
VSMDiagnostics
MediaControlDiagnostics
MEGDiagnostics
MEGRESDiagnostics
MEGTSADiagnostics
AUEDiagnostics
RadioResourceDiagnostics
BSMDiagnostics
CSIDiagnostics
RCMDiagnostics
RCIDiagnostics
GAPDiagnostics
IS41MSCTrunkManagement
IMPDiagnostics
MCIDiagnostics
MCIDiagnostics
ISIDiagnostics
ISIDiagnostics
IS41MobilityManagement
VLRDiagnostics
MIPDiagnostics
GSMMobilityManagement
MCGDiagnostics
111INSMAUDiagnostics
Thereareotherdiagnostictoolsinthetoolsdirectory;however,thesetools
providenoextrainformationthatcannotbeobtainedfromlookingatthetables
usingtheOMCConsoleorCLI.Also,sometoolsareforlabuseonlyandhave
severeimpactsonresources.
Warning:
Donotusectrlctostopacommandthatistakingtoolong.Thiscan
sometimesleavetableslocked.Letthecommandruntocompletion.
SystemDiagnostics
ALMDiagnostics
TheAlarmManager(ALM)subsystemimplementseventbasedalarmsinthe
MSC.Itcollectsalarmsfromapplicationsandforwardsalarmsfromother
NetworkElementstotheOMC.
ThebestwaytomonitoralarmsistousetheOMCConsole.(RefertotheOMC
UserInterfaceGuide.)
Table 73showstheoptionsforthealm_dumpcommand.
Table 73CommandOptionsforalm_dump
Option Description
A Displayall
a Displayactive(default)
g Displayconfig
v Displayverbose
k Displaycli
SeeExample80forsampleoutputofthecommand.
Example80alm_dumpCommandOutput
msc>alm_dumpA
ALMCONFIGTABLE
log_num_days=1oam_ipc_id=ID=0
CDRDiagnostics
DetailedCDRinformationisbestobtainedbyusingthecdr_reportertool.Refer
totheViewingandManagingCallDetailRecordschapteroftheMSCOperations,
Administration,andMaintenanceGuide.
Table 74showstheoptionsforthecdr_dumpcommand.
Table 74CommandOptionsforcdr_dump
Option Description
a Displayalltables
f Displayfiletable
N Displaymsctable(default)
P Displayparameterstable
u Displayinusefile(s)
SeeExample81forasampleoutputofthecommand.
Example81cdr_dumpCommandOutput
msc>cdr_dumpa
MSC_NODE=1NAME:CDR000002005752NUM=112FILL_STATE=1
XN_STATE=4RECORDS=83406START=115487631END=115571036
WRFD=1RDFD=1TIME=ThuOct2707:59:552011
MSC_NODE=1NAME:CDR000002005753NUM=113FILL_STATE=1
XN_STATE=4RECORDS=83378START=115571037END=115654414
WRFD=1RDFD=1TIME=ThuOct2708:59:552011
...
msc>cdr_dumpP
**************PARAMETERSTABLE************************
CON_HOST=0LOG_FLUSH=5XN_FLUSH=5
POLL_SEND=15FILE_INTERVAL=60MAX_FILES_KEPT=96
PRINT_FORMAT=0
TRANSACT_TO=2LOCAL_MSC=2
ROLLOVER_TIME=0B_CELL_ID=1
OUT_VERSION=14
HANDOFF=0EVENT=0
COMDiagnostics
TheCOMDprocessrunsunderSCCPLite.Itshowsprovisionedinformationfor
lowlevellinks:SCCPLite,IPaccesslinkstotheBSC,andMTPlinks.M3UAand
MTPlinksarebestmonitoredusingtheOMCConsole.RefertotheSignaling
ServerGuide.
Table 75showstheoptionsforthecom_dumpcommand.
Table 75CommandOptionsforcom_dump
Option Description
a Displayalltables
f Displaydescriptortable
l Displaylinktable(default)
Example82com_dumpCommandOutput
msc>com_dumpa
COMLINKTABLE
LINK_DESCLINK_TYPEREAD_FUNCAPP_FUNCD1D2D3D4
=======================================================================
1COM_LINK_WJT_CAP_M0xff312c080xff35c5f40000
2COM_LINK_SIG_CAP_M0xff2f2bfc0xff35ccc00000
3COM_LINK_WJT_CAP_M0xff312c080xff35c5f40000
COMFDTABLE
LINK_DESCFDFD_TYPE
=======================================================================
115COM_FD_READ_M
EVLDiagnostics
Theeventloggermodule(EVL)isresponsibleforprintingeventsfromother
systemmodules.
Theeventloggercollectseventsfromtheapplicationsandwritesthemtoafile.It
isusedmainlyforbasicsysteminformationsuchasapplicationsstartingupor
stopping,aswellassomeerrormessages.Usethevlogtooltoviewtheevents.
(RefertoChapter 13,ViewingLogsandEvents.)
Table 76CommandOptionsforevl_dump
Option Description
a All
Table 76CommandOptionsforevl_dump
Option Description
N Notifytable(default)
Example83evl_dumpCommandOutput
msc>evl_dump
EVLNOTIFYTABLE
MSG_IDIPC_IDTYPE
================
CPU_CM_STARTUP_MEVL_SBINARY_TYPE
IPC_TCP_HOST_DOWN_MVSMBINARY_TYPE
IPC_TCP_HOST_UP_MVSMBINARY_TYPE
IPC_TCP_HOST_DOWN_MASPBINARY_TYPE
IPC_TCP_HOST_UP_MASPBINARY_TYPE
GSSDiagnostics
Thegss_dumptooldisplaysinformationfromtheGSSSAPtable.
Table 77showstheoptionsforthegss_dumpcommand.
Table 77CommandOptionsforgss_dump
Option Description
s DisplaySAPtable(default)
SeeExample84forasampleoutputofthecommandofthecommand.
Example84gss_dumpCommandOutput
msc>gss_dump
010)GSSSAP=11DEST_SAP=11INDEF=FSWTYPE=GSM_NA
STADiagnostics
Thesta_dumptooldisplaysstatisticsparametersandinformationfromteh
staconfigtable.
Table 78showstheoptionsforthesta_dumpcommand.
Table 78CommandOptionsforsta_dump
Option Description
A Displayalldata
G Displaystatisticsdata
P Displayparameterstable
Example85sta_dumpCommandOutput
msc>sta_dumpA
key105:name(cpgroup:group=1_vsmTrkGrpUsageOutgoing)type(4)
clear(0)data(6809)start(20)der(0)
key106:name(cpgroup:group=16_vsmTrkGrpICComplete)type(1)clear(1)
data(1063)start(0)der(0)
key107:name(gsmbsc:bsc=3_pageRsp2BSC)type(1)clear(1)data(20)
start(0)der(0)
key108:name(_succInterVLRLocationUpdates)type(1)clear(1)data(0)
start(0)der(0)
key109:name(cpgroup:group=40_vsmTrkGrpUsageOutOfService)type(4)
key110:name(_vlrMaxSubActive)type(5)clear(1)data(25000)start(0)
der(0)
key111:name(gsmbsc:bsc=3_terminationUsageBSC)type(4)clear(0)
data(8478)start(27)der(0)
key112:name(cpgroup:group=1_vsmTrkGrpUsageOutOfService)type(4)
total113statisticsrecords
Collection:initialized(1),periodstart(ThuOct2711:05:002011),
periodid(240)
STAPARAMETERTABLE
PARAM::m_interval=5,file_period=10,output_format=1
STAPARAMETERTABLE
PARAM::m_interval=5,file_period=60,output_format=1
TPMDiagnostics
Table 79showstheoptionsforthetpm_dumpcommand.
Table 79CommandOptionsfortpm_dump
Option Description
a Displayalltables
s Displaystatistics
t Displaytimers(default)
v Displaytimerswithdatafield
Example86tpm_dumpCommandOutput
msc>tpm_dumps
Inuse3071
SmallStarts577899908
Largestarts839044868
Stops1366986284
Expires52532291
Ticks1331501
Maxsmalltimerdelay170
Maxlargetimerdelay678400
Maxinuse5641
Avgsmalltimerdelay1
Avglargetimerdelay1
CallProcessingDiagnostics
ASPDiagnostics
TheASPsubsystemispartofCallProcessing.
Usetheasp_dumpcommandtoreportthenumberofcallscurrentlybeing
handledbythesystem,andtheoriginatingandreceivingpartydetails.asp_dump
canrevealtrafficpatternsandnetworkusage.
Table 80showstheoptionsfortheasp_dumpcommand.
Table 80CommandOptionsforasp_dump
Option Description
a Displayallloadeddatabasetables
B Displaybillingrecords
C Displayallcallbuffers(default)
D Displayallpartybufferswithmoredetail
e Displayanilookuptable
f Displaynumberoffreebuffers
g Displaycardlookuptable
G Displayglobalstable
j Displayprepaidpartytables
l Displaycallprogresstable
O Displayclassofservicetable
P Displaypartybuffers
r Displaytranslationtable
s Displaysystemfeaturetable
t Displaytreatmentstable
u Displaybearercapabilitypatterntable
v DisplayBearercapabilityrecontable
x Displaysystemfeaturetimerstable
z Displayprotocolinterworktable
tags... Displaylistofobjects(calltakesprecedenceoverparty)
Example87asp_dumpCommandOutput
msc>asp_dump
PARTIES:2CALL:2911CONNECTION:2911
Party:3045Ti:0ID:ASP_PARTY_STATE_WAIT_FOR_ANSWER_M
Party:3340Ti:0ID:160455500990ASP_PARTY_STATE_ALERTING_M
Party:3082Ti:8ID:123010000000208ASP_PARTY_STATE_WAIT_FOR_ANSWER_M
AUADiagnostics
TheAUAsubsystemispartofCallProcessing,andisresponsibleforestablishing
andmaintainingconnectionswithmultiplemediagateways.
Usetheaua_dumpcommandtodisplaythestatusofeachMediaGatewaycardin
thechassisoftheIntelligentMediaGateway(IMG).
Table 81showstheoptionsfortheaua_dumpcommand.
Table 81CommandOptionsforaua_dump
Option Description
d Displayboardtable
Example88aua_dumpCommandOutput
msc>aua_dumpd
AUABOARDTABLE
TypeIDStateHandleIP SubnetMask Gateway Trans
==================================================================
14 1 OPEN 0 172.25.2.58 255.255.255.0 172.25.2.1 TCP
AURDiagnostics
TheAURsubsystemisapartofCallProcessingthathelpsimplementCAStrunk
setupbetweentheMSCServerandtheMediaGateway(MGW)functiononthe
IntelligentMediaGateway.
Table 82showstheoptionsfortheaur_dumpcommand.
Table 82CommandOptionsforaur_dump
Option Description
a Displayalltables
C Displaycalltable
d Displaycausetable
k Displayclg_pty_cattable
g Displaygrouptable
P Displayporttable
Example89aur_dumpCommandOutput
msc>aur_dumpa
AURPORTTABLE
================
PORTGROUPSRV_STATECODECSTATE
AURCALLTABLE
=============
PORTCALL_NOSESSION_IDORIG
AURGROUPTABLE
================
GROUPTYPECOS
AURCALLINGPARTYCATEGORYTABLE
================================
GROUPKAKDDIRCLGPTYCAT
AUSDiagnostics
AUSispartoftheAUAprocess.ItisresponsibleforprocessingPRIsignalingon
theAudiocodesmediagatewaysviatheproprietaryTPNCPprotocol.
Table 83showstheoptionsfortheaus_dumpcommand.
Table 83CommandOptionsforaus_dump
Option Description
a Alltables
P Porttable(default)
C Calltable
g Grouptable
Example90aus_dumpCommandOutput
msc>aus_dump
AUSPORTTABLE
================
PORTGROUPSRV_STATECODECSTATE
1000010HW_OOS1IDLE
1000110HW_OOS1IDLE
1000210HW_OOS1IDLE
1000310HW_OOS1IDLE
1000410HW_OOS1IDLE
1000510HW_OOS1IDLE
1000610HW_OOS1IDLE
1000710HW_OOS1IDLE
1000810HW_OOS1IDLE
LFMDiagnostics
TheLogicalFacilityManager(LFM)subsystemispartofCallProcessing.Usethe
lfm_dumpcommandtoreportthecurrentstatusoffacilities(includingportsand
connections).
Table 84showstheoptionsforthelfm_dumpcommand.
Table 84CommandOptionsforlfm_dump
Option Description
d[portnumber...] Displayportbuffersforaspecificport(s)
C[A] Displayconnectionbuffer[includeidlebuffers]
g[w] Displaydialoguetable[widedisplay]
G Displaytimerstable
L Displayglobalstable
P Displayportbuffer(default)
Example91lfm_dumpCommandOutput
msc>lfm_dump
udid=8947pdid=41124port=73017party=1ti=8
msc>lfm_dumpd73017
==============STARTPort73017StatusDump============
ObjectInformation:
port=73017ti=8party=4796serv_tag=0
clnt_tag=254445006
StateInformation:
currentstate=LFM_PS_CONNECTED_Mnextstate=LFM_PS_SELECTED_M
replycause=DNA_UNDEFINED_M
portfaulted=NOmobilereleased=NOabortingpage=NO
current_request=0call_progress_pending=NO
pending_request=0
connection_event_pending=NOpending_event=0
treating_orig_reply=NO
PortData:
port_type=LFM_MOBILE_M
trunk_group=1trunk_type=0
cell=0imsi=
TreatmentData:
class=0cause=Id(0)signal=VSM_SIG_IDLE_M
tone=Id(0)cycles=0wait_tone=Id(0)
==============ENDPort73017StatusDump============
PPXDiagnostics
Table 85showstheoptionsfortheppx_dumpcommand.Thisisfortheoriginal
Prepaidfeaturewhichhasnowbeendiscontinued.
Table 85CommandOptionsforppx_dump
Option Description
a Displayalltables.
s Displaystatisticstable.
t Displaytransactiontable.
Example92ppx_dumpCommandOutput
msc>ppx_dumpa
TransactionIdSvIdClIdTypeOperationState
Total:0transactions
===========SENT======================RCVD===========
RQRSABEVINRQRSABEVIN
05400000000000000000000000000000000000000000000000000
SSFDiagnostics
TheServiceSwitchingFunction(SSF)ispartofCallProcessing.Someofthe
optionsonthisdumpcommandsimplydisplaythecontentsoftables.This
informationcanbeobtainedfromtheCLIorOMCconsole.
Table 86showstheoptionsforthessf_dumpcommand.
Table 86CommandOptionsforssf_dump
Option Description
a Displayallloadeddatabasetablesandshowstatistics
A Displaysummary
d DisplayTBCSMstatistics
e DisplaySSMEstatistics
f DisplaySSFstatistics
g Displaydialoguetable
h Displayinterfacehandler
l Displaycallsegmentassociationstatistics
L Displaydetailinfoofthespecifiednodenuminsatable
r DisplayOBCSMstatistics
s Displaycallsegmentstatistics
t Displaycontentsofofficetriggersandssftimertables
Example93ssf_dumpCommandOutput
msc>ssf_dumpd
TBCSMCSFSMStatePartyTI
357545041246085PIC_T_ALERTING47410
400642941246384PIC_T_ALERTING47890
409543581246445PIC_T_ALERTING48160
415748491245880PIC_T_ACTIVE41400
418443371246053PIC_T_ACTIVE41590
419144621246261PIC_T_ACTIVE48100
421347371246348PIC_T_ACTIVE43000
429147501246292PIC_T_ALERTING46180
429539111246023PIC_T_ALERTING45110
430127611246252PIC_T_ACTIVE45370
430244971246359PIC_PRESENT_CALL35340
431041211246483PIC_T_ACTIVE41320
msc>ssf_dumpf
SSFCSFSMState
43114311988963IDLE
43954395983221IDLE
49394939989312MONITORING
54915491988865IDLE
55165516989413IDLE
55295529986966IDLE
55615561989082IDLE
55655565987402IDLE
55785578989028IDLE
56195619989365IDLE
56515651989299IDLE
msc>ssf_dumpt
SSF_OFFICE
ScpModeLegCriteriaSWTypePC
6010171005
SSF_OFFICE12512727
SSF_TIMERS
Tssf1Tssf2Tssf3Tssf4
1000100010001000
SSF_TIMERS10121
SSPDiagnostics
TheSSPdumptoolsimplyshowsthecontentsoftablesassociatedwiththe
ServiceSwitchingPoint(SSP)functionality:aninternalglobalstable,costable,
gtttable,andtsaptable.
Table 87showstheoptionsforthessp_dumpcommand.
Table 87CommandOptionsforssp_dump
Option Description
a Displayalltables
C DisplayClassofservice(cos)table
g DisplayGTTtables(default)
Table 87CommandOptionsforssp_dump
Option Description
s DisplayGlobalsandSAPtables.
Example94ssp_dumpCommandOutput
msc>ssp_dumpa
GlobalsTable
INIT(COSSAPGTT)MAXCOSMAXSAPMAXGTT
YYYY20106500
TransportlayerSAPTable
SAPUSAPSSNSS7LDCONXNSAPSTATECOUPLETYPEPROVIDUSERID
118010BOUNDTIGHTIPCID=1ID=0
PC:71002NIIND:NATLSWTYPE:ITUGT_IND:SSNENC_SCH:DIGITS:
500010INITTIGHTIPCID=1ID=0
GlobalTitleTranslationTable
TranslationClass1
PATTERN
XXXX??????????????
TYPENIINDRTEINDPCGLTINDTTYPENPLANNAINDSSN
ANSINATLSSN000.000.012n/an/an/an/a6
PATTERN
604XXXXXXX
TYPENIINDRTEINDPCGLTINDTTYPENPLANNAINDSSN
ANSINATLSSN000.000.012n/an/an/an/a6
COSPURPOSE
VSMDiagnostics
TheVirtualSwitchMatrix(VSM)subsystemrunsontheMSCandisusedto
managetonesandannouncements.Usethevsm_dumpcommandtoreportthe
servicestateofEl/TlportsfromthePSTN.Forexample,ifcustomerscannotmake
landcalls,youcanchecktoseeifaportisinservice,outofservice,orblocked,
andyoucanseethetrunkgrouptowhichitbelongs.
Table 88showstheoptionsforthevsm_dumpcommand
Table 88CommandOptionsforvsm_dump
Option Description
B Displaynumericonly
Table 88CommandOptionsforvsm_dump
Option Description
g[group] Displayportsinspecifiedgroup
G Displayglobalstable
O Displaycustomervrasoffsettable
P Displayporttable
r Displayvsmgroupframetable
s DisplaystoredISUPmessages
u DisplayISUPgrouptable
Example95vsm_dumpCommandOutput
msc>vsm_dumpu
ISUPGROUPTABLE
GROUPVARIANTSTATUSCIRCUITCICPORTRANGEUSAPSPIDSUID
======================================================================
15ITURESUMED15000130215100043100 1
16ITURESUMED16000130220100043100 1
27ITURESUMED27000146500049500 1
28ITURESUMED28000148100049500 1
29ITURESUMED29000149700046400 1
60ITURESUMED6000026000400 1
msc>vsm_dumpg60
PortCPStateServiceStateMediaStateAnsFeatGroup
=========================================================
6000NULLSTARTUPOOS0060
msc>vsm_dumps
iIAM(11)ACM(12)ANM(13)REL(14)KEYIDPORT
00010130170011317001
10010130170131317013
20010130180011318001
30010130190031319003
40010130190131319013
50010130190151319015
60010130200071320007
70010130210211321021
80010130220151322015
90010130230081323008
100010130230101323010
110010130230141323014
120010130160051316005
130010130160071316007
140010130160091316009
MediaControlDiagnostics
MEGDiagnostics
TheMediaGateway(MEG)subsystemispartofMediaControl.
Table 89showstheoptionsforthemeg_dumpcommand.
Table 89CommandOptionsformeg_dump
Option Description
C Displaycallentities
e DisplayeventID
g Displaymgiandmditransactions
l Displaygateways
P Displayppidataentities
r Displaycallresource
s Displayspansofparticulargateway
Example96meg_dumpCommandOutput
msc>meg_dumpe
MDI_EVENTIDTRANSID
=========================
MRAActiveSessions:MEG_MRA_SP_SESSION
SP_CONN_IDSU_CONN_IDCP_STATEURI_ROUTE
=================================================================
60970418010PLAYINGsip:annc@172.25.142.46:5061
60970518012PLAYINGsip:annc@172.25.142.46:5061
60971118016PLAYINGsip:annc@172.25.142.46:5061
6097136016PLAYINGsip:annc@172.25.142.46:5061
6097145017PLAYINGsip:annc@172.25.142.46:5061
6097155001PLAYINGsip:annc@172.25.142.46:5061
6097168000PLAYINGsip:annc@172.25.142.46:5061
60971719010PLAYINGsip:annc@172.25.142.46:5061
6097185007PLAYINGsip:annc@172.25.142.46:5061
60971919002PLAYINGsip:annc@172.25.142.46:5061
60972019004PLAYINGsip:annc@172.25.142.46:5061
msc>meg_dumpr
res_id=2039
port=173022res_id=2039call_id=13869557use_count=2
res_id=2040
res_id=2539
msc>meg_dumpl
IDGATEWAY_NAMESVC_STATEAUDIT_REPLIEDGATEWAY_SYNCED
AUD_TID
===============================================================
=======
2mag196INS1128208
3mgs169INS1128361
4virtualINS000
5mag87INS112227
6silverINS1128519
MEGRESDiagnostics
Table 90showstheoptionsforthemeg_res_dumpcommand.
Table 90CommandOptionsformeg_res_dump
Option Description
A Displayalltables
C DisplayMGCPresourcetable
t DisplayMEGACOresourcetable.
Example97meg_res_dumpCommandOutput
msc>meg_res_dumpC
13430302005870OOS6@amigo67.comIDLE
msc>meg_res_dumpt
MEGACORESOURCETABLE
res_idportgwidcallIDstatetermcp_statelast_rep_stateR2_status
============================================================
MEGTSADiagnostics
ThemegtsadumptoolprovidesdiagnosticsfortheinterfacebetweenCall
ProcessingandtheSIPstack.Table 90showstheoptionsforthemeg_tsa_dump
command.
Table 91CommandOptionsformeg_tsa_dump
Option Description
p DisplaytheSIPstackaccesspointsusedbyCallProcessingtocontrol
MRFsorCRBTservers
R DisplaytheroutestoSIPsignalingdestination
U Displaytheuser(CallPorcessing)oftheSIPstackaccesspointsfor
controllingMRFsorCRBTservers
s DisplaytheCallProcessingsessionsthatareusingtheSIPstackto
controlMRFsorCRBTservers
Example98meg_tsa_dumpCommandOutput
msc>meg_tsa_dumpP
TSAAccessPointInfo:MEG_TSA_ACCESS_POINT
IDBind_StatusType Registered_UsersAddressPort
====================================================
0 YES UA 1 172.25.141.8 5061
2 YES NS_STATELESS1 172.25.141.86 5062
msc>meg_tsa_dumpR
TSARouteInfo:MEG_TSA_ROUTE
IDROUTEQ_VALUEEXPIRYAPPL_IDCURR_SESSIONSMAX_SESSIONSENABLED
===================================================================
1annc@172.25.140.34:506110PERMANENT000N
2annc@172.25.142.46:506110PERMANENT02348Y
=========================================================
TSAStatistics:MEG_TSA_ROUTE
IDROUTEATTEMPTSUCCESSFAIL
ABORTED
===================================================================
1annc@172.25.140.34:5061000
2annc@172.25.142.46:50616127706127700
msc>meg_tsa_dumpU
TSAAccessPointUsers:MEG_TSA_ACCESS_POINT_USERaccess_point_id=ALL
SU_IDSP_IDACCESS_POINT_IDCOUPLING_TYPEREG_UNSOLICITED_MSGS
==============================================================
1 0 0 TIGHT ALL
msc>meg_tsa_dumps
TSAActiveSessions:MEG_TSA_SESSION_CONN_KEYaccess_point_id=ALL
LOC_SU_CONN_IDSP_CONN_IDTSA_SU_CONN_IDEVT_TYPECP_STATECALL_ID
=============================================================
842989568614976614978SOT_ET_INVITEESTABLISHED
0200BF3CCA814000000AC9EF@twin.teloseng.com
0200BF3CCE814000000ACA13@twin.teloseng.com
0200BF3CCE814000000ACA15@twin.teloseng.com
AUEDiagnostics
TheAUEsubsystemisresponsibleformonitoringIMGboardeventsand
maintaininghardwarestatesthataffectmultiplechannels(forPRI,R1,andR2).
AUEmonitorsboardconnections,spanstatus,andDchannelstatus(forPRI).
AUEputsportsinandoutofservicebasedonreceivedTPNCPevents.Aportis
uniquelydefinedbythecombinationofboard,span,andchannelnumbers.
Table 92showstheoptionsfortheaue_dumpcommand.
Table 92CommandOptionsforaue_dump
Option Description
Nooption Displayall.
Example99aue_dumpCommandOutput
msc>aue_dump
PortDChannelBoardTrunkBChannelState

800000101Inservice
800010102Inservice
800020103Inservice
800030104Inservice
800040105Inservice
800050106Inservice
800060107Inservice
800070108Inservice
800080109Inservice
8000901010Inservice
8001001011Inservice
8001101012Inservice
8001201013Inservice
8001301014Inservice
8001401015Inservice
8001501016Inservice
8001601017Inservice
8001701018Inservice
8001801019Inservice
8001901020Inservice
8002001021Inservice
8002101022Inservice
8002201023Inservice
RadioResourceDiagnostics
BSMDiagnostics
Table 93showstheoptionsforthebsm_dumpcommand.
Table 93CommandOptionsforbsm_dump
Option Description
B Displaycelltable
C DisplayLACBSCtable
Example100bsm_dumpCommandOutput
msc>bsm_dumpB
cellbscmscIdmccmnclacciis41typeIPC
==================================
111410043109911416171617Id(897583105)CSI
221410043109910816331633Id(897583105)CSI
331410043109912816491649Id(897583105)CSI
441410043109910916651665Id(897583105)CSI
551410043109911616811681Id(897583105)CSI
661410043109911516971697Id(897583105)CSI
CSIDiagnostics
UsetheIS41csi_dumpcommandtoreportthestatusoftheAinterface
(interfacebetweentheBaseStationController(BSC),andtheMSC).
Table 94showstheoptionsforthecsi_dumpcommand.
Table 94CommandOptionsforcsi_dump
Option Description
a DisplayallrecordsforallattachedBSCs
A0 Displayconnectionhashrecords
A1 Displaynumberofconnectionrecordsonfreelist
B[N] DisplaydumpforBSCNonly(defaultisallBSC)
C Displayconnectionrecords(default)
d DisplayPortGatewayinformation
f DisplayFLASHdialogs
F DisplayBSCportownershiplist
g DisplayPagerecords
h Displaywithoutanycolumn/outputtitles
H DisplayPortHashrecords
L DisplayInserviceportsperBSC
O DisplayBSCportlistrecords
P DisplayPortrecords
r Displayp_cref(call_id)keytable
s DisplaySMSdialogs
S DisplaystateinnumericGBIformat
u DisplayUSAP,DPCtoBSCkeytable
U DisplayAUTHENTICATIONdialogs
x DisplayPageHashrecords
X DisplayCICHashrecords
y DisplayTAGdialogs
Y DisplaySMSreply_keykeytable
Z DisplayBSCdefinitions
Example101csi_dumpCommandOutput
msc>csi_dumpZ
(1)BSC=1default_lac=1SSNinservice=FALSE
AIF_codec=IOS4AIF_msgflow=IOSAIR_type=IS95
srvc_option=3speech_type=8kbEnhanced
sms_page_max=200cic_retry=1band_class=0
cell_definition=STATIC
SCCPType=GAP(hostname=tnvbsc1)(qos.ret_on_err=NONE)
usap=9psap=9bound=TRUE
DADITU,routeusingSSN,addressindicatorisNATL
SSN:254PC:00322
GlobalTitleisnotpresent.
SADITU,routeusingSSN,addressindicatorisNATL
SSN:254PC:00641
(2)BSC=2default_lac=2SSNinservice=FALSE
AIF_codec=IOS4AIF_msgflow=IOSAIR_type=IS95
srvc_option=3speech_type=8kbEnhanced
sms_page_max=200cic_retry=1band_class=0
cell_definition=STATIC
SCCPType=GAP(hostname=tnvbsc2)(qos.ret_on_err=NONE)
usap=9psap=9bound=TRUE
DADITU,routeusingSSN,addressindicatorisNATL
SSN:254PC:00323
SADITU,routeusingSSN,addressindicatorisNATL
SSN:254PC:00641
RCMDiagnostics
TheGSMRadioControlsubsystemcommunicateswiththeCSIprocessinorderto
handleallrequestsfrom/totheBSC.ItforwardsrequestsfromCSItoMCI,orMCI
toCSI.
Table 98showstheoptionsforthercm_dumpcommand.
Table 95CommandOptionsforrcm_dump
Option Description
a Displayallinformation(default)
d Dumpallcontrolblocks(CB)
Cn DumpCBsfordialogueinvokeid(did)nonly
g DisplaygatewayIPtable
H Donotprintanycolumn/outputheaders
t Displaytimerinformation
l Displayactivelinklists
L Displayalllinklists
v DisplayhandoverlistofMSCs
Example102rcm_dumpCommandOutput
msc>rcm_dumpg
...
cb_id=1048mobile_id=1048conn_index=2217
queue_ci=1lac=2serving_bsc=1
is_serving_bsc=TRUE
cur_context_id=$cur_termination_id=023/004@172.25.142.169
new_context_id=new_termination_id=
handovertype=UNKNOWN
created_time=TueNov815:46:322011
cb_id=1051mobile_id=0conn_index=2219
queue_ci=1lac=1serving_bsc=1
is_serving_bsc=TRUE
cur_context_id=cur_termination_id=018/005@172.25.142.169
new_context_id=new_termination_id=
handovertype=UNKNOWN
created_time=TueNov815:46:332011
...
msc>rcm_dumpg
PortGatewayIP
=======================
1024172.25.140.144
1025172.25.140.144
1026172.25.140.144
1027172.25.140.144
1028172.25.140.144
1029172.25.140.144
1030172.25.140.144
1000172.25.140.144
1001172.25.140.144
1002172.25.140.144
1003172.25.140.144
1004172.25.140.144
1005172.25.140.144
...
msc>rcm_dumpv
DumpMSCinformation
====================
MCCMNCLACMSCAddress(E.164)CI_METHOD
=====================================
Totalentries:0
RCIDiagnostics
TheIS41RadioControlmodulecommunicateswiththeCSIprocessinorderto
handleallrequestsfrom/totheBSS.ItforwardsrequestsfromCSItoMCI,orMCI
toCSI.
Table 96showstheoptionsfortherci_dumpcommand.
Table 96CommandOptionsforrci_dump
Option Description
a Displayallinformation
d Displayallcontrolblocks
e DisplayMCBkeybyMINtable.
f DisplayallAinterfacecontrolblocks.
g Displayallhandoffcontrolblocks.
h Displayhelptextofthecommand.
H Displaywithoutcolumn/outputheaders.
k DisplayMSChandoffglobalstable.
l DisplayallMSClistentries.
r Displayallmastercontrolblocks.
Example103rci_dumpCommandOutput
msc>rci_dumpk
MSCHOGlobalsTable
====================
handoff_depth=0
tandem_depth=0
MHOT=7
HOT=12
MAT=9
CTT=8
T11=4
T1124=7
guard=120
msc>rci_dumpl
MSCHOList
===========
msc_index=1
MSCID=3078
handback_allowed=FALSE
handthird_allowed=FALSE
Totalentries:1
WJTDiagnostics
TheWJTmoduleresidesintheCSIprocess.ItisresponsibleforTCPbased
communications.CurrentlyitisonlyusedfortheSCCPLiteinterface.
Table 97showstheoptionsforthewjt_dumpcommand.
Table 97CommandOptionsforwjt_dump
Option Description
a Displayalltables
B DisplayBSCtable
g Displayglobalstable
Example104wjt_dumpCommandOutput
msc>wjt_dump
WJTBSCTABLE
LINKDESCBSCTYPEHOSTSTATUSACC_FDFDR_CNTW_CNT
=================================================================
1000MSCsparkCONNECT_OPEN1150000000000
3002BSkimCONNECT_FIRST110000000000
GAPDiagnostics
Table 98showstheoptionsforthegap_dumpcommand.
Table 98CommandOptionsforgap_dump
Option Description
a Displayalltables
C DisplayBSCtable
r Displayconnectionreferencetable
R DisplayGAPRTDinformation
Example105gap_dumpCommandOutput
msc>gap_dumpC
BSC:001TYPE:BSTRANSPORT:GAP_TRANS
SS7PC:065801SSN:0000254HOST:amigo27IPC:COMD
LINKDESC:01
LINKDESC:02
...
LINKDESC:99
LINKDESC:100E
IS41MSCTrunkManagement IMPDiagnostics
TheIS41InterMachinePort(IMP)subsystemmanagestheconfigurationfor
portsandtrunksforanadjacentMSC.
Table 99showstheoptionsfortheimp_dumpcommand.
Table 99CommandOptionsforimp_dump
Option Description
a Displayalltables
B Displayportstates
d Displaydialogtable
f DisplayMSCtableandportlists(FIFO,FILO,ROT,RROT)
P Displayportbuffers
t Displaytrunkgroupmembertoporttable
x Displayoutstandingtransactions
Example106imp_dumpCommandOutput
msc>imp_dumpa
CellidxPortRfSccVmaxCtmrLtmrTrcAckEakOrgTypSwRlLocTSt
====================================================
TOTALNUMBEROFPORT:0
IMPDIALOGTABLE
==================================================================
TOTALNUMBEROFIMPDIALOGINUSE:0
PORTTABLE
==================================================================
TOTALNUMBEROFPORT:0
MSCKEYANDPORTLISTS(FIFO,FILO,ROT,RROT)
==================================================================
TOTALNUMBEROFMSCKEY:0
TRANSACTIONTABLE
==================================================================
TOTALNUMBEROFTRANSACTION:0
MEMBERNUMBERTOPORTLOOKUPTABLE
==================================================================
TRUNK_GROUPMEMBER_NUMBERPORT_NUMBERMSCID
==================================================================
TOTALNUMBEROFTRUNKGROUPMEMBERKEY:0
MCIDiagnostics
TheIS41MobilityControl(MCI)moduleisresponsibleforhandlingtheMAP
signalingforbasiccalls.
Table 100showstheoptionsforthemci_dumpcommand.
Table 100CommandOptionsformci_dump
Option Description
a Displayalltables
c DisplayActivationMintable
C Displayclasstable
d Displaydialogs
D DisplaytheSCPAddress
h Displaywithoutheaders
l Displaydatadialogs
L DisplayLACnametable
m DisplayMessageCenteraddresstable
r Displaytemporaryreferencenumbertable
t Displaytransaction/WIN/CALEAcapabilitiytable
x Displaytransactiontable
Example107mci_dumpCommandOutput
msc>mci_dumpL
LACNAMETABLE
=============================================
LAC=1,NAME=AmigoBSC1
LAC=4,NAME=AmigoLAC4
LAC=9,NAME=test5678901234567890123456789012
LAC=22,NAME=Q_CELL_ginger_rohan
msc>mci_dumpt
IS41Transaction/WINCapabilitiyTable
=======================================
TransactionCapability
Profile:1
BusyDetection:1
Announcements:1
RemoteUserInteraction:0
SubscriberPINIntercept:0
MultipleTerminations:0
TerminationList:1
UserZoneCapabilityIndicator:0
NDSSCapability:0
NameCapabilityIndicator:0
Winaddressing:1
OTAParameterAdministration:1
WINCapability
IntroducingStar/Pound:1
Kdigit:1
AllCalls:1
RevertiveCall:1
CallTypes:1
UnrecognizedNumber:1
PriorAgreement:0
AdvancedTermination:1
TerminatingResourceAvailable:0
TBusy:0
TNoAnswer:0
ConnectResource:1
OriginationAttemptAuthorized:1
OAnswer:1
ODisconnect:1
CallingRoutingAddressAvailable:1
InitialTermination:1
CalledRoutingAddressAvailable1
TAnswer:1
TDisconnect:1
CallControlDirective:1
ISIDiagnostics
ISIdefinescommonstructuressharedbytheIS41trunkmanagementmodulesso
thatthesemodulescancommunicatewitheachother.
Table 101showstheoptionsfortheisi_dumpcommand.
Table 101CommandOptionsforisi_dump
Option Description
a Displayalltables
g Displayglobaltable
k DisplaybillingIDRecordtable
l DisplayadjacentMSCAddresslist
Table 101CommandOptionsforisi_dump
Option Description
s Displayallsubscribers
s[MIN/MDN] Displayallsubscribers
s[MIN/MDN][min/mdn] Displayindividualsubscriberbasedonmin/mdn
s DisplayrangeofsubscribersfromminA/mdnAto
[MIN/MDN][minA/mdnA] minB/mdnB.Notrecommendedtouseswithoutthe
[minB/mdnB] MIN/MDNoption,sincethiswilldisplayALLusersonthe
system,whichcanbeinexcessof100,000users.
t Displaytimerstable
v Displayverboseoutput
Example108isi_dumpCommandOutput
msc>isi_dumpg
MSCGlobalTable
============================================
MAPSAP:2
DefaultLACName:default8901234567890123456789012
NoServiceLACName:test
SMSPageMax:110
SMSSrvcOption:6
OTAPASrvcOption:18
SMSUseDirect:FALSE
msc>isi_dumps
MobileSubscriberRecord:
CreateTime:MonNov2110:57:192005
MINESNIMSIMDN
60420068050xfe6068051020060420068056042006805
LACBindCountActive
601
PWRMACDTXESAMPSCDMATDMAIS136
2000100
SLOT_MODECELLSLOT_CYCLESTATUSAUTH_DENIEDLIFETIMESM
DPF
136110000
...
IS41MobilityManagement
VLRDiagnostics
TheIS41VisitorLocationRegister(VLR)subsystemispartofMobility
Management.
Table 102showstheoptionsforthevlr_dumpcommand.
Table 102CommandOptionsforvlr_dump
Option Description
a Displayalltables
g Displayglobalstable
r Displaytimerstable
t Displaytdntables
v Displayvirtualmscidtable
x Displaytransactiontable
s Displayallsubscribers
s[MIN/MDN] Displaysubscriberinspecifiedrange
s[MIN/MDN][min/mdn] Displaysubscribersinspecifiedrange
s[MIN/MDN][minA/mdnA][minB/mdnB] Displaysubscribersinspecifiedrange
Warning:
Usingthesoptionwithno[MIN|MDN]willtakealongtime.Donotctrlcoutof
thiscommandasthismayresultintablerecordsbeinglocked.
Example109vlr_dumpCommandOutput
msc>vlr_dumpsMIN60420504446042050446
VISITORLOCATIONREGISTER
==================================================================
MIN=6042050444IMSI=ESN=0xFE650444
CARR_D=1234DEST_D=000000000000000
MDN=6042050444HAVE_PROFILE=TRUEAUTH_DENY=0OIND=7TERM_RESTRICT=2
CFU=1CFB=1CFNA=1CW=1TWC=1CD=3VP=0CNIP1=1CNIR=1CNIR_OVERRIDE=0
DMH_BILL_D=6042050444
PREFERRED_LANG_IND=0
NOANSWERTIMER=0
MSG_WAIT_TYPE=0x00MSG_WAIT_CNT=255255255255255255
WINTRIGGERADDRESSLIST

AddrType:PCSSNPC:121SSN:14
TDPRTYPE:

OAA:0x36
CALLING_RAA:0x37
IT:0x38
O_DISC:0x41
T_DISC:0x70
CALLED_RAA:0x39
TDPNTYPE:

T_ANSWER:0x69
O_ANSWER:0x40
status=0x0party=0lai=5expiry=INDEFINITE
ACTV=TRUEACCESS=FriDec2308:24:442005
SMS_ADDRnature=49plan=13encoding=3bcd_digit=4addr=0x5128080000
SMS_ORIG_RESTdefault=0direct=0force_message_center=0
SMS_TERM_RESTdefault=0reverse_charges=0
SMSDPF=0
MIN=6042050445IMSI=ESN=0xFE650445
...
SMSDPF=0
MIN=6042050446IMSI=ESN=0xFE650446
...
SMSDPF=0
MIPDiagnostics
TheIS41MobilityInterfaceProtocol(MIP)providesservicessuchasIS41invoke,
returnresult,returnerror,rejectmessages,internalerrorhandlingprocedures,
localrecovery,andtimeoutmanagement,tousersoftheMobilityInterface.
Table 103showstheoptionsforthemip_dumpcommand.
Table 103CommandOptionsformip_dump
Option Description
a Displayalltables
B DisplayMINGLT/DNblocktables
d Displaydispatchtable
N DisplaynetworkMSCtables(detailed)
r Displayroutingtable
s DisplayMAPSAPtable
w DisplayMSCtables
x Displaytransactions
Example110mip_dumpCommandOutput
msc>mip_dumps
<<IS41MAPSAPCONFIGURATIONTABLE>>
MIP_SAPIAU_SAPSSNBINDSTATUSUSER_ID
=================================================
115BOUND61(MIPD)
228BOUND76(IMP)
337BOUND73(VLR)
msc>mip_dumpw
<<LOCALMSCTABLE>>
MARKETSWITCH(MSCID)SS7PC
=====================================================
10010(25610)0.40.81(10321)
<<NETWORKMSCTABLE>>
MARKETSWITCH(MSCID)MANUFACTURERIS41REV.TDN_BIN
====================================================================
ALLALL(285212672)MIP_MSC_TELOS_MMIP_IS41D_M1
DiagnosticTools 343
<<IS41PMSCTABLE>>
MARKETSWITCH(MSCID)SS7PC
=====================================================
GSMMobilityManagement TheGSMmobilitymodulescommunicatedirectlywiththeGSMprotocols.
MCGDiagnostics
TheMCGdiagnosticsshowtheDialogueControlTable:theFSMControlBlock
tableandtheMAPtable
Table 105showstheoptionsforthemcg_dumpcommand.
Table 104CommandOptionsformau_dump
Option Description
a Displayall(default)
C DisplayClass(mscclass)table
d DisplayDialogControltable
g DisplayGlobals(mscmapconfig)table
O DisplayMobileandCallobjecttable
r DisplayMSCSubscriberRecordtable
B DisplayODBConfigurationtable
t DisplayTimers(mscmaptimers)table
l Displayactivelinklists
L Displayalllinklists
P Displaypointcodestatus
Example111mcg_dumpCommandOutput
msc>mcg_dumpd
DialogueControlTable
====================================================================
**********FSMControlBlock1**********
did:3862
alt_did:0
rsv_did:0
map_did:3382
fsm_id:1900582
imsi_str:123010000000113
tmsi:0x00000000
imsi_str:16041110113
req_smg_in_tag:0
req_in_tag:0
req_out_tag:0
csg_did:1191
sdl_idx:ProcessOCS_MSC
cm_srv_type:MO
apn:V3_ROAM_NUM_ENQ
gsm_v1:GSM_V2
gsm_variant:GSM_TYPE

344 Chapter:
call_reference_number:36044
FSMControlBlockcreatedat:2011/11/0817:07:29
**********FSMControlBlock2**********
did:2082
alt_did:0
rsv_did:0
map_did:2512
fsm_id:854617
imsi_str:123010000000513
tmsi:0x00000000
imsi_str:16041110513
req_smg_in_tag:0
req_in_tag:0
req_out_tag:0
csg_did:0
sdl_idx:ProcessICS_MSC
cm_srv_type:MT
apn:V3_ROAM_NUM_ENQ
gsm_v1:GSM_V2
gsm_variant:GSM_TYPE
call_reference_number:36045
FSMControlBlockcreatedat:2011/11/0817:07:29
...
**********MAPDialog54**********
cbid:7070
did:1729
MAPdialogueinvokeidlist:
invoke_id/OP:0/FORWARD_SHORT_MESSAGE
MAPDialogcreatedat:2011/11/0817:10:43
**********MAPDialog55**********
cbid:7070
did:1923
MAPdialogueinvokeidlist:
invoke_id/OP:1/PAGE
MAPDialogcreatedat:2011/11/0817:10:43
...
msc>mcg_dumpP
PointCodeStatusTable
=============================
pointcodestatus

111INSMAUDiagnostics
MAUDiagnostics
TheMAUdiagnosticsareusefulinthecaseofadialogleakage.StarSolutions
technicalsupportmayaskyoutorunthisdiagnosticifthisoccurs.Amongstother
things,itliststheversionusedforeachdialog.

DiagnosticTools 345
Table 105showstheoptionsforthemau_dumpcommand.
Table 105CommandOptionsformau_dump
Option Description
a Displayalltables
d DisplayMAUdialogtable(default)
f Displayfunctionpointers
g DisplayMARgentable
l DisplayMAPdialoguetable
r DisplayMARroute(sap)table
s DisplayMAPssmstable
t DisplayTRIactivationtasks
u DisplayMAPqueuetable
v Displayverboseoutput
w DisplayMAUdialogtableandMARversiontable
y DisplayMARapntable
msc>mau_dumpw
MARDIALOGBUFFERS
==================================================================
DID=2166RTE=LOCALEDID=0LDID=1523UDID=62VER=2PSAPU/L/E=1/1/1
...
MARVERSIONTABLE
PATTERNHIGHESTVERSIONSUPPORTED
==================================================================
XXXXXXX????????VER=2P

ACRONYMS
D
Thisappendixdefinesacronymsusedinthisguide.
Table 106ListofAcronyms
Acronym Definition
AIN AdvancedIntelligentNetwork
ALM AlarmModule
AMPS AdvancedMobilePhoneService
ANSI AmericanNationalStandardsInstitute
ASCII AmericanStandardCodeforInformationInterchange
ASN AbstractSyntaxNotation
ASP ApplicationServiceProvider
ATM asynchronoustransfermode
ATP AppleTalkTransactionProtocol
AUP AcceptableUsePolicy
AUR AccessUsageRecord
BASIC BeginnersAllPurposeSymbolicInstructionCode
BCH Base,Chaudhuri,Hocquenghem
BS BaseStation
BSC BaseStationController
BSS BaseStationSystem
CALEA CommissiononAccreditationofLawEnforcementAgencies
CAS ChannelAssociatedSignaling
CD CompactDisk
CDMA CodeDivisionMultipleAccess
CDR CallDetailRecord
CFB CallForwardBusy
CFNA CallForwardNoAnswer
CFU CallForwardUnconditional
CIC CircuitIdentificationCode
CLASS CustomLocalAreaSignalingService
CLI CommandLineInterface
CMS CallManagementServices
CNIR CallingNumberIdentificationRestriction
CODEC CoderDecoder

348 AppendixD:Acronyms
Acronym Definition
COS ClassofService
CPU CentralProcessingUnit
CW CallWaiting
DAT DigitalAudioTape
DB Database
DID DirectInwardDialing
DN DirectoryNumber
DPC DestinationPointCode
DTAP DataTransferApplicationPart
DTE DataTerminalEquipment
DTMF DualTone,MultiFrequency
ECC ExchangeCarrierCode
ESD ElectrostaticDischarge
FCAPS Fault,Configuration,Accounting,Performance,andSecurity
Management
FIFO FirstInFirstOut
FILO FirstInLastOut
FTP FileTransferProtocolorFoilTwistedPair
GAP GenericAddressParameter
GB Gigabyte
GMT GreenwichMeanTime
GPS GlobalPositioningSystem
GUI GraphicalUserInterface
HA HighAvailability
HLR HomeLocationRegister
IMSI InternationalMobileSubscriberIdentity
IN IntelligentNetwork
INS InService
IO InputOutput
IP InternetProtocol
IPC InterprocessCommunication
IPCD InterProcessCommunicationDaemon
ISP InternetServiceProvider
ISUP IntegratedServicesDigitalNetworkUserPart
IT InformationTechnology
ITU InternationalTelecommunicationsUnion
IUA ISDNUserAdaptationLayer
LAC LocationAreaCode
LDAP LightweightDirectoryAccessProtocol

349
Acronym Definition
LFM LogicalFacilityManager
LIA LawfulInterceptAccess
LNP LocalNumberPortability
MAC MediumAccessControl
MAP MobileApplicationPart
MB Megabyte
MCC MobileCountryCode
MDN MobileDirectoryNumberorMobileDialingNumber
MGCP MediaGatewayControlProtocol
MGW MediaGateway
MID MessageIdentification
MIN MobileIdentificationNumber
MM MobileManagement
MMI ManMachineInterface
MNC MobileNetworkCode
MRF MediaResourceFunction
MS MobileStation
MSC MobileSwitchingCenter
MSCID MobileSwitchingCenterIdentifier
MSID MobileStationIdentifier
MWI MessageWaitingIndication
NAP NetworkActionPointorNetworkAccessPoint
NE NetworkElement
NFS NetworkFileSystem
NIC NetworkInterfaceCard
NMS NetworkManagementStation
NO NetworkObject
NOC NetworkOperationsCenter
NSAP NetworkServiceAccessPoint
NSP NetworkServiceProvider
NTP NetworkTimeProtocol
NVRAM NonVolatileRandomAccessMemory
OMC OperationandMaintenanceCenter
OPC OriginatingPointCode
OS OperatingSystem
OTA OvertheAir
OTAPA OvertheAirProvisioningAdministration
OUI OrganizationallyUniqueIdentifier
PC PersonalComputerorPointCode

350 AppendixD:Acronyms
Acronym Definition
PID ProcessIdentificationorProtocolIdentifier
PIN PersonalIdentificationNumber
PL PatchLevel
PM PerformanceMonitoring
PRI PrimaryRateInterface
PROM programmablereadonlymemory
PSAP PublicSafetyAnsweringPoint
PSTN PublicSwitchedTelephoneNetwork.
RCP RemoteCopyProgram
RJE RemoteJobEntry
RLC ReleaseComplete
ROM ReadOnlyMemory
SA SourceAddress
SAP ServiceAccessPoint
SCCP SignalingConnectionControlPart
SCF ServiceControlFunction
SCP ServiceControlPoint
SCSI SmallComputerSystemInterface
SCTP StreamControlTransmissionProtocol
SIGTRAN SignalingTransport
SIP SessionInitiationProtocol
SLR SendLoudnessRating
SM SwitchModule
SMS ShortMessageService
SN SequenceNumber
SNA SystemsNetworkArchitecture
SP SignalingPointorServiceProvider
SPID ServiceProfileIdentifier
SS7 SignalingSystemNumberSeven
SSD SharedSecretData
SSF ServiceSwitchingFunction
SSN SubsystemNumber
SSP ServiceSwitchingPoint
SSVR SignalingServer
STA SpanningTreeAlgorithm
STP SignalingTransferPoint
SU SubscriberUnit
SWAP SharedWirelessAccessProtocol
TAR TapeArchive

351
Acronym Definition
TCP TransmissionControlProtocolorTestCoordination
Procedure
TDMA TimeDivisionMultipleAccess
TDP TriggerDetectionPoint
TMSI TemporaryMobileStationIdentifier
TO TransmitOnly
TPNCP TrunkPackNetworkControlProtocol
TRC TransitRoutingControlTable
TWC ThreeWayCalling
UCT UniversalCoordinatedTime
UDP UserDatagramProtocol
URL UniformResourceLocator
UTC UniversalCoordinatedTime
VAR ValueAddedReseller
VLR VisitorLocationRegister
WIN WirelessIntelligentNetwork
WYSIWYG whatyouseeiswhatyouget

INDEX
ASCII db_restore 140

Symbols exportreporttoExcel 114 kill 154
.rhosts ASPSubsystemDiagnostics 332 less 220
configurationfile 154 ASPSubsystemEvents 281 mt 164
Editing 157 asp_dump 321 netstat 160
AUASubsystemDiagnostics 322 ntpq 133
aua_dump 322 save 243
A AUESubsystemDiagnostics 331 snoop 224
Ainterface aue_dump 331 tm 240
Usagecounters 276 AUPSubsystemDiagnostics 332 top 117
account AURSubsystemDiagnostics 322 useradd 156
root 83 aur_dump 322 vmstat 117
superuser 83 authoritativetime 129 zic 136
accounts automaticalarmclearing 206 commandlineinterface
drum 67,70 addcommand 45
pro 67,70 cancelcommand 45
root 67,70 B cancelingacommand 45
changecommand 45

activestatus backup
checkingstatusofactiveSSVR 96 datalocation 141 copycommand 45
add,incommandlineinterface 45 managingdatabasebackupfiles 144 deletecommand 45
AirFilters namingconvention 141 exitcommand 45
Changing 271 backups navigating 46
alarmclearing manualdatabasebackups 143 overview 43
automatic 206 whentomanuallybackup 143 querycommand 45
manualalarmclearing 207 BSMSubsystemDiagnostics 323 syntax 44
alarmtype bsm_dump 332 verifycommand 45
communications 206 Commandoption
equipment 206 alm_dump 317
processingerror 206 C asp_dump 321
alarmtypes 206 cancel,incommandlineinterface 45 aua_dump 322
alarms capacityplanning 105 aue_dump 331
cpuCMSoftFail 211 CDRSubsystemDiagnostics 332 aur_dump 322
cpuCMVitalDeath 211 cdr_dump 317 bsm_dump 318,323,332
cpuPMCMDDeath 212 change,incommandlineinterface 45 cdr_dump 317
cpuPMHWDDeath 212 changes com_dump 318
DblDbRecErr 212 todatabaseschema 140 csi_dump 333
DiskAlarm 214 ChangingAirFilters 271 gap_dump 334,336,337
listofprobablecauses 208 child_mon 118 gss_dump 319
lpcTcpHostDown 213 clearing imp_dump 337
NERestart 211 automaticalarmclearing 206 isi_dump 339
probablecauses 208 manualalarmclearing 207 lfm_dump 324
RootDiskAlarm 214 CLI.seecommandlineinterface mau_dump 343,345
tpmTmrDrop 213 clock mci_dump 338
ALMSubsystemDiagnostics 332 selfreference 131,132 meg_dump 329
alm_dump 317 clocksource meg_res_dump 330
application NTP 129 meg_tsa_dump 330
starting 79 com_dump 318 mip_dump 342
stopping 79 Command ppx_dump 325
viewingstatus 57 ufsrestore 183 rci_dump 335
applicationtracing 234 command ssf_dump 325
Architecture crontab 142 ssp_dump 326
HighAvailability 87 db_auto 141 sta_dump 319
STA R SO LU TIONS CO N FI DE N TI A L SonataCoreVoiceNetworkIS41R4.5.3|GSMR4.5.4Operations,MaintenanceandProvisioning

Guide
354 Index
tpm_dump 320 CPUSubsystemEvents 279 versioning 140

vlr_dump 341 CPUutilization daylightsavingstime
vsm_dump 327 checking 117 configuring 134
commandoption collectedperformancedata 109 db_auto
useradd 156 graphsandreports 116 command 141
commonservices performancestatistics 111 db_restore
starting 58 CPUutilization,verifying 119 command 140
stopping 58 cpu_abort 60 DblDbRecErralarm 212
communications cpu_idle delete,incommandlineinterface 45
alarmtype 206 CPUstatistic 276 DER
communicationsprotocolerror cpu_status DiscreteEventRegistration 276
probablecause 208 checkingactivestatus 96 DGTSubsystemDiagnostics 324
componentsoftware 249 cpu_sys dimensioning 105
CONF CPUstatistic 276 Directory
mscworkingdirectory 83 cpu_usr CONF 83
Configuration CPUstatistic 276 EXEC 83
Reapplychangesafterfailover 87 cpu_wio LIBR 83
configuration CPUstatistic 276 RJE 83
restoringdatabase 145 cpuCMShutDown 215 RUN 83
configurationdatabase alarmgeneratedduringSSVRshutdown 97 TMP 84
backups 139 cpuCMSoftFailalarm 211 TOOL 84
configureautomaticbackups 141 cpuCMVitalDeathalarm 211 Disaster
manualbackups 143 cpuPMCMDDeathalarm 212 Recoveringfrom 233
Restoring 145 cpuPMHWDDeathalarm 212 Recoveryfrom 233
configurationfile crontab DiscreteEventRegistration 276
.rhosts 154,157 Understandingcrontab 142 disk
hwd.conf 95 crontabfile block 81
inetd.conf 154 automaticsystembackups 141 cylinder 81
ntp.client 130 CSISubsystemDiagnostics 332 disklabel 81
ntp.conf 130 CSISubsystemEvent 296 partition 81
configurationmanagement 139 csi_dump 333 sector 81
configurationorcustomizationerror CumulativeCounters 275 slice 81

probablecause 208 customerservice track 81
Configuring contacting 28 diskI/O
NTPserversandclients 130 checking 123
configuring collectedperformancedata 109
daylightsavingstime 134 D graphsandreports 122
timezones 134 Daemon performancestatistics 111
UTCtimedisplay 135 Hardwaredaemon 90 disk_io
congestion hwd 90 I/Ostatistic 277
probablecause 208 daemon DiskAlarmalarm 214
connectingtotheSSVR 71 exec 155 display
contacting inetd 154 UTCtime 135
technicaldocumentationdepartment 29 login 155 drift
conventions,text 24 NTP 129 systemtime 129
convertingtracefiles 245 shell 155 driftfile 131,132
copy,incommandlineinterface 45 xntpd 130 drum 70
corruptdata datamodel
probablecause 208 databaseschema 140
Counter database E
DiscreteEventRegistration 276 backuplocation 141 EmergencyPatchreleases 250
Gauge_Max 275 backupnamingconvention 141 equipment
Gauge_Min 276 databasebackup alarmtype 206
Usage 276 filenamesyntax 141 Ethereal
CounterNaming 276 managingbackupfiles 144 downloading 227
Counters databasebackups eventlog
Cumulative 275 about 140 checkingrecords 100
Gauges 275 configureautomaticbackups 141 EXEC
Typesofperformancecounters 275,279 manualbackups 143 mscworkingdirectory 83
CPUprocesslists,viewing 127 overview 139 exit,incommandlineinterface 45
CPUstatistic databaseschema
cpu_idle 276 definition 140
cpu_sys 276 databasetable F
cpu_usr 276 incrementversion 140 Failover
cpu_wio 276 databasetables Controlledbyhardwaredaemon 90
CPUStatistics 276 versionmismatch 140 Equivalenttorestart 88

Guide
Index 355
Reapplyunsavedconfigurationchanges 87 IMPSubsystemDiagnostics 337 Memoryutilization 277

failover 146 imp_dump 337 mem_total
causedbyfault 99 inetd Memoryutilization 277
duetohardwaremalfunction 99 daemon 154 Memoryutilization
duetosoftwarefailure 99 inetd.conf mem_free 277
investigatingfailovercause 99 configurationfile 154 mem_total 277
operatorinitiated 96 Input/Outputstatistics 277 swap_free 277
faultlocalization 210 Interface swap_total 277
faultmanagement 201 hme0 89 memoryutilization
faults hme1 89 collectedperformancedata 109
recordingfaults 52 Physicalinterfaceandhardware graphsandreports 119
fileerror daemon 89 performancestatistics 111
probablecause 208 IPaddress MemoryUtilizationStatistics 277
fsck 151,163 127.127.1.0 131,132 MIPSubsystemDiagnostics 342
clockselfreferenceIP 131,132 mip_dump 342
displayingactive/standbyIPaddresses 94 Modules
G IPnetwork Traceablemodules 235
GAPSubsystemDiagnostics 337 qualityofservice 106 mt 164
gap_dump 334,336,337 ipc_remoted 246 multiusermode 77
Gauge_Max 275 IpcTcpHostDownalarm 213 multiusermode 68
Gauge_Min 276 IpcTcpHostUp 216
Gauges 275 ISISubsystemDiagnostics 339
globalpositioningsystem 129 isi_dump 339 N
GPS Naming
globalpositioningsystem 129 Counternaming 276
systemtime 129 J NAUSubsystemDiagnostics 324
graphsandreports JavaWebStart 42 NERestartalarm 211
about 109 jitter net_io
gss_dump 319 systemclockerrors 129 I/Ostatistic 277
networkI/O
checking 124
H K collectedperformancedata 110

Hardware kill 60 graphsandreports 123
Systemperformancestatistics 276 command 154 performancestatistics 111
Heartbeat Networkoutage
Physicalinterfaces 89 Recoveringfrom 233
HighAvailability L networkprotocolanalyzer 224
Implementation 87 LFMSubsystemDiagnostics 324 networksynchronizationandclocking 137
Managementview 88 lfm_dump 324 networktimeprotocol 129
Physicalview 87 LIBR URL 130
Serviceview 88 mscworkingdirectory 83 notices 24
highavailability link_status 127 NTP
checkingactivestatus 96 logviewer daemon 129
NTPconfiguration 132 vlog 217 networktimeprotocol 129
verifyingredundancy 91 loggingin universaltimecoordinated 129
hme0 aspro 77 NTPClient
Physicalinterface 89 asroot 77 Configuring 130
hme1 Logginginandout 77 NTPclient 130
Physicalinterface 89 checkstate 133
homedirectory 83 NTPconfiguration
superuserhomedirectory 83 M inhighavailabilityconfiguration 132
hostsfile insingleavailabilityconfiguration 131
MACAddress
IPaddresses 95 NTPServer 130
virtualMACaddress 95
HUPsignal 154 MACaddress Checkstate 133
hwd Configuring 130
VirtualMACaddress 87
Daemon 90 NTPserversandclients
manualalarmclearing 206,207
hwd.conf MAUSubsystemDiagnostics 344 synchronizing 133
configurationfile 95 ntp.client
mau_dump 343,345
configurationfile 130
MCISubsystemDiagnostics 338
mci_dump 338 ntp.conf
I configurationfile 130
meg_dump 329
I/Ostatistic ntpq
meg_res_dump 330
disk_io 277 meg_tsa_dump 330 command 133
net_io 277 MEGRESSubsystemDiagnostics 330
I/OStatistics 277 MEGTSASubsystemDiagnostics 330
icons 24 mem_free

Guide
356 Index
fileerror 208 Sharedaddress

O outofmemory 208 Swappingafterfailover 88
offset powerproblem 209 singleavailability
systemclockoffset 129 resourceatornearingcapacity 209 NTPconfiguration 131
OMC responsetimeexcessive 209 singleusermode 69,77
accessing 42 softwareerror 209 SMISubsystemDiagnostics 339
application 41 softwareprogramabnormally snoop
launching 42 terminated 209 command 224
URLformat 42 transmitfailure 209 MEGACOportforpacketcapture 225
operatingsystem,stopping 61 underlyingresourceunavailable 209 MGCPportforpacketcapture 225
OperationsandMaintenanceCenter.seeOMC unspecifiedreason 209 packetcapturepoint 225
oscillator processingerror SCTPportforpacketcapture 225
inSSVR 129 alarmtype 206 SIPportforpacketcapture 225
outofmemory PROTO snoopfiles
probablecause 208 databasefile 140 binary 224
Outage protocolanalyzer 224 softwareerror
Recoveryfrom 233 probablecause 209
softwareprogramabnormallyterminated
Q probablecause 209
P qualityofservice softwareversion,displaying 66
partition definition 106 Solaris
terminology 82 IPnetwork 106 universaltimecoordinated 130
passwords query,incommandlineinterface 45 SS7
account 70 packetmonitoringandcapture 224
changing 79 SSFSubsystemDiagnostics 325
PatchLevelreleases 251 R SSFSubsystemEvents 310
Peerprocess RCISubsystemDiagnostics 338 ssf_dump 325
HeartbeatandmonitoringinMSC 90 rci_dump 335 SSPSubsystemDiagnostics 326
Performance rebooting 61 ssp_dump 326
Cumulativecounters 275 recordingfaults 52 STASubsystemDiagnostics 319
DiscreteEventRegistration 276 Redundancy sta_dump 319
Gauge_Max 275 Physicalview 87 Statistics

Gauge_Min 276 redundancy CPUperformance 276
Gauges 275 verifyinghighavailability 91 I/Operformancestatistics 277
Usagecounter 276 redundantconfiguration Memeoryutilization 277
performanceanalysis 107 checkingredundancy 94 Systemhardwareperformance 276
Performancecounter report,incommandlineinterface 45 stoppingtheoperatingsystem 61
Types 275,279 resourceatornearingcapacity storage
performancedata probablecause 209 performancereports 113
CPUutilization 109 resourcetracing 234 su 78
diskI/O 109 responsetimeexcessive Subsystemdiagnostics
memoryutilization 109 probablecause 209 ASP 332
networkI/O 110 Restart AUA 322
swaputilization 109 Andfailover 88 AUE 331
performancemanagement 105 restore AUP 332
activities 108 verifyvolumelabel 141 AUR 322
overview 105 restoringsystembackup 178 BSM 323
performancemonitoring 107 RJE CDR 332
PerformanceStatistics mscworkingdirectory 83 CSI 332
Systemhardware 276 root 70 DGT 324
performancestatistics rootaccount 83 GAP 337
CPUutilization 111 RootDiskAlarmalarm 214 IMP 337
diskI/O 111 RUN ISI 339
memoryutilization 111 mscworkingdirectory 83 LFM 324
networkI/O 111 MAU 344
swaputilization 111 MCI 338
powerproblem MEGRES 330
S
probablecause 209 MIP 342
save
PPXSubsystemDiagnostics 325 command 243 NAU 324
ppx_dump 325 PPX 325
schema
pro 70 RCI 338
databaseschema 140
probablecause securitymanagement SMI 339
communicationsprotocolerror 208 SSF 325
SSVRuseraccounts 67
configurationorcustomizationerror 208 SSP 326
Serviceoutage
congestion 208 Recoveringfrom 233 STA 319
corruptdata 208 TYP 320
settingsystemtime 134

Guide
Index 357
swaputilization tracefiles 238 messageformat 217

checking 121 traceableprocesses 236 search 220
collectedperformancedata 109 traceableresources 238 severitylevels 218
graphsandreports 120 tracingmodules 234 vlr_dump 341
performancestatistics 111 viewingtracefiles 244 vmstatcommand 117
swap_free transmitfailure volume
Memoryutilization 277 probablecause 209 volumelabelverifiedduringrestore 141
swap_total Trunkstatistics VSMSubsystemEvents 301
Memoryutilization 277 Usagecounters 276 vsm_dump 327
Swapping TYPSubsystemDiagnostics 320
Ofsharedaddress 88
swapping W
paging,memory 119 U wander
symbolforhomedirectory 83 ufsdump 147 unstableoscillator 129
synchronize ufsrestore 182 warrantysupport 29
NTPclientandserver 130 underlyingresourceunavailable workingdirectory 83
synchronizing probablecause 209 WYSIWYGformat
NTPserversandclient 133 universaltimecoordinated 129 performancegraphs 111
SystemAdministrationAccount 68 anddaylightsavingstime 134
systemadministrationaccount 68 usedbySolaris 130
systemadministrator UNIX X
superuseraccount 68 Solaris 82 xntpd
systembackup UNIXfilesystem daemon 130
restoringbackup 178 Description 82
ufsdump 147 description 82
systembackups 147 dump 147 Z
systemmirror unspecifiedreason zic 136
databaserestore 146 probablecause 209
systemtime uplink
checking 134 checkingstatus 91
drift 129 URL
GPS 129 http

offset 129 //www.ntp.org 130
Usagecounter 276
useraccount
T proaccount 69
tableversioning 140 root 83
technicalsupport superuser 83
hours 28 useraccounts 67
website 28 usermanagement 70
templatefile useradd 156
ntp.client 130 UTC
timezonefiles universaltimecoordinated 129
compiling 135 UTCtime
timezones configureUTCtimedisplay 135
compiler 136 utility
configuring 134 fsck 151,163
tm less 220
command 240 systemmirror 146
TMP ufsdump 147
mscworkingdirectory 84 ufsrestore 182
TOOL vlog 57,217
mscworkingdirectory 84 zic 136
topcommand 117 Utilization
tpm_dump 320 Memoryutilizationstatistics 277
tpmTmrDropalarm 213
tracefiles 238
syntax 238 V
tracetypes 234 verify,incommandlineinterface 45
tracing 246 version,displayingsoftware 66
archivingtracefiles 245 versioningdatabasetables 140
checkingtracestatus 239 Virtual
convertingtracefiles 245 VirtualMACaddress 87
removingtracefiles 246 virtualMACAddress 95
runningtraces 240 vlog 217
settingtracelevels 239 checkingeventLog 100

Guide
358 Index

Guide

Copyright2012 StarSolutionsInternationalInc.
1204600JacombsRoad
Richmond,BritishColumbia V6V3B1 Canada
www.starsolutions.com

Core Voice O&amp;M Guide

Загружено:

Сведения о документе

Авторское право

Доступные форматы

Поделиться этим документом

Поделиться или встроить документ

Параметры публикации

Этот документ был вам полезен?

Это неприемлемый материал?

Авторское право:

Доступные форматы

Core Voice O&amp;M Guide

Загружено:

Авторское право:

Доступные форматы

AllIPCDMA2000|AllIPGSM

U N I T E D S TAT ES G OVE R N M E N T LEG E N D S

STA R SO LU TIONS CO N FI DE N TI A L SonataCoreVoiceNetworkIS41R4.5.3|GSMR4.5.4Operations,MaintenanceandProvisioningGuide

PARTIII OPERATIONS ANDADMINISTRATION

STA R SO LU TIONS CO N FI DE N TI A L SonataCoreVoiceNetworkIS41R4.5.3|GSMR4.5.4Operations,MaintenanceandProvisioningGuide

STA R SO LU TIONS CO N FI DE N TI A L SonataCoreVoiceNetworkIS41R4.5.3|GSMR4.5.4Operations,MaintenanceandProvisioningGuide

STA R SO LU TIONS CO N FI DE N TI A L SonataCoreVoiceNetworkIS41R4.5.3|GSMR4.5.4Operations,MaintenanceandProvisioningGuide

10 DATABASE ANDSYSTEMBACKUP ANDRESTORE

STA R SO LU TIONS CO N FI DE N TI A L SonataCoreVoiceNetworkIS41R4.5.3|GSMR4.5.4Operations,MaintenanceandProvisioningGuide

STA R SO LU TIONS CO N FI DE N TI A L SonataCoreVoiceNetworkIS41R4.5.3|GSMR4.5.4Operations,MaintenanceandProvisioningGuide

STA R SO LU TIONS CO N FI DE N TI A L SonataCoreVoiceNetworkIS41R4.5.3|GSMR4.5.4Operations,MaintenanceandProvisioningGuide

STA R SO LU TIONS CO N FI DE N TI A L SonataCoreVoiceNetworkIS41R4.5.3|GSMR4.5.4Operations,MaintenanceandProvisioningGuide

STA R SO LU TIONS CO N FI DE N TI A L SonataCoreVoiceNetworkIS41R4.5.3|GSMR4.5.4Operations,MaintenanceandProvisioningGuide

STA R SO LU TIONS CO N FI DE N TI A L SonataCoreVoiceNetworkIS41R4.5.3|GSMR4.5.4Operations,MaintenanceandProvisioningGuide

STA R SO LU TIONS CO N FI DE N TI A L SonataCoreVoiceNetworkIS41R4.5.3|GSMR4.5.4Operations,MaintenanceandProvisioningGuide

STA R SO LU TIONS CO N FI DE N TI A L SonataCoreVoiceNetworkIS41R4.5.3|GSMR4.5.4Operations,MaintenanceandProvisioningGuide

Table 50 cpuCMShutDownClearingInformation 215

STA R SO LU TIONS CO N FI DE N TI A L SonataCoreVoiceNetworkIS41R4.5.3|GSMR4.5.4Operations,MaintenanceandProvisioningGuide

Table 105 CommandOptionsformau_dump 345

STA R SO LU TIONS CO N FI DE N TI A L SonataCoreVoiceNetworkIS41R4.5.3|GSMR4.5.4Operations,MaintenanceandProvisioningGuide

STA R SO LU TIONS CO N FI DE N TI A L SonataCoreVoiceNetworkIS41R4.5.3|GSMR4.5.4Operations,MaintenanceandProvisioningGuide

STA R SO LU TIONS CO N FI DE N TI A L SonataCoreVoiceNetworkIS41R4.5.3|GSMR4.5.4Operations,MaintenanceandProvisioningGuide

Notices Table 1listsnoticeiconsusedinthisguide.

Icon NoticeType Description

Text Table 2liststextconventionsinthisguide.

STA R SO LU TIONS CO N FI DE N TI A L SonataCoreVoiceNetworkIS41R4.5.3|GSMR4.5.4Operations,MaintenanceandProvisioningGuide

RelatedDocumentation Figure 1mapstherelationsamongthedocumentsintheStarSolutionsSonata

STA R SO LU TIONS CO N FI DE N TI A L SonataCoreVoiceNetworkIS41R4.5.3|GSMR4.5.4Operations,MaintenanceandProvisioningGuide

STA R SO LU TIONS CO N FI DE N TI A L SonataCoreVoiceNetworkIS41R4.5.3|GSMR4.5.4Operations,MaintenanceandProvisioningGuide

STA R SO LU TIONS CO N FI DE N TI A L SonataCoreVoiceNetworkIS41R4.5.3|GSMR4.5.4Operations,MaintenanceandProvisioningGuide

STA R SO LU TIONS CO N FI DE N TI A L SonataCoreVoiceNetworkIS41R4.5.3|GSMR4.5.4Operations,MaintenanceandProvisioningGuide

STA R SO LU TIONS CO N FI DE N TI A L SonataCoreVoiceNetworkIS41R4.5.3|GSMR4.5.4Operations,MaintenanceandProvisioningGuide

STA R SO LU TIONS CO N FI DE N TI A L SonataCoreVoiceNetworkIS41R4.5.3|GSMR4.5.4Operations,MaintenanceandProvisioningGuide

STA R SO LU TIONS CO N FI DE N TI A L SonataCoreVoiceNetworkIS41R4.5.3|GSMR4.5.4Operations,MaintenanceandProvisioningGuide

STA R SO LU TIONS CO N FI DE N TI A L SonataCoreVoiceNetworkIS41R4.5.3|GSMR4.5.4Operations,MaintenanceandProvisioningGuide

STA R SO LU TIONS CO N FI DE N TI A L SonataCoreVoiceNetworkIS41R4.5.3|GSMR4.5.4Operations,MaintenanceandProvisioningGuide

STA R SO LU TIONS CO N FI DE N TI A L SonataCoreVoiceNetworkIS41R4.5.3|GSMR4.5.4Operations,MaintenanceandProvisioningGuide

STA R SO LU TIONS CO N FI DE N TI A L SonataCoreVoiceNetworkIS41R4.5.3|GSMR4.5.4Operations,MaintenanceandProvisioningGuide

STA R SO LU TIONS CO N FI DE N TI A L SonataCoreVoiceNetworkIS41R4.5.3|GSMR4.5.4Operations,MaintenanceandProvisioningGuide

STA R SO LU TIONS CO N FI DE N TI A L SonataCoreVoiceNetworkIS41R4.5.3|GSMR4.5.4Operations,MaintenanceandProvisioningGuide

STA R SO LU TIONS CO N FI DE N TI A L SonataCoreVoiceNetworkIS41R4.5.3|GSMR4.5.4Operations,MaintenanceandProvisioningGuide

STA R SO LU TIONS CO N FI DE N TI A L SonataCoreVoiceNetworkIS41R4.5.3|GSMR4.5.4Operations,MaintenanceandProvisioningGuide

Command Description Syntax Example

STA R SO LU TIONS CO N FI DE N TI A L SonataCoreVoiceNetworkIS41R4.5.3|GSMR4.5.4Operations,MaintenanceandProvisioningGuide

STA R SO LU TIONS CO N FI DE N TI A L SonataCoreVoiceNetworkIS41R4.5.3|GSMR4.5.4Operations,MaintenanceandProvisioningGuide

STA R SO LU TIONS CO N FI DE N TI A L SonataCoreVoiceNetworkIS41R4.5.3|GSMR4.5.4Operations,MaintenanceandProvisioningGuide

Check CPU Utilization

Check Disk Space

Check Log Usage

STA R SO LU TIONS CO N FI DE N TI A L SonataCoreVoiceNetworkIS41R4.5.3|GSMR4.5.4Operations,MaintenanceandProvisioningGuide

OperatorTasks ThissectiondescribestheoperatortasksoutlinedinFigure 6,OperatorTask

Step Task Done Record/Comment

STA R SO LU TIONS CO N FI DE N TI A L SonataCoreVoiceNetworkIS41R4.5.3|GSMR4.5.4Operations,MaintenanceandProvisioningGuide

STA R SO LU TIONS CO N FI DE N TI A L SonataCoreVoiceNetworkIS41R4.5.3|GSMR4.5.4Operations,MaintenanceandProvisioningGuide

STA R SO LU TIONS CO N FI DE N TI A L SonataCoreVoiceNetworkIS41R4.5.3|GSMR4.5.4Operations,MaintenanceandProvisioningGuide

STA R SO LU TIONS CO N FI DE N TI A L SonataCoreVoiceNetworkIS41R4.5.3|GSMR4.5.4Operations,MaintenanceandProvisioningGuide

STA R SO LU TIONS CO N FI DE N TI A L SonataCoreVoiceNetworkIS41R4.5.3|GSMR4.5.4Operations,MaintenanceandProvisioningGuide

STA R SO LU TIONS CO N FI DE N TI A L SonataCoreVoiceNetworkIS41R4.5.3|GSMR4.5.4Operations,MaintenanceandProvisioningGuide

STA R SO LU TIONS CO N FI DE N TI A L SonataCoreVoiceNetworkIS41R4.5.3|GSMR4.5.4Operations,MaintenanceandProvisioningGuide

STA R SO LU TIONS CO N FI DE N TI A L SonataCoreVoiceNetworkIS41R4.5.3|GSMR4.5.4Operations,MaintenanceandProvisioningGuide

STA R SO LU TIONS CO N FI DE N TI A L SonataCoreVoiceNetworkIS41R4.5.3|GSMR4.5.4Operations,MaintenanceandProvisioningGuide

STA R SO LU TIONS CO N FI DE N TI A L SonataCoreVoiceNetworkIS41R4.5.3|GSMR4.5.4Operations,MaintenanceandProvisioningGuide

STA R SO LU TIONS CO N FI DE N TI A L SonataCoreVoiceNetworkIS41R4.5.3|GSMR4.5.4Operations,MaintenanceandProvisioningGuide

Memory Data Lines Test, PASSED.

Core Voice O&M Guide

Core Voice O&M Guide