How to troubleshoot my sap system when it was down? What are the things I have to check?

A Basic Technical Troubleshooting:

Perform initial analysis.

Identify if something has changed (software or hardware installation, scripts, profile parameters,
database structure).

System (hardware) is down:

- Check console messages.
- Reboot and keep important system log files.
- Check hardware components.
- Analyze with diagnosis tools.
- Escalation procedure: call hardware support.
- Check main log files.
- Analyze with diagnostic tools.
- Start up/shut down log files.
- Check operating system log files.
- Identify other problems (startup/shutdown, disks and operating systems, database, operational R/3,
performance).

Startup and shutdown problems (R/3 and application servers):

1. Check processes or services at operating system level.
2. Check whether database is up.
3. Check connection with database (i.e., tnsping, R3trans).
4. Check whether file systems are full.
5. Check for network problems in the server network.
6. Check backup log for problems after backup.
7. Check for problems after system crash.
8. Check for problems with PDC file server.
9. Check files permissions and authorizations.
10. Manually start up or shut down database and then the application server.
11. Check hosts and services files.

Operating system problems:

1. Check operating system log and event viewers.
2. Check disks and file systems.
3. Check directories and permissions.
4. Check path names.
5. Check security.

Database problems:
- Starting and stopping database
- Refer to startup problems.
- Check database specific error log file.
- Database storage status
- Check database error numbers.
- Check database alerts.

R/3 application operational problems:

1. Check whether all users are affected.
2. Check whether there is no work at all, or whether work is just limited.
3. Check whether application servers are available (SM51). Check with standard monitoring transactions:
SM50, SM04, SM12, SM13, ST22, SM37, and SP01.
4. Check trace files on work directory (/usr/sap/<SID>/<instance>/work).
5. Check SAP systems alerts.
6. Analyze database problems.

Specific operational problems:-

Problems with printing:

A. Check network.
B. Check SP01/SPAD spool request logs.
C. Check system log.

Problems with batch input:

A. Check directories and files.
B. Check SM35 logs.
C. Check system log.

Problems with lock entries:

A. Check lock entries.
B. Check update process.

Problems with update process:

A. Check update process.
B. Check database.
C. Check network connections.

4. Problems with enqueue:

A. Check database.
B. Check instance parameters.
C. Check operating system processes.

5. Problems with message server:

A. Check processes.
B. Check network.
C. Check log files at work directory.

6. Problems with background processing:

A. Check job log.
B. Check background system consistency.
C. Check file systems.
D. Check system log.

7. Performance problems:
1. Perform workload analysis.
2. Perform debugging.
3. Perform runtime analysis.
4. Perform SQL trace.
5. Check for operating system problems.
6. Check disks for input/output bottlenecks.
Resolution process:
1. Search or SAPnet notes for hints, solutions, or workarounds.
2. Escalation: enter message on SAP hotline or SAPnet.
3. Enter problem on hardware partner hotline.
4. Follow problem actions and SAP indications.
5. Apply patches and corrections; test and verify results.

SAP Administrator Daily Activities

SAP DAILY ACTIVITIES

1] Check that all the application servers are up:

sm51 SAP Servers
sm04/al08 Logon Users

2] Check that daily backup are executed without errors

db12 Backup logs: overview

3] SAP standard background jobs are running successfully. Review for cancelled and critical jobs.
sm37 Background jobs--- Check for successful completion of jobs. Enter * in user-id field and verify that all
critical successful jobs and review any cancelled jobs.

4] Operating system Monitoring

st06

5] Extents monitoring
db02 Database monitoring--Check for max-extents reached

6] Check work-processes(started from sm51)

sm50 Process overview-- All work processes with a running or waiting status.

7] Check system log

sm21 System log-- Set date and time to before the last log review. Check for errors ,warning, security,
message-bends, database events.

8] Review workload statistics

st03 Workload analysis of <sid>
sto2 tune summary instance

9] Look for any failed updates

sm13 update records

10] check for old locks

sm12 lock entry list

11] Check for spool problems

sp01 spool request screen-- check for spool that are in request for over an hour.
12] Review and resolve dumps
st22 ABAP Dump analysis

13] Checking .trc file in SAP trace directory for block corruption on daily basis.
C:\ORacle\sid\saptrace

14] Archive backup

brarchive -f force -cds -c
Insert the archive backup tape

15] Review NT system logs for problem

-> NT system log- look 4 errors or failures
-> NT security log- failed logon 2 sap servers
-> NT Application log -look 4 errors or failures

Tcodes used for Daily System Monitoring

After running daily system monitoring transaction, what should we check for:

In st22 look for the core dumps if any and report to the respective consultants and try to know why it happened.

In sm21 try to check for errors.

In sp01 try to see if any spool jobs have failed.

In st02 look if any swaps are happening, swaps are not good for performance.

In st04 look for Database alert logs and Performance.

In st03 look for ratio hits.

In sm59 look for connectivety tesing if there are other systems also connected to your SAP R/3 system

In db13 look if the jobs have run successfully.

In sm37 look for any cancelled scheduled job and take action appropriately.

In sm12 look for any pending locks from the previous days.

In sm13 look for any hanged updates, or updates pending for long or updates in PRIV mode.