Introduction to L1-FM BSS Troubleshooting

AP ANTIVIRUS FUNCTION FAULT

Connect to RDC In the Start menu select All Programs | Computer Associates | eTrust | eTrust Antivirus | eTrust Antivirus The Local Scanner window is opened In the Local Scanner window, in the Scanner menu, select Signature Update Options. The Signature Update Options window is opened. In the Signature Update Options window, select the Incoming tab. The Download Sources list is opened. In the Download Sources list, select the predefined item and click the Edit button. The Source Select window is opened. In the Source Select window, in the Proxy Name field, enter the address and port number of the proxy server. Click the Ok button. Now the antivirus software can access the internet via the proxy. Ping the proxy server to verify that it is responding. Use command - ping <address>[:<port number>] In the Signature Update Options window, Click the Download Now button. The Antivirus Download Monitor window is displayed. Cease the AP ANTIVIRUS FUNCTION FAULT alarm.

AP FILE PROCESSING FAULT

It indicates that the AFP function has failed to transfer a file or directory. There is backup data stored in the AP that the user has to transfer manually.

Check destination path details cdhls -l ossdest1 verify if the connection is working properly cdhver cletus see all file details afpls -ls see files of cletus tranfer queue, If status is FAILED then problem arise. Try to send Force fully Afpls -l cletus Forcefully transfer to OSS. READY state will come. afpfti -f cletus

AP REBOOT
AP node was rebooted because of a fault in the AP node. The cause parameter shows the reason for the AP Reboot. When the cause is Fault initiated, Cluster failure, Unexpected shut down

Take basic health check of APG Cease the alarm if APG health is fine

AP SYSTEM ANALYSIS
The Object is Logical Disk and the Counter is % free space. Drive- C,D,G,K The Object is Memory and the Counter is Available Bytes. change the timeout value of the telnet server - Remote connection via pcANYWHERE for Windows NT -4 -Remote connection via RDC for Windows 2003

Memory Increase session timeout counter Check node is 2003/NT and find Node IP Connect RDC-->BSC IP->Login (administrator)PW. StartAll program->Program telnet server>Program telnet manager. Explorer-->User-->Default-->Enter. Increase the Idle session time out value as per DT given. Don't forget to LOGOFF from RDC Check ALIST if Alarm is there CEASED it from alarm list.

Drive cleaning M drive delete old APG backup files K drive delete old APG backup files delete old ACS logs from location K:\acs\logs\alog\logfile C drive c drive contains system software, so next level should be referred before deleting anything from c drive.

G drive from G drive delete old file (APG or CP backup) in ftpvol Aploc; G: Cd ftpvol Cd lm cd.. Cd apg40 Dir Del old apg/cp backups mml

D drive D is duplicate of C
Delete temporary internet files & application data from local setting. Del *.bak/s/a Del *.evt /s/a D:\winnt\profiles Dir*nt86*/a/s Del *nt86*/s

AP PROCESS STOPPED
Consult the next level and start the stopped process cluster res <resource name> /on /wait Cease the alarm, if alarm is not cleared itself

AP SYSTEM CLOCK NOT SYNCHRONIZED

Check the cp apg timing Caclp; - shows cp time and date Time /t shows apg time Date /t shows apg date Caclc change cp date and tme time 11:14 PM apg time is set 11:14 PM Note CP time should be changed during night followed by a small restart and while changing APG time make sure operation is done on passive node, if time of active node needs to be changed then first reboot the system by prcboot so that sts are not affected.

AP Not Redundant
telnet the node which is down If unable to telnet node then escalate to next level for power cycle. Else Restart ACS_PRC_Clustercontrol and clussvc If node not comes up try by telneting the active node and reset other node by fcc_reset other If node is still down escalate to next level

APPLICATION DETECTED SOFTWARE ERROR

Check the severity of software block and if the error intensity is high recommend for software upgrade else cease by syrae
Commands Syrip:survey; LASIP:BN=H'025C; RAEPP:ID=BSSRELEASE; IOEXP; SYRAE:EVENT=53; PCORP:BLOCK=ALL; SYELP; pcorp:block=missra; syrtp;

AUDIT LOG DEACTIVATED

Use command alogact to activate alog

BACKUP INFORMATION FAULT

Check the fault code and act accordingly Commands Sybfp:file; Sybue; Sybup:file; Sytuc; Sybui:disc; cpfmkfile -l 2048 -c RELFSWX RELVOLUMSW cpfcp RELFSW2 RELFSWX cpfrm -r RELFSW2 Cpfls L:\FMS\data\CPF\RELVOLUMSW>DIR

COMMAND LOG BLOCKED

Check the alarm and fault code, generally FAULT CODE 155 is seen. allip:alcat=apz,acl=a2; Activate the command log using command sycli; Check the alarm will be cleared allip:alcat=apz;

CP Fault
Take cp backup and cp health check Give repair to cp (repci, repce and recci) Again take cp backup and cp health check If fault is not rectified than escalate to next level for card replacement.

CPT Fault
Make the resources PMCrHW_1 and PMCrSDLC_1 offline and online cluster res PMCrHW_1 /off /wait cluster res PMCrHW_1 /on /wait cluster res PMCrSDLC_1 /off /wait cluster res PMCrSDLC_1 /on /wait Check alarm in allip, if alarm is still persisting escalate to next level

IO Printout Destination Faulty

Delete and recreate the faulty destination
cpdlist -l cpdremove AD-4 cpdlist AD-4 cpdinsert -f k:\mcs\logs\pds\SystemSBDev.log -v 0-255 AD-0 cpdinsert -i 10.34.20.21:50000 -b -v 0-53,55-255 -e crlf -n 04 AD-4 cpdinsert -f g:\ftpvol\UEBGKP2.log -v 0-255 AD-8

RP Bus Fault
Replace all the faulty RPs. Take cp backup and cp health check Give repair to cp (sarpi, repci, repce, recci and sarpe) Again take cp backup and cp health check

RP Fault
Give block/deblock to RP or RP can be repaired by repri, remri and recri. If fault is not resolved than RP card need to be replaced. allip:alcat=apz; exemp:rp=40,em=all; Exrpp:rp=rp no; blrpi:rp=RP NO,forced; blrpe:rp=RP NO; repair rp repri:rp=rpno; remri:rp=rpno,pcb=<>; recri:rp=rpno;

RP Card Replacement
Find the position of faulty RP expop:rp=361; dbtsp:tab=rpsrpbspos; Check The status of RP exrpp:rp=361; Block the RP if it is not MBL blrpi:rp=361,forced; Inform the CP that regarding maitence work so that it may not switch. sarpi; Turn ON and OFF LEDs tebic:led=on,rp=361; - It will ON all LED tebic:led=off; - It will off all LED but Faulty LED will glow. NOW CHANGE THE RP - ask the BSC coordinator to replace the card. Deblock the RP blrpe:rp=361; Check The status of RP exrpp:rp=361; END sarpi sarpe;

BLOCKING SUPERVISION OF DEVICE

Find the devices which are in blocked state and if the device are unused then make devices out of service else deblock them. Circle team coordination is taken for detail of devices. BLOCKING SUPERVISION OF DEVICE DETY DETY LVB NDV BLO dety lvb ndv blo limited value blocking end

Fan Magazine Fault

Check the alarm allip:alcat=apt,acl=a2; find the RP of FAN MAGAZINE and blk/dblk it blrpi:rp=240,forced; blrpe:rp=240; Check the alarm, alarm will be cleared if not than escalate to next level. allip:alcat=apt,acl=a2;

Group Switch Fault

Give block/deblock the faulty group switch unit and observe it for sometime, if it malfunctions then inform circle team to jack-out/jack-in the card or to replace it if it again malfunctions.
gdstp; gdbli:unit=<unit name>; gdtei:unit=<unit name>; gdble:unit=<unit name>; Gsstp Gsbli gsble

Radio Transmission Transcoder Pool Self Configuration Timeout

The status in the rrpsp:trapool=all has been changed from halted to active, after confirming that RNOTRA and POOLACT are equal in rrtpp:trapool=all, now the alarm rrpsp:trapool=all; rrpar; check alarm in allip , alarm will be cleared

Radio Transmission Transcoder and Rate Adaptor Fault

find the mux & associated associated devices to the blocked devices and blck/dblck the associated devices with the mux device.
stdep:dev=RRTGD-9417; stdep:dev=RRTGD-9410&&-9420; rrtbi:dev=RRTGD-9413&&-9417,forced; rrtbe:dev=RRTGD-9413&&-9417;

if fault not removed , blck/dblck the RP and check the status of SNT also , if devices are still coming in ABL then change the card

radrp:dev=RTTGD-4101; ntstp:snt=RTTGS-16 ;

Radio Transmission Transcoder Pool Mean Hold Time Supervision

Check the devices and reset them
rrmap:trapool=all; rrtbi:dev=RTTG1D-7323,force; rrtbe:dev=RTTG1D-7323; stdep:dev=RTTG1D-7315&&-7328; rrmap:trapool=all;

LICENSE MANAGEMENT EVENT LOG OUTPUT ERROR

FILENAME LMAFILE FAULT TYPE FILENAME ERROR

either LMAFILE is incorrectly defined or is not defined in EXCHVOLUME if incorrectly defined than delete and redefine , else define it cpfls cpfls -ls LMAFILE cpfmkfile -f inf -l 125 -r LMAFILE EXCHVOLUME

MIRROR DISKS NOT REDUNDANT

Take APG health check and give raid rebuild consulting the next level support
for C/2 raidutil -L logical raidutil -L physical raidutil -a rebuild d#b#t#d#
for C/4 megarc -ldInfo -a0 -Lall megarc -doRbld -a0 -RbldArray[0:02]

Command Log Output Error FC 31

cluster res FMS_CPF_server /off /wait cluster res CPS_BUAP_parmgr /off /wait cluster res CPS_BUAP_filemgr /off /wait cluster res CPS_BUAP_loader /off /wait cluster res CPS_BUFTPD /off /wait cluster res CPS_BUSRV /off /wait cd L:\FMS\Data\CPF\RELVOLUMSW del RELCMDHDF.attribute rd /s /q RELCMDHDF cluster res FMS_CPF_server /on /wait cluster res CPS_BUAP_parmgr /on /wait cluster res CPS_BUAP_filemgr /on /wait cluster res CPS_BUAP_loader /on /wait cluster res CPS_BUFTPD /on /wait cluster res CPS_BUSRV /on /wait cpfmkfile -c -l 512 RELCMDHDF RELVOLUMSW -Alarm should be ceased.

RELOAD PARAMETERS INVALID

Print the RELCMDHDF file details in APG40, and note down the last subfile number cpfls -ls RELCMDHDF Initiate the creation of a new subfile by command, nnnnnnn should be 1 higher than the last subfile noted in the previous step. Also it should be of seven digits. SYCSI:SFN=nnnnnnn; Check the alarm allip:alcat=apz;

CP File Output, AP CP File System, Transfer Queue Fault

afpls afprm -z BCDCFILE OSSDEST1 afpls -ls BCDCFILE cdhls -l ossdest1 cdhver ossdest1 cluster res stmmp -L cluster res FMS_CPF_server /Off /WAIT cluster res FMS_CPF_server /On /WAIT afprm MTRFIL62 afpdef -r -1 -t 300 -d 60 -g pppp -j local MTRFIL61 OSSDEST8 cdhls -l

CDH RESPONDING READY DIRECTORY WARNING

connect to RDC go to K:\aes\data\cdh\ftp\OSSDEST9\Ready path delete all old files cease the alarm

SIZE ALTERATION
check sae dbtsp:tab=saactions; saaep:sae=500; INCREASE NI accordingly SAAII:SAE=348,NI=450; dbtsp:tab=saactions; SAADI:SAE=348,NI=450; SAAII:SAE=501,BLOCK=C7CO,NI=55; saaep:sae=466; dtqsr:dip=all,es,ses,sf; dtqsr:dip=all,unacc,degr; dtqsr:dip=all,unacc,degr;

LICENSE MANAGEMENT EVENT LOG OUTPUT ERROR

check if file is defined which is in alist , else File was not found: LMAFILE cpfls -ls LMAFILE define the file LMAFILE cpfmkfile -f inf -l 125 -r LMAFILE EXCHVOLUME cpfls -ls LMAFILE now alarm will be cleared from alist alist

STATISTICS AND TRAFFIC MEASUREMENT FILE ACCESS FAULT

chech the alarm allip:alcat=apt,acl=a3; Check the alarm in APLOC alist Check all the MP are Running or not. It may be Waiting state for next stats stmmp -L go S drive and go the given path S:\STS\data\Deliverybuffdir Check Recent STS are coming in Every Hour or Not. CD STS\data\Deliverybuffdir>dir Make OFF the Resources cluster res stsmain /off /wait Check that some other Res become OFFLINE ,Which have to make ONLINE cluster res | findstr -ive online cluster res stsmain /on /wait cluster res stsprov /on /wait cluster res stsconv /on /wait cluster res stsopcf /on /wait NOTE:if stmmp -L FAULTY comes then try to OFF/ON the RES or give a BOOT to the NODE. stmmp -S 1010 stmmp -R 1010

SWITCHING NETWORK TERMINAL FAULT

block/deblock snt ntstp:snt=ETM2-4; ntbli:snt=ETM2-4; ntble:snt=ETM2-4; block/deblock mux device ntcop:snt=ETM2-4; gdcop:unit=XM-A-0-0; gdcop:unit=MUX34-A-0 gdstp block/deblock RP blrpi:rp=RP NO,forced; blrpe:rp=RP NO;

EXTERNAL ALARM RECEIVER FAULT

state for all reciever should be WO exalls all MR and AC should be N N , N Y means fault is there exalls -l block exal exalblk exal-1-0 delete exal exalrm exal-1-0 redefine exal exaldef -o yes -s "MAINS FAIL" -C 8 -c 0 exal-1-0 exalclear exal-1-0 deblock excal exaldeblk exal-1-0

cluster res MCS_ALEC_ADM /off /wait cluster res MCS_ALEC_ADM /on /wait