Advantage Faults Overview

Table of Contents
Advantage Faults Overview ...................................................................................................................................... 1 Functional Faults .................................................................................................................................................. 2 J2EE faults ............................................................................................................................................................. 3 Web Services Faults.............................................................................................................................................. 4 Database Faults .................................................................................................................................................... 4 .NET Faults ............................................................................................................................................................ 5 SA Fault ................................................................................................................................................................. 5

Functional Faults
These faults are turned on/off through admin page/functional faults section 1. Calculate balance incorrectly in the money transfer mode. When enabled, balance will be calculated incorrectly for each money transfer made. To visually check this fault: a. turn the fault on and go to the money transfer page and transfer some amount from one account to another b. go to accounts page and open transactions for one or both of the accounts you used for transaction and observe incorrect balance on the accounts

2. Break the Continue image button on the transfer page a. turn the fault on and go to the money transfer page and try to transfer some amount from one account to another b. When you click Next you can observe an error 500 with Java exception 3. Break the Account Links on the summary page a. turn the fault on and go to the account summary page. Click on one of the accounts or account numbers and observe an error 500 message instead of the account transactions 4. Expose the Account Numbers a. turn the fault on and go to the account summary page. Instead of the masked account numbers (xxxxxxx874) you will see full account numbers 5. Hide Credit Accounts a. This fault is supposed to influence the way list of available accounts is displayed. Since we don't have credit accounts in this release, this fault has no visual effect.

J2EE faults
These faults are turned on/off through "admin page/J2EE faults" section 1. Memory leaks a. turn on the fault and go to bill pay / payment history page (just click on Bill Pay link) b. This will trigger the memory leak logic that will allocate additional memory every second until stopped. c. Memory leaks can be observed in the application server logs and in windows task manager for application server's java.exe process. Please note that due to the nature of JVM additional memory will be shown in task manager only when leaked memory will go over the currently allocated JVM memory allowance. In this case you can observe java.exe process to consume more memory with each allocation. d. Every second, after memory leak thread has started, a chunk of 16MB is allocated but not more than 1GB e. To stop memory leaks - turn off the fault and visit Bill Pay page. This will trigger the memory leak logic to release all allocated memory 2. Activate a delay inside the servlet layer for Tx over specified amount a. Turn this fault on and specify threshold amount (e.g. 100) and delay (e.g. 20 seconds). Also you can include random delay (e.g. from 20 to 30 seconds) by clicking Include Random checkbox and specifying Min and Max fields b. then try to transfer amount that is larger than $100 c. Observe page loading time increase by the number of seconds specified d. To verify the delay check application server logs for line like this: INFO: ==> Delaying execution for: 31562 msec 3. Create a long and inefficient SQL call for Tx over specified amount a. Turn this fault on and specify threshold amount (e.g. 100) b. then try to transfer amount that is larger than $100 c. If the DB server is not very fast then you might be able to visually see a delay of 2-5 seconds d. Inefficient query can be found in application server logs. It uses RIGHT JOIN. Usually it looks something like this: SELECT DISTINCT COUNT(*) FROM APP_TRANSFER T RIGHT JOIN FAULT_SUPPORT_DATA FSD ON (T.REFERENCE_NUMBER NOT LIKE '%1%' AND T.REFERENCE_NUMBER = FSD.STRING_ONE) WHERE FSD.STRING_ONE NOT LIKE '%FAULT%'") e. Also if DB tracing is enabled on your DB server you can observe all queries that are executed and check for queries like one mentioned 4. Create unnecessary allocation of connections for Tx over specified amount a. Turn this fault on and specify threshold amount (e.g. 100) b. then try to transfer amount that is larger than $100 c. There will be 13 additional connections open to the database d. you can look for line like "Created unnecessary connection #13 of 13" in e. Also, you can check how many connections are open on the database side

Web Services Faults

1. Activate memory leak a. Turn on this fault and try to pay any bill b. Each time you pay a bill - additional memory is allocated (currently 16MB) but not more than 1GB c. This behavior is made specifically different from memory leaks in servlets/JSPs to cover more cases of memory leaks in our application. This memory leak is request driven while servlet memory leak is time driven. 2. Activate slow performance a. This fault works the same way as J2EE delay works. Enable it and specify number of seconds or random number of seconds and observe execution delayed in the web services layer for amount specified 3. Activate failure of CreateCheckingAccount service a. Enable this fault and try to create a new account b. Observe following error in your browser: EJB Exception: : java.lang.RuntimeException: Checking account creation failed. Reason: Fault 14 is enabled at com.advantage.businesslogic.createCheckingAccountService.ws.CreateCheckingAccountService.create Account(CreateCheckingAccountService.java:52)

Database Faults
1. Activate Database index corruption a. Turn on this fault and see that index FAULT_SUPPORT_DATA_IDX1 is dropped b. Now try to create new account and see that the same query takes 10 seconds instead of 5-15 msec c. Fault_Support_data is a large "artificial" table with lots of data. This table is needed because by default our application does not have a lot of data so broken indexes will not have any visual effect. d. Also you can observe longer queries by employing Oracle/MS SQL monitoring and logging tools e. Query executed: select fsd from FaultSupportData fsd where fsd.stringOne=:value 2. Activate exhaustion of database locks a. Turn on this fault and try to transfer money from one account to another b. Using DB monitoring tools observe that a very lange number of DB locks is generated in stored procedure fault_support.generate_locks. 3. Activate 100% CPU Utilization on DB server a. Turn on this fault and observe CPU usage spikes on your DB server every time user clicks on the account link on account summary page to see account transactions. DB usage spike depends on the performance of the CPU installed and number of CPUs/cores. Sample behavior: quadcore Phenom II Black will consume only 30% of all cores for about 12-15 seconds, while dual core Opteron will consume 67% of both cores for about 16 seconds.

.NET Faults
1. Activate slow performance a. This fault works the same way as other "slow performance" faults. b. Activate fault and try to load Order checkbooks page c. Every time the page is loaded delay for specified number of seconds is activated d. Also when this fault is activated "CreditRiskService" activates delay every time it is called e. To check/verify the delays: i. Order Checkbooks section: 1. Turn on the fault 2. Load Order checkbooks page 3. Observe slow performance and check IIS logs on the front-end server for delay statements ii. CreditRiskService 1. Turn on the fault 2. Try to create a new account with ownership "Joint" 3. Specify all co-signer data and try to created account 4. Observe slow performance and check IIS logs on the back-end server where this service is hosted for delay statements

SA Fault
This fault is activated by replacing "creditcards.jsp" file with a "faulty" one. "Faulty" JSP throws an exception that can be detected by SA tool. JSP file along with a sample BAT file to automate it's deployment is located in Advantage distribution package under /SA folder. Please follow the instructions on how to find a temporary WL or WAS folder where unpacked WARs are located and locate creditcards.jsp, then replace it to activate the fault.