Anthony B Lawal

STE Feb 2014

Static analysis

2. Static analysis
2.1 Introduction Static analysis is a software testing technique designed to analyse a program with the aid of a tool, without actually running the program. Static analysis checks the programs assembly to verify that the code conforms with relevant design guidelines, coding standards, code quality formula, system architecture and design documents1. Some of the wide range of features provided by static analysis include2 type checking i.e. checks that the appropriate types are assigned to variables, style checking that best practice rules are complied with. Static analysis provides artifacts to help with program understanding and importantly helps find bugs in the code. Using some of the aforementioned features of the tools, PriorityQueue program is tested with to measure its quality of the program, as well as demonstrate the strenght and the weakness of static analysis in general. The program is analysed with the following tools - FindBugs, PMD and Checkstyle. A brief description of these tools and a report containing the results from the analysis is reported in the essay. The part is concluded with presenting the merits and demerit of using static analysis in professional practice, using the result of testing PriorityQueue as a casestudy.

2.2 PriorityQueue Code Analysis 2.2.1 Analysis with FindBugs FindBugs analyses a java codes .JAR files by looking for issues and comparing the code against a certain bug patterns. Analysing PriorityQueue code with FindBug analysis, with the most issue sensitive configuration did not produsce any result.

1. Ritchie, S. D. (2011) Pro .NET best practices. Berkeley, CA: Apress, pg235 2. Chess, B. and West, J. (2007) Secure programming with static analysis. Upper Saddle River, NJ: Addison-Wesley. pg23

Anthony B Lawal

STE Feb 2014

Static analysis

2.2.2 Analysis with PMD PMD is a static analysis tool developed to test Java codes by scanning the source code for common coding mistakes. Some of the issues the tools is designed to find include1, empty try/catch/finally/switch blocks, dead codes containing unused variables, parameters and private methods as well as highlighting overcomplicated expression that could be simplified. PriorityQueue code was analysed with PMD. Some violations are spotted by the tool and highlighted in the result. The result shows 30 methods with violations, 352.9 violation/KLOC and 3 violations per method. Information about the violations provided by PMD reports are documented in Table 1: PMD Violations Overview on page 4. The most serious violation is ranked Blocker. A blocker rule violation was found in PriorityQueue. The violation relates to AvoidThrowingRawExceptionTypes violation. Get() method is one of the two methods with this violations:
public String get() throws Exception { if (empty()) throw new Exception(Priority queue empty); }

The standard practise prescribes that a sub class exception or an error is more appropriate than throwing a generic exception. Throwing generic exception means that a caller class will not know the expected exception from the method. PMD found 3 instances where systemPrintln was used in the code. The better practise is to use a Logger. Logger is better for processing messages, as it offers flexibility, in terms of control over the message. The message can be classified, as error, warning and info, and a rule can be set on the visibility of the messages across the project. Methods with V i o l a t i o n s / Violations/Meththe violation KLOC od 2 23.5 0.20

Priority

Rule violations AvoidThrowingRawException Types SystemPrintln SimplifyBooleanReturns AvoidLiteralsInIfCondition

Blocker

Critical

35.3

0.30

Urgent

11.8

0.10

Urgent

11.8

0.10

1. http://en.wikipedia.org/wiki/PMD_(software)

Anthony B Lawal

STE Feb 2014

Static analysis

Urgent

ShortVariable MethodArgumentCouldBeFinal LawofDemeter L o c a l Va r i a b l e CouldBeFinal RedundantFieldInitializer UnusedLocalVariable SignatureDeclearThrowsException BeanMemberShouldSerialize OnlyOneReturn AvoidFieldNameMatchingMethodName PackageCase

35.3

0.30

Urgent

35.3

0.30

Urgent

23.5

0.20

Urgent

35.3

0.30

Urgent

11.8

0.10

Urgent

11.8

0.10

Urgent

23.5

0.20

Urgent

47.1

0.40

Urgent

11.8

0.10

Urgent

23.5

0.20

Urgent

11.8

0.10

Table 1: PMD Violations Overview

Anthony B Lawal

STE Feb 2014

Static analysis

2.2.3 Analysis with Checkstyle Checkstyle was used to analyse the code for compliance with Java coding standards. Rules are already defined in the tool and the user have the option to define their custom rules. Coding styles can be effectly enforced in a development team. The result of testing PriorityQueue with CheckStyle is shown in Figure 1: Checkstyle violation graph.

Figure 1: Checkstyle violation graph

PriorityQueue code contains 601 violations in 20 categories of violation. Categories contain stylistics issues which range from methods not designed for extension, to missing comment. The analysis highlights the quality of PriorityQueue code. With an average of 3 violation per method, 352.9 per violations per KLOC as shown in PMD report. Furthermore, the programs 123 line of code contains 601 violations. In practice these violations and issues will be analysed within the context of the projects quality agenda and priorities. However, given the size of PriorityQueue code, the number of violations shows that the quality is low.

Anthony B Lawal

STE Feb 2014

Advantages of static analysis

2.3 Advantages of static analysis Coding standards are mostly derived from the intensive study by industry experts. The experts analysed the sources of bugs and then correlated the bugs and specific coding practices1, coding standards are devised to safeguard against common coding mistakes causing these bugs. Static analysis helps in ensuring that these standards, the fundamental rules of the language are complied with by developers. StyleChecker found 20 class of violations containing 601 violations One of the violations highlighted by PMD is the use of raw exceptions. The specific type of exception expected from this methods in this violation is not defined. Using raw exception will not provide helpful information or expectation to the caller of this method, therefore the caller may not be able to handle the exception and consequently crash. Resolving these violations and issues will certainly improve the quality of the code. Human perception are biased and distorted by their beliefs, expectations, context, needs, motives and desire2. Weaknesses in human perception are eliminated by using static analysis. As aforementioned, the analysis discovered hundreds of violations in a small program like PriorityQueue, a manual review of this code size will not achieve this same quality of analysis. Moreover, it will be practically impossible to manually review an enterprise scale application with million of KLOC. Static analysis offers the precision and scalability a manual analysis can not achieve. Static analysis reduces development costs by enforcing the development of a readable, maintainable and efficient code. Developers spend time and increases cost by trying to locate the part of the code containing the defects found through Black-box testing. By contrast, Static analysis of PriorityQueue demonstrates that the code containing a violation can be reached easily. Therefore, the timescale for debugging and development is significantly reduced. The efficient use of static analysis can reduce development cost as bugs are found and fixed earlier, support efficiently spending development time and the quality of the program will be improved

2.4 Disadvantages of static analysis False positives from static analysis can provide a serious challenge. False positives are violations or defects found where there is none. The most effective static analysis tools can still produce false positives of between 30 to 100 %.3 False positives can be a distraction and causing loss of time and money. The result from these analysis, as indicated above, are mostly borderline violation cases. Therefore this creates the risk of subjective judgement of the person analysing the result. This could result in serious issue being declassified as low priority. Also this further analysis is time consuming and has the potential to reduce the value of the entire analysis. Only 2 of 30 violations reported by PMD is categorised as Blocker (severe)

1 http://www.parasoft.com/aep/aep_practices.jsp?practice=CodingSt 2 http://psych.princeton.edu/psychology/research/pronin/pubs/2007%20Bias%20Perception.pdf 3 Engler, D., Chelf, B., Chou, A., Hallem, S.: Checking system rules using system- specific, programmer-written compiler extensions. In: Proceedings of Operating Systems Design and tion (OSDI). (2000) Implementa

Anthony B Lawal

STE Feb 2014

Conclusion

False negatives in the result could make the exercise seem less productive. FindBugs did not report a single bug in the code. Comparing the number of bugs found in this program measured with the number of bugs found by these static analysis tools shows that static analysis is not as effective as other testing methods in finding bugs in the code. Static analysis uses approximation to determine the goal of the program and this approximation algorithm does not always determine the specification for which the code is written. Specification being the standard against which the programs quality should be measured. This weakness is shown in the bug in size() and capacity(), and some of the bugs missed by the tools.

2.5 Conclusion Static analysis certainly will add value to the quality of developed software, as highlighted by the advantages of using it. However, an outstanding technical skill will still be required in both setting up the test and the analysis of the result.