JOURNAL OF COMPUTING, VOLUME 4, ISSUE 10, OCTOBER 2012, ISSN (Online) 2151-9617 https://sites.google.com/site/journalofcomputing WWW.JOURNALOFCOMPUTING.

ORG

66

Enhanced Quality Metrics for Identifying Class Complexity and Predicting Faulty Modules Using Ensemble Classifiers
C. NEELAMEGAM Dr. M. PUNITHAVALLI Abstract Software industry is increasingly relying on software metrics to determine the extent of software quality through defect identification. Detection of faults help to reduce time and cost spent during every phase of software life cycle. Identifying quality metrics is a challenging task due to the ever changing software requirements and increasing complexity and size of the software application. Using quality metrics for fault detection is a two-step process, where the first stages measures the complexity of the software module, which is used in the second step to predict faulty modules. In this paper, along with the traditional object oriented metrics, four new metrics are proposed. An ensembling classification model that combines three classifiers is proposed to predict faulty modules in C++ projects. The performance of the proposed system is analyzed in terms of accuracy, precision, recall and F Measure. The experimental results showed positive improvement in the performance of prediction with the inclusion of the proposed metric and ensemble classifier. Index Terms Class Complexity, Defect Detection, Ensemble Classification, Objected Oriented Software, Quality Metrics, Software Quality.

1 INTRODUCTION
Software quality metrics are methods that quantitatively determine the extent to which a software process, product or project possess a certain quality attribute. They are used to measure software engineering products (design, source code, etc), processes (analysis, design, coding, testing, etc.) and professionals (efficiency or productivity of an individual designer). Techniques and methods that identify and predict faults using these quality metrics has gained wide acceptance in the past few decades (Catal and Diri, 2009; Chowdhury an dZulkernine, 2011) as they have direct impact on the software products time, cost and scope. The high usage of software system poses high quality demand from users, which results in increased software complexity. Fault prediction is a strategy to identify faulty parts of a program, so that the testing process can concentrate only on those regions. This will improve the testing process and indirectly help to reduce development life cycle, project risks, resource and infrastructure costs. Fault prediction models can be either process oriented (development and maintenance) or product oriented (design and usability). Usage of software metrics to evaluate the quality of software design has attracted software industries as they help to assess large software system quickly at low cost. Increased usage of Object Oriented (OO) paradigm has been envisaged in recent software products, which has increased the need for new quality metrics to be devised. Existing metrics for fault module detection include CK metrics and Mood metrics along with traditional general metrics like simple metrics and program complexity measures. Traditional metrics do not consider OO paradigms like inheritance, encapsulation and passing of message and therefore do not perform well with fault prediction. The OO metrics have been developed specifically to analyze the performance of OO system. But, the increase in software complexity and size is increasing the demand for new metrics to identify flaws in the design and code of software system. This demand has necessitated the researchers to focus on adopting new metrics for which established practices have yet to be developed. This paper focuses on such needs through the development of four metrics for OO design. In particular, this work analyzes metrics for measuring class complexities that can be used as a medium to identify design defects. For this purpose, four metrics based on flow of information, friend class/function, inheritance and cohesion are proposed. Several studies have focused on evaluating the usefulness of software metrics to predict software design faults (Damm and Lundberg, 2007). These techniques can be loosely categorized as statistical techniques, structural patterns based techniques, metrics based techniques, formal / relational concept analysis and software inconsistency management techniques. Classification, a frequently used data mining technique, has found wide usage in a range of problem domains such as finance, medicine, engineering, geology and physics. Combining soft----------------------------------------C.Neelamegam is with the Computer Applications at Sri Venkatesara Col- ware metrics and single classifiers is a methodology that lege Computer Applications and Management,Coimbatore , Tamilna- has gained attention recently. This study proposes a methodology that combines software metrics and a suite du,India-641 112. M.Punithavalli is with the Computer Application at Sri Ramakrishna of classifiers (ensembling) to design a fault prediction model. The rest of the paper is organized as follows. SecCollege of Engineering,Coimbatore,India. tion 2 presents the four proposed metrics to calculate the 2012 Journal of Computing Press, NY, USA, ISSN 2151-9617 complexity of the class. The methodology used by pro-

JOURNAL OF COMPUTING, VOLUME 4, ISSUE 10, OCTOBER 2012, ISSN (Online) 2151-9617 https://sites.google.com/site/journalofcomputing WWW.JOURNALOFCOMPUTING.ORG

67

posed prediction based ensemble classification that uses existing and proposed metrics to predict faulty class is presented in Chapter 3. Several experiments were conducted to analyze the performance of the proposed metrics and ensemble classifier to predict faulty modules. The results are presented and discussed in Section 4. Section 5 concludes the work with future research directions.

and ICM=Product of DCM of all methods in the length of two entities and C is the class and m is the method. Now, Equation (1) can now be rewritten as CMFCM = (FI + FO) * CAE * MCL (5) Here, the multiplicative operator in the traditional complexity measure is replaced by an additive operator. This modification was done to accommodate coupling among entities computation. This has the added advantage of reducing computation complexity. In the equation, MCL is the module code length and is calculated using Equation (6). MCL = LOC + MLOC + CLOC + (CL * j) + BL (6) where LOC is the line of codes with comments and blank lines, MLOC is the multiline of code which is calculated as LOC * number of separate statements in the same line, CLOC is the line of code that contain comments and is calculated as the sum of LOC and number of comment lines. CL * j expression denotes the number of lines that contain more than one comment statement and BL denotes the blank lines. The proposed CMFCM metric is a method level metric.

2 PROPOSED CLASS COMPLEXITY METRICS

This section presents the four proposed complexity metrics. With all the metrics a high value denotes high functional complexity and point towards serious design flaws that requires extensive testing and redesigning.

2.1 Class Method Flow Complexity Measure (CMFCM)

The two famous metrics, Cyclomatic complexity and the structural fan-in/fan-out, are concerned with the control flow of a program and ignore the data or information flow complexity. Two measures that are used during information flow complexity are Fan-In and Fan-Out. FanIn measures the information flow into the procedure, that is, it is the sum of number of parameters passed to a module from outside and global variables read by the same module. Fan-out, on the other hand, indicates the sum of number of return values of a module and global variables written by the same module. According to Henry and Kafura (1981), the module complexity can be calculated as in Equation (1). CMFCM = (Fan-In * Fan-Out)2 + Code Length

2.2 Friend Class Complexity Metric (FCCM)

A friend class is defined as a function or method that can access all private and protected members of a class to which it is declared as a friend. While considering complexity measure for friend classes, the following charac(1) teristics have be noted. 1. On declaration of friend class, all member functions It is known fact that in object-oriented systems, the priof the friend class become friend of the class in which vate data (internal data) of an object cannot be directly the friend class is declared. accessed by other objects and therefore programmers use 2. Friend class cannot be inherited and every friendship parameter passing and return values. The Fan-In (FI) and has to be explicitly declared. Fan-Out (FO) measures for a method m should taken 3. The friendship relation is not symmetric into consideration these values and can be calculated us- In the field of OO metrics for fault detection, studies on ing the following Equations (2) and (3). friend classes are minimum, inspite of its extensive usage (Counsell and Newson, 2000; Counsell et al., 2004). Friend FI = 1+ Nm1 + (NIP+ NPV+ NPU+ NLV + NGVR) + f( ) (2) constructs are violation of encapsulation and will compliFO = = 1 + Nm2 + (NOP+ NGVW) + f( ) (3) cate a program, which in turn, makes debugging more where Nm1 is the number of objects called, Nm2 is the difficult. Moreover, the task of tracking polymorphism number of objects that call this method, NIP is number of also becomes more complex while using friend classes. input parameters, NPV is the number of protected varia- According to Chidamber and Kemerer's principle only bles, NPU is number of public variables, NLV is the number those methods which require additional design effort of Local variables, NOP is the number of parameters writ- should be counted for metric measurement and inherited ten to, NGVR and NGVW are number of global variables read methods or methods from friend classes are not defined and written to and f( ) is a function which returns a value in the class and therefore, need not be included. However, it has been proved that the coupling that exists be1 if method m returns a value, zero otherwise. Another property that has to be considered while consid- tween friend classes increase fault proneness of a class ering OO systems is the coupling among entities. The (Briand et al., 1997). These methods consider relationship Coupling Among Entities (CAE) is calculated as the sum and type of association between class attributes and of indirect coupling metric and direct coupling metric methods and do not consider the relationship between friend attributes and external attributes. This section pro(Equation 4). poses a modified version, which considers this relationCAE = DCM + IDCM (4) ship and extends coupling metrics to use these friend metrics. Using these metrics, a new coupling measure to where determine the class complexity is proposed. Coupling measure can be either Direct Coupling (DC) or parameters of Computing Press, NY, USA, ISSN 2151-9617 DCM= No. of Methods in C No. of 2012 Journalin the C Indirect Coupling (IDC). DC here refers to the normal No. of Methods in m No. of Parameters in m

JOURNAL OF COMPUTING, VOLUME 4, ISSUE 10, OCTOBER 2012, ISSN (Online) 2151-9617 https://sites.google.com/site/journalofcomputing WWW.JOURNALOFCOMPUTING.ORG

68

coupling factor, while IDC refers to coupling while friend functions or classes are used. Thus, the new coupling factor is defined as a sum of DC and IDC (Equation 7). CFNew = DC + IDC (7) DC is calculated using the method specified in Mood metric suite. The IDC of a class is calculated as average of Method IDC (MIDC) factor. The MIDC is modified to identify a factor called actual friend methods, which is introduced because generally, a friend class declaration grants access to all methods in a class but in reality only a few of these methods are actually called by other classes. The MIDC combined with this factor is calculated using Equation (8).

pects of a program in terms of depth, breadth in a hierarchy and its overriding complexity. It can be used to measure class complexity as a measure of data / method shared from ancestor classes. The class complexity while taking inheritance into consideration depends mainly on the inheritance type (single, multiple, multi-level inheritance). Apart from this, while calculating the class complexity with respect to inheritance, the complexity imposed by inherited methods and inherited attributes should also be considered. Thus the proposed CCI metric considers the individual complexity of a class while taking the properties of inheritance into consideration (ICC), inherited method complexity (IMC) and inherited attribute complexity (IAC) and is calculated using Equation (12). CCI = ICC + IMC + IAC (12) where ICC of a class i is calculated as ICC = NA +
NA i 2

N MC

MIDC

i 1

(N GVR i

N GVWi ) N GFi N MC

N PCi

N Vi

(8)

ICC i

(13)

where NGF is the number of global functions, NPC is the number of messages to other classes and NV is the number of references to instance variables of other classes and NMC is the number of actual methods in the class which is calculated as the difference between the number of methods (NM) and Number of Hidden Methods (NHM) in a class (Equation 9). Hidden methods are methods that cannot be called on an object of the class without the use of the friend construct. NMC = NM NHM (9) The number of hidden methods is calculated as the number of methods in a class that access hidden members of classes which declare the current class as a friend (English et al., 2005). NHM is calculated as the sum of two measures. The first is the number the hidden methods belonging to other classes accessed by the class. This measure is called in this study as Number of EXternal Hidden Methods NHME. The second measure is the number of hidden methods that are invoked by other classes from the class. This measure is referred in this study as Number of Internal Hidden Methods NHMI. Thus NHM is calculated as NHM = NHME + NHMI (10) Using the above metric, the complexity measure can be calculated by modifying Equation (5) as given below FCCM = (FI + FO) * CFNEW * MCL (11) Again, this metric is a method level metric, where a high value indicates design flaws.

Here, the ICC of the root of the inheritance tree is zero as it has no parent. The ICC measure thus takes into consideration the depth of the class in the inheritance hierarchy, number of parents of the class and their depth in the inheritance hierarchy along with the type of inheritance. The IMC is calculated as IMC = (NPD * 1) + (NDD * 2) + (NUD* 3) (14) where NPD is the number of primary data variables, NDD is the number of derived data variables and NUD is the number of user defined data type variables. The classification of data types is similar to the one proposed by Arockiam and Aloysius (2011), who defined PD as inbuilt data types like int, float and char, DD as in-built structures like arrays and UD as user designed structures which are formed by combining PD and DD. Examples for UD includes structure, union and class. As suggested by the same author, a cognitive weight of 1, 2 and 3 are used along with NPD, NDD and NUD respectively. These cognitive weights are assigned according to the cognitive phenomenon suggested by Wang (2002) which assigns weight for PD=1, DD=2 and UD=3. Finally, IAC is calculated again by assigning cognitive weights to the control structures in the method. The control structures considered are sequence statements, branching statements, iterative statements and call statements. As suggested by Wang (2002) a value of 1, 2, 3 and 2 are assigned to these statements respectively.

2.4 Class Complexity From Cohesion Measure(Cccm)

Cohesion of a class describes how the methods of a class are related to each other. In general, a high cohesion is desirable as it promotes encapsulation, while a low coheInheritance a powerful mechanism in OO program- sion indicates high likelihood of errors, design change ming provides a method for reusing code of existing ob- and high class complexity. This section presents a metric jects or establishes a subtype from an existing object, or to calculate class complexity through cohesion measure. both. Inheritance metrics are used to analyze various as- Four typesISSN 2151-9617 2012 Journal of Computing Press, NY, USA, of cohesion methods are used, namely, Cohe-

2.3 Class Complexity from Inheritance (CCI)

JOURNAL OF COMPUTING, VOLUME 4, ISSUE 10, OCTOBER 2012, ISSN (Online) 2151-9617 https://sites.google.com/site/journalofcomputing WWW.JOURNALOFCOMPUTING.ORG

69

sion Among Attribute in a class Measure (CAA), Ratio of Cohesion Interactions (RCI), Cohesion Among Methods in a class (CAMC) and Normalized Hamming Distance (NHD) Metric. Here RCI, CAMC and NHD are calculated using steps as provided by (Briand et al., 1999; Bansiya et al., 1999; Counsell et al., 2005). The RCI considers the data to method relationship, the CAMC considers the methodmethod interactions The CCCM metric is included in this study to measure the degree of similarity among methods while considering attribute usage. Thus, the CCCM is calculated as using the procedure given in below. CCCM = CCM + RCI + CAMC / 3 (15) 1) Calculate the number of methods in a class, M (={m 1, m2, }) 2) Calculate the number of instance variables in each method, Vi ({=V1,V2, V3, }, i M) 3) Calculate number of methods using each instance from V, NVi, as (M Vi)-1. The value is 1 used to remove the attributes similarity dependency from the method it is declared. Calculate CCCM as

Operands) 5) RE (Readability with Comment percentage) 6) B VO (Volume) Mood Metrics 1) MHF (Method hiding factor) 2) AHF (Attribute hiding factor) 3) MIF (Method inheritance factor) 4) AIF (Attribute inheritance factor) 5) PF (Polymorphism factor) 6) CF (Coupling factor) D

classes) 5) RC (Response for a Class) 6) LCM (Lack of Cohesion in Methods) Program Complexity Measure 1) CC (Cyclomatic Complexity) 2) FI-FO(Fan-In Fan-Out)

NV i ( .M
i 1

1 ) xV

3.2

DIMENSIONALITY REDUCTION

3 FAULT PREDICTION USING OBJECT ORIENTED METRICS

The present study proposes the use of machine learning algorithm to analyze the performance of the proposed metrics in predicting design flaws in OO programs. The proposed method consists of four steps. (i) Selection of metrics (ii) Dimensionality Reduction (iii) Normalize the metric values and (iv) Implement prediction model. Here the prediction model is proposed as an ensemble-binary classification task, where a module is predicted as either faulty (complex) or not-faulty (normal).

3.1 SELECTION OF METRICS

The four proposed metrics are combined with twenty existing metrics (Table 1) during fault prediction. The selected existing metrics were chosen because of their wide usage in fault detection.

The vital step in designing a classification model is the selection of a set of input metrics, which unless selected carefully will result in Curse of dimensionality. This phenomenon can be avoided by the use of dimensionality reduction procedure, which aims to reduce the number of input variables by removing irrelevant data and retaining only those data which are most discriminating. In the present study, Sensitivity Analysis of data is used for this purpose. Sensitivity analysis analyzes the importance of each input data in relation to a particular model and estimates the rate of change of output as a result of varying the input values. The resulting estimates can be used to determine the importance of each input variable (Saltelli et al., 2000). This study adopts the Sensitivity Casual Index (SCI) proposed by Goh and Wong (1991).

3.3

NORMALIZATION

TABLE 1 : LIST OF SELECTED EXISTING METRICS Chidamber & A Simple metrics C Kemerer's Metrics 1) WMC 1) LOC (Total num(Weighted Methods ber of lines) per Class) Vj min( Vj ) (17) ' 2) BR (Number of 2) DIT (Depth of Vj max( Vj ) min( Vj ) methods) Inheritance Tree) 3) NOP (Total Num3) NC (Number ' ber of Unique Operawhere Vj is the normalized or scaled value, min(Vj) of children) tors) and max(Vj) are the maximum and minimum bounds 4) NOPE (Total 4) COC (Coupling 2012 Journal of Computing Press, NY, USA,metric j from n observations respectively. of the ISSN 2151-9617 Number of Unique between object

This step is used to normalize each input to the same range and makes sure that the initial default parameter values are appropriate and every input at the start has equal important. Further, normalization is performed to improve the training process of the classifier. Normalization is performed by estimating the upper and lower bounds for each metric value and then scale them using Equation (17).

JOURNAL OF COMPUTING, VOLUME 4, ISSUE 10, OCTOBER 2012, ISSN (Online) 2151-9617 https://sites.google.com/site/journalofcomputing WWW.JOURNALOFCOMPUTING.ORG

70

real-time project written in C++ consisting of 43000 LOC. The dataset has a total of 1571 modules out of which 319 are faulty modules while 1252 are non-faulty modules. The feature vector created has 24 dimensions each repre3.4 PREDICTION MODEL senting one selected metric. This vector was first normalThe steps to build the prediction model are given below. ized to an interval [0, 1] to ensure that all the 20 values Step 1 : Identify a classifier have equal importance. Dimensionality reduction was Step 2 : Identify the feature vector to be used as input next performed on this set to select discriminating metrics Step 3 : Partitioning method (Training and Testing sets) by calculating SCI of each input dimension over the entire Step 4 : Train and test the classifier using n-fold cross- normalized dataset with =0.1. After calculation of SSI, the metrics were arranged in descending order of SSI and validation method The present study uses ensemble classifier for fault pre- the top 15 metrics were selected. The resultant feature diction. Three classifiers, namely, Feed Forward Back vector, after dimensionality reduction consists of Propagation Artificial Neural Network (BPNN), Support CMFCM, FCCM, CCI, CCCM, LOC, BR, RE, WMC, DIT, NC, COC, RC, LCM, MHF, AHF, MIF, AIF, PF and CF. It Vector Machine (SVM) and K-Nearest Neighbour (KNN) can be seen that the resultant reduced dataset consists of are considered during ensembling. Input feature vector is only those metrics which has impact on complexity created by combining the proposed metrics (Section 2) measure and all the four proposed methods were selectwith traditional metrics and salient data is identified us- ed. This indicates that the proposed metrics can be used ing the procedures described in Sections 3(A) and 3(B). as a good fault indicator. The reduced dataset with 19 The resultant dataset is then partitioned into training and metrics is then divided into training (943 modules) and testing set using hold method. The holdout method ran- testing (628) datasets. To evaluate the effect of the prodomly partitions the dataset into two independent sets, posed metrics, a 15 metric feature set without the proposed metrics was also considered. training and testing. Generally, two-thirds of the data are Four classification performance metrics were used during allocated to be the training set and remaining one-third is evaluation. They are accuracy, precision, recall and Fallocated as test set. The method is pessimistic because measure, which are derived from the confusion matrix. A only a portion of the initial data is used to derive the 10-fold cross validation method was used with all expermodel. Thus, given a set of input data set (combined pro- iments. The performance of the single classifiers was posed and traditional metrics) and the ensemble classifier compared with that of ensemble classifiers. For SVM clas(BPNN + SVM + KNN) marks each given input as belong- sifier, the regularization parameter was set to 1, the kernel function used was Gaussian and bandwidth of the kernet ing to one of two categories (faulty or not-faulty). The result of the classifiers in the ensemble model is com- was set to 0.5. For K-NN classifier, k was set to 3. For BPNN classifier, 2 hidden nodes with learning rate of 0.2 bined using a combination of majority voting and were used. T-Test was performjed at 95% confidence levweighting scheme. The modified majority vote scheme el (0.05 level) to analyze the significant difference bethat combines weighting scheme is explained below. Let tween SVM and BPNN, SVM and KNN. The T-test meththe decision of the ith classifier be defined as dt, j {0, 1}, t od adopted was proposed by Nadeau and Bengio (2003). = 1, , T and j = 1, , C, where T is the number of classi- This method was adopted because it is more suited for fiers and C is the number of classes. If the ith classifier classifiers adapting 10-fold cross-validation method (Dietchooses class j, then dt,j = 1 and 0, otherwise. In majority terich, 1998). The traditional student t test, method produces more false significant differences due to the devoting scheme, a class j is chosen, if pendencies that exists in the estimates. Further, the affect c T T d t, J max d t , j * w t (18) of the proposed metrics in classification performance is j 1 t 1 t 1 ascertained by running the experiments with the existing metric set containing 24 metrics and analyzing the classiHere wt (weight assigned to classifier t) is calculated us- fication accuracy. From the three single classifiers, four 2ing Kuncheva (Kuncheva, 2004) method (Equation 19). classifiers PEMs (BPNN + KNN, BPNN + SVM, KNN + pt (19) wt log SVM) and one 3-classifier PEMs (BPNN + KNN + SVM) t 1 p were built. Tables 3 to 5 shows the 1-classifier, 2-classifer and 34 EXPERIMENTAL RESULTS classifier PEM performance of the proposed BPNN, KNN and SVM based ensemble predictors based on Accuracy, The proposed fault-detection classifier systems using Precision, Recall and F Measure. To analyze the adsoftware metrics was developed using MATLAB 2009 vantage obtained by the proposed predictors the proand all the experiments were conducted on a Pentium IV posed models are compared with their traditional single machine with 4GM RAM. The NASA IV & V Facility classifier counterparts. In these tables, SD denotes the MDP data (http://mdp.ivv.nasa.gov/repository.html), standard deviation and the column Sig denotes the status consists of error data from several projects. This study of significance. In the Sig column, Yes denotes that there of records related to Press, a USA, ISSN 2151-9617 uses KC1 project, which consist 2012 Journal of Computing a is NY,significance performance difference between single The result of normalization thus, maps each input value to a closed interval [0, 1].

JOURNAL OF COMPUTING, VOLUME 4, ISSUE 10, OCTOBER 2012, ISSN (Online) 2151-9617 https://sites.google.com/site/journalofcomputing WWW.JOURNALOFCOMPUTING.ORG

71

prediction model and the corresponding ensemble prediction model, while a No represents insignificant performance. A + sign at the end denotes that ensemble prediction model has outperformed the corresponding single prediction model, while sign denotes the oppo-

site.

TABLE 3 : PERFORMANCE OF BPNN BASED ENSEMBLE PREDICTION MODELS

Feature Set 19Metrics Feature set Model BPNN BPNN+SVM BPNN+KNN BPNN+SVM+KNN BPNN BPNN+SVM BPNN+KNN BPNN+SVM+KNN Accuracy Mean SD Sig 77.38 3.562 84.26 2.100 Yes(+) 81.92 0.960 Yes(+) 82.74 1.703 Yes(+) 82.16 1.201 Yes(+) 89.91 1.236 Yes(+) 94.55 1.579 Yes(+) 96.17 1.314 Yes(+) Precision Mean SD Sig 80.12 2.981 85.74 2.441 Yes(+) 84.11 1.569 No(-) 85.18 2.258 No(-) 84.76 2.697 No(-) 97.36 0.899 Yes(+) 98.93 0.371 Yes(+) 99.94 0.012 Yes(+) Mean 84.01 89.87 88.14 88.57 88.22 93.44 92.94 94.16 Recall SD 3.015 2.640 1.010 1.270 1.180 0.587 1.574 1.122 Sig Yes(+) Yes(+) Yes(+) Yes(+) Yes(+) Yes(+) Yes(+) F Measure Mean SD Sig 82.02 3.298 87.76 2.221 Yes(+) 86.08 0.674 Yes(+) 86.84 1.188 Yes(+) 86.46 1.047 Yes(+) 95.36 0.745 Yes(+) 95.84 0.361 Yes(+) 96.96 0.202 Yes(+)

24 Metric Feature set

TABLE 4 : PERFORMANCE OF KNN BASED ENSEMBLE PREDICTION MODELS

Feature Set 20 Metric Feature set Model BPNN BPNN+SVM BPNN+KNN BPNN+SVM+KNN BPNN BPNN+SVM BPNN+KNN BPNN+SVM+KNN Accuracy Mean SD Sig 84.98 2.416 89.26 1.841 Yes(+) 87.89 0.306 Yes(+) 88.98 0.566 Yes(+) 87.81 0.382 Yes(+) 89.91 1.236 Yes(+) 90.26 1.077 Yes(+) 96.17 1.314 Yes(+) Precision Mean SD Sig 89.72 0.126 91.76 0.441 Yes(+) 89.97 0.314 Yes(+) 91.12 0.876 Yes(+) 90.76 0.924 Yes(+) 97.36 0.899 Yes(+) 97.94 0.821 Yes(+) 99.94 0.012 Yes(+) Mean 95.42 96.42 95.89 96.16 96.02 93.44 92.67 94.16 Recall SD 0.124 0.441 0.467 0.978 0.997 0.587 0.687 1.122 Sig Yes(+) No(-) No(-) No(-) Yes(+) Yes(+) Yes(+) F Measure Mean SD Sig 92.48 0.397 94.03 0.241 Yes(+) 92.84 0.978 Yes(+) 93.57 0.618 Yes(+) 93.32 0.344 Yes(+) 95.36 0.745 Yes(+) 95.23 0.798 Yes(+) 96.96 0.202 Yes(+)

24 Metric Feature set

TABLE 5 : PERFORMANCE OF SVM BASED ENSEMBLE PREDICTION MODELS

Feature Set 20 Metric Feature set Model BPNN BPNN+SVM BPNN+KNN BPNN+SVM+KNN BPNN BPNN+SVM BPNN+KNN BPNN+SVM+KNN Accuracy Mean SD Sig 90.62 1.161 93.99 1.991 Yes(+) 92.96 0.989 Yes(+) 93.41 1.562 Yes(+) 93.16 1.199 Yes(+) 90.26 1.077 Yes(+) 94.55 1.579 Yes(+) 96.17 1.314 Yes(+) Precision Mean SD Sig 90.34 0.040 92.34 1.461 Yes(+) 91.27 0.785 Yes(+) 92.08 1.318 Yes(+) 91.76 0.978 Yes(+) 97.94 0.821 Yes(+) 98.93 0.371 Yes(+) 99.94 0.012 Yes(+) Mean 98.43 98.77 98.01 98.54 98.12 92.67 92.94 94.16 Recall SD 0.068 0.241 0.114 0.981 0.457 0.687 1.574 1.122 Sig Yes(+) No(-) No(-) No(-) Yes(+) Yes(+) Yes(+) F Measure Mean SD Sig 94.21 1.014 95.45 0.166 Yes(+) 94.52 0.045 Yes(+) 95.20 0.681 Yes(+) 94.83 0.457 Yes(+) 95.23 0.798 Yes(+) 95.84 0.361 Yes(+) 96.96 0.202 Yes(+)

24 Metric Feature set

From the results, it could be seen that the inclusion of the proposed metrics has increased the efficiency of the ensemble classification model with all performance parameters. Further, the application of ensembling concept has also improved the performance of the fault prediction. This is evident from the significant difference obtained when compared with single classifier models., Among the four data selection algorithms, the Sequential Selection method produced significant improvement to classification performance. While comparing the three classifiers, the performance of SVM-based prediction models is better when compared with BPNN and KNN. While considering the number of classifiers, the 3-classifier ensemble model ranked first when compared with all other models. The best performance was produced by the model that combines BPNN, KNN and SVM classifiers with 19 metrics feature set.

5.CONCLUSION This paper proposed four new metrics for evaluating the complexity of object oriented software products. Further, the usage of these metrics on OO fault module detection was also analyzed using ensemble prediction classifiers. For this purpose, 24 metrics that are related to with the complexity factor of a system were selected. Sensitivity index was used to select relevant metrics for classification after normalization. Three classifiers, namely, BPNN, SVM and KNN were used to generate ensemble classifiers. These classifiers are termed as 1-classifier ensemble prediction models. The three classifiers were grouped together to form five ensemble models (identified as 2classifier and 3-classifier models). The performance was analyzed using accuracy, precision, recall and F-measure. When comparing with single classifier systems, all the proposed models produced improved classification performance and among the 16 models, the 3-classifier model that combined BPNN, SVM and KNN produced best re 2012 Journal of Computing Press, NY, USA, ISSN 2151-9617

JOURNAL OF COMPUTING, VOLUME 4, ISSUE 10, OCTOBER 2012, ISSN (Online) 2151-9617 https://sites.google.com/site/journalofcomputing WWW.JOURNALOFCOMPUTING.ORG

72

sults. Moreover, the results also prove that the proposed complexity measuring metrics improve the faulty module prediction irrespective of the classifier used. Future research is planned in the direction of development of new design metrics and their use with the proposed classifiers.

[16] [17] [18] [19] [20] [21] [22] [23]

Kuncheva, L.I. (2004) Combining pattern classifiers: Methods and algorithms, Wiley-Interscience, New Jersey. Nadeau, C. and Bengio, Y. (2003) Inference for the generalization error, Machine Learning, Vol. 52, Pp.239281. Saltelli, A., Chan, K. and Scott, E.M. (2000) Sensitivity Analysis, John Wiley & Sons. Wang. Y, (2002) On Cognitive Informatics, IEEE International Conference on Cognitive Informatics, Pp. 69-74. Kuncheva, L.I. (2004) Combining pattern classifiers: Methods and algorithms, Wiley-Interscience, New Jersey. Nadeau, C. and Bengio, Y. (2003) Inference for the generalization error, Machine Learning, Vol. 52, Pp.239281. Saltelli, A., Chan, K. and Scott, E.M. (2000) Sensitivity Analysis, John Wiley & Sons. Wang. Y, (2002) On Cognitive Informatics, IEEE International Conference on Cognitive Informatics, Pp. 69-74.

6. References
[1] Arockiam, L. and Aloysius, A. (2011)Attribute Weighted Class Complexity: A New Metric for Measuring Cognitive Complexity of OO Systems, World Academy of Science, Engineering and Technology, 58,Pp. 808-813. Bansiya, J., Etzkorn, L., Davis, C. and Li, W. (1999) A class cohesion metric for object-oriented designs, Journal of Object-Oriented Program, Vol. 11, No. 8, Pp. 47-52. Briand, L.C., Devanbu, P.T. and Melo, W.L. (1997) An Investigation into Coupling Measures for C++, International Conference on Software Engineering, Pp.412-421. Briand, L.C., Morasca, S. and Basili, V.R. (1999) Defining and validating measures for object-based high-level design, IEEE Transactions on Software Engineering, Vol. 25, No. 5, Pp. 722-743. Catal, C and Diri, B. (2009) Investigating the effect of dataset size, metrics sets, and feature selection techniques on software fault prediction problem, Information Science, Elsevier, Vol. 179, Pp. 1040-1058. Chowdhury, I. and Zulkernine, M. (2011) Using complexity, coupling, and cohesion metrics as early indicators of vulnerabilities, Journal of Systems Architecture, Elsevier, Vol. 57, Pp. 294-313. Counsell, S. and Newson, P. (2000) Use of Friends in C++ Software: An Empirical Investigation. Journal of Systems and Software, Vol.53, No.1, Pp.15.21. Counsell, S., Newson, P. and Mendes, E. (2004) Design Level Hypothesis Testing Through Reverse Engineering of ObjectOriented Software, International Journal of Software Engineering, Vol.14, No.2, Pp.207.220. Counsell, S., Swift, S. and Crampton, J. (2006) The interpretation and utility of three cohesion metrics for object-oriented design, ACM Transactions on Software Engineering and Methodology (TOSEM), Vol. 15, No. 2, Pp.123-149. Damm, L.O. and Lundberg, L. (2007) Company-Wide Implementation of Metrics for Early Software Fault Detection, Proceedings of the 29th international Conference on Software Engineering (ICSE '07), IEEE Computer Society, USA, Pp. 560-570. Dietterich, T. (1998) Approximate statistical tests for comparing supervised classification learning algorithms, Neural Computation, Vol. 10, Pp. 18951924. English, M., Buckley, J., Cahill, T. and Lynch, K. (2005) An Empirical Study of the Use of Friends in C++ Software, International Workshop on Program Comprehension, Pp. 329.332. Goh, T.H. and Wong, F. (1991) Semantic extraction using neural network modeling and sensitivity analysis, Proceedings of IEEE International Joint Conference on Neural Networks, Pp. 1821. Henry, S.M. and Kafura, D. (1981) Software structure metrics based on information flow, IEEE Transactions on Sofware Engineering, Vol. SE-7, Pp. 510-518.

[2]

[3]

[4]

[5]

C.Neelamegam received M.ScDegree from Bharathidasan University ,Trichy,India.He is the head of the computer Application department at Sri Venkatesara College of Computer Application and management,Coimbatore,India. His research work has appeared in Global journal of computer science ant technology, International journal of Computer Applications, Journal of Computer Science. M.Punithvalli holds a Ph.D Degree in Computer Applications from Alagappa University , Tamilnadu,India. She is the Director of Computer Application Department of Sri Ramakrishna College of Engineering ,Coimbatore,India. She has produced some Ph.D scholar. she is a Editorial Bord member of various publications.She has published many research paper in different journals.

[6]

[7]

[8]

[9]

[10]

[11]

[12]

[13]

[14]

[15]

http://mdp.ivv.nasa.gov/repository.html 2012 Journal of Computing Press, NY, USA, ISSN 2151-9617