From CMMI Maturity Level 3 to Maturity Level 5

R.T. Sundari
Abstract: In September, 2007, C-DAC, Noida initiated the project for achieving CMMI Maturity Level 5. Within a period of about 3 years it was accomplished. The ML 3 reached in, about 18 months from the start of the project. The overview of the transition of the software engineering practices prevalent in the centre, prior to implementing CMMI, till reaching of ML3 is given in [9]. After this the major restructuring done, is the introduction of centralized verticals for Design, Development, Testing and Implementation activities of all projects. This facilitated the generation of more appropriate documentation of these activities. It also resulted in generation of more flexibility in the system and horizontal movement of staff between projects within the vertical and also across verticals. We believe that this accelerated our pace towards ML5. This paper presents our roadmap from CMMI ML3 to the ML5.

Key words: CMMI Implementation, Root Cause Analysis, OID I Introduction Software quality assurance through software process improvement and certification is one of the strategies with which software companies are engaged with twofold objectives. The first one is to make their projects more efficient by improving their internal processes and the second is the improvement in customer satisfaction. Quality cannot be copied. There is no step by step work book that applies equally to all company situations and cultures. The quality of the product/service is largely determined by the quality of the processes that are adopted by the organization. Three elements for the Project Success revolve around Process (a defined method involving steps or operations), Technology (Application domain, Tools, Languages, Information environment) and People (skills, management). Any deficiency in any of the elements will affect the quality of the product/service. We should also note that the above three factors reinforce each other. A set of good processes will ensure generation of appropriate technologies and will attract competent people towards the organization. Similarly, competent people will ensure that the processes are strengthened and appropriate technologies are brought in. Adoption and adaptation of new technologies will influence the organizational culture and improve the processes. Thus, any improvement in one section of the triangle will correspondingly improve the other sections as well. The primary purpose of the project for implementation of CMMI ML3 was to restructure the existing processes into more appropriate ones and strengthen these. The journey has been satisfying. We were able to quantitatively bring out the improvement in all spheres of software development, management and customer satisfaction. This paper describes our experience in achieving the ML5. The companion paper [1] describes the overview of the transition prior to the

Proceedings of ASCNT-2011, CDAC, Noida, India, pp

R.T. Sundari

introduction of CMMI standards till the time of reaching ML3. Accordingly, in this paper, we are concentrating more on our journey from ML3 to ML5.

The remaining section of this paper is organized as follows: Section II gives the Overview of the CMMI framework. Section III describes the Roadmap upto CMMI ML3. Section IV gives the Roadmap from CMMI ML 3 to CMMI ML5. Section V describes the restructuring strategy that we have adopted for accelerating the process to reach ML5. Section VI describes the ML5 Processes, Section VII describes the Process Adaptation, Section VIII describes the Appraisal Process. Finally, Section IX concludes with a list of lessons learnt. II Overview of CMMI Framework CMMI (Capability Maturity Model Integrated) is the structured collection of software engineering best practices that has been published by the Software Engineering Institute (SEI) of Carnegie Mellon University (CMU), USA. CMMI is the registered trademark of SEI, CMU, USA. This comes with two different representations- staged and continuous. The staged model groups process areas into five maturity levels. The continuous representation defines capability levels within each profile. The CMMI uses a common structure to describe each of the 22 process areas (PAs). These process areas are build upon three key concepts: Process areas, goals, and practices. CMMI helps in integrating traditionally separate organizational functions, set process improvement goals and priorities, provides guidance for quality processes and a point of reference for appraising current processes. The primary goal of CMMI is to Improve processes for better products as per providing certification to any organization is just a byproduct. The CMMI Frame work makes it easy for the organization to define and deploy measurable and achievable goals. The capability levels in the framework are: Initial Level 1: Process performed but unpredictable, poorly controlled and reactive. Managed Level 2: Processes are characterized for instance and is of reactive. Defined Level 3: Processes are characterized for instance and is of proactive. Quantitatively managed Level 4: Processes are measured and controlled statistical methods are used. Optimizing Level 5: Focus is on continuous improvement. Table 1 gives the details of the processes that are organized by maturity levels. In initial stage there is no processes adopted. In level 5 the organizations innovation , deployment and causal analysis helps to optimize the processes followed in the organization. Table 1 : Processes handled in Various CMMI level S.No. 1. 2. Level Initial Managed Details Project Planning, Monitoring and Control; Requirements Management;

From CMMI Maturity Level 3 to Maturity Level 5

Configuration Management Measurement and Analysis; Process and Product Quality Assurance; Supplier Agreement Management. 3. Defined Requirements Development; Technical Solution; Product Integration; Verification; Validation; Organizational Process Focus; Organizational Process Definition; Organizational Training; Integrated Project Management ; Integrated Supplier Management; Decision Analysis and Resolution; Organizational Environment for Integration and risk management. Organizational Process Performance; Quantitative Project Management. Organizational Innovation and Deployment; Causal Analysis and Resolution.

4.

Quantitatively Managed

5.

Optimizing

III Road Map upto CMMI ML 3 The software development activities to be undertaken in research establishments like C-DAC is significantly different from those of other software development companies who develop the modules as per well defined customer requirements. The planning is more difficult and specification changes during implementation are unavoidable. This is normally used as an excuse for not implementing the strict structures like CMMI in research organizations. It is our experience that implementing software development standards like CMMI will, in fact be an advantage to research organizations as well. However, this has to be done by taking care of activities going on in the organizations in its proper perspective. These processes developed during the initial period enabled people to develop their potential more prominently and are more effective within the organization. By defining, measuring, and controlling the process, improvements are more successful and sustained. The likelihood that appropriate technology, techniques, and tools are introduced successfully; increases predictability. Our processes have become consistent and proactive. The planning and estimation of project scope, schedule, resource and cost are clearer and better controlled, based on the quantitative management architecture. The major lessons learnt during our experience over the last 2 years are given below:-

R.T. Sundari

Keep sizable goals. Clear understanding for targets and percolate it down. Everyone needs to be engaged train the masses. Specific training for process owners/subject matter experts. Utilize Teams as process champions. Multi-department representation. Process-focused charters. Proper utilisation of Resource. Start the implementation with the groups that want to improve, which eager to benefit from efforts and are able to recognise their own weaknesses. Provide skilled coaching, resources, support, and tools. Publicize successes; Share best practices.

IV Roadmap from CMMI ML3 to ML5 At ML3, an organizations processes are rigorously defined, tailored for individual projects from a common set of standard processes, and are well managed. The benefits at ML3 include improved process performance predictability. At CDAC, we already have established a process data base, as a part of ML3 initiatives and the processes are defined and templates are used. At ML4, quality and process performance objectives are expected to be quantitatively predictable. There is considerable emphasis on establishing and using current process performance baselines along with process performance prediction models. Through the statistical management of selected sub-processes variations and capabilities, key quality and process performance is understood, controlled and managed throughout the life of the processes and project. However, current process performance may not be sufficient to meet changing business objectives. At ML5, the organization focuses on techniques to improve and/or optimize the performance of processes, based on a quantitative understanding of the common causes of variation inherent in the processes. This optimization is accomplished by applying root cause analysis techniques and managing continuous process improvements through the use of Organization Improvement and deployment. At both ML4 and ML5, the use of statistical and quantitative analysis is essential to understand as well as to improve the business. Implementation of Maturity Level 5, enables an organization to truly optimize its performance. This distinction is even more noteworthy when an organization is able to successfully mature rapidly. To achieve a maturity level 5, one must practice the 22 process areas at that level. Following are the major requirements for reaching ML5: Each management layer is and must behave as the process owner. Senior managements support and commitment. Adequate resources and training. Infrastructure groups (SEPG, MC, Working Groups) active. Clear Roles and Responsibility with the defined process and project across the organization.

From CMMI Maturity Level 3 to Maturity Level 5

By conducting regular Internal Audit the processes were monitored and any gap in the processes was improved. During Internal Audits we found that Non Compliances in Project Management Plan and Control was more. The root cause analysis was done and it was found that there was less understanding in this area. So the Project Management training based as per the method of Project Management Institute was planned and conducted. After the Project Management Training there is a considerable improvement in the Project management practice. It has been observed that the average Non Compliances has reduced from 30 to 10 after the PMI training. The pain was felt while collecting the data for the analysis and model building. During the data collection and internal audits we became aware that in the case of many projects, Design documents and review reports were prepared after the activity; defeating the very purpose of preparing such documents. V Restructuring Strategy The existing structure consisted of groups organized around technical groups like Health Informatics, e-Governance, Language Technology and Embedded Systems. Project teams were formed within these groups, who were responsible for design, development, testing and implementation. This resulted in excessive coding and little importance was given to design, testing and documentation. A root cause analysis of the above problem brought out that it can be solved only with the proper restructuring of the existing system. Accordingly, a restructuring strategy was adopted by which the existing groups were demolished, paving a way to the creation of four verticals viz. Design, Development, Testing and Implementation. Due to this structure, the documentation has become a must. It became a deliverable from one vertical to the other. Project leaders were drawn from the Implementation Group and they are responsible for interaction with the customers for the requirements and finalizing the same with them. They prepared the SRS and submit it to the Design Team for designing. The Design Team generates the HLD and LLD for review by the Review Committee before they are handed over to the Development Group. After the coding, the executable was deployed in the test server and the testing team worked on this by testing the artifacts. The common repository system was created for easy handing of the documents from various groups. The baseline documents were uploaded in project repository for the stakeholders to use. This enabled the project leaders to monitor the activity, systematically. The vertical functional group and horizontal project team of matrix helped us to speed up our journey towards CMMI ML5.

VI ML5 Process The QA group is responsible for collection of the data and prepares the statistical models. The workshop approach was followed to make all the practitioners to understand the models and control charts. The tools and techniques are appropriately explained in the workshop. For easy reference a hand book was introduced wherein the statistical techniques and Root Cause analysis was explained. This handbook was made available for all practitioners in the centre. Level 5 requires Organizational Innovation and Deployment (OID) process to be implemented. That calls for a plan to coordinate resources; pilot the improvements, analyze the results and

R.T. Sundari

successfully deploy the new and/or revised process assets. OID is to select and deploy incremental and innovative improvements that can measurably improve the organizations processes and technologies. The improvements support the organizations quality and processperformance objectives as derived from the organizations business objectives. Quality and process-performance objectives that this process area might address include the following: Improved product quality (e.g., functionality, performance). Increased productivity . Decreased Efforts. Greater customer and end-user satisfaction. Reduce delivery time. We have successfully implemented five OIDs during six months time and sample results are given in next section. VII Process Adaptation Re usable Component (RUC) was one of the OID which has resulted in major reduction in the software development efforts. Prior to this implementation and its adaptation, similar modules / functionalities developed by a number were not reused in similar other projects. This has resulted in duplication of work. Also, defects detected and rectified in similar projects earlier, were resurfaced in some other projects, since the information were not spanning across project groups. In QMS portal Core Library link is used for RUC, which is made part of the QMS Site. This facilitated all users to contribute to the library. The library can be accessed by them for using the RUC from their working projects. We conducted the pilot study to determine the effectiveness of the RUC. The results of this study are given in Table II. Table II : Improvement in Organizational PCB June 2010 Mean Effort variance Schedule Variance Defect Density 19% 27% 2.53 SD 12% 6% Dec 2010 Mean 10% 9% 1.75 SD 8% 6% 1.14 Improvement in mean -9% -18% -0.78

It may be seen that the effort variance, schedule variance and defect density showed a reduction of 9%, 18% and 0.78 respectively.

From CMMI Maturity Level 3 to Maturity Level 5

The other three innovations introduced as a part of OID were Test Link Tool, Code Review Tool (PMD) and LLD creator. Test Link and PMD is an Open source tool. Using the Test Link test case execution rate is increased by 15%. Overall testing efforts like Test case Preparation, Review and Execution are managed in this. By using PMD in one of the project the code review effectiveness was increased from 25% to 53%. The LLD creator was developed by the design team and the details are presented in the companion paper [6]. VIII Appraisal Process Organizational Appraisals are used to identify strengths and weakness of current process and reveal the risk. The Appraisal method consists of the appraisal planning, preparation of onsite activities, preliminary observations, findings and ratings and final report. SCAMPI C appraisals are shorter and more flexible than SCAMPI A and B appraisals and are conducted to address a variety of special needs, from a quick gap analysis to determine an organization's readiness for a SCAMPI A. This was conducted in December 2010 to assess the readiness. SCAMPI A appraisal is typically conducted when an organization has implemented a number of significant process improvements and needs to formally benchmark its process relative to the CMMI. A SCAMPI A is the only appraisal method that provides CMMI Maturity Level or Capability Level ratings. Class A is conducted by SEI authorised Lead appraiser who uses Method Definition Document to conduct the appraisals. The CMMI Accreditation process Level with C-DAC, Noida is currently of ML5. This certification is one of the key milestones in our efforts and as a result of the certification process, our tools and software development processes has significantaly improved productivity, which will lead to greater customer satisfaction. IX Conclusion Organization capability to map with CMMI process area and how the requirements are managed to deliver quality product or services with extreme customer satisfaction drives the organization to handle the project with high predictability. The major lessons learnt during this process have been: Institutionalizing the practical process and measurement approach. Review Process to be strengthen. While Data collection, definition of the data measuring should be uniform throughout the project across the organization. Managing the project using data increase the visibility and helps to improve the product quality and process effectiveness. Introduction of the tools helps in turning the stable processes into capable processes Workshop Approach helped to achieve the goal in shorter period. Improved process + Competent Work Force + Appropriate Technology = Better Quality product, Higher productivity and reduced risk.

R.T. Sundari

ACKNOWLEDGMENT
M/s KPMG was our consultant for the project. We are thankful to Mr. Amir Khan, Shri Sudhir V Teotea and Shri. N.C.Chinthamani of KPMG for the help and association in the project. I am also thankful to Dr. Rajiv Nag, the Lead Appraiser and Dr. George Varkey, Executive Director CDAC Noida for valuable feedbacks.

REFERENCES
[1] Standard CMMI Appraisal Method for Process Improvement (SCAMPI),
[2] CMMI Distilled: A Practical Introduction to Integrated Process Improvement By Dennis M. Ahern, Aaron Clouse, and Richard Turner.

[3] [4] [5] [6]

Multiple views of CMMI approach: A case experience by Balasubramanian. S and Manivannan Reaching the peak of CMMI: How fast can you climb? From CEO World Magazine www.software-quality-assurance.org Revolutionizing and Standardizing Low Level Design Document using LLD Creation Tool by Umesh Sharma, Ajay Kumar Gupta, Siddharth Dixit [7] A case study presented in CMMI Annual Symposium 2007 by Gideon Smit, SITA [8] Effectively Deploying CMMI in a Large Organization b Sarah Bengzon. [9] Software Testing and Continuous Quality Improvement by R.T. Sundari R.T.Sundari (Consultant) has Post graduation in Management and holds more than 30 years experience in software designing, development and testing. She is currently heading the Quality Assurance division of CDAC, Noida.