WHITE P APER JBoss Operations Network: Measuring Business Impact and ROI

Sponsored by: Red Hat Tim Grieser July 2010 Randy Perry

INTRODUCTION
Linux-based applications have become essential to many organizations for delivering and supporting ecommerce, Web, and mission-critical business applications and databases. As Linux grows in importance and value to the business, IT must deliver high service quality for availability, security, and performance in order to support business success. Red Hat Inc. distributes and supports the Red Hat Enterprise Linux operating environment and JBoss Enterprise Middleware for development, deployment, and production operation of Java and Web-based applications. Red Hat provides the JBoss Operations Network (JBoss ON) management platform for centralized systems management (server-agent) of JBoss Enterprise Middleware. JBoss ON provides management functions for the JBoss SOA Platform, the JBoss Enterprise Application Platform, the JBoss Enterprise Web Server, and other JBoss components. JBoss ON management functions include discovery and inventory, configuration management, application deployment, availability management, performance management, and content management. This ROI study focuses on the quantitative benefits gained from using the JBoss ON management platform.

Global Headquarters: 5 Speen Street Framingham, MA 01701 USA P.508.872.8200 F.508.935.4015 www.idc.com

EXECUTIVE SUMMARY
IDC conducted a series of in-depth interviews with staff members of IT organizations that have deployed JBoss Operations Network to manage their JBoss Enterprise Middleware environments. A structured set of questions was used to assess and quantify the internal and external costs of performing certain administrative, operational, and support functions and how costs changed as a result of implementing JBoss Operations Network. Table 1 shows the key results of this study.

T ABLE 1
Three-Year ROI Analysis
Category Benefit Investment Net present value (NPV) ROI = NPV/Investment Payback = Investment/NPV
Source: IDC, 2010

Value $347,878 $47,378 $300,500 634% 5.32 months

The study was based on standard IDC ROI methodology used to calculate average cost savings resulting from higher availability, including changes in downtime, improved IT efficiency, and increased user productivity resulting from using JBoss Operations Network. Data from the survey was used as the basis for the IDC ROI analysis. IDC's ROI analysis methodology is described in the Appendix.

KEY FINDINGS
Based on data from interviews with IT managers from organizations using JBoss Operations Network, IDC's JBoss ON ROI analysis yielded an average 634% ROI more than six times the initial investment and an average payback period for the initial investment of a short 5.3 months for the IT organizations in this study, as shown in Table 1. A key metric was the improvement in the number of JBoss Middleware environments managed per system administrator. The number of Linux servers running JBoss Middleware managed per system administrator more than doubled, increasing from an average of 38 Linux servers per administrator before deploying JBoss ON to 84 Linux servers per administrator after deploying JBoss ON. The IT organizations reported a substantial savings in staff hours expended on managing JBoss Enterprise Middleware by using JBoss Operations Network, resulting in annual IT productivity increases. IT managers also claimed improved end-user productivity, while downtime hours per month were reduced. Participants also identified other annual savings through improvements in IT efficiency, absolute software/hardware savings, and other indirect cost savings. These results are illustrated in Figure 1 in terms of average annual savings per 100 users.

IDC's JBoss ON ROI analysis yielded an average 634% ROI more than six times the initial investment and an average payback period for the initial investment of a short 5.3 months. The number of Linux servers running JBoss Middleware managed per system administrator more than doubled, increasing from an average of 38 Linux servers per administrator before JBoss ON to 84 Linux servers per administrator after deploying JBoss ON.

FIGURE 1
Average Annual Benefits of JBoss Operations Network per 100 Users

Source: IDC, 2010

#224332

2010 IDC

Deployment of JBoss ON achieved a very impressive average annual benefit of $147,212 per 100 JBoss users. Revenue increase accounted for 40.5% of this benefit, while improvements in IT staff efficiency constituted 23.7% of the total. Other major benefits came from cost reduction (19.2%) and improved user productivity (16.6%).

STUDY DEMOGRAPHICS
To assess the benefits of Red Hat Systems Management, IDC conducted in-depth telephone interviews with staff members from IT organizations located in companies headquartered in North America; South America; Europe, the Middle East, and Africa (EMEA); and Asia/Pacific that have deployed JBoss ON. The industries of the companies supported by the IT organizations include financial services, media, and business services. Table 2 shows the survey demographic averages across the IT organizations.

T ABLE 2
JBoss ON Survey Demographics
Demographic Employees IT staff IT operations management staff before JBoss ON IT operations management staff after JBoss ON Servers managed per system administrator before JBoss ON Servers managed per system administrator after JBoss ON Regions
Source: IDC, 2010

Average 10,545 710 5 4 38 84 Asia/Pacific, EMEA, North America, South America

RE ASONS FOR IMPLEMENTING JBOSS OPERATIONS NETWORK

Survey respondents were asked about the reasons their organizations decided to implement JBoss Operations Network for managing JBoss Enterprise Middleware. As shown in Figure 2, all of the respondents cited "reduce costs" as a major reason for implementing JBoss Operations Network. Two-thirds of the respondents cited "increase efficiency" and "integrate processes," while one-third pointed to "competitive advantage" as a factor influencing their organization's decision.

2010 IDC

#224332

FIGURE 2
Reasons for Implementing JBoss ON

Source: IDC, 2010

BENEFITS OF JBOSS OPERATIONS NETWORK

Cost Reduction
Companies in this study were able to reduce costs with JBoss Operations Network by moving away from other more expensive or less efficient tools by avoiding increased staffing or new hires and by reducing costs for new hardware. One customer said, "In terms of JBoss Operations Network the centralized administration tool it is displacing would have a licensing fee close to $200,000 per year, and we are spending about $16,000 in licensing fees for JBoss ON." JBoss Operations Network helps IT organizations manage critical hardware resources. A senior middleware architect commented, "With JBoss ON we can do a better job of resource planning for our hardware; we can use the information from JBoss ON to determine if we need more hardware, memory, etc." JBoss Operations Network enables customers to contain costs by leveraging IT staff. As one senior engineer of middleware services said, "The IT budget going toward keeping the lights on definitely changed after implementing JBoss ON without it we would need an extra 34 people per department, or 12 across our entire IT group, to do the same work." Another example of leveraging IT staff with JBoss ON came from a senior manager of IT operations. "Two administrators' time was mostly spent managing the 85 servers, and now they can manage our entire infrastructure of 255 servers." On average, the companies surveyed in this study were able to reduce annual costs by $28,311 per 100 users with JBoss ON.
"The IT budget going toward keeping the lights on definitely changed after implementing JBoss ON without it we would need an extra 34 people per department, or 12 across our entire IT group, to do the same work."

#224332

2010 IDC

IT Staff Efficiency
IT staff efficiency is based on the amount of staff time needed to perform specific management tasks and the ability to achieve higher scalability in IT operations. One JBoss customer commented, "In the past we had to get into each console to do administration; now we have one central console that we use to manage all 76, so it's a huge time savings." Scalability achieved through the use of management software helps IT operations to keep up with rapidly growing infrastructure demands. For example, one respondent observed, "While the FTEs have remained the same [just me], I am now able to do the work that would have required 34 people [to manage our current environment]." JBoss ON customers are seeing significant efficiency benefits related to help desk operations. According to one user, "Before JBoss ON, our T1/T2 help desk was fielding 50 calls a day; now it's down to 10 calls per day. For T3 calls ... it now takes me about a half hour on average, and before JBoss ON, it used to take me about 1 hour on average per call." Another comment related to staff efficiency in a wider set of operations: "We also get greater efficiencies in our IT staff across our entire operation. I would guess of the 20 people in our IT organization who touch JBoss ON [besides myself], they are maybe 10% more efficient overall. Those 20 people consist of system administrators and DBAs."

User Productivity
User productivity is influenced by such factors as slowdowns, downtime, and loss of application availability. JBoss ON helps IT operations prevent incidents and respond more quickly to analyze and repair problems when they do occur. One key metric is the mean time to repair (MTTR). In terms of the study respondents, the average MTTR was reduced from 1 hour before JBoss ON to 15 minutes after JBoss ON an impressive 75% reduction. Downtime has a direct impact on user productivity: Fewer hours "in the dark" means users have more time to complete tasks. One user noted that with JBoss ON, "we have seen reduced downtime, as well as rapid relief in terms of things like patches for example, if we have a problem or need a particular update to our environment, we can go to Red Hat and they will do a one-off patch." Another user commented, "Before its implementation, we had probably 3 downtime incidents per month, and each incident probably averaged 23 hours of downtime. We even had some times we were down a whole day. Now with JBoss ON, in the past year, we had only one issue with downtime, and it lasted only an hour." On average, downtime hours per year were reduced by 83% with JBoss ON for the study respondents. User productivity is impacted by incidents requiring help desk support. JBoss ON customers have been able to reduce the number of support calls to the help desk and also shorten the amount of time spent on each support call. The average number of support calls per week was reduced by 73%, from 268.8 to 73.2, while the average amount of time spent per support call was reduced by 66%, from 25.8 minutes to 8.3 minutes. On average, in this study, customers experienced an equivalent of $24,392 per 100 users in annual user productivity increase.

2010 IDC

#224332

Revenue Increase
Revenue increase is driven by such factors as reduction in downtime and limitations in accessing key business applications. One respondent explained that before JBoss ON, "our call center reps couldn't access their systems for long periods of the day, and that gave us a lot of problems. If a customer called the insurance company with an insurance proposal in hand, and we told them, 'We can't help you,' it would definitely affect our business." For the survey population as a whole, the average yearly revenue loss from downtime was reduced from $5.7 million to $966,000 with the use of JBoss an 83% improvement.

Benefit and Cash Flow

The annual benefit, investment, and cumulative cash flow over three years are shown in Figure 3. Benefits tend to increase over time as the solution gains greater traction in the organization. Investments are highest in the year JBoss Operations Network is purchased. Once the initial cost is accounted for, investment in the following years declines and levels off over time. Ongoing investment includes annual subscription fees, standard hardware turnover, IT time required to maintain the solution, and staff training.

FIGURE 3
Benefit, Investment, and Cash Flow

Source: IDC, 2010

#224332

2010 IDC

Return on Investment
The three-year IDC ROI analysis on JBoss Operations Network is based on initial and annual investments compared with the benefit over the three years. Based on the data gathered from the customers in this study, this solution offers an ROI of 634% and payback occurs at 5.3 months. Table 1 displays the ROI results (details on how these figures are calculated are available in the Appendix).

NEED TO MANAGE OPEN SOURCE MIDDLEW ARE

This ROI analysis is presented in the context of the growing need to manage open source middleware. Linux has rapidly grown in importance as an operating environment and is being used to deploy an increasing number of applications and databases often replacing higher-cost Unix systems. Many applications are dependent on JBoss Enterprise Middleware for SOA, Web servers, and application servers. With increasing business value, Linux-based applications must be managed to achieve high-availability, security, and performance standards. Management software is needed to ensure service quality and to support such functions as provisioning, configuration management, and change and patch management.

MAN AGEMENT BENEFITS

Linux management software provides key benefits for managing Linux-based servers, middleware, and applications. These benefits include the following: Better service quality. Management software can improve availability and performance with fewer slowdowns and reduced downtime. Downtime has direct costs to the business that come from loss of business opportunity as well as decreased employee and end-user productivity. Leveraging of staff resources. Management software can help increase the amount of staff time that can be used for productive work. Automation. Management software can be used to automate routine or repetitive tasks, such as server provisioning or applying patches. Agility. Management software can facilitate and greatly reduce the time needed to implement system and application changes. Cost savings and improved ROI. Management software drives cost savings including reduced hardware and software costs, reduced IT operational costs, and reduced IT management costs. Benefits also come from increased end-user productivity and enhanced availability of business applications.

2010 IDC

#224332

RED HAT LINUX MAN AGEMENT

Red Hat Inc. develops, distributes, and supports the Red Hat Enterprise Linux operating environment, based on the open source Linux kernel, and JBoss Enterprise Middleware, which includes the JBoss SOA Platform, the JBoss Enterprise Application Platform, the JBoss Enterprise Web Server, and other middleware components. One of the key requirements for Red Hat is to provide management support for systems and applications deployed on Red Hat Enterprise Linux servers and JBoss Enterprise Middleware.

Red Hat Network and JBoss Operations Network

Red Hat Network (RHN) is a solution that provides system management services specifically for Red Hat technologies and for the distributed software that makes up the Red Hat Enterprise Linux operating environment, as well as for applications deployed on Red Hat Enterprise Linux. JBoss Operations Network is a management software platform for centralized systems management (server-agent) of JBoss Enterprise Middleware. JBoss ON provides management functions for the JBoss SOA Platform, the JBoss Enterprise Application Platform, the JBoss Enterprise Web Server, and other JBoss components. JBoss ON management functions include discovery and inventory, configuration management, application deployment, availability management, performance management, and content management.

CHALLENGES AND OPPORTUNITIES

As shown by the results of this ROI study, management software for open source middleware can provide a number of key benefits for IT organizations, including increased IT efficiency, operational cost reductions, and service improvements such as reduction in downtime. These factors contribute to making Linux an increasingly viable platform for deploying enterprise-class applications and databases. One challenge for Linux management is to extend functional capabilities to encompass the changes occurring in infrastructure technology, such as the rapidly spreading virtualized infrastructures in the x86 environments. Increasingly, Linux systems management will need to address both the Linux virtual images and the impacts of the virtual infrastructures in which the images are deployed. Another area of high interest is managing Linux in the cloud. As cloud architectures mature, IT organizations will want to manage Linux-based applications in a variety of cloud configurations with emphasis on self-service capabilities, including provisioning, patching, and security management. Ensuring service quality including performance and availability will be an important management concern for Linux-based applications deployed in the cloud.

#224332

2010 IDC

These technological advances pose opportunities for Linux management vendors to develop new and extended functionality for system management software. Other opportunities exist for increased integration of core Linux management capabilities with major application platforms, commercial databases, and enterprise management suites. The further development of Linux management software by companies such as Red Hat can offer IT organizations the opportunity to achieve positive ROI benefits in expanded areas of their operations.

SUMMARY AND CONCLUSION

This ROI study is based on data gathered from structured in-depth interviews with representatives from IT organizations that are using JBoss Operations Network to manage JBoss Enterprise Middleware. Based on this study, the IT organizations experienced strong positive returns on their investments, yielding an average 634% ROI over a three-year period over six times the initial investment. In today's economic climate, IT organizations are looking for ways to achieve cost savings in the short term. Based on this study, the IT organizations achieved this goal by realizing an average payback period for JBoss Operations Network of a short 5.3 months recovering their initial investments well within the first half-year of deployment. Another key metric that demonstrates the positive effects of using JBoss Operations Network is the impact on the number of Linux servers running JBoss Enterprise Middleware that can be managed by a single system administrator. Before deployment, the surveyed IT organizations averaged 38 Linux servers running JBoss Enterprise Middleware per administrator. After successful deployment, the organizations averaged 84 Linux servers running JBoss Enterprise Middleware per administrator, more than doubling the administrators' server management capabilities. In summary, the IT organizations that were interviewed for this study received positive ROI results, with short payback periods, from deploying JBoss Operations Network to manage their JBoss Enterprise Middleware environments.

2010 IDC

#224332

APPENDIX
IDC'S ROI Methodology
IDC's ROI methodology measures the efficiency of management software products and processes and uses the findings to calculate ROI for the deployed management software. The method does this in four steps: 1. Evaluates the internal and external costs of administering the systems, networks, and applications before deploying the management software tools. Ascertains the investment in the purchase, implementation, and deployment of the management software tools. It is important to estimate not only the initial purchase cost of software but also the required implementation, integration, and training costs. To measure the total deployment investment required, IDC is careful to include questions not only about the cost of purchasing and setup of the software but also about the integration and the annual software maintenance fees. Measures the cost savings and gains in productivity, availability, and efficiency achieved using the management software tools. Portions of the interviews are dedicated to the discovery of cost savings including both "hard" IT costs, such as savings in software rental and maintenance fees, and "soft" costs, such as IT staff productivity, IT management efficiency, and application availability. IT staff productivity. To measure changes in IT productivity, IDC asks about the use of staff time in such deployment and operational areas as setting up servers, deploying and updating software, tracking hardware and software assets, and dealing with user problems. Staff time for these tasks before and after implementation is recorded, together with the fully burdened (i.e., after fringe benefits and overhead) hourly staff salary rate. IT management efficiency. IT management efficiency pertains to efficiencies achieved in user administration and support by obtaining better management scalability. Some questions asked relate to the ability to centrally manage remote locations to achieve reductions in travel costs, while others relate to the additional staff that would be required to support expected growth in the user or server population, with and without the tools. Application availability and user productivity. To measure the effects of application availability, IDC concentrates on determining the effect on user productivity and business revenue caused by downtime by asking questions about systems, network, and application unavailability patterns before and after implementation. The fully burdened hourly salary rates of the user base are also required, and an estimate is sought of the loss of business that would be associated with an hour of downtime. 4. Calculates the payback period and ROI for the deployed enterprise management software. Based on the interview data, IDC calculates the average payback period and rate of return based on the overall cost savings resulting from the investments in JBoss Operations Network software. To normalize the data, IDC presents the results in terms of per 100 users.

2.

3.

10

#224332

2010 IDC

ROI and Payback Period Calculation Assumptions

IDC bases the payback period and ROI calculations on a number of assumptions, which are summarized below: Time values are multiplied by burdened salary (salary + 28% for benefits and overhead) to quantify efficiency and manager productivity savings. Downtime values are a product of the number of hours of downtime multiplied by the number of users affected. The impact of unplanned downtime is quantified in terms of impaired end-user productivity and lost revenue. Lost productivity is a product of downtime multiplied by burdened salary. Lost revenue is a product of downtime multiplied by the average revenue generated per hour. The net present value (NPV) of the three-year savings is calculated by subtracting the amount that would have been realized by investing the original sum in an instrument yielding a 12% return to allow for the missed opportunity cost. This accounts for both the assumed cost of money and the assumed rate of return. Because every hour of downtime does not equate to a lost hour of productivity or revenue generation, IDC attributes only a fraction of the result to savings. As part of our assessment, we asked each company what fraction of downtime hours to use in calculating productivity savings and the reduction in lost revenue. IDC then taxes the revenue at that rate. Further, because IT solutions require a deployment period, the full benefits of the solution are not available during deployment. To capture this reality, IDC prorates the benefits on a monthly basis and then subtracts the deployment time from the firstyear savings.

Copyright Notice
External Publication of IDC Information and Data Any IDC information that is to be used in advertising, press releases, or promotional materials requires prior written approval from the appropriate IDC Vice President or Country Manager. A draft of the proposed document should accompany any such request. IDC reserves the right to deny approval of external usage for any reason. Copyright 2010 IDC. Reproduction without written permission is completely forbidden.

2010 IDC

#224332

11