Proceedings of I-KNOW 04 Graz, Austria, June 30 - July 2, 2004

Measuring Knowledge Management at HP Services Consulting & Integration

Gita Haghi
(HPS Consulting & Integration, Austria gita.haghi@hp.com)

Abstract: Hewlett Packard services consulting and integration (HPS C&I) has had worldwide Knowledge Management programs for over 6 years, but until recently they had not developed a consistent set of monthly KM measures. In late 2003, HPS C&I established a set of 6 worldwide measures, which are described in this presentation. We also discovered that there was value in having uniform worldwide measures, but we could not establish uniform worldwide goals. The reason for this had to do with wide variability in business maturity and English language capability within our workforce. Consequently, we assigned each country into one of four "bands" and set different goals for each band. The presentation provides details on this aspect of our measurement program. Actual performance is not presented, because it is deemed company confidential. Keywords: Knowledge Management, Balanced Scorecard, Knowledge Measurement Categories: A, H

Introduction

HPS C&I is host to HP's Global Industries (Financial services, manufacturing, network and service providers, public sector) and encompasses four geographical regions (Americas, Asia Pacific, EMEA, Japan), two practices (enterprise application services and enterprise infrastructure), and 16,000 consulting professionals around the world. Below report presents the case study of knowledge management measurement program carried out at Hewlett Packard services consulting and integration (HPS C&I) department. It is built upon the balanced scorecard program, which should manage, track and measure HPs business and is supported by HPs CEO Carly Fiorina. The case study should reflect HPS C&Is approach to KM metrics selection, measuring them and finally analyzing them for improving the KM program.

Knowledge management measurement program

In September 2003 HPS C&I implemented a new KM measurement program to assess the health of its collaboration and KM activities. The purpose of the program was to improve the way KM operates in C&I. Over time, the KM metrics will be tied into more traditional business measurements: sales cycle, win rate, and project profitability. Prior to starting the measurement program, HPS C&I KM team developed a KM expectation document which defined the KM expectations that apply to practitioners, project managers, practice principals and senior leaders. This was

Haghi G.: Measuring Knowledge Management ...

89

reviewed and approved by senior management. Our choices of KM metrics took action items from this document into consideration. 2.1 How is the Balanced Scorecard implemented at HP?

Announced last spring, the Balanced Scorecard (BSC) is a strategic management system that uses both financial and non-financial measures to track HP's strategy execution and performance, set goals and report results. In HP's Leadership Framework, the Balanced Scorecard is part of the Metrics and Rewards component. During the second half of FY02, the company piloted the Balanced Scorecard with the Executive Council, basing members' variable pay on metrics in four key dimensions: financial; integration; customers and employees. In FY03, the company's businesses and functions rolled out the Balanced Scorecard, starting with senior levels of the organization. Ultimately, individual performance plans at all levels will be tied to dimensions and metrics of the Balanced Scorecard to ensure strategic alignment throughout the organization. In contrast to a more traditional system that focuses on financial metrics, a balanced scorecard "balances" both financial (e.g., revenue, profit) and other (e.g., customer, employee) measures. This model which allows companies to evaluate past performance and gain insight into future performance is considered a best practice in much of the business world. Approximately 50 percent of companies in the U.S. and Europe use a version of the balanced scorecard (Source: Mercer Delta). Prior to the merger, Compaq used a balanced scorecard, as did several of HP's businesses. Each group/function will have the ability to customize the Balanced Scorecard for its own strategic objectives, following a few simple guidelines. For example, corporate dimensions and metrics cannot be eliminated. In addition, business/function-specific metrics can be added to any of the existing dimensions as long as they do not conflict with the corporate metrics. These guidelines are communicated to group and function heads, as well as their direct reports. 2.2 Why are KM metrics important?

When an organization builds effective measures and tracks metrics around knowledge sharing, then tracking the success of implementation, identifying key milestones, and showing return on investment become possible [Wesley, 2002]. Like every business process, KM must be measured so it can be managed. Measurement tells us whether our KM programs are meeting their targets. Ongoing measurement and analysis will tell us if KM is doing what it is supposed to do: aiding the growth of business reducing the cost of business pursuit and delivery, and improving Total Customer Experience

The value and results of KM programs are only hypothetical unless their effectiveness and their impact on ones business are measured. By defining and publicizing our measures, we also encourage employees to behave in ways that contribute to the achievement of KM results.

90
2.3

Haghi G.: Measuring Knowledge Management ...

What is measured?

Measures are appropriate to a particular knowledge management approach, its objectives, and its stage of development [Crager et al. 2003]. Based on the knowledge management systems we currently use and their main focus on communities of practice and team collaboration, the worldwide KM team in agreement with regional KM leads selected 5 initial measures. They all deal with intellectual property, people and their expertise: Project profiles - what has HP done before - leverage for presales and delivery Communities of Practice (CoPs) - detailed material in specialized verticals and horizontals Forums - more personal interaction in form of discussions along the lines of the CoP Knowledge Briefs (KBs) - short, technical knowledge-sharing articles Project and Team Collaboration - sharing information and knowledge within Teams using two different tools offered by C&I

Our final goal is to measure active and passive participation at KM program in terms of knowledge creation, capture and reuse. So for instance on one side we use the number of users/visitors and on the other hand the number of submitted documents, postings, as well as the number of downloads. Thus you would get a complete picture of what is going on in each aspect of knowledge management. To achieve this purpose, we created 7 indices (see Figure 1) measured on a monthly basis. Three of them measure only active participation from users side- they are as follows: PPR index= (#of submitted project profiles Year-To-Date / #of new projects Year-To-Date) * 100 KB index= (#of submitted KBs / total number of consultants in this region) *12 (annualized) Forum contribution index= (#of postings to forums/ total number of consultants in this region)*12 (annualized) The other 4 indices reflect usage of available KM tools, no matter active or passive: Tool (WSS/CoP/forum) index= (# of active user/total # of consultants)*100 Whereby we consider an individual as active user of a specific tool (e.g. Community of practice CoP) only if she/he has been once viewing/downloading/submitting something on that tool during the past month. In other terms a user who had performed an activity on that tool regardless of its nature (active or passive participation). The raw data for these metrics are extracted from different sources. General numbers, like the number of consultants in different regions or number of new projects are retrieved from HP Services common platforms provided by human resource or project office department. Other data like number of active users from a

Haghi G.: Measuring Knowledge Management ...

91

certain country/region for a specific KM tool e.g. CoP/WSS/forums or number of postings to a forum or submissions are generated by running special statistics programs on those tools. 2.4 How do KM measures link to business goals?

The KM measurement program will use the balanced scorecard method that has been widely adopted in HP. The KM Balanced Scorecard will have customer, internal process, employee, and financial components that link to CI's Global KM program objectives and CI business goals. For some metrics we could of course see a direct relation to our business goals. For example the PPR( project profile repository) Index shows whether each project is documented in terms of knowledge sharing. Having a full list of projects serves as a reference list for future proposals. On the other hand, lessons learned from each project should prevent future failures. KB(Knowledge briefs) metrics gives us a picture how efficiently our expert knowledge is becoming available for everyones usage in a compact format. Increasing this number would benefit the whole business. Other metrics reflect at first place how our people collaborate. However it is proven that using easy platforms for collaboration save each consultant a lot of time and this has a major impact on our costs. Identifying strengths and weaknesses of the program should happen on 3 different levels: Country Practice Knowledge area (creation, capture, reuse) The selection of above parameters for KM metrics program correlates with the worldwide Balanced Scorecard program. Whereby BSC approach towards KM program is based on a document called KM expectation created by global KM team and approved by senior management. Wherein it is determined, what KM duties are expected from each group according to their job functions. For example depending on their job roles, employees are obliged to submit project profiles (for project managers, project assistants or project team members) or knowledge briefs (for technical consultants, solution architects). Currently negotiations with human resource department and senior management are under way to extend the evaluation of employees KM activities in the balanced scorecard. This would be in line with the KM expectations mentioned earlier. According to them, experts like solution architects, technical consultants, and senior consultants should respond to incoming questions posted to forums in their expertise area. This could happen on a rotational basis. On the other hand membership to at least one community of practice (preferably to the respective area of expertise or to more than one) should become mandatory in the near future. 2.5 What are the targets?

In order to have an equal comparison for all countries worldwide, we decided to classify them into country bands as follows:

92

Haghi G.: Measuring Knowledge Management ...

Band A Countries with a mature business with widespread accepted English language written culture (e.g., Canada, Germany) Band B Countries with a mature business where English is poor and a significant cultural barrier (e.g., Brazil) Band C Countries with an immature business with widespread accepted English language written culture (e.g., India) Band D Countries with an immature business and where English is poor and a significant cultural barrier (e.g., Vietnam) Upon this classification, we set targets according to the business and language situation. For those countries where English language is a barrier, we need to provide our localized KM tools. If these are not available, the targets are set lower. In each band we set up globally consistent targets for team collaboration, knowledge creation, knowledge reuse, and knowledge application. Thus we are able to break out and compare results by country and region. The report is designed to be easy to understand, and uses the common conventions of Green (meeting target), Yellow (somewhat below target), and Red (urgent attention required). The colored table actually gives everyone a lot of information about the countrys performance at first glance.

Figure 1: KM metrics chart for HPS C&I

Haghi G.: Measuring Knowledge Management ...

93

Each quarter, the goals themselves will be reviewed. As the KM program matures, it is likely that "the bar will be raised." Therefore, it is important for countries and practices to address the root cause of Yellow and Red measures quickly because expected performance levels will only increase in the quarters to come. 2.6 Who are the stakeholders and what do they need to know?

Senior management, KM leads in regions and countries, as well as KM leads for industries and practices are the first recipients of our monthly metrics. Senior management is primarily interested to see ROI measures, which cannot be retrieved from our chart. But an extensive use would at least ensure them that the investment is going into the right direction. By measuring each countrys, regions KM activity, we provide KM leads and operational managers an insight into their constituencys usage of KM tools. This could be leveraged primarily for improvements to the KM program. For example if the usage is very low, the question comes up whether it is because localization is needed for that particular country or do people face many problems using the tool or is it simply lack of awareness. By analyzing these data, each KM lead could take actions for increasing the usage of KM tools. On the other hand they could give us feedback why some countries make no use of the offered program and the worldwide KM team can better understand users needs and try to fulfil them as much as possible. We also hope that the comparison also boost competition between regions and finally encourages people to share their knowledge and take part at the KM program. Afterwards these numbers are published in our monthly KM newsletter, which is automatically sent to all KM interested parties through their subscription. KM leads in regions and countries are responsible for communicating KM expectations and the monthly metrics to their respective regions constituency. Within the global KM team one person is assigned to create monthly KM reports in collaboration with KM leads. Others assist in preparation of the raw data or communication of the results through the monthly newsletter.

Conclusions

We received very positive feedback from senior management on our metrics program. The colored chart was easy to understand and comprehensive. Nevertheless an email analyzing the results of the chart brought strengths and weaknesses into KM leads and their managers attention. This led those to take actions towards better promoting the KM program within their respective regions. All in all measuring KM activities within a large company is a time-consuming task, similar to every type of data mining and statistic analysis. Therefore all requirements for a KM metrics report and the purpose behind it should be well defined prior to initiating such a program. Otherwise it would be hard to justify the amount of time spent on preparation and creation of such statistics. It would be beneficial to include all stakeholders in the early stages of the decision process. Furthermore we would recommend investing more time on automation of data

94

Haghi G.: Measuring Knowledge Management ...

retrieval and design of templates at the very beginning of the process. Thus you would save plenty of time in the subsequent months. As stated in [Wesley, 2002] measures should help you manage the implementation of your KM initiative and help to change behaviour. Whereby KM should be considered as an enabler for other business processes, so we need first to understand critical success factors for that area of business before developing measures. Eventually this should lead to measuring business results rather than KM activities.

Future Work

We intend to expand the level of detail for our worldwide report to include Practice metrics, which should reflect HPS C&Is global industries and practices consisting of Financial services(FS), manufacturing, Network and Service Providers, Public sector, enterprise application services and enterprise infrastructure. Talks have been started with representatives from two Practices and KM Leads for a better understanding of requirements. This would mean that we would provide each Practice with usage numbers of its constituency from different regions. At the moment, we still have some obstacles, which we need to overcome. Such a report should help the Practices to better monitor their communities and determine whether these are fulfilling consultants' needs in terms of content, collaboration, etc. As soon as the prototype of the first two Practices is ready (FS and Enterprise Solutions), it will be communicated and applied to other Practices. Another improvement which is under way is output measures. This correlates to download number. Only by measuring this number we could really judge the value of the content we provide on various platforms.

References
[Crager et al. 2003] Crager, J.; Lemons, D.: Measuring the impact of knowledge management, executive summary, 2003; APQC (American Productivity and Quality Center) [Wesley, 2002] Wesley Vestal, August 2002: Measuring Knowledge Management; APQC (American Productivity and Quality Center)