!

Nugent 1

Dylan Nugent 13 November 2013 POLS 30595 World Bank & Stockholm University: Community-based Monitoring in Uganda I. Context & Institutional Setting World Bank studies have indicated that health systems monitoring and improved accountability could prevent the millions of annual childhood deaths from preventable diseases. In 2004, along with researchers from Stockholm University, the World Bank conducted a randomized field experiment on the community-based monitoring of healthcare providers in fifty rural communities across nine Ugandan districts. The initiative focused on public dispensaries in the selected communities, which fall within the lowest tier (below hospitals) of the countrys health system where patients still have face-to-face interaction with a physician or other healthcare professional. Utilizing local NGOs, the experiment was designed to improve the accountability relationship between citizens and primary health providers through citizen report cards and community meetings. Until this point, there was a dearth of relevant information on health service delivery and no widespread set of expectations for providers in these communities, which prevented monitoring at the communal or provider level (Bjrkman and Svensson 735-738). II. Project Design & Implementation Staff from the World Bank and Stockholm University designed the project in conjunction with eighteen Ugandan NGOs. The ultimate goal of the experiment was to increase the effort of health providers, thereby improving the quality of care available to the community. It involved fifty public dispensaries in predominately rural areas across Uganda. The dispensaries were selected in different catchment areas, which were five-kilometer radii around the specific facilities, and organized by location and population before being randomly assigned to treatment and control groups of 25 facilities each (Bjrkman and Svensson 739-740). Data collection for the project was designed to assess the quality and efficiency of dispensary services by contrasting the perspectives of the patients and providers, thus requiring the

Nugent 2

implementation of two simultaneous surveys (one for each side). Provider data was collected by acquiring patient records kept by dispensaries for their own uses, rather than through a reporting mechanism, to prevent intentional misreporting. Patient surveys were then carried out at a random sample of households within each catchment area, totaling roughly 5,000 households across the fifty communities. These surveys were conducted at the beginning and end of the intervention. The results were compiled into scorecards that were translated into local dialects and disseminated to each specific facility. To distribute the information in the report cards, participation was encouraged through a series of community meetings. In each village, roughly 100 participants would share their perspectives on patients entitlements, in addition to suggested improvements for each hospitals shortcomings and potential methods of future monitoring. These meetings were followed up with staff meetings to compare what the hospitals had reported and the community expressed grievances. Lastly, a representative group from each community and health provider would discuss the suggested improvements and create a contractually based shared action plan outlining the communitys specific needs (Bjrkman and Svensson 741-743). To enhance the monitoring and evaluation, Jakob Svensson, a collaborator on the Ugandan community-based monitoring project, and others later published their preferred indicators to assess hospital performance based on a service delivery system that create outcomes, rather than indicators focused solely on the eventual outcomes (Bold et al. 2). These delivery service indicators (DSI) were created meet the need for a supply side indicators on successful service delivery and health provider behavior. The DSI are then presented as a report card that seeks to measure the performance and quality of service delivery as experienced by citizens (3). It relies on both qualitative and quantitative information that includes tracking provider expenditures, quantitative statistics on services rendered, staff absenteeism, and observation. The DSI report card for healthcare was broken down into three major categories with several measurable statistics within each: 1. Clinics: infrastructure and inputs

Nugent 3

Access to basic infrastructure services including electricity, water, and sanitation at primary health facilities was evaluated. Similarly, inventories of basic medical materials and equipment were measured based on the availability of a thermometer, stethoscope, and weighing scale (Bold et al. 913). 2. Personnel: absenteeism, performance, and time Low incentives and little accountability to higher-level staff contributed to high absence rates among personnel. Clinical performance was evaluated on the bare-minimum basis of correctly diagnosing patients, providing appropriate treatment, and informing the patient about what actions to take. In addition to patient and provider perceptions, performance was evaluated by presenting the same case study patients to several physicians for quick comparisons. The time spent treating patients and facility efficiency was also measured (Bold et al. 9-13). 3. Funding: expenditures and staff pay While primarily related to the allocation of government-level sector funding to individual providers, expenditures were evaluated to determine if a leakage of resources was occurring. Staff pay was also monitored for delays to determine its impact on quality of service and morale. These final indicators are more relevant to the healthcare provider than the community, but can be readily incorporated into the surveys for data collection (Bold et al. 9-13). The DSI framework for report cards and community-based monitoring create a system that is evaluated based on the successful receipt of expected healthcare services. Effective facility governance and staff accountability are more immediately related to this service delivery than communal health outcomes; however, as shown below, this project also drove the desired improvements to public health. III. Results Rigorous statistical analysis exhibited improvements in nearly all areas measured in the treatment group only six months after the initial intervention. The absence rate of facility employees, as measured by physical presence at the time of the survey, improved by roughly 13%. A larger proportion of households had received information deterring them form self-treatment and

Nugent 4

encouraging visits to health providers. The treatment group reported less frequent stock outs of medications, despite equal inventories and more patients than dispensaries in the control group. Immunizations and nutrition also improved for children under five in the treatment group: twice as many newborns received vitamin A supplements, 46% more newborns received BCG vaccine, and 42% more newborns received the first dose of polio vaccine when compared to the control group. Qualitative post-intervention evaluations also revealed that facilities were cleaner and visits to traditional healers declined in the treatment group (Power to the People, 2009, 751-755). IV. Application to Catholic Relief Services Health Systems Strengthening in Zambia As the Notre Dame Initiative for Global Development and Catholic Relief Services look to address governance and accountability from the community level in four Zambian hospitals, the community-based monitoring project in Uganda provides a starting point for what a community scorecard should include to ensure that all aspects of reliable service and accountability are measured, as well as how it can be complemented through regular community feedback. The experiment carried out by the World Bank and Stockholm University is obviously on a much larger scale than what is currently being considered in Zambia. Nevertheless, it can be scaled down to the hospital level for CRS, as several elements of the survey and scorecard process are directly relevant. The World Banks approach in Uganda is community-driven, inherently tailored to each communitys expectations, applicable in resource-constrained environments, and can be readily integrated into other methods of monitoring and evaluation. It provides several indicators that should be included in community surveys and the compilation of report cards, while supplementing this method with open forums for service feedback. Svenssons indicators are dually focused on the immediate outcomesthe perceived quality of care receivedand the broader outputsimproved public health. It ties provider accountability to patient satisfaction, while also ensuring an effective long-term impact. The example provided is merely one intervention of this community-based monitoring approach, but its resource light nature allows it to be an iterative process that can be utilized for continual monitoring and governance.

Nugent 5 Works Cited

Bjrkman, Martina, and Jaokob Svensson. Power to the People: Evidence from a Randomized Field Experiment of on Community-based Monitoring in Uganda. The Quarterly Journal of Economics (May 2009): 735-769. 12 November 2013. Bold, Tessa, et al. Delivering Service Indicators in Education and Health in Africa: A Proposal. econ.worldbank.org. The World Bank. June 2010. 12 November 2013.