Вы находитесь на странице: 1из 15

Realist evaluation of the

Integrated electronic
Diagnosis Approach (IeDA)
for the management of
childhood illnesses at
primary health facilities
in Burkina Faso

Evaluation
October 2018
2 | London School of Hygene & Tropical Medicine 3

Contents

Executive Summary 4 Results 14


Lead institution:
Background 4 The management vision 14
London School of Hygiene & Tropical Medicine
Methodology 4 The actual intervention 14

Results 4 Intensity of implementation 14


Partner institution:
Centre Muraz Introduction 7 The political and policy context 16

Background 7 The political context 16


LSHTM investigators:
Simon Cousens (PI), Karl Blanchet (PI), The Electronic Register of The policy context 16
Consultations (REC) and the
James Lewis, Sophie Sarrassat, IeDA intervention 7 Free healthcare policy initiative 16

Blandine Binachon The research methodology 8 Comparative advantage 17

Centre Muraz investigators: Realistic Evaluation 9 Compatibility 19

Arsene Some Satouro (National Coordinator), Principles of realist evaluation 9 Complexity 19


Vincent-Paul Sanon
Formulation of our MRT 10 Testability 20

Implementation evaluation 10 Observability 20

The study was funded by The Bill & Melinda Context evaluation 10 Analysis 22

Gates Foundation and the Swiss Agency for Mixed methods: embedded A summary of the intervention
realistic evaluation 10 and its outcomes 22
Development and Cooperation
Realistic evaluation design and CMO configurations 23
research methods 12
The new MRT 24
Conceptual framework 12
Discussion and Conclusion 25
Study design 12
Lessons for policy and practice 25
Data collection 13
Methodological lessons 25
Data analysis 13
References 26

Intervention implemented by
© Tdh-Ollivier-Girard

Published October 2018


Copyright London School of
Hygiene & Tropical Medicine, 2018

Cover Images:
Ollivier-Girard
4 | London School of Hygene & Tropical Medicine 5

Executive Summary

Background was emphasized by the introduction


of a national free healthcare policy on
child health care during the course
Recent advances in Information and Communication Technologies (ICT) could potentially of the project. Second, the initial
transform health care services in low- and middle-income countries. However, the experience training of healthcare workers on
with using such technology to improve adherence to the Integrated Management of Childhood IMCI needed to be complemented
illness (IMCI) guidelines is limited. by regular supervisions and
coaching after initial training. Third,
staff turnover of nurses is very
From 2014, Terre des hommes, in intervention that may have acted opinions and understandings of
high in rural areas of Burkina Faso
partnership with the Burkinabe as moderators of implementation actors intervening in IeDA. In-depth
making challenging the progressive
Ministry of Health (MoH), and outcomes, i.e. as facilitators interviews were conducted with 154
development of nurses’ skills on
implemented the Integrated or barriers to IeDA implementation individuals including 92 healthcare
IMCI. Finally, the capacity of district
eDIagnosis Approach (IeDA) package and affecting REC use. In particular, workers from health centres,
health teams to conduct supervisions
of interventions in primary health we payed attention to potential 16 officers from district health
is quite limited due to their budget
facilities of two regions of Burkina unanticipated factors. Social, political, authorities, 6 members of health
restrictions and limited access to
Faso with the objective of improving resources and logistical factors were centre management committees. In
vehicles – volume of resources that

© Tdh-Ollivier-Girard
health care workers’ (HCW) screened and added throughout the addition, 5 focus groups (on average
will not increase in the near future.
adherence to the IMCI guidelines. study. Various sources of information 11 people per group) were organised
were used: individual interviews and with mothers and carers. During the later phases of the
An evaluation was performed by an document review (project documents analysis, we found that the adoption
independent team from the London as well as national policy documents). process can be grouped according
School of Hygiene and Tropical
managers to district health teams effective and regular support, dialog to their key mechanism and this led
Medicine (LSHTM), United Kingdom, For the implementation evaluation, Results and in a certain measure to Heads to identify their problems and needs to the description of parallel CMO
and Centre Muraz, Burkina Faso. The a set of process research questions
of health centres anytime support is in using the REC and full recognition configurations, each with their own
aim of the realistic evaluation was to were defined prior to the evaluation The actual activities of the
needed. of their role in improving child outcome.
identify the potential mechanisms for and according to the IeDA theory intervention can be summarised
consultations.
change within the IeDA programme of change. Most of the information as follows: (i) Development and Regarding the process of Our analysis identified three
and specify how they are able to collected was triangulated from implementation of improved versions implementation, we noted a good Regading the implementation CMO configurations that indicate
change existing social processes different sources. This was of the REC; (ii) Provision of a six-day coherence between the initial context, we found several potentially causal pathways between sets of
within primary health care facilities. particularly important as the data training course on IMCI guidelines theory of change developed by the important elements in the context management practices and use
were collected retrospectively and REC; (iii) Development of a implementers (MoH and Tdh), the of primary health care in Burkina of REC and we modified the MRT
hence subject to recall biases. quality assurance mechanism; (iv) project management team’s vision Faso in the two regions of Boucle accordingly:
A combination of electronically Monthly supervision of every health and district health teams and health du Mouhoun and Nord. First, all
Methodology documents review and in-depth centre benefiting by the district centres team’s practice. Indeed, in health centres in the two regions The adoption of a computer-based
interviews were conducted in 2017. health team; and (v) Development line with their vision, the project were staffed by at least one nurse decision support tool by health
The realistic evaluation was
of a health information system. We management team motivates the (depending on health centre’s size) staff at primary health care will be
embedded within the steeped In order to capture social phenomena also found important additional health centre staff involved in child who were all aware that management enhanced by having a leadership
wedge trial in order to explain such as management decisions and activities that organically appeared consultations (e.g. nurses, midwives of childhood illnesses is an important focusing on building wide consensus
some of the results of the trial and interactions between individuals, during the implementation of the and nurse assistant) through priority at primary health care, which from surrounding stakeholders (local
explore research questions identified an in-depth qualitative research intervention: the creation of a support
during the trial. The methodology approach was adopted. Various system to respond to breakdowns Table: the three CMO configurations related to IeDA
first focused on implementation sources of data were used by the and questions on the software and
evaluation and then on mechanism of investigator during data collection the tablet; district meetings at least Context Mechanism Outcome (students)
change and context. that took place between January 2016 once a year to enable district health C1. Availability of a support team M1. Promoting amongst healthcare workers O1. Notions of quality in
and October 2017. The combination of teams and heads of health centres to be responsive to healthcare “doing the right thing the right way” approach- childhood illnesses routinised
The implementation evaluation aimed
several data sources proved valuable to discuss about performance and staff questions. es during consultations
to document how the implementation
to the research. Direct observation in find concrete solutions; creation C2. In health centres where the M2. Clear distribution of roles before and O2. Efficient organisation of
of IeDA was organised and achieved,
health centres generated elements of eLearning modules on IMCI nurse is assisted by at least two during child consultations (including triage, the health team
and how the intervention was
of information that helped to identify available on tablets for continuous other members (nurse assis- weight and size measurements, consultation
received. This implementation
new issues or verify assumptions. knowledge development; an inclusive tants or outreach workers) and and counselling)
evaluation focused on fidelity,
The analysis of project reports from and team approach associating in where management flexibility is
dose delivered (completeness),
health facilities helped analyse the allowed
dose received (exposure), reach the implementation process not
implementation of IeDA and the only nurses but also any other staff C3. Strong consensus amongst M3. Introducing at primary health care level O3. Sustained use of REC as a
(participation rate) and recruitment.
vision of the project by managers. In who is directly or indirectly involved stakeholders on the benefits of the notion of individual accountability and routine practice
The context evaluation aimed to addition, interviews and focus group introducing REC responsibility and collective contribution to
in managing child consultations;
discussions provided evidence in the wider system.
document factors external to the and good accessibility of top Tdh
relation to the perceptions, in-depth
6 | London School of Hygene & Tropical Medicine 7

Introduction

and national authorities) on the


benefits of using such an innovation
On the other hand, the innovation,
REC, needs to be flexible enough
atmosphere and environment
(including community and policy Background
and having a wide of actors fully and to take into account the constant makers support), which can be
truly engaged in the directions the changing policy environment and the translated by peer support and Despite a large reduction in under- evaluation conducted in 2013 found proper nutrition, hygiene and safe
project could take. This necessitates emerging needs and requests from district authorities support, and five child mortality (from 180 per that only 22% of nurses working water) and curative measures (e.g.
a system promoting flows of its users. availability of support services 1,000 live births in 1990 to 83 per in primary care facilities had been treatment of common illnesses,
information between all levels of the responding to software or 1,000 live births in 2015), sub-Saharan trained in IMCI (Kouanda and Baguiya minor surgery, supply of essential
health system where transparency of The REC is adopted when perceived hardware issues. The supportive Africa failed to reach Millennium 2013). Only 28% of children were medicine, maternal and child
information is valued. by users and district managers as environment is based on reciprocity Development Goal 4 target of 60 assessed for three danger signs as consultations). CSPSs are governed
being encompassed within a broader and acknowledges individual deaths per 1,000 live births (United recommended by IMCI, and only 15% by a management committee (comite
The introduction of such innovation quality improvement strategy contributions to the wider system. Nations Inter-agency Group for of children were correctly classified de gestion) composed of members
needs to occur in an environment where health staff is sensitised to Conditions for such environment Child Mortality Estimation 2015). In (Kouanda and Baguiya 2013). About of the community. The district health
flexible enough to provide space the importance of quality and their to be promoted by a leadership 1999, the World Health Organisation 30% of children were correctly management team is in charge of
to staff make decisions on the capacity to address quality issues at that creates a decentralised (WHO) developed the Integrated prescribed an antibiotic for suspected supervising CSPSs and analysing
distribution of clearly-defined tasks their own level. decision space where initiatives are Management of Childhood Illness pneumonia or oral rehydration salts routine data collected in them
within the team in order to better respected.
The introduction of the REC needs strategy (IMCI) (Black, Morris et (ORS) for diarrhoea and 40% were (Ministere de la Sante 2011).
adapt their work to the new situation.
to be accompanied by a supportive al. 2003). This strategy provides an correctly referred (Kouanda and
algorithm to guide health workers Baguiya 2013).
through a systematic clinical The Electronic Register of
assessment of sick children with In 2014, Terre des Hommes (TdH), a Consultations (REC) and
Swiss non-governmental organisation,
the aim of improving the diagnostic the IeDA intervention
classification and the treatment of together with the Ministry of Health
these children (Boss, Toole et al. 1994, (MoH), launched the Integrated The “electronic register of
Nguyen, Leung et al. 2013, Rakha, electronic Diagnosis Approach consultations” or “Registre
Abdelmoneim et al. 2013) and hence (IeDA) intervention with the objective Electronique de Consultations” in
reducing mortality (Jones, Steketee of improving adherence to IMCI French (REC) was designed in 2010
et al. 2003, Rakha, Abdelmoneim et al. guidelines in public primary health by Tdh (Deflaux 2010). The REC
2013). centres in two regions of Burkina Faso. software, based on the CommCare
In this paper, we present the design software language, is installed on the
However, effective implementation of a mixed methods evaluation of this open access CommCare platform
of IMCI is often constrained by poor intervention. developed by Dimagi (Deflaux,
adherence to the guidelines (Bryce, Agagliate et al. 2014). The REC guides
Victora et al. , Derenzi, Parikh et al. Burkina Faso is composed of 13
health workers through the IMCI
2008, Horwood, Voce et al. 2009). regions and 70 health districts. The
algorithm. By doing so it aims to
Previous studies have reported public health system is characterised
improve adherence of nurses to the
that adherence to the guidelines by a three-tier service structure: (i)
clinical protocol and to provide the
decreases over time due to inadequate at the first level are the districts with
local health district and the MoH with
initial training, shortage of staff and 2,000+ health centres (Centre de
routine data on the management of
insufficient supervision (Chaudhary, santé et de promotion sociale (CSPS))
childhood illnesses. The first versions
Mohanty et al. 2005, Rowe, Onikpo and the 104 district hospitals (Centre
of the REC were piloted in 2011 and
et al. 2010). Takada et al. (2007) médical avec antenne chirurgicale
2012 in fifty-two primary health
have noted that health care workers (CMA)), (ii) at the next level are
facilities located in two districts in the
typically find the IMCI chart booklet the nine regional hospitals (Centre
Nord region and perceived by 90% of
burdensome and try to work from hospitalier régional (CHR)), and (iii)
users (nurses) as being a supportive
memory, resulting in a decrease in finally the third level is comprised of
tool during consultations (Yameogo,
quality of care. Chaundhary (2005) the three national teaching hospitals
StollL et al. 2011). An additional pilot
demonstrated that the adherence (Centre hospitalier universitaire
district, Yako, was added in the Boucle
of health workers improved with (CHU)) (DSS/DGISS 2012).
du Mouhoun region in 2014 and
supervision. However, regular 2015. Following the pilot phase, the
The IMCI approach was implemented
supervision of health workers after MoH requested Tdh to expand the
only at the first level of the pyramid,
training is often lacking, (Horwood, implementation of the REC to the
i.e. in health centres. These facilities
Voce et al. 2009, Mugala, Mutale et remaining health districts of both
deliver a minimum package of
© Tdh-Ollivier-Girard

al. 2010) partly due to the lack of regions.


services defined by the Ministry of
resources.
Health comprising both preventive
In order to do so, Tdh launched
Burkina Faso introduced the IMCI (e.g. vaccinations, antenatal care,
in 2014 the IeDA intervention,
strategy in 2003. However, an health education, and promotion of
which includes the following five
8 | London School of Hygene & Tropical Medicine 9

Realistic Evaluation

The research methodology Principles of realist evaluation


The IeDA intervention is being eval- We structured our study in four steps: Pawson and Tilley (1997) call the action. For the researcher, ‘identifying
uated using a mixed-methods study the formulation of the middle Range working hypothesis that emerge mechanisms involves the attempt to
design composed of the following Theory, the design of the study, during the analysis phase “Context- develop propositions about what it is
three interlinked studies (see Figure the data analysis and presentation Mechanism-Outcome configuration” within the program which triggers a
1) (Blanchet, Lewis et al. 2016): of the results. A realist evaluation (CMOC). For Pawson and Tilley, reaction from its subjects’.
1. a stepped-wedge trial to starts from a middle range theory issues of context and mechanism
evaluate the effect of IeDA on (MRT), which is understood as are crucial elements to consider in Consequently, adopting a realist
adherence to IMCI guidelines “theory that lies between the minor any realist evaluation as they help to approach means two things:
in primary health facilities; but necessary working hypotheses explain ‘what works, for whom and ◗◗ the researcher has to identify
2. a cost-effectiveness analysis (…) and the all-inclusive systematic in what circumstances. For these the potential mechanisms for
(CEA) to assess the value for efforts to develop a unified theory authors ‘what works’ is not of itself change within a programme/
money of the delivery of IeDA; that will explain all the observed a helpful question as: ‘programs intervention and specify how
3. a realistic evaluation uniformities of social behaviour, social work (have successful ‘outcomes’) they are able to overcome
to understand the organisation and social change” only insofar as they introduce the or change existing social
implementation process, (Merton 1968). The MRT states how appropriate ideas and opportunities processes.
Figure 1: Map of the two regions Boucle du Mouhoun and Nord and their districts the mechanisms by which the intervention leads to which (‘mechanisms’) to groups in the ◗◗ The researcher has to specify
(Source: Map Universal) the IeDA intervention leads effect in which conditions. The MRT appropriate social and cultural the social and cultural
to change and to identify can be formulated on the basis of conditions (‘contexts’)’.
components, delivered at district and and support to the health conditions necessary for the
factors that may affect these existing theory and past experience.
health centre levels: district authorities in their change mechanism(s) to
mechanisms at health centre If the latter is not available, They summarise their approach
annual supervision scheme. operate as well as locating
◗◗ An electronic Clinical Decision and community levels exploratory-on site research can to evaluation through the use of a
◗◗ A health information system them in different contexts.
Support System (eCDSS) These three studies are taking place be done to unearth the models formula: Context + Mechanism =
based on under-five child There may be multiple
available on tablets and in a total of eight health districts used implicitly by the stakeholders Outcome (CMO) and subsequently
consultation data collected combinations of mechanisms
provided to primary health across the Nord and Boucle du Mou- to make sense of the intervention, focus considerable attention
through the eCDSS. and contexts that could
facilities. The eCDSS guides houn regions, with the three districts what Pawson and Tilley called “folk to describing the specific
facilitate action in the desired
HCWs through the IMCI where Tdh piloted the REC excluded theories” (Pawson and Tilley 1997). characteristics of contexts and
direction.
protocol during under-five from the evaluation. Through individual interviews and mechanisms during the evaluation.
consultations, from the clinical focus groups discussions, the key The task for the researcher is to
For Pawson and Tilley, the context
assessment of the child, elements of the problem and the distil the key potential mechanisms
embraces a wide variety of elements
through the classification, intervention, the expected outcomes and contexts and examine how
such as: ‘Programs are always
prescription, referral and and potential moderating factors are they interact in practice. It is also
introduced into pre-existing social
counselling (Deflaux, 2010, to be identified (Donaldson 2007). important to note that the generation
contexts and…these prevailing social
Yameogo et al., 2011, Deflaux et Additional information was derived of particular ‘change mechanisms’
conditions are of crucial importance
al., 2014). from programme documents. A can fulfil different functions in
when it comes to explaining the
◗◗ A 6-day training course on literature review identified studies different evaluation contexts. For
successes and failure of social
IMCI guidelines, including 2 reporting causal chains, moderating example, realistic evaluation can
programs. By social contexts, we
days on the use of the eCDSS, factors and unintended outcomes, be used in circumstances where
do not refer simply to the spatial of
provided to HCWs. allowing a plausibility check of it is possible to establish more
geographical or institutional location
◗◗ A quality assurance coaching the preliminary MRT. The result control over the context. These may
into which programs are embedded.
system involving team is then again discussed with the be circumstances where there is
So whilst indeed programs are
meetings two to four times a stakeholders and results in the MRT a considerable body of evidence
initiated in prisons, hospitals, schools,
year through which districts that will be tested. about the performance of different
neighbourhoods, and car parks, it is
and their primary health interventions and what is sought is
Regarding design and research the prior set of social rules, norms,
facilities were encouraged to the opportunity to test the workings
methods, realist evaluation is clues and interrelationships gathered
find appropriate solutions to of those mechanisms within pre-
neutral (Pawson and Tilley 1997): in these places, which sets limits on
improve the functioning of specified and selected contexts.
the hypotheses as expressed by the the efficacy of program mechanisms’.
health facilities and the quality However, in other circumstances,
of health care. MRT are guiding the choice of data there may be relatively little evidence
Mechanisms in their turn are not
◗◗ A supervision system including to be collected and the methods about how a particular intervention
merely interventions or actions but
a monthly supervisory visit to and tools to be used. Most theory- will operate because it is new and
theories: ‘which spell(s) out the
Figure 1: Evaluation Mixed-methods Study Framework driven evaluations in healthcare use its use is not restricted to contexts
primary health care facilities potential of human resources and
the case study design and combine defined by the researcher. In these
reasoning’ and can be translated
quantitative and qualitative methods. circumstances, the task for the
into a programme of evidence-based
10 | London School of Hygene & Tropical Medicine 11

researcher is to first theorise possible Formulation of our MRT in the analysis the a limited number
C+M=O configurations and to explore of key interventions of IeDA, select-
the ways in which real-life experience We formulated our MRT on the basis ed according to the intervention
reflect and differ from these theories. of an explorative study of the pilot theory of change and the opinion of
Here evaluation outcomes are districts where IeDA was first test- main stakeholders: tablets and REC
focused on refined theories of action ed. During that study, interviewees availability, IMCI/REC training, super-
based on understandings gained indicated the importance of the vision and the sequencing of these
from empirical research. characteristics of the innovation as activities. (Bonell, Fletcher et al. 2012,
a driver for use and the importance Moore, Audrey et al. 2015).
Focusing on the realist dimensions of the facility setting as a physical Dose delivered is about identifying
enables us to examine the particular and organisational structure. We also the activities and material most and
significance of ‘mechanism’ and found indications that the perception least successfully delivered at all
‘context’ more closely. Specifically, by communities was a determinant levels.
it directs us to identify the specific factor that influenced health provid- Dose received is defined in the
mechanisms, defined as specific ways ers’ behaviour to adopt and use IeDA. present study as the number of par-

© Tdh-Ollivier-Girard
of introducing IeDA, that might be A second source of inspiration is ticipants to trainings, the number of
activated by the prospect of use of the realist synthesis of 36 peer- healthcare (HCW) workers effectively
REC and the contexts in which these reviewed papers we conducted on supervised and those taking part in
mechanisms might apply. the factors influencing the use of the Quality Improvement activities,
electronic Computer-based Decision and the REC utilisation rates.
The evaluation environment we are
SystemS (eCDSS). It highlighted the Mixed methods: embedded quantitative approaches alone; The mechanisms triggered are
working in is one in which relatively
interrelation between the properties Reach of IeDA is estimated through (iv) MM encourages collaboration determined together with the way in
little is known about ‘what works realistic evaluation
of the innovation itself with the the rate of HCW having been invited between researchers from various which they produce the outcomes
in what circumstances’ in relation
organisational environment. The to trainings, the rate of healthcare The realistic evaluation approach is disciplines; (v) MM is pragmatic as in each of these specific contexts.
to the use of the REC and more
contextual factors that influenced facilities supervised and the average defined by Pawson and Tilley (1997) it opens the possibilities of methods This leads to the definition of a
generally the use of electronic
negatively the use of eCDSS were: technical issues disrupting REC as a series of principles and theories. (Johnson, Burke et al. 2007, Bergman theory detailing which mechanisms
computer-based decision support
financial incentives; competing utilization. To analyse recruitment, However, the authors do not provide 2008, Teddlie and Tashakkori 2011, of the program work in which
systems. Therefore, a core task for
programmes; previous knowledge we gathered information about the any guidance on how to translate Denzin 2012). context to produce which outcomes
the research is to draw on existing
and use of IT; high clinician turnover; actions conducted to create and these principles into research and for whom. An MM approach is
data to theorise what seem to be MM research can have four different
link of eCDSS to an ordering system; maintain adherence and participation methods (Rycroft-Malone, Fontenla appropriate for realistic evaluations.
likely ‘change mechanisms’ and to objectives (Creswell and Plano Clark
and individual patient preferences to IeDA activities and use of REC. et al. 2010). Indeed, they advocate for This evaluation is concerned with
use the empirical study to explore 2007):
for treatment. The complexity, methodological pluralism and the use providing an overall understanding
the presence or otherwise of these
lack of a relative advantage, and of mixed methods. of the nature of the theory-of-change
C+M configurations, to examine 1. triangulation: the objective of
the nature of their interaction and
incompatibility of eCDSS with Context evaluation MM is to generate additional
model and how it actually operates. A
workflow, current practice and beliefs Using more than one research quantitative and qualitative approach
their consequences, both in terms The context evaluation aimed to method can generate a more and complementary evidence
of clinicians was associated with low is required to explore the research
of outcomes but also in terms of document factors external to the accurate analysis of the phenomena on the same topic using
use of eCDSS. Trialability and change questions and deal simultaneously
facilitating greater awareness of intervention that may have acted being studied (Morse 2003). Mixed different methods to better
valence did not influence eCDSS use. with the inductive and deductive
sustainability issues. Our aim is to as moderators of implementation methods (MM) research is defined capture phenomena;
theoretical drives (Marchal, Westhorp
make use of the early ‘demonstration and outcomes (Pawson and Tilley as “the combination of quantitative 2. embedded: one data set
et al. 2013).
projects’ (cases where we know provides the main set of
the REC has been successfully
Implementation evaluation 1997), i.e. as facilitators or barriers to and qualitative approaches that
evidence (e.g. quantitative
IeDA implementation and affecting provide a better understanding Quantitative data and qualitative
used) to help us define the various The implementation evaluation aimed REC use (Moore, Audrey et al. 2015). of research problems than either data) and the second set (e.g. data are collected concurrently:
C+M configurations and test these to document how the implementation In particular, we payed attention approach alone” (Creswell and Plano qualitative data) complements quantitative numerical data is
hypotheses through case studies of IeDA was organised and achieved, to potential unanticipated factors. Clark 2007). The advantages of MM the first one; collected from questionnaires and
carefully selected. and how the intervention was re- Social, political, resources and have been extensively described in 3. explanatory: qualitative clinical observation and qualitative
ceived. This implementation evalua- logistical factors were screened and the literature and be summarized as data contributes to provide data (text data, transcripts and
The emerging findings are compiled explanations to initial results
tion focused on fidelity, dose deliv- added throughout the study. Various follows: (i) MM gives the opportunity memos) from open-ended questions
as conjectural CMOCs that indicate collected with quantitative
ered (completeness), dose received sources of information were used: of mitigating the limitations of included in semi-directed interviews,
how the intervention led to particular methods;
(exposure), reach (participation rate) individual interviews and document both quantitative and qualitative focus group discussions, documents
outcomes in which context and by 4. exploratory: the results of
and recruitment (Saunders, Evans et review (project documents as well as research; (ii) combining quantitative review and observations. In this
which mechanism. Their fit with the the qualitative method help
al. 2005). national policy documents). with qualitative research enables research, qualitative and quantitative
data is checked to ensure internal elaborate the questions and
Fidelity is about comparing what the researcher to study phenomena methods are mixed throughout all
validity. The retained CMOCs are then tools for the quantitative
happened in practice in the four from different perspectives using phases of the project from the design
compared with the MRT, which in method that follows.
districts to what was planned in IeDA different paradigms; (iii) MM helps stage through data collection to data
turn is modified if necessary (Barnes,
project documents (Moore, Audrey answer questions that cannot interpretation.
Matka et al. 2003).
et al. 2015). We decided to prioritise be answered by qualitative or
12 | London School of Hygene & Tropical Medicine 13

Realistic evaluation design and research methods

The sampling procedure was chosen Table 1: Number and profile of individuals interviewed during the realistic
generalisation” (Yin 2003 p. 32). In according to the objectives of the evaluation
Healthcare workers Patients
real-life contexts, qualitative research study: generating theories and
Coges MoH and, more particularly, case study concepts rather than generalising Profile Number
Outcomes findings to a wider population.
methodologies are known to be Healthcare workers 92
appropriate for understanding and Therefore, a purposive rather than a
probabilistic sampling method was Health district officers 16
District Management team

interpreting complex causal links in


Attributes of the Implementation natural setting interventions (Keen deliberately used by the investigator COGES 6
tool and use and Packwood 2000). (Patton 1999, Bowling and Ebrahim Carers 5 Focus group discussions, 9
2005). Purposive sampling is used individual interviews
when researchers Drug stock managers 3
Data collection “seek out groups, settings and Village representatives 2
Various sources of data were used individuals where … the processes Community health workers 6
Impact of the tool
by the investigator during data being studied are most likely to Regional health authority 1
and unintended
collection that took place between occur” (Denzin and Lincoln 1994 p.
consequences Health centre maintenance officer 2
January 2016 and October 2017. The 202).
Tdh Traditional chief 2
combination of several data sources
For the implementation evaluation, MoH Officers 6
proved valuable to the research.
International partners a set of process research questions
Direct observation in health centres TDH 3
were defined prior to the evaluation
generated elements of information Total 154 Interviews + 5 focus group
and according to the IeDA theory of
Figure: Conceptual framework of the IeDA realistic evaluation (Adapted from Rycroft- that helped to identify new issues or discussions
Malone, Fontenla et al. 2008) change (Oakley, Strange et al. 2006).
verify assumptions. The analysis of
Most of the information collected was
project reports from health facilities the intervention in terms of content In terms of implementation
triangulated from different sources.
Conceptual framework Study design helped analyse the implementation
This was particularly important as the
and application, and the intended evaluation, we analysed the concepts
of IeDA and the vision of the project and actual outcomes. We drew on of fidelity (How much has IeDA been
The study’s theoretical framework In order to capture social phenomena data were collected retrospectively
by managers. In addition, interviews our interviews, observations and delivered as intended? What has
integrates various components, such as management decisions and hence subject to recall biases.
and focus group discussions document analysis to differentiate not been delivered?), dose delivered
including the four areas that play a interactions between individuals, A combination of electronically
provided evidence in relation to the the vision (what the team wants), (What parts of IeDA were delivered
role in protocol-based care in general an in-depth qualitative research documents review and in-depth
perceptions, in-depth opinions and the discourse (what they say) and most and least successfully to DHMT,
and related impact on stakeholder approach was adopted. According interviews were conducted in 2017.
understandings of actors intervening the actual practice (what they do). primary healthcare facilities and
outcomes: patients, health staff, to Fitzpatrick and Boulton (1994 p. in IeDA. We described the organisational healthcare workers?), dose received
service providers and policy makers: 107), qualitative research “is used climate perceived by staff in terms of (Which content and activities of IeDA
where it is important to understand In-depth interviews were conducted Data analysis
procedures, structures and incentives were attended best by participants?),
1. What are the properties of the the meaning and interpretation with 154 individuals including 92
The initial coding was based on a (Schneider, Gunnarson et al. 2004). In reach (What proportion of the
REC tool?  of human social arrangements healthcare workers from health
preliminary list of codes inspired order to indicate how the intervention intended healthcare workers was
2. How is the approach such as hospitals, clinics, forms of centres, 16 officers from district
by the MRT and on additional ideas worked, we analysed both the context effectively exposed to IeDA?) and
implemented and the tool management or decision making”. health authorities, 6 members
that emerged from the fieldwork. In and the intervening mechanisms and recruitment (What was conducted
used? Qualitative research aims to generate of health centre management
a second round of analysis, some attempted to identify the essential to encourage adherence to IeDA
3. What is the impact of the concepts and theories that can be committees. In addition, 5 focus
themes and patterns emerged. In conditions. activities and REC use?).
approach and its unintended generalisable (Green 2006) - what groups (on average 11 people per
consequences? order to structure them as CMO
Yin called “analytic generalisation” group) were organised with mothers
configurations, we found it useful to
in opposition to “statistical and carers.
borrow categories from theory-driven
evaluation (Chen 1990). We described
© Tdh-Ollivier-Girard
14 | London School of Hygene & Tropical Medicine 15

Results

The management vision Table 2: Number of healthcare workers trained on IMCI/REC in four districts

  Toma Solenzo Titao Ouahigouya All


We analysed the views of the aspect of IeDA that is central to them: The original activities identified
implementation team members at the REC gives them access to patient by the implementation evaluation HCW trained (N) 32 44 31 26 133
MoH and Tdh to understand their medical history: are as follows: (i) Delivery of IMCI
respective roles, the implementation and REC training; (ii) Provision of HCW, total 63 46 49 50 208
process and the effect they aim to “If a child comes for consultation, we tablets and solar kits; (iii) IMCI/REC HCW trained (%) 51% 96% 63% 52% 64%
achieve. The vision is well shared type his/her name and the name of supervision session at facility level; (iv)
between senior MoPH officers and his village, and we retrieve his history Organisation of Quality Improvement State enrolled nurses trained (N) 9 9 6 5 29
Tdh managers: the introduction of in a few seconds. This is impossible sessions; and (v) Use of REC during State enrolled nurses, total 10 11 6 6 33
the REC aims to support the scaling with the manual registry” (nurse, child consultations.
up of IMCI in Burkina Faso with the health centre) State enrolled nurses trained (%) 90% 82% 100% 83% 88%
ultimate goal of improving adherence Between May 2015 and December
Assistant degree nurses trained (N) 9 11 9 6 35
to IMCI protocol and quality of child 2017, the intervention evolved.
health services at primary health care The actual intervention Several activities and tools were Assistant degree nurses, total 10 10 8 11 39
level. But this vision has evolved over added during the year 2017 primarily
Based on the analysis of interviews Assistant degree nurses trained (%) 90% 110% 113% 55% 90%
time with the implementation of IeDA to improve knowledge and data use
to encompass new dimensions. As and project documents, we found for management and clinical care: (i) Outreach health workers trained (N) 10 13 10 11 44
well as improving adherence to IMCI that the list of activities that really development of dashboards at health
constitute what the intervention Outreach health workers, total 19 10 13 13 55
protocol at health centre level, the centre level; (ii) supply of a second
vision of the project also included is about goes beyond the original tablet to larger health centres; and Outreach health workers trained (%) 53% 130% 77% 85% 80%
aspect related to centralisation and vision. The project has been very (iii) online learning modules on IMCI.
dynamic experiencing several stages Midwives trained (N) 1 7 6 1 15
availability of patient data to decision The REC itself evolved several times
makers for remote monitoring, of changes mainly guided by the during the project period experiencing Midwives, total 7 7 6 5 25
targeted supervisions and tailored feedback received from the users. Tdh several software improvements on
has put in place dialog mechanisms Midwives trained (%) 14% 100% 100% 20% 60%
trainings. As managers describe the tablet and the backend of the tool
it, it also fills a gap in the health with healthcare staff in order to (data analysis) resulting in several Auxiliary midwives trained (N) 2 1 0 3 6
information system in Burkina Faso ensure that the evolution of the tool consecutive versions of the REC.
and project take into account users’ Auxiliary midwives, total 17 8 16 15 56
and aims to influence the way health
centres are managed. Data is put feedback. Auxiliary midwives trained (%) 12% 13% 0% 20% 11%
at the centre of the management
processes and made available from Intensity of implementation was extended to other cadres as Following the launch of IeDA in each
2017 to district managers and health the implementers realised that, on district, supervisors from the district
We analysed the actual the ground, other cadres take over health management team (DHMT)
centre managers in order to guide their
implementation of IeDA using the for consultations when nurses are were assigned to conduct IMCI/
decisions and tailor their supervision
framework described in the method not available at the health centre REC supervision activities. It was
and coaching visits.
section. Each activity was analysed (on holiday, sick leave or training). reported that after a few months it
At the health centre level, there is clear in terms of implementation. Between The innovation is that the project would usually be noticed that those
consensus with management level March 2015 and December 2017, all the managers did not hesitate to negotiate DHMT members did not have enough
on the IMCI adherence aspect of the health centres in the 17 districts were with MoH an exception measure for time to deliver IMCI/REC supervision
project. supplied with one tablet and one solar the project to adjust the national activities. Hence, it was eventually
kit. In total, 608 health centres were policy on IMCI, based on information decided that DHMTs had to assign
“When we use the REC, we have to covered. Regarding the provision of from health centres on whom is other supervisors, for instance HCW
follow each single step of IMCI. This a second tablet, only 25 tablets were involved in conducting consultations. from health centres.
means that we scan all the health distributed as a second tablet with As a result, the project managers
problems of a child. This requires that only one supplied during the year 2017. In terms of Quality Improvement
included in the initial training
we ask all the IMCI questions and (QI), at district level, representatives
In terms of training, it was originally midwives, midwives assistants and
help us have a global diagnosis. This from all health centres were invited
planned to train all HCW on IMCI/ even outreach workers.
is a better management of the child” to attend QI sessions two to four
(Nurse in health centre). REC. by the end of 2017, Tdh and In terms of supervision, 90% of times a year. During these sessions,
MoH managed to train 88% of the supervision was conducted as performances of health centres
However, the vision of management nurses (both State and Certificate), planned every month. All stakeholders regarding REC use and IMCI
on centralisation of data for better their primary target as they are the reported that recruiting and retaining adherence were presented and
management is not really as being ones who are supposed to conduct supervisors who had enough time to discussed.
part of the vision of health centre staff child consultations and use IMCI, dedicate to supervision activities was
(nurses and nurse assistants). They according to national MoH policies. highly challenging in most districts.
rather mentioned another data-related Over the years, the training strategy
16 | London School of Hygene & Tropical Medicine 17

The political and policy context

The political context district managers confirmed that


the rate is constant every year. This
of the adopters or the adopting
system and does not require
means that every 12 months, around significant modifications from
A Coup d’Etat occurred in September were paused for five weeks, which IeDA project would be implemented, 40% of the nurses or midwives move the adopters (Aubert and
2015 and caused a few interruptions primarily impacted the supervisions as the whole government was to another facility (most of the time Hamel 2001, Denis, Hebert et al.
in the implementation of IeDA and the training of healthcare workers. changed. outside the district). 2002);
activities. At that time, IeDA had been It also created uncertainties amongst 3. complexity - the perceived
deployed in two new districts, Toma project managers and health district The innovation attributes difficulty in understanding
and Solenzo districts. All activities managers on the future and how the Health systems are viewed as a new idea or using a new
complex systems (Institute of technology. A complex
Medicine 2001, Plsek and Greenhalgh innovation can also be an
The policy context 2001). Complex systems are systems
with a high number of elements
intervention which involves a
high number of actors (Grilli
or actors that interact with each and Lomas 1994, Denis, Hebert
other in ways that are not always
Free healthcare policy (UHC) for which willingness of the and the systematic use of REC, which et al. 2002);
predictable following the introduction
initiative Government had been officially might have been the case in Toma 4. triability - the notion that an
formulated in September 2015 with district. The introduction of the new of an innovation (e.g. a new health innovation can be tested on a
During the course of the project, a a Law establishing a compulsory policy coincided by the start of the Staff turnover intervention) (Borgatti, Everett et al. small scale (Yetton, Sharma et
new policy emerged, which would Universal Health Insurance (Conseil malaria season. According to nurses IeDA was implemented in rural remote 1990, Rihani 2002). Introducing an al. 1999); and
potentially directly affect the National de la Transition 2016). interviewed, the period of adaptation areas, where healthcare workers innovation into a complex adaptive 5. observability - the degree
utilisation of health services at health Following the implementation of this to the new workload lasted around 7 usually do not want to spend more system (e.g. a health centre) can to which the results of the
centre level. A free healthcare policy policy, the number of consultations months as soon as the malaria season than a few years and where staff produce extensive changes in innovation are visible (Grilli and
for children under 5 was nationally in health centres increased as shown ended and hey had time to reorganise turnover was anecdotally said to be various socio-technical aspects Lomas 1994, Rogers 1995).
launched in April 2016 (Ministere in Figure 3.6. It is likely to have had their services. In other words, the high. For example, in Titao district, it of the system, including tasks of
2016) (Gouvernement Burkinabe a negative impact on the workload utilisation of REC was not deeply and was reported that up to 95% of newly individuals, relationships between The IeDA intervention and more
2016). This decree was one step of health care staff in health centres durably impacted by the new policy. transferred staff were healthcare actors or management mechanisms specifically the technological
towards Universal Health Coverage workers coming straight from nursing (Greenhalgh 2008, Blanchet 2013). innovation, the REC, the CDSS
school with no primary experience. An innovation is defined in this paper provided to nurses at the start of the
During interviews, district managers as “an idea, practice or object that project on small laptops and later
estimated that newly arrived staff is perceived as new by an individual on tablets was analysed in relation
worked during an average of three or other unit of adoption” (Rogers to Rogers’ attributes: comparative
years in the district before asking to 1995). The process generated by advantage, compatibility, complexity,
be transferred to another district. Staff the introduction of an innovation is triability and observability.
turnover was also seen as a challenge described by Rogers (1995) as an
for Tdh who worried about training “innovation-decision process”.
staff and sustaining the utilisation of Comparative advantage
Diffusion of innovation theory can
REC in each health centre. help explain how the continuation of In terms of comparative advantage,
In July 2017, all health care workers activities is related to the attributes the REC was compared by healthcare
working in the four districts where of activities as innovations. Beyond workers to the previous situation
IeDA had been implemented, 31% the description of an innovation as a where only paper-based version of
of healthcare workers (62 out of newness, Rogers (1995) showed that the IMCI was available. We know
198) who were asked to use the REC innovations are characterised by five from the trial that IMCI paper-form
had not benefited from the IMCI/ attributes: was used for 68% (916/1,343) of
REC training. This was exclusively the consultations in the control arm
1. relative advantage - individuals respectively, while the REC was used
explained by staff turnover: nurses assess innovations by
who had been trained by the IeDA in nearly all consultations (97%,
comparing the expected 674/694) The healthcare workers
project had been transferred to other advantage of the new initiative
districts and replaced by staff who highlighted the advantages of the
with the benefits provided REC, which is described as a tool
had not received the initial IMCI/ by the previous one that it
REC training. To triangulate the covering several functions. The REC
replaced; has been well accepted by healthcare
information, all 40 health centres were 2. compatibility - an innovation is
surveyed to understand staff turnover. workers and has become a routine
perceived as compatible when tool in their practice to the point that
It was found that 36% of nurses had the new idea or technology
been changed within the last 12 years, healthcare workers contribute to
introduced by the innovation the maintenance of the tool, regular
period of time corresponding to the is consistent with the mandate
Figure 2: Total number of consultations by district between May 2015 and June 2017 first IMCI/REC training. Anecdotally, synchronisation and do not hesitate
18 | London School of Hygene & Tropical Medicine 19

sometimes to use their own money to the single symptom expressed by the des DOnnées Sanitaires; District centre but we needed to move around was that the REC would increase
cover internet costs. carer. Health Information System 2 (DHIS2), furniture.” (Healthcare worker) the physical distance between the
which will be a critical key to scaling patient and the healthcare worker. In
The REC is, above all, an eCDSS “The REC pushes us to be more “At the start, we through that the REC
up. fact, we observed in several centres
tool that guides healthcare workers scrupulous. So it takes more time was asking for drugs that we do not that one agent moves away from his/
in their clinical decisions and help and mothers complain about it”. In the eyes of the community, the have in stock. We then realised that her desk to sit down next to the child
them respect the recommended IMCI (Healthcare worker) REC has generated indirect benefits. these drugs were part of the essential in order to consult the child and ask
protocol. Step-by-step decisions the Tdh provided solar kits to ensure list of medicine. We had to order questions to the carer. The healthcare
The REC is described by many inter-
clinicians need to make throughout autonomous supply of energy for the them.” (Drug stock manager) workers noticed the satisfaction of
viewees as a living entity with its own
the course of the consultation are running of the tablet. The community the community in this new approach
autonomy and decision power. As a In terms of team organisation, health
guided by the software that forces made direct link between the and feel a gain of trust from the
result, the “machine”, brings its own staff realised that the use of REC was
the consultant to follow each step introduction of the REC with supply of community. The acceptance of the
independent opinion on the top of the easier with several health agents
of the protocol in order to be able electricity at the health centre, which REC by the community cannot be
healthcare worker’s opinion. involved with an efficient use of
complete the consultation. The brings security around the health better assessed when it is missing:
software developed for the project centre during the night. several personnel. For example, one
“It is the REC that help quickly find when the REC is not functioning
uses conditional logic where the agent stays in the waiting room and
the right products that are needed or out of battery, the community
following questions and steps depend “This tool is great since it was take basic measures (weight and
to treat my child when he is sick.” notice it, ask for explanations and
on the answers previously registered brought. Power is not in the village size). When possible, two agents
(Mother) demand the use of the REC during
by the healthcare worker and no skip but light and the machine are here. manage the consultation as a team.
consultation time.
option where every variable needs to The night guard can now sit down in One person close to the child
“The machine gives more information
be recorded to enable the healthcare advantage of having access to the the light and even if a patient comes and a second person guiding the The strict adherence to the
than the nurse”. (Father)
worker to progress to the next step. medical history of patients. Access during the night, there is light. It is consultation with the tablet through IMCI protocol also means that
The district officers as well as the In a sense, the presence of the REC to the medical history of the child is great.” (healthcare worker) each step of the IMCI protocol and the prescription of drugs is not
healthcare workers recognised that is reassuring for the community as it probably the most visible function recording data on the tablet. In other systematic, which contrasts with
the tool is well designed and enables is a way to guarantee and triangulate from the perspective of the carer. health centres, usually large health current practice in Burkina Faso
the healthcare workers to directly the diagnosis provided by a nurse. To Compatibility centres, the consultation of children is where each patient expects to receive
go further, it is as if the community “Once they [the nurses] type the conducted in one room in parallel to
have access to the protocol without a prescription of drugs. This change
name of the child, they can see a lot In terms of compatibility, we consultations for adults in a second
searching for the right information. had more trust in the REC viewed as of practice puts a lot of pressure on
of information as they already can tell investigated the compatibility with room. We observed that the outreach
generating a non-biased opinion: healthcare workers and requires from
the age”. (Mother) the infrastructure, the use of IMCI, health agents were mobilised for
“When using the REC, we have them perseverance and conviction.
“To me it is like a machine. It is a the health team and the relationship consultations, often in charge of
to follow each step [of the IMCI “When the child is here, you click
computer. This will diminish the patient/clinician. the triage and measuring weight “The person expects to be prescribed
protocol], which means we are to here to see past treatments. You can
screen all the potential problems errors. When I see some work done and size of the children outside the drugs like in the old times. This was
see when he came and what reason. In terms of infrastructure, the REC,
of the child, even the ones not with a machine, I have no fear. I consultation room. We observed routine practice during consultations.
With the registry, it is very difficult. which consists of a tablet, does not
mentioned by the carer. The REC respect this work.” (Father) several times the involvement of People are used to drugs. For people
And we change registry all the time create any specific challenge for the
pushes us to ask the right questions.” one member of the health centre who are illiterate, you explain but they
“In the REC, there is no lie”. (Father) as soon as the pages are finished. health centres, whatever their size.
(healthcare worker) management committee when staff will go to another facility to ask for
But here, even one year later, you see More than being seen as an obstacle,
are overstretched. drugs. It is about trust between us
“I think that it is in the REC that the everything.” (Healthcare worker) the REC actually highlights in each
“If you directly register the child in and the patient.” (Healthcare worker)
healthcare workers see the diseases facility the minimum equipment There are situations when the use
the REC, it [the REC] provides the Another important function of the
of the child. Every time you come and drugs required to run child of REC was challenged by the
classification, the medicine you REC is the centralisation and sharing
back to the health centre, they find consultations according to IMCI population: when the agent was on
need to prescribe, even the dose. So
the information inside the machine”. of data (including monthly reports). standards. The introduction of
Complexity
no need anymore to search in the his/her own and during the malaria
(Father) The patient registry is saved on the the REC systematically generates
documents [i.e. IMCI paper protocol]. season. Complexity of REC was one of the
tablet, saved on a cloud and shared amongst the health team an
So to me, it is much easier like this: main concerns from the national
Another advantage of the REC is with district and national authorities. inventory of equipment missing or not “If I take months such as September-
you ask questions, record the answer policy makers considering the limited
the capacity to generate a patient functioning and the list of essential October-November, when the waiting
and this is finished. You get the “At the end of each week, data are level of computer literacy of their staff
registry and even the medical history medicine. For example, in many health room is full of patients, people are
treatment and the prescription. Huge sent to the district – very quickly at primary health care level. It appears
of the child. The information from the centres, after the IMCI training and vomiting, people are on the floor with
advantage!” (healthcare worker) – from the tablets without leaving that the use of the tool is perceived
previous consultation are recorded in the introduction of REC, we observed fever, it is very challenging when staff
the health centre. We can say that as being easy to understand after
the patient file. The paper-based filing the creation of oral rehydration is limited. The population would insult
The community really perceived a what we save is time.” (Townhall initial training. We have also observed
previously used could not make this therapy (ORT) corners with plastic us if we are slow.” (healthcare worker)
change in the way consultations employee) that new comers in a health centre
task possible. Thanks to the patient containers and oral rehydration
are delivered. The fact that the IMCI are immediately trained by their peers
history function, the healthcare solution (ORS). In terms of patient/clinician
protocol is followed step-by-step The negotiations between the MoH on how to conduct consultation
worker can refine his/her consultation relationship, the REC introduced a
takes more time than only focusing on and Tdh focus on adapting the REC with the REC and use the tablet.
and ask further questions to the carer. “IMCI requires a consultation room new way of interacting with patients.
database to the national health All healthcare workers trained on
This function is well appreciated by dedicated to child consultations, One concern at the start of the project
database, ENDOS (Entrepôt National REC recognise the importance of
the community who understands the which was possible in our health
20 | London School of Hygene & Tropical Medicine 21

coaching following the training and worker complaining about the For the district health team, the “For example, in terms of respiratory practice and behaviour change. A services provided. Even most Heads
after practising in order to be able to innovation was not excluded from introduction of the REC does not infections, to check whether a child real support system was put in place of the health centres have a sense
ask questions, understand some of the intervention but on the contrary, only mean the introduction of a new has a stridor, you can click on the engaging each level of the health that it is their responsibility to monitor
the troubleshooting methods when their concerns were embraced by the technology but is really rather seen as REC to watch a short video with a system in the implementation and the quality of the consultations
the software or tablet has issues and coaches in order to build their skills the scaling up of the IMCI strategy. specific case of stridor. The REC promotion of quality of care practices. performed by their team. This
verify they are doing the correct tasks. and later n their confidence. provides a few more extra details on This required the involvement of a integration of the governance system
“We have to say that before we what information we need to check wide range of actors ranging from to the lowest level of the health
The feedback loops established One limitation of the current version introduced the REC, even if some IMCI to confirm a stridor. They are plenty of national actors from all levels and system is a real achievement and
by Tdh to understand users’ of REC is the absence of correction training took place at some point, details provided.” (Healthcare worker) sectors of the health system (different contribution towards the routinisation
perspectives is also well valued as function. The information cannot be there were only two health centres departments at MoH including family of the REC at the primary health care
after each software version healthcare changed as soon as the section is [out of 28] that were using IMCI and Another new management practice medicine, statistics and information) system of Burkina Faso.
workers can see the improvements validated. This lack of flexibility makes within the health centre, only nurse introduced by IeDA is team work. and international donors and United
made compared to the previous in typing mistakes very unforgiven and had been trained in IMCI and tried to The utilisation of the REC is more Nations agencies as well non- “I keep an eye on the ones who use
order to facilitate their work. The may lead to incorrect diagnosis. used it during consultations. So you efficient when two staff are involved governmental organisations and civil REC and when I go around, I select
healthcare workers really understand can imagine the proportion of children to conduct consultations. In several society organisations and individuals the patients already consulted and
that they are the key players in this consulted with IMCI was quite low. health centres, we even observed (opinion leaders, religious leaders). check whether their name is recorded
project and that their voice and views Observability After the introduction of REC, which that peer support was organised in Many of these actors were involved or not. If they are not registered, I
are recorded and analysed to improve was preceded by an IMCI training order to enhance individuals practice and engaged at each stage of the call the agent and explain that all
the usability of the tool. In terms of observability (i.e. the for all agents, I can say that now all and knowledge. Peer support was the patients need to be registered.”
project to share views on the next
possibility for the users to perceive CSPS use IMCI during consultations suggested by the Quality Assurance (Head of health centre)
steps of the implementation and
The utilisation of REC becomes visible benefits), interviewees and more than one agent per health activities and coaches as an approach scaling-up of IeDA. The recognition
very complex when the system listed quite a few aspects. First, the centre.” (District Officer) to enhance quality. “The person who leads the
of everybody’s voice created an
breakdowns. It happened that healthcare workers realised that the consultation has to provide his
atmosphere of mutual support and
in the middle of the consultation, use of REC lead to a more rational “It is really a positive change because The IeDA is seen by the health personal details, which helps identify
trust within health centres and
the software froze or the system prescription of medicine and with the REC it is really the IMCI district officers as the introduction who is in charge of the consultation,
between health centre staff and
shut down deleting all information reduced over-prescription, which is strategy that is rolled out, which of a quality of care approach at so we know the proportion of
district health teams. There is general
registered during the consultation. usually the result from community means that we introduce the IMCI primary health care level. The IeDA consultations performed by nurses, as
perception that the IeDA has been
We also observed that in some pressure. The presence of the tablet form during every consultation.” project through higher adherence they are the ones who supposed to do
implemented with genuine will from
health centres, nurses were using the provides vis-à-vis the community (District Officer) to IMCI promotes a comprehensive it. And when there are problems, we
health staff to make it work.
paper registry as they had serious arguments and a rationale for the assessment of the child health, can identify which person has difficult
issues with the battery of the tablet. healthcare worker for not prescribing The REC is also a dynamic tool, which moving away from a classification conduct correct consultations.”
“The culture of performance and
The point here is to highlight that drugs when not necessary (for evolves with the policies and can based on the most visible and (District Officer)
quality needs to start from the
the introduction has become so example when the child has a simple support the dissemination of new apparent symptoms. institutional level. We need to be
much part of routine practice that cough). policies at reduced cost. For example,
able to support the institutional level,
its absence due to a breakdown is during the course of the project in “With IeDA, it helps screen more
which means the national, district and
noticed by the healthcare workers On the other hand, the healthcare 2016, a revised version of the national comprehensively the health
health centre levels.” (Tdh)
and disrupts the organisation of worker through the use of REC IMCI protocol was introduced conditions of the child. It helps
consultations. has a better understanding of and by MoH. The protocol was then change the practice of health agents. The behaviour of health workers
adherence to the IMCI protocol as supposed to be rolled out by the MoH, Sometimes, there are consultations is also influenced by the new
skipping steps are impossible with which requires dissemination of the when, maybe. We don’t take enough accountability system introduced
Testability REC. The healthcare workers have document and ideally refreshers for all time to consult the child or ask all the de facto by the REC. Indeed, every
the feeling that they really follow the health staff. With the REC, a revision right questions to the mum. But with healthcare worker needs to log in
In terms of testability, we observed in IMCI protocol as they should do. As a of the protocol in the software and that [REC], as everything is indicated, every time they use the REC. The
a few health centres some resistance result, healthcare workers feel more the upload of the revised protocol we have to follow each step and this information officer at the district level
from heads of facilities. It was mainly confident in their own classification on each tablet were the only tasks contributes to improve the behaviour can easily retrieve this information in
due to the lack of self-confidence and prescription. necessary to a full roll out of the of healthcare workers. So this is a case of problem. This is a significant
and literacy on using tablets and revised protocol. great advantage for us.” (District change in the Burkinabe public
softwares. The coaching played a “Without the REC, there are many Officer) service culture as for the first time this
key role in accompanying individuals questions we used to forget. But here, From the perspective of the
information can be used to identify
who had some reluctance in using all the questions are listed and you healthcare workers, nurses or nurse “When you see on the walls the ideas
malpractice (if needed).
REC and building their confidence. cannot skip any of them. So to me, I assistants, the REC also represents of changes and the solutions. You
Coaches did not consider resistance think that we better manage patients. a tool supporting continuous can see a weekly programme and the The high level of commitment from
as an exceptional event but rather For example, when a child comes development through the eLearning indicators displayed.” (Head of health a wide range of actors generated
assumed that resistance was the with a simple malaria, you can without tools. Indeed, in 2017 were introduced district) more legitimacy for the project and
norm. As a result, any healthcare the REC forget to identify anaemia.” online training modules with short created a devolution of powers and
(Healthcare worker) demonstration videos. The quality of care approach
responsibilities within the health
promoted by MoH and Tdh goes
system to monitor the quality of the
beyond the improvement of individual
22 | London School of Hygene & Tropical Medicine 23

Analysis

Table: the three CMO configurations related to IeDA


After categorising the primary data in the form of CMO configurations, a realist evaluation seeks
to examine the link between these findings and the middle range theory it set out to examine. In Context Mechanism Outcome (students)
practice, we searched the causal pathways between the adoption and utilisation of the REC and C1. Availability of a support team M1. Promoting amongst healthcare workers O1. Notions of quality in
prescription and the various management practices and dynamics in health centres. to be responsive to healthcare “doing the right thing the right way” childhood illnesses routinised
staff questions. approaches during consultations
A summary of the find concrete solutions; creation of different cadres in the project is C2. In health centres where the M2. Clear distribution of roles before and O2. Efficient organisation of
nurse is assisted by at least during child consultations (including triage, the health team
intervention and its of eLearning modules on IMCI based on correct assessment of the
two other members (nurse weight and size measurements, consultation
outcomes available on tablets for continuous context in rural Burkina Faso. IMCI is
assistants or outreach workers) and counselling)
knowledge development; an inclusive supposed to be exclusively used by and where management
Our interviews and document review and team approach associating in nurses, according to MoH policies. flexibility is allowed
showed that the MoH and Tdh the implementation process not However, the implementation teams
C3. Strong consensus amongst M3. Introducing at primary health care level O3. Sustained use of REC as a
managers defined good performance only nurses but also any other staff realised at the very early stage of the stakeholders on the benefits of the notion of individual accountability and routine practice
as health teams who systematically who is directly or indirectly involved implementation that a nurse (often introducing REC responsibility and collective contribution to
used the REC, had a good score in in managing child consultations; the unique nurse in a health centre) the wider system.
correct classification and prescription and good accessibility of top Tdh has other responsibilities outside
and were able to improve over time managers to district health teams the health centres (e.g. training,
through self-learning. In contrast (?), high in rural areas of Burkina Faso prescription. This also concerns include initial IMCI/REC courses,
and in a certain measure to Heads meeting days at district health
district health teams defined good making challenging the progressive the shift from output indicators to peer pressure/support mechanisms
of health centres anytime support is bureau) and cannot permanently
performing health centres as health development of nurses’ skills on quality and outcome indicators, and personnel development
needed. be physically present at the health
centres who were systematically IMCI. Finally, the capacity of district which implies an organisational opportunities through eLearning
centre to conduct child consultations.
using the REC, experienced limited Regarding the process of health teams to conduct supervisions culture change within MoH staff. So modules. Availability of a support
As a result, the nurse delegates the
service interruption due to system implementation, we noted a good is quite limited due to their budget many practice changes expected by team to be responsive to healthcare
responsibility of child consultations to
breakdown or drug stockout and coherence between the initial restrictions and limited access to the implementers can be achieved staff questions and needs is an
other cadres who may be, depending
were able to report their results theory of change developed by vehicles – volume of resources that through the introduction of innovative important context element and make
on health centres, a midwife, a nurse
in time. The district health teams, the implementers (MoH and Tdh), will not increase in the near future. management approaches. The possible the combination of all these
assistant or a midwife assistant.
compared to MoH and Tdh managers, the project management team’s project management team set up management processes.
As a result of this assessment, the
had less emphasis on quality of vision and district health teams an initial training programme, which
project management team decided The second CMO configuration
care but rather concentrated on and health centres team’s practice. to extend the inclusion in project CMO configurations guides every health staff through
the IMCI guidelines emphasizing can be summarised as follows:
the activities directly related to the Indeed, in line with their vision, activities (including the initial IMCI/ During the later phases of the on the importance of good practice a health centre team where the
district health teams’ responsibilities the project management team REC training) to all cadres working in analysis, we found that the adoption and quality of care. In addition to the nurse is assisted by at least two
for which they are accountable to the motivates the health centre staff health centres. process can be grouped according initial training, regular supervision other members (nurse assistants or
Regional Directorate and MoH. involved in child consultations
to their key mechanism and this led was put in place to complement initial outreach workers) and management
(e.g. nurses, midwives and nurse Realistic evaluation improves external
The actual activities of the to the description of parallel CMO training with in service-supervision. flexibility is permitted (C2) can be
assistant) through effective and validity of a case study by describing
configurations, each with their own This was accompanied by quality better organised and efficient (O2)
intervention can be summarised regular support, dialog to identify the implementation context.
as follows: (i) Development and outcome. assurance sessions where staff in when the roles of each member are
their problems and needs in using During the study, we found several
implementation of improved versions health centre were asked to find well distributed before and during
the REC and full recognition of their potentially important elements in The first CMO can be summarised
of the REC; (ii) Provision of a six-day solutions as a team. There was child consultations (M2). Key
role in improving child consultations. the context of primary health care in as: with the support of a support
training course on IMCI guidelines also much attention for a clear role practices in this set include creating
All these practices conducted by Burkina Faso in the two regions of team responsive to healthcare staff
and REC; (iii) Development of a distribution. The notion of teamwork open discussions and dialog between
the project management team are Boucle du Mouhoun and Nord. First, questions and needs (C1), promotion
quality assurance mechanism; (iv) was emphasized by the project all health team members on how the
applied to all health centres (good all health centres in the two regions amongst healthcare workers of
Monthly supervision of every health management team recognising consultation should be organised
coverage) without any distinction were staffed by at least one nurse “doing the right thing the right way”
centre benefiting by the district the value and role of each member, considering the introduction of
of location and health cadres (good (depending on health centre’s size) approaches (M1) in order to routinise
health team; and (v) Development whatever the title and background. a new tool, the electronic tablet,
external fit) and these practices who were all aware that management notions of quality in childhood
of a health information system. We In summary, both “hard” and “soft” and quality assurance session.
are reinforcing each other (good of childhood illnesses is an important illnesses during consultations (O1).
also found important additional management mechanisms were In order to be more efficient, a
internal fit). Concerning the latter, priority at primary health care, which The project is trying to influence
activities that organically appeared used to influence the organisational triage is conducted in the waiting
this was achieved by Tdh and was emphasized by the introduction practice of health care workers by
during the implementation of the culture. The former includes task room by a nurse assistant or an
MoH by regular assessment of the of a national free healthcare policy on moving away from “simply doing”.
intervention: the creation of a support distribution between health care outreach agent who identifies the
situation and identification of models child health care during the course The awareness from MoH and Tdh
system to respond to breakdowns staff by task - pre- (e.g. triage, children in critical condition and
of good practice at health centre of the project. Second, the initial that reducing the child mortality with
and questions on the software and child measurements), during (e.g. take child measurement (e.g. size).
and district levels, identification training of healthcare workers on the same level of resources from
the tablet; district meetings at least consultation and prescription), and This reinforces open relationships
of constraining factors to REC use IMCI needed to be complemented government can only be achieved
once a year to enable district health post- (e.g. counselling) consultation between health centre staff and
and dissemination of good practice by regular supervisions and by improving quality of care, which
teams and heads of health centres tasks - and between clinical and contributes to solving practical
through local implementation coaching after initial training. Third, in the context of the project relates
to discuss about performance and administrative activities. The latter problems and build solidarity
teams. For example, the inclusion staff turnover of nurses is very to correctness of classification and between staff members. The quality
24 | London School of Hygene & Tropical Medicine 25

Discussion and Conclusion

assurance sessions are built around and no disruption of availability of (including community and policy This study offers interesting insights on how the introduction of one computer-based decision
specific concrete issues experienced the tablet are considered as essential makers support), which can be
by the health centre team and context elements in the three
support tool combined with management support practices created new work practices.
translated by peer support and
elaborate realist solutions and action configurations. district authorities support, and
points, which achievement depends availability of support services Lessons for policy and to a stronger organisational culture motivation. Assessing the exact
on how the members will work as responding to software or practice (Granovetter 1973, Eisenberger, contribution of a set of management
a team. In turn, it stimulates the The new MRT hardware issues. The supportive Hutchinson et al. 1986). Finally, a practices to overall performance
feeling of perceived organisational environment is based on reciprocity This project reinforces the point balanced management approach is virtually impossible. What realist
system and team mechanism. The Our analysis identified three that in a successful diffusion of
and acknowledges individual is costly, especially in management evaluation can do is to stimulate the
leadership and management style CMO configurations that indicate innovations (such as in the case of
contributions to the wider system. time (supervision, dialog, problem- researcher to describe a detailed
introduced by Tdh is perceived by causal pathways between sets of IeDA), it is necessary to combine
Conditions for such environment solving sessions). It requires picture of the causal web that
health centre staff to be effective and management practices and use the introduction of technology
to be promoted by a leadership reasonable financial resources and includes the multiple determinants
supportive. In ensuring that all staff of REC and we modified the MRT with support and management
that creates a decentralised a management capability to deal and to categorise these as
have access to improved working accordingly: mechanisms. It also shows that in
decision space where initiatives are not only with administration but intervention, underlying mechanism
environment, their work tools are respected. management of healthcare workers, also with the less tangible issues of and context. In our case, we argue
The adoption of a computer-based
repaired, and their requests listened it is important to mix different relationships, organisation culture that open dialog, training and support
decision support tool by health The introduction of the innovation
contributes to create reciprocity and management practices. It also and motivation of staff. services are essential, but we don’t
staff at primary health care will be is combined with a multiplicity of
organisational commitment. This is in important to highlight that managers’ know which among these sets is the
enhanced by having a leadership management practices including
turn contributes to good performance attitude plays a great place in the most important and in which setting.
focusing on building wide consensus role distribution, team work, problem
in terms of utilisation of the tablet
from surrounding stakeholders (local solving approach and task monitoring
success of the intervention: open Methodological lessons
and adherence to the IMCI protocol. dialog and respect are crucial The MRT is used in realist evaluation
and national authorities) on the (hard) and training, supervision,
dimensions. This is aligned with the We used a realist evaluation to clarify key findings. An MRT cannot
The third CMO configuration can be benefits of using such an innovation support and recognition (soft).
findings from other studies. approach as we see health facilities cover all possible explanations of
formulated as after creating strong and having a wide of actors fully and
as primarily being social entities. change. A realist evaluator does
truly engaged in the directions the Based on the mechanism of
consensus at all levels on the benefits Regarding the mechanisms, our Pawson argues that realist evaluation not pretend to provide the ultimate
project could take. This necessitates perceived organisational support,
of using REC (C3), sustaining the findings relate to the analysis of is well suited to investigate change evidence that the intervention
a system promoting flows of such combinations lead to a
use of REC as routine practice (O3) Evans and Davis (2005) who situated in such social system (Pawson and works. Rather, the MRT aims at
information between all levels of the reorganisation of the health team
requires introducing the notion the underlying mechanisms of high Tilley 1997). However, appealing as it enlightening the decision-maker, a
health system where transparency of and the distribution of roles before
of individual responsibility and commitment management at the level is, realist evaluation poses a number process of utilisation of research that
information is valued. and during the consultation, and
accountability (M3). The members of of the internal social structure of the of challenges for the researcher. may be the most frequent in case
positive atmosphere that includes
the health centre, the primary users of The introduction of such innovation organisation. Such practices improve of social research. In such cases, a
recognition of each team member, The most critical issue is the
the REC, have the feeling of belonging needs to occur in an environment knowledge, practice and skills but pragmatic position should be taken
organisational commitment and attribution paradox. In complex
to a system that is wider than their flexible enough to provide space also exert effects at the level of whereby one tries to refine the MRT
sense of belonging. Every new comer systems, the behaviour of people
health centre and contributing to to staff make decisions on the relationships between team members as much as practically possible
starting in the health centre or the is determined by many interlinked
a bigger enterprise than their own distribution of clearly-defined tasks but also with line managers (in this with the explicit aim of providing
district are fully integrated into factors. Health professionals act
district. This is the result of early within the team in order to better case, the district health managers). options for improvement or scaling
this new organisational culture and under influence of their professional
and ongoing engagement with a adapt their work to the new situation. Weak ties are strengthened, up rather than reaching a perfect
benefit from the same support and norms, social pressure, management
wide range of actors ranging from On the other hand, the innovation, reciprocity is established and understanding of the intervention as
recognition. interventions and their intrinsic
national and district authorities to REC, needs to be flexible enough shared mental models contribute such.
opinion leaders at community level to take into account the constant Conditions for such management
(Yukl 1999). REC users feel strong changing policy environment and the changes to work include open dialog
and wide consensus on the necessity emerging needs and requests from at all levels of the system, a minimum
of testing and using REC – a unique its users. of resources to cover the support
message sent by a multiplicity of services and supervision and regular
key stakeholders influencing the The REC is adopted when perceived
discussions focusing on solving
environment of healthcare workers. by users and district managers as
problems faced by health centre
The introduction of the notion of being encompassed within a broader
teams.
individual accountability in public quality improvement strategy
services through personal login where health staff is sensitised to
on the software also contributed the importance of quality and their
to enhance a sense of individual capacity to address quality issues at
responsibility and contribution to the their own level.
wider system. The introduction of the REC needs
to be accompanied by a supportive
In terms of context, availability of
atmosphere and environment
well-trained staff in IMCI and REC
26 | London School of Hygene & Tropical Medicine 27

References

Aubert, B. A. and G. Hamel (2001). Paediatrics 72(9): 735-739. organisational performance: the mediating research. Qualitative research in health Plsek, P. E. and T. Greenhalgh (2001). “The Yin, R. (2003). Case study research: design
“Adoption of smart cards in the medical role of internal social structure.” Journal of care. C. Pope and N. Mays. London, BMJ challenge of complexity in health care.” and methods. London, Sage Publications.
sector: the Canadian experience.” Social Management 31(5): 758-775. books. British Medical Journal 323: 625-628.
Science & Medicine 53: 879-894. Chen, H.-T. (1990). Theory-driven
evaluations. Newbury Park, California, Yukl, G. (1999). “An evaluation of
Sage Publications. Fitzpatrick, R. and M. Boulton (1994). Kouanda, L. and A. Baguiya (2013). Rakha, M., A.-N. Abdelmoneim, S. Farhoud conceptual weaknesses in transformatinal
Barnes, M., E. Matka and H. Sullivan (2003). “Qualitative methods for assessing health Evaluation de la qualite des soins and et al. (2013). “Does implementation and charismatic leadership theories.”
“Evidence, understanding and complexity. care.” Quality in Health 3: 107-113. prodigues aux enfants de moins de cinq of the IMCI strategy have an impact on Leadership Quarterly 10(2): 285-305.
Evaluation in non-linear systems.” Conseil National de la Transition (2016). (05) ans dans les formations sanitaires child mortality? A retrospective analysis
Evaluation 9(3): 265-284. Loi numero 060-2015/CNT portant regime des regions du nord et du centre-nord du of routine data from Egypt.” BMJ Open
d’assurance maladie universelle au Gouvernement Burkinabe. (2016). Burkina Faso. Burkina Faso, WHO, UNICEF, 3(e001852). Bergman, M. M. (2008). Advances in
Burkina Faso. JO numero 07 du 18 Fevrier “Gratuité des soins pour les enfants UNFPA. mixed methods research: Theories and
Black, R. E., S. S. Morris and J. Bryce 2016. de moins de cinq ans : Une réalité au applications, Sage.
(2003). “Where and why are 10 million Burkina Faso à partir du 02 avril 2016.” Rihani, S. (2002). Complex systems theory
children dying every year?” Lancet Retrieved 15/08/2017, from http://www. Merton, R. K. (1968). Social theory and and development practice. London, Zed
361(9376): 2226-2234. Deflaux, G. (2010). Registre Electronique sig.bf/2016/04/gratuite-des-soins-pour- social structure. New York, The Free Press. Books. Creswell, J. and V. Plano Clark (2007).
de Consultations (REC) - Cahier des les-enfants-de-moins-de-cinq-ans-une- Designing and conducting mixed
charges. Ouagadougou, Burkina Faso, realite-au-burkina-faso-a-partir-du-02- methods research. Thousand Oaks, Sage
Blanchet, K. (2013). “How to facilitate Terre des hommes. avril-2016/. Ministere de la Sante (2011). Plan National Rogers, E. M. (1995). Diffusion of Publications.
social contagion?” Int J Health Policy de Developpement Sanitaire 2011-2020. Innovations. New York, The Free Press.
Manag 1(3): 189-192. Ouagadougou, Ministere de la Sante
Deflaux, G., T. Agagliate, J.-E. Durand Granovetter, M. (1973). “The Strength of Burkina Faso. Denzin, N. K. (2012). “Triangulation 2.0.”
and P. Yamaogo (2014). Computerization Weak Ties.” American Journal of Sociology Rowe, A., F. Onikpo, M. Lama and M. Journal of Mixed Methods Research 6(2):
Blanchet, K., J. J. Lewis, F. Pozo-Martin, of Medical Consultation for Children 76: 1360-1380. Deming (2010). “The rise and fall of 80-88.
A. Satouro, S. Somda, P. Ilboudo, S. Under Five Years of Age in Rural Areas Ministres, P. d. F. P. d. C. d. (2016). Decret supervision in a project designed to
Sarrassat and S. Cousens (2016). “A mixed of Burkina Faso. Technologies for 2016-311_PRES/PM/MS/MATDSI/MINEFID strengthen supervision of Integrated
methods protocol to evaluate the effect Sustainable Development. J.-C. Bolay et al. Green, J. (2006). Analysing qualitative portant gratuite des soins au profit des Management of Childhood Illness in Johnson, R., A. Burke, J. Onwuegbuzie and
and cost-effectiveness of an Integrated Swistzerland, Springer. data. Principles of social research. J. femmes et des enfants de moins de cinq Benin.” Health Policy Plan 25(2): 125 - 134. L. A. Turner (2007). “Toward a definition
electronic Diagnosis Approach (IeDA) for Green and J. Browne. Maidenhead, Open ans vivant au Burkina Faso. JO numero 22 of mixed methods research.” Journal of
the management of childhood illnesses at University Press. du 02 Juin 2016. Ouagadougou, Burkina mixed methods research 1(2): 112-133.
primary health facilities in Burkina Faso.” Denis, J. L., Y. Hebert, A. Langley, D. Lozeau Faso. Saunders, R., M. Evans and P. Joshi (2005).
Implementation Science 11(1): 1-9. and L. H. Trottier (2002). “Explaining “Developing a process-evaluation plan
diffusion patterns for complex health care Greenhalgh, T. (2008). “Role of routines for assessing health promotion program Marchal, B., G. Westhorp, G. Wong, S. Van
innovations.” Health Care Management in collaborative work in healthcare Moore, G., S. Audrey, M. Barker, L. implementation: a how-to guide.” Health Belle, T. Greenhalgh, G. Kegels and R.
Bonell, C., A. Fletcher, M. Morton, T. Review 27: 60-73. organisations.” British Medical Journal Bond, C. Bonell, W. Hardeman and et al. Promot Pract. 6(2): 134-147. Pawson (2013). “Realist RCTs of complex
Lorenc and L. Moore (2012). “Realist 337: 1269-1271. (2015). “Process evaluation of complex interventions - an oxymoron.” Soc Sci Med
randomised controlled trials: a new interventions: Medical Research Council 94: 124-128.
approach to evaluating complex public Denzin, N. and Y. Lincoln (1994). Handbook guidance.” BMJ 350. Schneider, B., S. Gunnarson and K. Niles-
health interventions.” Soc Sci Med. 75(12): of qualitative research. Thousand Oak CA, Grilli, R. and J. Lomas (1994). “Evaluating Jolly (2004). “Creating the climate culture
Sage Publications. the message: the relationship between of success.” Organisational Dynamics Morse, J. (2003). Principles of mixed
2299-2306.
compliance rate and the subject of a Mugala, N., W. Mutale, P. Kalesha 23(1): 17-29. methods and multimethod. In. Handbook
practice guideline.” Medical Care 32: 202- and E. Sinyinza (2010). “Barriers to of mixed methods in social and behavioral
Borgatti, S. P., M. G. Everett and P. Shirey Derenzi, B., T. Parikh, M. Mitchell, M. 213. implementation of the HIV guidelines research. A. Tashakkori and C. Teddlie.
(1990). “LS sets, lambda sets and other Chemba, D. Schellenberg, N. Lesh, C. Sims, in the IMCI algorithm among IMCI Takada, S., B. Oudavong and C. Kurolwa Thousand Oaks, Sage Publications: 189-
cohesive subsets.” Social Networks 12: W. Maokola, Y. Hamisi and G. Borriello trained health workers in Zambia.” BMC (2007). “The successes and challenges 208.
337-357. (2008). “e-IMCI: Improving Pediatric Horwood, C., A. Voce, K. Vermaak, N. Paediatrics 10(93). of the IMCI training course in Lao PDR.”
Health Care in Low-Income Countries.” Rollins and S. Qazi (2009). “Experiences Southeast Asian Journal Trop Med Public
CHI 2008. of training and implementation of the Health 38(1). Pawson, R. and N. Tilley (1997). Realistic
Boss, L. P., M. J. Toole and R. Yip (1994). integrated management of childhood Nguyen, D., K. Leung, L. McIntyre, W. Ghali evaluation. London, Sage Publications.
“Assessments of mortality, morbidity, illness (IMCI) in South Africa: A qualitative and R. Sauve (2013). “Does Integrated
and nutritional status in Somalia during Donaldson, S. (2007). program theory- evaluation of the IMCI case management Management of Childhood Illness (IMCI) United Nations Inter-agency Group for
driven evaluation science. Strategies and Child Mortality Estimation (2015). Levels Rycroft-Malone, J., M. Fontenla, D. Bick
the 1991-1992 famine. Recommendations training course.” Social Science and Training Improve the Skills of Health
applications. New York, Lawrence Erlbaum and trends in child mortality - Report 2015. and K. Seers (2008). “Protocol-based care:
for standardization of methods.” JAMA Medicine 70(2): 313-320. Workers? A Systematic Review and Meta-
Associates. New York. Impact on roles and service delivery.”
272(5): 371-376. Analysis.” PLoS One 8(6): e66030.
Journal of Evaluation in Clinical Practice
Institute of Medicine (2001). Crossing the 14: 867 - 873.
Bowling, A. and S. Ebrahim (2005). DSS/DGISS (2012). Enquete SARA. Quality Chasm: A New Health Care System Oakley, A., V. Strange , C. Bonell, E. Allen Yameogo, P., B. StollL, N. A. Zonon, T.
Handbook of health research methods. Ouagadougou, Burkina Faso, Ministere de for the 21st Century. Washington DC, and J. Stephenson (2006). “Process Agagliate and G. Viala (2011). REC:
la Sante. Résultats d’utilisation d’un logiciel Rycroft-Malone, J., M. Fontenla, D. Bick
Maidenhead, Open University Press. National Academy Press. evaluation in randomised controlled
informatique pendant la consultation and K. Seers (2010). “A realistic evaluation:
trials of complex interventions.” BMJ
des enfants de moins de 5 ans selon la the case of protocol-based care.”
332(7538): 413-416.
Bryce, J., C. G. Victora and R. E. Black “The Eisenberger, R., R. Hutchinson, S. Jones, G., R. W. Steketee, R. E. Black, Z. A. stratégie PCIME. Burkina Faso, Terre des Implementation Science 5(1): 38.
unfinished agenda in child survival.” The Hutchinson and D. Sowa (1986). Bhutta, S. S. Morris and G. Bellagio Child hommes.
Lancet 382(9897): 1049-1059. “Perceived organisational support.” Survival Study (2003). “How many child Patton, M. (1999). “Enhancing the quality
Journal of Applied Psychology 71(3): 500- Teddlie, C. and A. Tashakkori (2011).
deaths can we prevent this year?” Lancet and credibility of qualitative analysis.”
503. Yetton, P., R. Sharma and G. Southon “Mixed methods research.” The Sage
362(9377): 65-71. Health Services Research 34(5 Pt 2):
Chaudhary, N., P. Mohanty and M. Sharma (1999). “Successful IS innovation: the handbook of qualitative research: 285.
1189-1208.
(2005). “Integrated Management of contingent contributions of innovation
Childhood Illness. Follow up of basic Evans, W. and W. Davis (2005). “High- Keen, J. and T. Packwood (2000). Using characteristics and implementation
Health Workers.” Indian Journal of performance work systems and case studies in health services and policy Pawson, R. and N. Tilley (1997). Realistic process.” Journal of Information
evaluation. London, Sage Publications. Technology 14: 53-68.
London School of Hygiene
& Tropical Medicine

Keppel Street, London WC1E 7HT


United Kingdom

Switchboard: +44 (0)20 7636 8636


Fax: +44 (0)20 7436 5389
www.lshtm.ac.uk