Академический Документы
Профессиональный Документы
Культура Документы
Background
World Bank and National Governments carry out many finance and private sector projects every year, but to date few have been rigorously evaluated. This makes it difficult for us to know what works, why, and what could we do better Purpose of this workshop is to take a big step towards rectifying this status quo. => But first lets take stock of what has been possible, and in the process illustrate different methods of evaluation you will be learning in the workshop, as well as issues to keep in mind this week.
E.g. project disbursed $7 million in loans, trained 2,000 firms, etc. This tells us nothing about effectiveness.
Rigorous impact evaluation compares the outcomes of a program or policy against an explicit counterfactual of what would have happened without the program or policy, and takes seriously the issue of why some firms, regions etc. participate in the program and others do not.
Perception that many finance and private sector policies lend themselves less to formal evaluations:
Changes in laws or regulations may occur at economy-wide level Large loan may be given only to one or two banks
Lack of incentives for both Bank operational staff and Governments to do Lack of knowledge as to what is possible => Hopefully this week we can overcome these barriers.
Firms
randomly chosen to receive these grants can be compared to firms which were randomly chosen not to receive them (Male Results)
Baseline Follow-up
Treatment
Control
(Female results)
4500 4000 3500 3000 2500
Baseline
2000 1500 1000 500 0
Follow-up
Treatment
Control
Low returns to female businesses in Sri Lanka provides reason to question exclusive focus on women Experiment in Philippines found converting group to individual liability led to faster client growth and no change in default (among existing clients) Experiment in South Africa found marginal loan recipients taking high interest consumer loans were better off 6-12 months later along with high returns to capital, rejects argument that poor cant pay high rates. Large randomized trial of microfinance in India and Philippines finds fairly modest results after 1 year, with little impact on poverty reduction.
General experience has been low take-up: e.g. 4.6% for one product in India, 17% in Malawi. Research suggests trust and financial literacy both play roles
Implications for designing products should be simple, and pay out with reasonable frequency.
Setting up one-stop shops to improve process of starting a business is a popular reform Often reforms like this are phased in at a municipal or regional level, offering scope for a difference-in-differences evaluation
Example: Mexico simplified the process of business registration Due to staffing constraints, the federal agency could not implement the reform in all priority municipalities at once, but instead staggered the reforms, introducing them first in some municipalities and then later in others. Among the municipalities identified as priorities for implementation, there was no specification of which should go first. A Difference-in-differences estimation can then be used to estimate the effect of the reform by comparing changes in business start-ups in the municipalities which got the reforms earlier to the changes in business start-ups in the similar municipalities which were to receive this reform later.
Results show increase in registration and employment, but coming from new entry, not conversion of existing firms.
reform introduced into entire country at once, but only applies to subset of population.
Regression
possible. E.g. evaluation of bankruptcy reform in Colombia which reduced costs of re-organizing a bankrupt firm. Active, non-bankrupt firms not affected by law, so can use difference-in-differences comparing active to bankrupt.
Lessons
1)
Why have these topics been the ones where progress has been made?
Close ties to theoretical work, and in many cases, theoretical effects unclear. Have been feasible for researchers to do with existing data sets or implementing with an NGO => Big opportunity is to do this with larger projects.
-
Lessons
2) Variety of tools means evaluation of many finance and private sector programs is possible.
Lessons
3) Ex-ante design beats ex-post - Huge issues with data availability, understanding what program did, how it selected people, etc. ex post. => Need to get researchers involved from initial design of new projects, rather than coming ex post.
Scope for much more work in the areas where existing efforts have occurred: microfinance, microenterprises, insurance, regulatory reform. Still only a handful of rigorous studies. Big gains to be had from looking at impacts of policies and projects designed to benefit large numbers of consumers and firms:
Financial literacy and consumer protection SME sector policies training, loans, enterprise support Housing finance => Major area is big policies where too expensive for NGOs or researchers to run themselves should be in comparative advantage of the World Bank to work with National Governments to do.