Evaluate the influences of the two
Impact evaluation framework
Biases are normally visible in two situations: when the measurement of the outcome with program exposure or the estimate of what the outcome would have been without the program exposure is higher or lower than the corresponding "true" value p For instance, in a 5-year agricultural project, seeds are inputs, farmers trained in using them our outputs, changes in crop yields as a result of the seeds being planted properly in an outcome and families being more sustainably food secure over time is an impact. A theory-based approach enables policy-makers to understand the reasons for differing levels of program participation referred to as 'compliance' or 'adherence' and the processes determining behavior change. Public policy will therefore be successful to the extent that people are incentivized to change their behaviour favourably. Factors such as exchange rates would also impact objectives concerned with international marketing. Other forms of bias[ edit ] There are other factors that can be responsible for bias in the results of an impact assessment. Click an approach on the left to navigate to it Empowerment Evaluation A stakeholder involvement approach designed to provide groups with the tools and knowledge they need to monitor and evaluate their own performance and accomplish their goals. In contrast with 'black box' impact evaluation approaches, which only report mean differences in outcomes between treatment and comparison groups, theory-based impact evaluation involves mapping out the causal chain from inputs to outcomes and impact and testing the underlying assumptions. Click an approach on the left to navigate to it Horizontal Evaluation A particular type of case study used to jointly develop an agreed narrative of how an innovation was developed, including key contributors and processes, to inform future innovation efforts. Group comparisons that have not been formed through randomization are known as non-equivalent comparison designs Rossi et al. In addition, interventions are often active rather than passive, requiring a greater rather than lesser degree of participation among beneficiaries and therefore behavior change as a pre-requisite for effectiveness. COSA aims to facilitate this by enabling them to accurately calculate the relative costs and benefits of becoming involved in any given sustainability initiative. Impact Evaluations which compare outcomes among beneficiaries who comply or adhere to the intervention in the treatment group to outcomes in the control group are referred to as treatment-on-the-treated TOT analyses. These biases affect the interest of the stakeholder. Theory-Based approaches use both quantitative and qualitative data collection, and the latter can be particularly useful in understanding the reasons for compliance and therefore whether and how the intervention may be replicated in other settings.
Other forms of bias[ edit ] There are other factors that can be responsible for bias in the results of an impact assessment. While the first three appear during the project duration itself, impact takes far longer to take place.
Examples[ edit ] While experimental impact evaluation methodologies have been used to assess nutrition and water and sanitation interventions in developing countries since the s, the first, and best known, application of experimental methods to a large-scale development program is the evaluation of the Conditional Cash Transfer CCT program Progresa now called Oportunidades in Mexico, which examined a range of development outcomes, including schooling, immunization rates and child work.
Rigorous evaluation of impact using a credible counterfactual as discussed above. Not only that, but the direction the bias may take cannot usually be known in advance Rossi et al.
Impact evaluation template
Approaches on this site refer to an integrated package of options methods or processes. Factors such as exchange rates would also impact objectives concerned with international marketing. Ignoring confounding factors can lead to a problem of omitted variable bias. Secular trends or secular drift[ edit ] Secular trends can be defined as being relatively long-term trends in the community, region or country. If selection characteristics are known and observed, they can be controlled for to remove the bias. White  and Ravallion  discuss alternate Impact Evaluation approaches. Matching involves comparing program participants with non-participants based on observed selection characteristics. Different designs require different estimation methods to measure changes in well-being from the counterfactual. High quality impact evaluations will assess the extent to which different groups e. Furthermore, it is possible that program participants are disadvantaged if the bias is in such a way that it contributes to making an ineffective or harmful program seem effective. Subscribe to email updates from tutor2u Business Join s of fellow Business teachers and students all getting the tutor2u Business team's latest resources and support delivered fresh in their inbox every morning. A theory-based approach enables policy-makers to understand the reasons for differing levels of program participation referred to as 'compliance' or 'adherence' and the processes determining behavior change. Click an approach on the left to navigate to it. Definitions[ edit ] The International Initiative for Impact Evaluation 3ie defines rigorous impact evaluations as: "analyses that measure the net change in outcomes for a particular group of people that can be attributed to a specific program using the best methodology available, feasible and appropriate to the evaluation question that is being investigated and to the specific context". Internal Influences on Marketing Objectives Corporate objectives As with all the functional areas, corporate objectives are the most important internal influence.
It is the only evaluation design which can conclusively account for selection bias in demonstrating a causal relationship between intervention and outcomes. Interfering events[ edit ] Interfering events are similar to secular trends; in this case it is the short-term events that can produce changes that may introduce bias into estimates of program effect, such as a power outage disrupting communications or hampering the delivery of food supplements may interfere with a nutrition program Rossi et al.
This may be because of participant self-selection, or it may be because of program placement placement bias. Click an approach on the left to navigate to it Innovation History A way to jointly develop an agreed narrative of how an innovation was developed, including key contributors and processes, to inform future innovation efforts.
Click an approach on the left to navigate to it Beneficiary Assessment An approach that focuses on assessing the value of an intervention as perceived by the intended beneficiaries, thereby aiming to give voice to their priorities and concerns. The 'counterfactual' measures what would have happened to beneficiaries in the absence of the intervention, and impact is estimated by comparing counterfactual outcomes to those observed under the intervention.
This could possibly make the accomplishments of program seem small or even insignificant therefore forcing the personnel and even cause the program's sponsors to reduce or eliminate the funding for the program Rossi et al.
based on 87 review