Impact Assessment

2008 Reader on Measuring and Reporting Results, by Jim Tanburn (English/French/Spanish)

    In the absence of much discussion on the subject, it remains rather sensitive, and one that people therefore try to avoid. Meanwhile, external pressures are growing, for more information; they are coming from donors (e.g. through the Paris Declaration, the MDG deadline), new players and aid models (e.g. social investors) and increased visibility (e.g. Live8). This Reader argues that practitioners need to seize the initiative and to develop answers, before someone else does it for them. In the absence of good data, critics will always be able to say: 'if you cannot measure it, maybe it is not there'.

    A brief overview is therefore given of current understanding in the field, including particularly the terms, indicators and methodologies in use. It is argued that multi-agency agreement in these areas would yield very important benefits, in addition to an approximate comparison of performance; for example:

    - agencies could add impacts achieved across all of their country programmes, enabling them to report results for the agency as a whole;
    - agencies would also be able to make informed choices about which intervention strategies to fund

    Examples are given of impacts measured in a standard format, including for example cost per job created; since the resulting numbers are very different in magnitude, they make a rational conversation about strategy choice possible, even if they are only correct to within +/- 50%. Agreement now needs to be built around the key parameters for formulating these numbers, including for example the multipliers to use for indirect impacts.

    Approximate measures do not replace the need for rigorous impact assessments. But agreement between agencies on a small number of indicators, and their application across a wide range of interventions, would win recognition for the achievements of the PSD community. Affordable mechanisms are needed, to ensure that the numbers produced are credible - for example through certification of the methodologies used.

    Associated Activities and Documents
    Impact Assessment
    »Why have a Standard for measuring results? Donor Committee for Enterprise Development, 2011 (English/French)
    »DCED Results Measurement Initiative, 2011 (English/French/Spanish)
    Final Documentation
    »Accelerating Growth in the Pond Fish Sector, Katalyst, 2007
    Synthesis Documents
    »Making Markets Work (for the Poor)/M4P: Some key links, 2011
    »Top ten entries, in terms of total page views since January 2010
    »Top ten entries, in terms of page views in September 2012