Barry Carin, Senior Fellow at the Centre for International Governance Innovation (CIGI)
But can anyone do better? Can we devise a development framework for the post-2015 era that incorporates the many advantages but avoids the disadvantages of the MDGs? The way ahead is a challenging labyrinth, with many difficult decisions:
- Should post 2015 Goals, our aspirations for what is most important to accomplish, apply to the whole world, or focus on the poorest and least developed?
- What is the maximum number of post 2015 Goals, given the many competing priorities and the virtue of a limited number?
- Should goals be included for issues like security, human rights, democracy, climate change, water, gender, consumption?
- Should goals be devised for secondary and tertiary education; global financial system; urbanization; skill development; anti-corruption; tax evasion; land mine clearance?
- Should Goals and Targets (the specific levels of global and national ambition) be adapted to local contexts and country specific situations?
- What indicators should be selected to measure the success for each goal?
- What process of consultation and decision will satisfy the wide range of stakeholders, from research and policy communities, to government officials, to the world’s poor and vulnerable communities?
A meeting in June 2011 of a consortium of think tanks and organizations at Rockefeller’s Bellagio Centre resulted in "the Bellagio Goals," twelve potential successor goals to the MDGs. The list of goals relate to livelihoods and income levels, food and water, education and skills, health, gender equality, security, resilient communities, connectivity, civil rights, sustainable management of the biosphere, good global governance and rules of the world economy.
The availability of clear, practical ways to measure progress in agreed areas should be a dominant criterion in the selection of post-2015 Goals from among the menu of potential candidates. Proposed 2015 goals should be assessed based on the strengths and weaknesses on indicators available to measure performance.
Measurement is important. Sceptics are fond of quoting the aphorism legend ascribed to be on a plaque in Einstein’s office "Not everything that counts can be counted; not everything that can be counted counts." But without indicators we cannot monitor; we cannot report on progress. Indicators also inform debate and raise awareness. Progress reports on indicators will sensitize public opinion and highlight countries with best practices. Measurement provides an opportunity for improved coordination and will influence development priorities. Perhaps most important, measurement affects behaviour "Tell me what you are going to measure, I will tell you how I am going to behave."
But there are intimidating challenges in selecting indicators. Ideally, they should measure outcomes and outputs, not inputs and processes. Education provides the classic example of the danger of measuring inputs. Student enrolment can be very misleading - in some countries ‘performance’ was improved by moving to shifts, cutting the length of the school day in half. The danger of focusing on the output of graduation rates is that administrators sometimes respond by lowering standards to meet the targets. But outcome measures are controversial. The most likely candidate is the OECD’s Programme for International Student Assessment (PISA), a standardized test applied in 65 countries that could perhaps be extended globally. In the civil rights area, voter turn-out is suggested, but there are examples of countries with high rates where the elections are a sham.
Direct measures are preferable to indices. Indices are tempting because they reduce the number of indicators to a manageable number by summarizing a wide range of information, but the arbitrary choice of weights can distort results.
Seductive indices include the Global Footprint, the Political Terror Scale, and WHO has introduced the DALE (disability-adjusted life expectancy), or HALE (a summary measure of the equivalent number of years in full health that a newborn can expect to live, based on current rates of ill health and mortality). The methodology for the Global Footprint calculation has too many assumptions to easily display sensitivity calculations. The WHO reports that population health data have been particularly vulnerable to distortions owing to fear, denial, or lack of capacity to report. The Political Terror Scale applies arbitrary weights to aggregate rates of executions and of incarceration - two incommensurable variables.
If at all possible, we should avoid perception-based measures - but in the absence of administrative data, surveys are the only recourse. (Care must be taken in survey design - recall the cliché "Tell me the answer you want, then I’ll tell you the survey question".) Data must be available at reasonable cost, with disaggregation possible for several dimensions. Choice of indicators must be sensitive to potential behaviour response. Ideally, given the need to minimize the number of indicators, they should be "summative" reflecting whole sector outcomes (e.g. maternal mortality as an indicator for the effectiveness of the public health system). Perhaps most important, indicators should be accessible to lay readers.
Our research consortium reviewed the framework of the "Bellagio Goals" and the quality of available indicators in a series of consultations around the world. Insightful points raised include:
- Process matters; beware of focusing only on outcomes. Provisions for participation, transparency and accountability influence outcomes.
- There should be a minimum measure for every individual - perhaps the poverty line - which differs in each country. One suggestion was to select a target percentage of people to reach half the median income in each country.
- There is a risk that universal global goals will de-emphasize the focus on the poorest countries.
- There is a concern that early consideration of post 2015 successor goals will divert attention from efforts to achieve the current MDGs.
- While there is a wealth of potential indicators for health goals, there is a paucity of reliable indicators for equitable economic rules, civil rights and global governance.
- We should not be captive to existing data sources; if necessary we can mandate new statistical activity or surveys.
To paraphrase Tom Peters, if at this point, reader, you are not confused, then you have not been paying attention. The powers that be at the United Nations will be tested in consulting and negotiating through the maze of complexities and interests. One approach that will help them navigate the future course is to insist that practical and cogent indicators are available for candidate goals. In fact, this line of attack will reduce the number of suggestions to a manageable number - since valid indicators do not exist for many worthy aspirational goals. Trying to please everybody is a formula for failure. The UN architects can argue persuasively that future Goals require solid information to measure where we are and to prescribe what needs to be done.
* Barry Carin is a Senior Fellow at the Centre for International Governance Innovation, one of the leaders of a consortium with the Korean Development Institute, the International Federation of Red Cross and Red Crescent Societies, the Getulio Vargas Foundation, the International Poverty Reduction Center in China, the University of Manchester, the Tata Institute of Social Sciences, and the University of Pretoria, working together on post 2015 Development Goals.