What can be measured can be improved. But what if the measurement is flawed, outdated or biased? This is the situation confronting governments, businesses and other organisations in Africa. Simon Allison looks at data on Africa and discusses how it could be made more reliable.
Statistics are important, particularly for gauging development, and particularly in Africa. If we cannot measure and compare, we cannot understand, analyse and look for solutions. Governments, donors and businesses rely on numbers measuring healthcare, poverty, education, the economy and other issues. These numbers determine policy. How reliable are they?
In short, not very. Despite the plethora of facts and figures available, Africa in particular suffers from a dearth of good data. Some of it is old, some of it is biased, some of it just does not exist. Until the continent can rectify this situation, policy development is based on as much guesswork as fact.
“Without data, policy debate and creation cannot be based on facts and policy- making tends to remain in the realm of commitments,” said Elizabeth McGrath, director of the Ibrahim Index of African Governance, which collates much of the reliable data in Africa (and sometimes, by virtue of necessity, some of the less reliable data) into an index that measures governance. “There is no doubt that with better data, governments and partners alike could identify which policy interventions work most efficiently.”
One of the biggest issues facing the continent is poverty. Billions of dollars are poured every year into poverty alleviation projects. There is only one problem: no one has a clue how poor Africa really is. The statistics are so flawed that they are nearly meaningless. The absence of good poverty statistics raises serious questions about resource allocation, accountability and cost-effectiveness.
One example stands out as particularly disturbing. The World Bank’s World Development Report is one of the primary sources for poverty data. It is enormous. Businesses use it to make investment decisions, donors to determine funding requirements and finance ministries to make economic policy. It matters.
But sometimes, World Bank numbers can be dangerously misleading. Look at the report’s most headline-grabbing indicator: how many people in a particular country are living under the infamous poverty line, defined now as $1.25 a day?
For Algeria, the answer to that question would be 6.8% of the population, according to the report. But do not take that at face value. Dig a little deeper and it becomes apparent that this data was collected in 1995, 17 years ago. Since then, the World Bank has had no new data measuring the number of Algerians below the poverty line. It is forced to rely on extrapolation to estimate what the actual numbers might be. Extrapolation or related techniques are standard statistical practices and can often be quite accurate. But they are no substitute for collected data. The longer one has to extrapolate the greater the probability of error.
Algeria is the most extreme example, but there are others. The newest data in the report on South Africans and Tunisians below the poverty line is 12 years old; for Sierra Leoneans, it is eight years; for Nigerians, it is seven years. Even in the best-case scenario, the data is at least three years old, thanks to the time lag which is a feature of most national data, given the practicalities of data collection.
What all this means is that the World Bank’s main indicator of poverty in Africa is at least three years out of date, usually more. A lot can happen in three years, as any president trying to explain away his country’s unflattering numbers will admit. Essentially, what we think we know about poverty in Africa today is, in the best cases, a reasoned estimate. In others it is guesswork.
Other flawed statistics reveal how wrong these guesses can be. One of the most quoted and relied upon statistics in the world is that of gross domestic product (GDP), used to measure the size of an economy. GDP is calculated from what is known as a base year. This base year should be regularly updated to compensate for unanticipated variations; if not, GDP figures can be wildly skewed.
Take Ghana, which rebased its GDP in 2010, going from a base year of 1996 to 2006. Suddenly, the economy grew by 60%. The average per capita income jumped from under $800 to $1,318, enough to catapult Ghana from being a low-income to a middle-income country. Not a bad return for a few relatively straightforward calculations. Numbers, especially in Africa, are not sacred; they can often be misleading. Why? Rural communities in Africa are generally difficult to access, thus hard for data collectors to reach. Much commerce is conducted informally, leaving no records. Government resources are thinly-stretched and directed at material outputs like infrastructure or healthcare, rather than the less obviously valuable task of collecting statistics. There is a shortage of trained staff to collect data. When data is collected, it is often at irregular intervals and inconsistent with what came before, making it impossible to produce meaningful comparisons.
There are also some less valid reasons. Bias is a huge problem, particularly when so many statistics come from so-called “official data”, numbers produced by a government. What is to stop presidents from telling people what they want to hear rather than the real facts, particularly when the made-up or massaged statistics make the government look so much better? Equally, what is to stop non-governmental organisations (NGOs) from manipulating their figures to make a situation look so much worse—and therefore so much more in need of both attention and funding—than it really is?
The underlying point of these examples is that the less reliable the statistics, the greater the chance of misinterpretation or manipulation. So how can statistics in Africa be made more reliable?
There are three types of data, all of which are unreliable in different ways and therefore require unique solutions.
Outcome data is the most valuable, but also the most difficult and expensive to collect. It measures tangible outcomes such as the number of children in school or the prevalence of a certain disease. Even in a perfect world, this requires huge investment and all the data comes with a two- to three-year time lag. Usually, governments manage the collection of outcome data, which is the clue for how to improve it. Governments and donors should invest real resources into national statistics offices capable of serious research, and ensure the data gathered is comparable across countries and time. This is already happening, to an extent. The African Union Commission and the United Nations Economic Commission for Africa, in partnership with the Organisation for Economic Co- operation and Development, are focusing on strengthening national statistical capacity and harmonising data collection techniques. But along with strengthening the national offices, donors need to stop trying to replace them.
“Donors flood money into conducting independent, small sample surveys in a country, usually to assess the impact of their own projects. But this can undermine national statistics offices, and removes any incentive for the government to fund them,” said Richard Watts, a specialist in African data who consults for international NGOs and the United Nations. “More should be done to build the capacity of government statistical departments, rather than usurping them.”
Process data assesses laws and regulations and is more subjective. A good example is the World Bank’s Doing Business Report, which uses in-country experts to answer hypothetical questions such as the number of days it takes to establish a business in a country. These answers are then translated into scaled numbers and compared with other countries. But subjectivity is a problem. Almost all process data on Africa is compiled by western institutions, often using western rather than local experts, and this produces a certain cultural bias in the statistics.
Africa needs to commission its own process data and use its own experts if it wants a more relevant picture. Moves to address this are being made. The Mo Ibrahim Foundation, for example, is working with Afrobarometer, a Benin-based research company, to scale up Afrobarometer’s existing citizen surveys to cover two-thirds of Africa; they have commissioned the South Africa-based Global Integrity Trust to create a network of experts in each African country to provide assessments of key political, social and economic indicators. Finally there is input data, which is the easiest to measure but often the least useful. This is usually about the money that is allocated into a specific area or project. How much has the government spent on healthcare? How much did the United Kingdom donate to Tanzania last year? These numbers are easily manipulated, especially given the lack of accountability in the development sector. If an NGO says they spent $100 fighting poverty last year, how do we know how much of that went to the intended recipients and how much was spent on flights, hotels and per diems for the NGO staff?
The key to improving input data is improving accountability. NGOs need to be held to the same accounting standards as listed companies and governments must be encouraged to be transparent with their finances.
All this is easier said than done; data collection is hard, expensive work. But there are two trends that might encourage more reliable and timely statistics. The first is improved and more widespread education across the continent. It is producing people able to collect information and, more importantly, people able to use it effectively.
The second is Africa’s increasing commercial potential. Like countries, companies need current and accurate statistics to make effective business decisions. As a result, corporations will use their influence and money to improve data collection. Hopefully, when governments realise that accurate statistics can not only help them govern, but also attract investors, they too will put more emphasis on producing reliable and up-to-date numbers.
Still, it will be many years before Africa consistently produces statistics that governments, investors and donors can trust. Until then, policy and business decisions will all too often be based on misinformed conjecture.
