• University Module Series: Anti-Corruption

• Module 1: What Is Corruption and Why Should We Care?

 

 

  This module is a resource for lecturers  

 

Measuring corruption

 

Although not the main focus of this Module, the subject of measuring corruption is closely related to the discussions so far. Attempts to measure corruption are intended to reveal the nature and impact of corruption, and are necessary for developing anti-corruption responses. Measurements of corruption can be used to identify trends and illustrate the scale and scope of particular types of corruption. They can help policymakers, analysts and scholars to develop tools to reduce corruption effectively. For a further discussion on the importance of measuring corruption, see National Anti-Corruption Strategies: A Practical Guide for Development and Implementation (UNODC, 2015, chapter II).

While measuring corruption is essential, it is also a challenging task. As discussed below, there are different methods for measuring corruption and each has its own merits and drawbacks. Furthermore, each method is designed to detect certain things and ignore others. To appreciate the strengths and weaknesses of measurement methodologies, one must ask what exactly each methodology is claiming to measure and how its parameters are phrased and constructed? The latter part of the question goes to what each index or ranking is actually measuring, as opposed to what it claims to measure. Before discussing the pros and cons of different measurement methods, we must have an idea of what these methodologies are. There is a growing body of academic literature that compares and critiques the different approaches to measuring corruption (see, e.g., Holmes, 2015; Graycar and Prenzler, 2013; Schwickerath, Varraich and Lee-Smith, 2017). As a basic framework, Holmes (2015) divides the various types of measurements into three categories: official statistics, perception surveys, and experience-based surveys.

The different approaches to measuring corruption are discussed in detail in the Manual on Corruption Surveys (UNODC, UNDP and UNODC-INEGI, 2018, pp. 20-29). The Manual was developed to provide guidance on monitoring progress towards achieving target 16.5 of the SDGs, which calls on States to "[s]ubstantially reduce corruption and bribery in all their forms". The Manual stresses that experience-based and contextualized country-based measurement methods are far more precise than indirect or composite methods, or those that attempt to produce international rankings. The Manual classifies the various approaches to corruption measurement into direct and indirect methods as follows:

• Direct methods of measuring corruption aim at collecting evidence-based information on corruption through statistical and standardized procedures. They measure actual experiences of corruption, rather than perceptions of corruption, and can include official data(such as reported cases of corruption, conviction figures, electoral scrutiny findings) and experience-based sample surveys (which collect data on the experience of representative samples of a given population).
• Indirect methods of measuring corruption do not gauge the actual occurrence of corruption, but are rather based on perceived levels of corruption. They are often used because actual occurrences of corruption are difficult to measure. Indirect methods can be based on expert assessments (where selected experts are asked to assess corruption trends and patterns in a given country or group of countries) or other types of surveys that focus on perceived levels of corruption rather than on actual levels. They are sometimes composite measurements or "surveys of surveys" combining a variety of statistical data into a single indicator.

The indirect methods are usually based on subjective opinions and perceptions of levels of corruption among citizens, business representatives, civil servants or other stakeholders in a given country. While perception surveys can be useful tools to advocate internationally for the fight against corruption, they cannot be used as a proxy for actual levels of corruption. After all, people's opinions are affected by many factors, and their views on corruption may not be primarily informed by actual experience of corruption (Johnston, 2002; Olken, 2009). As shown by Byrne, Arnold and Nagano, "[w]hen perception-based and experience-based surveys have been compared, vast discrepancies have been found between people's perceptions and people's actual experience of the extent of corruption in a given country" (2010, p. 20). Another critique to the perception-based methods is that they can be influenced by the work of anti-corruption bodies. Active anti-corruption agencies may reduce corruption in reality, even though the headlines they generate about new corruption cases may drive perception-based indicators upwards. Moreover, despite their lack of accuracy, the media around the annual release of perception-based measurements can chase away investors and donors and thus have a detrimental effect on a country's economic development and capacity to fight poverty (this is further discussed in Byrne, Arnold and Nagano, 2010, pp. 19-20).

Composite indices could, in principle, derive from experience-based indicators. However, most of them use expert assessments and perception surveys as their primary sources of data. Therefore, the Manual on Corruption Surveys warns about their weaknesses in terms of validity and relevance as they are "based on a number of subjective assumptions, such as the selection of variables or sources and the determination of the algorithm used to combine heterogeneous data" (UNODC, UNDP and UNODC-INEGI, 2018, p. 21). The most widely known perception-based composite index is the Corruption Perceptions Index (CPI) of Transparency International (TI), which lists countries along a continuum of perceived levels of corruption. It is a composite index of 13 other indices from 12 organizations. Its use of data from other expert assessments and perception surveys raises questions about biases in its methodology (Donchev and Ujhelyi, 2014; Knack, 2007). The CPI has also received criticism owing to its exclusion of ordinary citizens and victims of corruption from its pool of respondents (Graycar and Prenzler, 2013, p. 15). Moreover, some of the expert assessments on which it relies are based on interviews with stakeholders that are not from the state in question, such as a European businessman being asked about corruption in an African country. More detailed discussions on these and additional critiques of the CPI are available in Thompson and Shah (2005) and in the Manual on Corruption Surveys (UNODC, UNDP and UNODC-INEGI, 2018). It is noted that TI also developed a corruption survey that combines perception-based and experience-based questions regarding the prevalence of bribery, namely the Global Corruption Barometer, which has also received criticism.

An interesting type of composite index, which relies on proxy indicators such as judicial independence and freedom of the press, is the Index of Public Integrity (IPI). The IPI aims to give an objective and comprehensive picture of the state of control of corruption in over a hundred countries. The index is based on evaluating a set of six components (judicial independence, administrative burden, trade openness, budget transparency, e-citizenship and freedom of the press) that help to clarify the institutional framework which empowers public integrity. While this index is not based on perceptions, it provides more of a risk assessment than a measure of the actual level of corruption. Additional examples of composite indices that rely on proxy indicators are the Control of Corruption Indicator of the World Bank Governance Indicators, the Bertelsmann Stiftung's Sustainable Government Indicators, and consulting firms' corruption scores such as PRS Group's International Country Risk Guide.

The idea of country ranking has also been challenged. Certain countries, such as New Zealand and Singapore, often score best on such rankings. Similarly, there is a fair degree of repetition in terms of the countries ranked as most corrupt - typically the poorest and those most affected by conflict and natural disasters. Yet, such rankings could be misleading because they do not provide a full picture. For example, they ignore the fact that the biggest bribe payers in the countries at the bottom of those indices are often multinational companies coming from the top ranked countries. As of July 2019, statistics show that 8 of the top 10 cases that resulted in settlements and fines under the USA's Foreign Corrupt Practices Act (FCPA) involved companies based in the least corrupt countries according to the CPI (Cassin, 2019). With respect to the vast majority of countries occupying the middle ground between these two extremes, methodology can make a significant difference as to a country's relative standing. Country rankings therefore raise important questions: What explains different levels and different types of corruption in countries across the world? Is it a function of political systems, culture, size and homogeneity of the population, history and stage of development, legal systems (or anti-corruption legal frameworks in particular), economic systems, natural resources, or some combination thereof, or some other set of factors? Should countries be considered corrupt if their citizens or legal entities engage in corrupt practices abroad? Depending on the diagnosis, what would the solutions to corruption be?

Turning to the direct methods, such as official data and experience-based sample surveys, the Manual on Corruption Surveys considers them as "the most reliable approach to producing the detailed information on corruption necessary for policymaking purposes (e.g., identifying corruption-prone areas, procedures or positions at risk, or monitoring trends over time)". The Manual cautions, however, against relying exclusively on official data regarding reported cases, as many victims do not report corruption. Official data may capture something other than corruption, such as how efficient the judicial system is, and, at best, give a minimum idea of corruption in the country at hand. Instead, the Manual recommends using surveys that collect data on the experience of representative samples of a given population. The Manual furthermore offers a methodology for measuring the prevalence of bribery through experience-based sample surveys.

The experience-based surveys attempt to measure actual personal experience of corruption by, for example, asking citizens or businesses if they have paid a bribe or were involved in other forms of corruption. Such a method is useful for overcoming under-reporting problems of official statistics and allows comparability of data and disaggregation of information for different population groups. At the same time, while a potentially rich source of information, sample surveys focused on bribery are not well calibrated to discover grand corruption or embezzlement. Few citizens come into contact with high-level officials and those who participate in corrupt schemes with such officials are unlikely to report them, even anonymously.

Conducted in the 1990s, the first sample surveys on corruption mainly targeted the perception of corrupt behaviours, but were eventually broadened to include the measurement of the experience of bribery. The International Crime Victims Survey, one of the best-known sample surveys measuring direct experience of crime in different countries, includes a focus on measuring bribery experiences among the population. The World Bank's Enterprise Surveys and Business Environment and Enterprise Performance Survey (BEEPS) are considered the largest firm-level survey data on experiences of bribery. Another development was the inclusion of Governance Modules in the 1-2-3 Surveys targeted at citizens of West African capitals and Andean countries' surveys.

To the measurement methods above, we can add the more recent experimental approaches to measuring corruption, which have gained popularity both in the field and in the lab. Creative designs in field studies have allowed assessing corruption, for example, by observing missing public expenditures (Olken, 2007). Zooming in on the behaviour of individuals, lab research has used a wide array of corruption games that model features of corrupt behaviour (Wantchekon and Serra, 2012). The decisions by participants engaging in these games have enabled causal insights into the micro-drivers of corruption (Köbis and others, 2019). It is also worth considering Internet- and social media-based measurements. While not a survey, Internet and social media platforms have been used to allow people to report on their experiences with corruption. In India, I Paid A Bribe allows for self-reporting of bribes paid and information about the bribe. On April 2019, the site contained 177,384 reports from 1,073 cities around India. The site is a statistical treasure trove and provides extensive reports on everyday corruption for researchers and the public.

Other methods of measuring corruption include public expenditure tracking surveys (Messick, 2015); focus groups involving dialogues between ordinary people; the Delphi method featuring opinions from researchers and experts; interviews of police officers, journalists, judges, and anti-corruption NGOs; content analysis of newspaper articles or NGO reports over a particular time span; statistical analysis of actual cases of corruption; and the proxy approach, which measures not corruption but the efforts being undertaken to combat it as an indication of how seriously political elites and active citizens take corruption. The auditing of governments and corporations is another method for measuring corruption (for a related discussion see this short clip on corruption measurements based on audits (11 minutes)). Additional surveys that are worth mentioning are the Global Competitiveness Index of the World Economic Forum, Latinobarometro, Eurobarometer, Afrobarometer, and World Values Survey. These surveys are broader in scope but include questions about corruption, enabling rich analysis of the relationship of corruption to other variables, such as attitudes toward democracy (see, e.g., Canache and Allison, 2005).

Each type of measurement has its own limitations. Corruption is, by its nature, a secretive activity that is often not in the self-interest of participants to report. Thus, self-reporting may not be honest, even in an anonymous format, because those involved may prefer to avoid drawing attention to and attracting scrutiny of their areas of corrupt activity. The amount of corruption that is uncovered by journalists and law enforcement agencies may not bear any stable relationship to the total amount of corruption in existence. Documented cases could represent the proverbial tip of the iceberg or a healthy percentage of the total sum, depending on the sophistication of the actors involved and the strength of the monitoring and enforcement efforts in play in the jurisdiction in question. Such data can even lead to misrepresentation about who is corrupt - for example, politically motivated allegations with no factual grounding published in state-controlled media and tried by judges for hire. Allegations of corruption may be a way for unsuccessful parties to save face and avoid responsibility for failure. They may also be politically charged in the sense of opposition parties and dissatisfied citizens having a natural human motivation to discredit adversaries, suspect the worst about those who oppose them or who have simply failed to take them into account. Moreover, as noted above, the perceptions of ordinary citizens, government officials and economic actors need not be especially accurate either. Finally, measurements of corruption are particularly sensitive to the definitions of corruption employed. For example, if political corruption is narrowly defined as a quid pro quo, involving the exchange of a tangible item of value for a particular political action or omission, the level of corruption in effect would be clearly lower than if corruption was defined more broadly in terms of trading in influence, undue influence of party and campaign donors, or the dependence of parties and candidates on such donors.

One lesson that emerges from diverse efforts to measure corruption is that those who read such content must make every effort to employ their critical thinking skills. Another lesson is that experience-based corruption measurements provide more valuable information than the perception-based tools. They provide systematic and comprehensive evidence which we may use as a basis for further investigation or policymaking purposes. A third lesson is that considering a variety of corruption measurements instead of just one or two certainly provides a better basis for approximating the truth. Finally, given the negative impact of corruption on most of humanity's concerns, it is also important to contemplate what lies at the opposite end of the spectrum. What positive goals are bound up in the struggle against corruption? Of particular relevance are notions of integrity (including personal, political, economic and organizational integrity), virtue, justice, peace, prosperity, citizen empowerment and satisfaction, and, ultimately, human flourishing.

 

Next: References

Back to top