Health systems strengthening:

How can it be monitored?

Von Carla AbouZahr / Weltgesundheitsorganisation WHO

Recent increases in funding for health and growing attention to the importance of health systems, are accompanied by demands for better statistics to track health-system performance and ensure accountability. Currently, few low and middle income countries are able to produce data of sufficient quality to permit such regular monitoring. A framework and toolkit to facilitate such monitoring is under development.

There is growing interest in the potential of incentive mechanisms, such as conditional cash transfers and performance based funding, for increasing the quality, coverage and use of priority health interventions. Performance-based funding mechanisms can only work with better data. Sound data are needed to identify appropriate performance indicators, to track progress towards benchmarks and targets, to monitor both intended and unintended effects, and to manage potential conflicts of interest which financial incentives may aggravate. The implementation of performance based mechanisms requires attention to the selection of performance metrics and identification of related measurement strategies, particularly in terms of health system functioning and performance. Yet currently, health information and statistical systems in many developing countries are not able to generate the kind of timely and reliable statistics needed for monitoring health systems. For example, there are multiple problems in clinic and programme-based reporting systems which often form the basis for the assessment of performance against benchmarks.

The need to develop better metrics of health system performance was first addressed by the World Health Organization in its 2000 World Health Report (World Health Report 2000). The framework for health system performance covered the range of indicators from health system inputs and processes (human resources, infrastructure, essential medicines etc.), through programme output measures (coverage of interventions) to health outcome or impact (mortality, morbidity, financial contribution, and responsiveness). The report did not attempt to quantify the full range of indicators, but ranked country health system performance using five outcome level indicators: population health, inequalities in health status, level and distribution of responsiveness and distribution of financial burden. The country rankings led to considerable debate and distracted attention from the value of the unifying framework for monitoring health systems.

Not until 2007 did WHO returned to the issue of monitoring health systems with the publication Everybody's Business (WHO 2007). Its health system framework was similar in terms of overall components to the earlier framework but paid more attention to the health system inputs and processes, summarized as six core building blocks of health systems: service delivery, health workforce, information, medical products, vaccines and technologies, financing, and leadership and governance.

The relevance of this effort has become ever more apparent with the growing attention of the health and development community on the health-system constraints that impede the attainment of disease- or programme-focused goals. Health-system strengthening is now recognized as central to strategies for accelerating the achievement of the MDGs – as noted by, for example, the Toyako Framework for Action at the G8 Summit. The International Health Partnership Plus (IHP+) promotes support to national health strategies and plans (including strengthened information systems for monitoring and evaluation) as a means of increasing alignment on national priorities. At the same time, virtually all health initiatives and funds call for better information in order to effectively steer and track health-systems strengthening efforts.

Feasible and realistic measurement methods

These developments provide a welcome rebalancing in terms of the contribution of "vertical" and "horizontal" approaches to health improvement. But from a monitoring perspective, they present a challenge in terms of the kinds of measurement strategies for health systems that would be commensurate with those used by categorical, or disease programmes. Health systems are complex; measuring and monitoring their performance will need to take this complexity into account.

Initial responses to these challenges started as long ago as 2004, when several global health initiatives, notably the GAVI Alliance and the Global Fund to fight AIDS, TB and Malaria, realized the need to tackle the constraints to the achievement of their own goals presented by weak country health systems. In 2004, WHO and the World Bank convened a series of meetings, bringing together country experts, academics, donors, funds and development partners, around a joint effort to improve health systems monitoring. The focus of the effort was on low and lower-middle income countries. The aim was not simply to define long lists of health system indicators and leave it at that. Rather, the strategy was to reach consensus on a parsimonious set of indicators for each health system building block and to identify feasible and realistic measurement methods for each indicator. It was anticipated that the effort would help countries establish their own benchmarks and targets, and that progress would be reflected in country progress reports or "dashboard". The dashboard could serve as a powerful advocacy tool for health system investments.

Arising out of this work is a draft Toolkit on monitoring health systems strengthening (WHO 2008).
The Toolkit describes a set of indicators, measurement approaches and strategies that


  • permit establishment of country health system statistical profiles;
  • permit monitoring of health systems and guide country and partner investments;
  • highlight gaps in terms of data availability and quality and point to needed investments in measurement strategies;
  • contribute to a global consensus around how to monitor and benchmark health systems strengthening. (see Table 1).
Health system building block Indicators Data sources
Governance Policy index (index to score whether certain policies are in place)
Implementation indicators for building blocks
CPIA and CPIA score for health
Expert review
Key informant surveys
Uses all data sources from other building blocks
Financing Total health expenditure per capita
% General Government expenditure on health
Out of pocket as % of total health expenditure
Total health expenditure as % of GDP
National Health Account
Public expenditure reviews
Expenditure reporting systems
Household surveys
Workforce Health care professionals per 10,000 population with distribution within country
Graduates of health training institutions per 100,000 population
Facility assessments
Administrative data
Professional society databases
Information Health Information System Performance Index (HISPIX) Expert review
Medical products Tracer drug availability in health facilities
Median drug price ratio for tracer drugs
Facility assessments
Facility reporting systems
Service delivery Health facilities per 10,000 population
Inpatient beds per 10,000 population
OPD visits per 100 population per year
General service capacity – proportion of facilities with basics such as clean water, basic equipment
Specific service capacity – proportion of facilities with key components for IMCI, FP, AIDS etc.
Facility assessments
Facility reporting systems

In working with countries to measure and compare health systems functioning, it is important to strike a good balance between avoiding blueprints that do not allow for country contexts and specificities, while simultaneously encouraging a degree of standardization that enables comparisons within and between countries and over time. Standardized indicators allow comparisons between countries and can help mutual learning, including identification of bottlenecks and sharing lessons learnt. However, measurement should be attuned to a country's health strategy objectives. Thus, each toolkit section has proposed "core indicators" that all countries are encouraged to collect and a wider set of indicators that users can chose or modify as needed. Countries should integrate new indicators with existing indicators of their health sector and statistical strategies and plans.

Poor availability and quality of health systems data

Following the development of the Toolkit, WHO and other partners collaborated with country partners to populate the indicators and generate the country dashboard with regular statistics for a minimum set of country health system indicators. The aim was to bring together data that would be comparable over time and between countries, and to include both national and sub-national or distributional data. In practice, the country dashboard proved to be more difficult than anticipated because of the poor availability and quality of health systems data.

A recent review of current health system monitoring practices in selected countries – Burkina Faso, Cambodia, Ghana, Kenya, Tanzania and Zambia – identified opportunities, challenges and best practices relevant to each building block (Monitoring health systems strengthening) Major concerns are evident regarding the availability and quality of data. Growing demand for results in the context of the MDGs has led to substantial investments in population-based health surveys to improve the availability and quality of health statistics on coverage of interventions, risk factors, nutrition, disease prevalence and mortality. By contrast, only fragmented efforts have been made to improve data on the health system building blocks. Yet both countries and development partners have a common interest in generating international consensus on comparable methods and measures for monitoring health systems strengthening.

In fact, countries are themselves taking the lead in implementing strategies to monitor health systems performance. Several high income countries are conducting various forms of performance assessment, often focusing on benchmarking and comparisons with other countries. For example, the Organisation for Economic Cooperation and Development (OECD) has been studying ways of measuring the performance of different health systems and of using such information to support on-going health systems improvement since 2002 (OECD 2002). The 2008 Dutch Health Care Performance Report presents a broad picture based on 110 indicators and includes comparisons over time and between countries where possible (Westert et al, 2008). In the USA, the Commission on a High Performance Health System has developed a National Scorecard, summarizing U.S. health system performance on 37 core indicators and providing comparisons with other countries (The Commonwealth Fund 2008). Authorship of such reviews varies, from multilateral organizations such as OECD, to public health bodies (for example in the Netherlands) and to independent bodies (as in the USA).

Investing in appropriate measurement strategies

Cross country comparisons are possible when data availability and quality are relatively complete. However, this is not the case for many developing countries. Here, there is much to be gained from an examination of sub national variations in health system indicators. For example, the South Africa District Health Barometer monitors a selected set of socio-economic and health care indicators in order to highlight inequities in health resource allocation, inputs, outputs and outcomes as well as the efficiency of health processes between provinces and between districts (Health Systems Trust 2007).

These country experiences demonstrate that progress towards more effective and efficient monitoring of health systems strengthening and health systems performance is possible, given the necessary political commitment and resources. Countries have much to learn from each other. It is clear that the main impediment towards better monitoring of health systems is not the lack of indicators but the failure to invest in appropriate and sustainable measurement strategies, and the generally poor quality of available data.

What is now needed to support countries is to further refine and promote standardized approaches, tools and methods for generating and analyzing health systems data and to build institutional and individual capacities for the collection, analysis, sharing and use of such data. Progress in these areas will require the mobilization of technical and financial support for countries and other stakeholders to support the process of improved health-system monitoring. Any approach will also need to find the right balance between standardization and country specificity, and will require a set of specific outputs that will improve the monitoring and strengthening of health systems. WHO and other partners are now working closely with funds, foundations, development agencies and countries, especially those involved in the International Health Partnership, to implement such approaches and to share lessons learnt.

*Carla AbouZahr is a statistician and public health expert working at the World Health Organization, Geneva, in the Department of Health Statistics and Informatics. She is currently working with a number of countries and partners on strategies for monitoring health systems strengthening. She is responsible for the views in this paper which do not necessarily represent the decisions or the stated policy of WHO.

References

  • World Health Organization 2000. The World Health Report. Health systems: improving performance. The work of WHO. ISBN-13 9789241561983
  • World Health Organization (2007) Everybody's business - Strengthening health systems to improve health outcomes - WHO's framework for action. ISBN 978 92 4 1596'77
  • World Health Organization (2008) Toolkit on monitoring health systems strengthening. http://www.who.int/healthinfo/statistics/toolkit_hss/en/index.html
    accessed 20 January 2009
  • Monitoring health systems strengthening: country practices and impact assessment. Unpublished paper
  • OECD (2002) Measuring Up. Improving Health System Performance in Developing Countries.
  • Westert GP, Berg MJ van den, Koolman X, Verkleij H (editors) (2008) Dutch health care performance report 2008 http://www.rivm.nl/vtv/root/o33.html accessed 20 January 2009
  • The Commonwealth Fund (2008) Why Not the Best? Results from the National Scorecard on U.S. Health System Performance
  • Health Systems Trust (2007) The District Health Barometer 2006-7
    http://www.hst.org.za/publications/717
    accessed 20 January 2009