Definition

Health metrics and evaluation is an emerging field used to evaluate overall health and treat quality effectiveness as well as determine resource allocation, program and system selection, and policy choices. Collecting and measuring health data provides a solid scientific foundation for decision makers to make educated, cost-effective choices regarding program quality and improvement.

In the past, the costliness of developing measurement tools and the inability to compare unique tools across projects limited much needed progress in this field. However, there is a real need to establish uniform health metrics, and doing so would serve a number purposes. Standardizing health metrics would:

• Provide valid, reliable and comparable information, at low cost and supplement the information provided by routine health information systems.
• Build evidence-base information for policy-makers to monitor health outcomes and assess whether investments are achieving desired outcomes.
• Provide policy-makers with necessary evidence to adjust policies, strategies and programs.

Note: The case studies put forth here will primarily be limited to the challenges of and solutions to measuring disease outcomes and management rather than global health policy choices and program implementation.

Relevance

Establishing common metrics will allow us to measure health outcomes of specific projects and to compare results across similar health improvement initiatives. However, institutional and cultural challenges have hindered the use of evidence-based metrics. This section outlines a number of relevant barriers:

• Insufficient funding limits the ability of national institutions to collect data either because data are not valued or inadequate funding produces poor data.
• Institutional entanglements limit the production of quality metrics. Conflicts of interest arise when institutions are responsible for advocating for funding, implementing programs, or providing technical assistance as well as monitoring and evaluating their progress.
• Replicability of data serves as another barrier to acceptance of health data. Too often scientific results cannot be replicated by other researchers because underlying data are not in the public domain and present a barrier for other scientists to conduct their own research.
• Perceived risk limits the collection and evaluation of health related data. Scientific findings often spark open debate and critique that can pose challenges to decision makers who want to know how their actions compare with institutions, communities, or nations in similar circumstances, but are also burdened with the choice of publically disclosing findings and opening themselves up to critique.

Context

Expanding and standardizing the use of health metrics will require policies and practices to address the barriers mentioned above.

• Insufficient funding. Sources for funding should be expanded beyond public dollars. Businesses and insurance payers have begun to recognize they have a vested interest in determining the efficacy and efficiency of health programs. The World Health Organization has established the Health Metrics Network (HMN), a global partnership to facilitate better health information at country, regional and global levels; and a business coalition of 50 purchasers founded the Pacific Business Group on Health (PBGH) to improve the quality and availability of health care while moderating cost.
• Institutional entanglements. Developing a network of third party researchers would eliminate institutional entanglements and improve data quality because third party researchers would: 1) not have a vested interest in a program’s success or failure; 2) serve as knowledge brokers for program developers and decision-makers; and 3) develop common metrics to compare data across projects.
• Replicability of data. Efforts by insurance payers and national institutions to mandate data transparency will facilitate replicability evaluations and measurements. For projects where privacy is an issue, safeguards can be implemented to ensure anonymity, such as grouping data by cohorts or communities.
• Perceived risk. Decision-makers must be re-educated to set aside personal concerns about success or failure and focus on the benefits of obtaining meaningful data.. Researchers like John Lavis at McMasters University argue that evaluative data should be presented to health care managers and policymakers in a way that is most useful to them. It should consider potential risks, costs, uncertainties, and any differential effects on population subgroups. Further, by presenting results in user-friendly formats like searchable online databases would allow managers or policymakers to rapidly assess relevance and access further research as needed.

Innovative Options - Health Metrics Research Institutions

In recent years, both emerging and developed countries have been under enormous pressure to improve healthcare performance and allocate resources as efficiently as possible. To facilitate this process a number of independent organizations have initiated their own outcome measurement projects as well as establishing organizations to support other countries’ efforts. Examples include:

Healthcare Productivity Study (October 1996)
McKinsey Global Institute
http://www.mckinsey.com/mgi/publications/healthcare.asp

This extensive study assessed the United Kingdom, United States and Germany’s health care systems to assess productivity levels at the disease level, the major source of these differences, and implications. The study identified common inputs for four common diseases (diabetes, gall stones, breast cancer, and lung cancer) and compared these metrics across countries. For example,

• The diabetes research found with better outcomes and fewer inputs, the U.K. was more productive than the U.S. in diabetes treatment. The U.K.'s productivity efficiency stemmed from its consistently lower complication rates.
• In breast cancer treatment, the U.S. had better productivity levels than the U.K. due in large part to greater frequency screening in the U.S. Germany's productive efficiency relative to the U.S. was lower due to longer hospital stays.

Counting drugs to understand the disease: The case of measuring
the diabetes epidemic
Henrik Støvring, et al. (Denmark)
http://www.pophealthmetrics.com/content/5/1/2

This project used pharmacoepidemiological databases –readily available in the Nordic countries – as a proxy for incidence rates. The researchers sought to determine if such databases could be used instead of costly population studies on the incidence rate and mortality of diabetes.

The researchers obtained data on all 470,000 inhabitants in Funen County, Denmark, in the period 1992–2003, including gender, date of birth, death and migration to and from the county, and any filled prescriptions of an anti-diabetic medication from the Odense Pharmaco-Epidemiological Database.

Researchers concluded that the pharmacoepidemiological databases provide a useful tool for monitoring pharmacologically treated diabetes (e.g. to obtain incident rates and severity measures); however, a dedicated diabetes database covering all prevalents and incidents is needed for a more detailed analysis of underlying causes and trends.

Institute for Health and Productivity Management (USA)
http://www.ihpm.org/info/background.php

The Institute for Health and Productivity Management was created in 1997 to make employee health an investment in corporate success through enhanced workplace performance. The effort is based on a joint effort by the National Business Coalition on Health and the National Association of Managed Care Physicians.

Employers' desire for value has led them to insist that costs be controlled while quality is measured and improved. But their concept of value has been limited mostly to the results of care being delivered to "patients" who happen to be their employees. A larger concept of value looks beyond just getting sick people well or even back to work. It expands the definition of value to include employee performance on the job, i.e. productivity. Health promotion and care delivery models organized to produce outcomes that have a positive impact on the bottom line of total labor costs--rather than just health care costs--get us on to the next higher level of value for the health benefit dollar.

Institute for Health Metrics and Evaluation
University of Washington
http://www.healthmetricsandevaluation.org/

The Institute for Health Metrics and Evaluation (IHME) is funded in part by a grant from the Bill and Melinda Gates Foundation and the State of Washington. IHME monitors global health conditions and health systems, as well as evaluates interventions, initiatives, and reforms. IHME provides high quality and timely information on health so that policymakers, researchers, donors, practitioners, local decision-makers, and others can better allocate limited resources to achieve optimal results. The Institute uses quantitative analysis and other analysis techniques to investigate: Health Outcomes; Health Services; Resource Inputs; Decision Analytics; and Evaluations.

Pacific Business Group on Health
http://www.pbgh.org/

The Pacific Business Group on Health (PBGH), a business coalition of 50 purchasers, seeks to improve the quality and availability of health care while moderating cost. Since 1989, PBGH has worked with state and national organizations to promote health care measurement, trend moderation, and system accountability through public reporting of data. PBGH seeks to 1) increase the availability and usability of quality and economic efficiency performance information for all levels of care: health plans, hospitals, medical groups and individual physicians and 2) identify high impact methods to improve performance and create market demand for adoption by plans and providers through effective value purchasing and consumer engagement efforts.

The Health Metrics Network (HMN)
World Health Organization
http://www.who.int/healthmetrics/en/

The Health Metrics Network (HMN) is a global partnership that facilitates better health information at country, regional and global levels. Partners include developing countries, multilateral and bilateral agencies, foundations, other global health partnerships and technical experts. HMN seeks to bring together health and statistical constituencies to build capacity and expertise and enhance the availability, quality, dissemination and use of data for decision-making.

Institute for Health Metrics (US)
http://www.healthmetrics.org/index.html

The Institute for Health Metrics is a privately funded not-for-profit organization focused on developing an electronic data analytics system to support quality and operational improvement in hospitals and research in public health and health care services.

IHM's goal is to become a national, collaborative organization working with and on behalf of community hospitals to improve the quality, safety and efficiency of health care. Using its unique ability to organize and analyze data from disparate hospital clinical information systems, IHM hopes to transform data into actionable insight that enables hospitals to understand and improve all aspects of their care processes.