106 C.C. Greenberg and J.B. Dimick

Intellectual Disciplines and Research Tools

Policy Evaluation

There are numerous national, state, and local health care policy changes each year that influence surgical practice. Oftentimes, the impact of these policies on surgical outcomes and costs are not adequately evaluated. As just one example, the Center for Medicare and Medicaid Services (CMS) issued a national coverage decision for bariatric surgery in 2006. CMS ruled that it would only reimburse for bariatric surgery performed in a Center of Excellence, as defined by criteria set forth by professional organizations. The impact of this policy is unclear. Did it improve outcomes for bariatric patients in the Medicare population? Were there spillover effects, with improved outcomes in younger patients? Rigorous policy evaluation research can provide these answers.

One recent example of policy evaluation was conducted to evaluate pay-for-performance in the national Medicare population. Using the national Medicare database, Ryan published in Health Services Research evaluated the impact of the Premier Hospital Incentive Demonstration.15 This Medicare demonstration project provided up to 2% bonuses to hospitals performing in the top decile on a composite measure of process and outcome. Two surgical procedures were included, coronary artery bypass and hip replacement. Ryan used sophisticated econometric techniques, known as “difference- in-difference” analysis, to adjust for secular trends in outcomes. This technique turned out to be important. A naïve analysis that simply looked at mortality before and after the implementation of the pay-for-performance program would have shown a significant reduction in mortality and Medicare payments. However, after accounting for preexisting trends using the methods described above, pay-for-performance had no impact on mortality or payments. This study dampened enthusiasm for this approach and caused policymakers to rethink how they construct incentives in pay-for-performance programs.

Chapter 7. Health Services Research

107

Surgical Workforce

There is perhaps no policy issue with more divergent opinions than surgical workforce.16 On one hand, many educators and clinical leaders believe there is a severe looming shortage of surgeons. Evidence in favor of this position includes an aging population, increasing rates of surgical procedures, and the decline in interest among medical students of surgical residency. On the other hand, leading policy experts believe there is more of a geographic imbalance in the workforce, with relative shortages in rural areas. Evidence in favor of this argument comes from the Dartmouth Atlas of Health Care, which shows dramatic variability in the per capita surgeon workforce in the USA.16 Experts from Dartmouth argue that it’s difficult to focus on a 10% shortage when there are presently 50% differences in rates of per capita surgeons across regions of the USA. The key question, which remains unanswered, is “how many surgeons per capita is the right number?” This area of scientific study is incredibly important but understudied, especially among surgeon-scientists.

It is necessary to have a working knowledge of health policy, economics, econometrics, and sociology/anthropology to conduct influential research in the health care delivery system. There are also a variety of research tools necessary to conduct research in the health care delivery system (Table 7.2). Econometrics provides key tools for evaluating policies in large databases, including panel data analysis, which provides many techniques for dealing with confounding, including the difference-in-difference approach, fixed effects regression, and instrumental variable analysis. Each of these provides sophisticated analytic tools for accounting for observed and unobserved confounding factors, such as secular trends toward improvement or differences in baseline hospital performance.At the other end of the spectrum, there are qualitative techniques from sociology/anthropology that help us understand the impact of these macro-system factors on individuals within the system. The mechanisms underlying intended and unintended consequences of changes in the