Making sense of policy evaluations for state policymakers

Policymakers, regulators, and other stakeholders need reliable evidence about which state policies are truly effective to help them make informed decisions that affect people’s health. Published scientific studies that evaluate the impact of state policies can answer important questions about policy effectiveness. For example: “Did state-level Medicaid expansion lead to more people getting substance use treatment?” or “Do naloxone access laws help to reduce fatal opioid overdoses?”

 

However, accurately determining the extent to which specific outcomes changed because of a policy is fundamentally challenging. Not all policy evaluation studies are equally rigorous—for example, some have major flaws in their analytic approach or limitations due to the data used. Findings from rigorous, high-quality evaluation studies are more trustworthy and provide a more reliable foundation for decision-making.

 

This Perspective – written by Bloomberg Professor of American Health in Evidence Dr. Stuart and colleagues at RAND -- is intended to help policymakers and other stakeholders assess policy evaluation studies, with a focus on health policy studies at the state level. They offer a few rules of thumb to identify policy evaluation studies that provide stronger evidence about whether there is a cause and effect relationship between a policy and an outcome. They highlight aspects of a study’s design and data measures to help assess how much to trust study findings regarding policy effectiveness. They also highlight some red flags indicating that a study’s findings should be viewed with caution. Although the primary motivation and application relates to policies related to addiction and overdose the lessons are broadly relevant for other Initiative areas as well, and for evaluation of policies at other (e.g., local) levels.

Learn more here: Policy Evaluation Guide