Benedikt Koch (Applied Stats Workshop)

Speaker & Title

Benedikt Koch (Harvard), "Statistical Decision Theory with Counterfactual Loss"

Abstract

A good outcome does not necessarily imply a good decision. For example, forcibly relocating a family from a disadvantaged neighborhood to improve children's employment prospects may be unjustified if the same result could have been achieved had the family been allowed to stay. Yet classical statistical decision theory, widely used to evaluate treatment choices and learn optimal policies, typically relies on losses that assess decisions based on realized outcomes alone. These standard losses leave such counterfactual considerations unaccounted for. Counterfactual losses incorporate them by evaluating decisions based on the joint vector of potential outcomes. The resulting risks, however, are generally unidentified.

We study policy selection under counterfactual risk based on observables. We illustrate how ambiguity criteria such as minimax, minimax regret and Pareto can lead to undesirable decision behavior. To avoid relying on such criteria, we study identification of counterfactual risk. We show that, under weak unconfoundedness, counterfactual risk is identifiable if and only if the loss is additive in the potential outcomes. Although more restrictive, additive losses can still express counterfactual preferences. They can capture both decision accuracy and difficulty through counterfactual regret, while standard losses reflect only accuracy. Furthermore, with three or more treatment options, additive losses can induce policy rankings that no fixed standard loss can reproduce. We characterize support restrictions under which every counterfactual loss admits an additive representation and provide a symbolic algorithm for verifying additivity. Finally, in an application to a food assistance program in Mexico, we illustrate counterfactual policy selection that balances paternalistic goals with freedom of choice.