Social Science Statistics Blog

I wonder if people won't care as much if few enough people are chosen to receive the benefits. If people see it as a kind of thing that you'd be really lucky to get -- not as any sort of entitlement or realistic expectation -- then they'd probably be a lot more apt to shrug and let it go than if, say, 50% of the people they knew got the benefit and they didn't. I'm totally guessing, but I think framing it in those terms might do something to avoid disgruntlement from building too much before the study is over.

If that weren't possible, one could also use as the control not doing nothing, but doing some other program whose efficacy is also uncertain. Then people would be assigned to one or the other randomly, and since it's obvious that nobody knows which is better, there would be no grounds for resentment. This has the disadvantage of having no true control (and costing more), but if the disgruntlement were a large enough problem then that might be outweighed by the advantages, especially since you might "approximate" a control by comparing your sample either to families before the benefit was introduced, or to families in a roughly-similar city with roughly-similar demographics. Obviously not ideal, but might be worth it in the balance.

But I really think it would probably not be a huge factor as long as the sample sizes were small enough.

Why not do what economists do? Create a lottery to balance out the other side. That is, for those who are not going to be in the experimental group, enter them in a lottery. That at least nets out the benefits of the control and experimental groups. One could use previously surveyed performance of the families as the baseline for setting the lottery. This at least incentivizes both sides in the experiment -- something that experimental economists do better and care more about than psychologists (and maybe political scientists). It also rationalizes the statistical manipulation, for the equity concerns of the political scientists.

I wonder if the "budgetary constraints" of pilot programs could be stressed more strongly in combination with a less precise control group of "losers". If the control group was much larger than the program group it might dilute some of the sense of unfairness and make it clear that the program can't be offered to everyone who participates. The perception is that it is unfair to the 2,500 who are surveyed but don't get anything out of it. No one seems to suggests that it's unfair to the other 6 billion people not offered the program. That would be patently silly rather than just silly.

Perhaps there could be a staggered pilot over several years with several waves, where each control group becomes the next wave's program group. That would probably reduce attrition rates in the control group as well.

Ryan's solution seems about right.

A study about music lessons and IQ gave free music lessons to one group and no lessons to the control group for one year. The next year, the control group got free lessons too.

http://scienceblogs.com/cognitivedaily/2005/06/music_and_iq.php

I agree with Ryan that using budget constraints is a good rhetorical device to persuade people that randomization is "fair" in some sense. Sendhil Mullainathan makes a similar argument about approaching organizations when you would like them to use randomization; if they are going to have to restrict access in some way, then why not randomize? I also like the lottery idea; it might allow you to distiguish between the income effect of the proposed program and the incentive effects. Or, you could just give everyone in the control group $5,000, which would really drive a wedge between the two effects.

Forgive me for chiming in here. I just stumbled across this post and I found the topic very interesting. Are they mirroring this project after the Low Income Mexican Health Program?
I would think that a better approach to the Anit Poverty would be keeping the parents involved and making them accountable for the attendance of their children instead of bribing them with tax dollars or am I missing somethng?