“Recently, Martin Ravallion passed away. Martin created the $1/day poverty line used by the World Bank. Few Australian economists have left such a profound legacy”, Labor MP in the Australian Parliament Andrew Leigh said on Twitter, adding that although we disagreed on the randomista issue, his interaction was always kind and considerate.
“We are witnessing a positive change toward a culture of experimenting with overcoming poverty and other development-related problems. Randomized controlled trials (RCTs) are one of the methods that can be used for this. They do not, however, merit the elevated status that advocates have granted them”, the MP said.
He added that this has profoundly impacted scholars, development organizations, philanthropists, and the broader development community. We require much more information than just the fact that one of the two evaluation schemes uses randomization to establish a confidence ranking of the two.
RCTs’ increasing popularity has been based on a hierarchy of methodologies that places RCTs at the top as the “gold standard.” This cannot withstand careful examination. An RCT does not compare counterfactual outcomes between treated and control units, despite frequent assertions to the contrary. The lack of systematic bias does not prove that a one-off RCT’s experimental error is smaller than that of comparative observational research. We are unable to know that. The RCT option does not have to reduce mean squared error among the practical approaches in any particular application (with a fixed budget for the evaluation). A little observational research may be more accurate than an RCT in the same situation if it can be conducted with a larger sample size.
There is still plenty of room for practical theory-driven observational studies. Yes, there is model uncertainty, but usually not to the extent that the randomizes belief. Additionally, when we observe RCTs in action, they deal with contamination, selective compliance, and miss-measurement issues. Then it becomes evident that the tool must make the same assumptions found in observational studies—beliefs that the randomized pledged to avoid—to address the issues we ask about poverty and the policies designed to combat it.
In contrast to experimentation utilizing observational studies, RCTs are also ethically debatable. Without evaluating the anticipated advantages of new knowledge given what is currently known, the ethical case against RCTs cannot be effectively assessed. The ex-ante case for purposefully withholding an intervention from those who need it and delivering it to some who do not, for the sake of learning, has to be given more consideration by review boards. Based on the constraints of current knowledge, there might be a strong argument in some circumstances, but it needs to be made convincingly and not just assumed.
