Abstract:

Regression discontinuity design (RDD) has gained traction across social science fields as a leading quasi-experimental strategy to assess policy impacts. Graphical representation is to a large extent responsible for this growing popularity, and it is now an integral part of any well-executed RDD study. The key graph in any RD paper plots the bivariate relationship between the outcome variable Y and running variable X, and is meant to convey a discontinuity or the lack thereof in the conditional expectation function as X crosses a policy threshold. A number of recommendations have been made on how to construct these graphs, but it is unclear what researchers should do in practice.
 
Since the success of a graphical technique ultimately lies in the correct subjective perception of the reader, we propose an experimental approach to evaluate competing RD graphical methods by aggregating readers' perceptions through crowdsourcing. We conduct a series of randomized experiments where we present RD plots based on data generating processes specified on datasets from published papers and created with different graphical parameters. We show each participant in the experiment a series of graphs, ask them to classify the existence of a discontinuity in the graphs, and pay them bonuses based on the number of correct classifications. We also study graphical representation in the regression kink design with an analogous approach.
 
In addition to documenting the merits and drawbacks of different graphical representation techniques, we also assess the accuracy of visual inference. We recruit both experts and nonexperts to participate in our study, and we compare their performances to those of popular econometric estimators, including those proposed by Imbens and Kalayanaraman (2012), Calonico, Cattaneo, and Titiunik (2014), and Armstrong and Kolesár (2018).  

Finally, in order to better understand visual inference, we collect data on 16 participants using eyetracker technology. We investigate the extent to which eye patterns are predictive of correct discontinuity classification in RD graphs.