You can add location information to your Tweets, such as your city or precise location, from the web and via third-party applications. You always have the option to delete your Tweet location history. Learn more
New (replication) paper alert!
“Which prisoner reentry programs work? Replicating and extending analyses of three RCTs” by me, @chelsea_econ, @1DavePritchard, & @ar8roberts.
http://jenniferdoleac.com/wp-content/uploads/2019/10/Doleac_Temple_Pritchard_Roberts_Oct2019.pdf …
Thread to follow.pic.twitter.com/EZNrMc6pWa
In the course of writing a review article on desistance from crime (link below), I came across dozens of empirical papers where some features of the analysis made it difficult to interpret the estimates as causal. Many of these were RCTs. http://jenniferdoleac.com/wp-content/uploads/2019/02/Doleac_Desistance_Feb2019.pdf …
For this replication study, we contacted the authors of 9 of these RCTs where it seemed that at least deidentified data should be available. Just 3(!) were willing and able to share their data with us. This new study replicates and extends the analyses from those 3 studies.
We want to thank the authors who shared their data, for their help replicating their findings & for answering questions along the way. They provide stellar examples of how social science should work. We’re all in this together, in pursuit of the right answers.
First, we consider a study of Decide Your Time (DYT), a swift-certain-fair program for drug-involved offenders. The original study included endogenous controls. We removed those controls and use one (employment during the program) as an outcome. https://onlinelibrary.wiley.com/doi/abs/10.1111/1745-9133.12246 …pic.twitter.com/3w8cHxgoUM
The orig study concluded that DYT had no effect. Our results are qualitatively similar, mostly bc the RCT was underpowered. We interpret the large coeffs as suggestive evidence that DYT reduced recidivism & increased employment, but none of the effects are statistically signif.pic.twitter.com/IK0C0TKeCO
Second, we consider an RCT of two aftercare programs for ppl w substance use problems who were recently released from prison: Therapeutic Communities and Oxford Houses. https://journals.sagepub.com/doi/abs/10.1177/0022042614552019?journalCode=joda …pic.twitter.com/0Kl7ZUGgIc
The randomization produced unbalanced treatment/control groups, and the outcome data are a panel of (snapshot) survey responses. The orig analysis compared simple trends over time; we convert this to a difference-in-difference model to more clearly estimate the causal effects.
We also add individual fixed effects to adjust for survey non-response. Finally, we remove an endogenous “dose” variable controlling for amount of participation, and use participation (staying 30 days) as a first stage to measure TOT effects.
We find that Oxford Houses increase recidivism (days incarcerated), and that Therapeutic Communities reduced employment and earnings, with suggestive evidence that they also increased recidivism. These are very different from the original study's conclusions.pic.twitter.com/5dxsJiicIB
Finally, we consider a study of a comprehensive/holistic prisoner reentry program in Minnesota. It included case management for offenders before they were released, to prepare them for success after leaving prison. https://www.tandfonline.com/doi/abs/10.1080/1478601X.2013.850081 …pic.twitter.com/fCR4QTjApy
The original study was designed as an RCT, but a large share of those originally assigned to treatment/control were later deemed ineligible, & those individuals were dropped from the analysis. More people from the treatment grp were dropped, raising concerns about selection bias.
Ideally we would have re-analyzed the data with those individuals included (with orig treatment assignment as the treatment or as an IV for treatment). Unfortunately the data on these `non-compliers’ was not available.
In our extension, we use matching methods to refine the comparison group. We also remove controls that may have been outcomes of treatment. We kept age at release & release year to adjust for imbalances that may affect outcomes (experiment occurred before/during the recession).
Note that deciding which covariates to include was complicated; in some cases, different data (e.g. age at randomization) would have helped. But it’s likely that using the full sample (as originally randomized) would have avoided these dilemmas.
Our findings are generally in line with those of the original study, but we interpret these results with caution since we were not able to conduct standard ITT or TOT analyses based on the full sample.pic.twitter.com/lxr9ovXpcm
Another RCT of the MCORP program would be helpful — if it is truly having the effects we estimate, then this program should be replicated and expanded!
The paper also includes a brief guide to analyzing data from RCTs, power analyses showing the sample sizes that would have been needed for each study to detect meaningful effects (see table), & a discussion of our results in the context of the broader prisoner reentry literature.pic.twitter.com/07WoN2pEQi
This paper was written for the Program on Empirical Legal Studies (PELS) replication conference, & will be published in a special section of the International Review of Law and Economics. Big thanks to PELS organizers for giving us an excuse to write this paper! We learned a lot.
Twitter may be over capacity or experiencing a momentary hiccup. Try again or visit Twitter Status for more information.
