TY - JOUR TI - Learning Curve: Progress in the Replication Crisis AU - Angrist, Noam AU - Cullen, Claire AU - Ainomugisha, Micheal AU - Bathena, Sai Pramod AU - Bergman, Peter AU - Crossley, Colin AU - Letsomo, Thato AU - Matsheng, Moitshepi AU - Panti, Rene Marlon AU - Sabarwal, Shwetlena AU - Sullivan, Tim T2 - AEA Papers and Proceedings AB - We present detailed monitoring data across a five-country randomized trial of phone-based targeted tutoring–one of the largest multicountry replication efforts in education to date. We study an approach shown to work in Botswana and replicated in India, Kenya, Nepal, the Philippines, and Uganda. While the existing literature often finds diminishing effects as proof-of-concept studies are replicated and scaled, we find the opposite: implementation fidelity (the degree of targeted educational instruction) improves across replications and over time. This demonstrates that replication is not intractable; rather, equipped with mechanisms to learn from experience, organizational “learning curves” can enable effective replication and scale-up. DA - 2023/05/01/ PY - 2023 DO - 10.1257/pandp.20231009 DP - DOI.org (Crossref) VL - 113 SP - 482 EP - 488 J2 - AEA Papers and Proceedings LA - en SN - 2574-0768, 2574-0776 ST - Learning Curve UR - https://pubs.aeaweb.org/doi/10.1257/pandp.20231009 Y2 - 2023/10/09/10:40:32 ER - TY - RPRT TI - Building Resilient Education Systems: Evidence from Large-Scale Randomized Trials in Five Countries AU - Angrist, Noam AU - Ainomugisha, Micheal AU - Bathena, Sai Pramod AU - Bergman, Peter AU - Crossley, Colin AU - Cullen, Claire AU - Letsomo, Thato AU - Matsheng, Moitshepi AU - Panti, Rene Marlon AU - Sabarwal, Shwetlena AU - Sullivan, Tim CY - Cambridge, MA DA - 2023/05// PY - 2023 DP - DOI.org (Crossref) SP - w31208 LA - en PB - National Bureau of Economic Research SN - w31208 ST - Building Resilient Education Systems UR - http://www.nber.org/papers/w31208.pdf Y2 - 2023/10/09/10:45:45 ER - TY - JOUR TI - Experimental evidence on learning using low-tech when school is out AU - Angrist, Noam AU - Bergman, Peter AU - Matsheng, Moitshepi T2 - Nature Human Behaviour DA - 2022/06/13/ PY - 2022 DO - 10.1038/s41562-022-01381-z DP - DOI.org (Crossref) VL - 6 IS - 7 SP - 941 EP - 950 J2 - Nat Hum Behav LA - en SN - 2397-3374 UR - https://www.nature.com/articles/s41562-022-01381-z Y2 - 2023/10/09/10:36:48 ER - TY - RPRT TI - School’s Out: Experimental Evidence on Limiting Learning Loss Using “Low-Tech” in a Pandemic AU - Angrist, Noam AU - Bergman, Peter AU - Matsheng, Moitshepi AB - Schools closed extensively during the COVID-19 pandemic and occur in other settings, such as teacher strikes and natural disasters. This paper provides some of the first experimental evidence on strategies to minimize learning loss when schools close. We run a randomized trial of low-technology interventions – SMS messages and phone calls – with parents to support their child. The combined treatment cost-effectively improves learning by 0.12 standard deviations. We develop remote assessment innovations, which show robust learning outcomes. Our findings have immediate policy relevance and long-run implications for the role of technology and parents as partial educational substitutes when schooling is disrupted. DA - 2020/12// PY - 2020 DP - National Bureau of Economic Research M3 - Working Paper PB - National Bureau of Economic Research SN - 28205 ST - School’s Out UR - https://www.nber.org/papers/w28205 Y2 - 2022/01/11/19:05:05 KW - ___working_potential_duplicate ER - TY - RPRT TI - Stemming learning loss during the pandemic: A rapid randomized trial of a low-tech intervention in Botswana AU - Angrist, Noam AU - Bergman, Peter AU - Brewster, Caton AU - Matsheng, Moitshepi AB - The COVID-19 pandemic has closed schools for over 1.6 billion children, with potentially longterm consequences. This paper provides some of the first experimental evidence on strategies to minimize the fallout of the pandemic on education outcomes. We evaluate two low-technology interventions to substitute schooling during this period: SMS text messages and direct phone calls. We conduct a rapid trial in Botswana to inform real-time policy responses collecting data at fourto six-week intervals. We present results from the first wave. We find early evidence that both interventions result in cost-effective learning gains of 0.16 to 0.29 standard deviations. This translates to a reduction in innumeracy of up to 52 percent. We show these results broadly hold with a series of robustness tests that account for differential attrition. We find increased parental engagement in their child’s education and more accurate parent perceptions of their child’s learning. In a second wave of the trial, we provide targeted instruction, customizing text messages to the child's learning level using data from the first wave. The low-tech interventions tested have immediate policy relevance and could have long-run implications for the role of technology and parents as substitutes or complements to the traditional education system. DA - 2020/// PY - 2020 DP - DOI.org (Crossref) LA - en ST - Stemming Learning Loss During the Pandemic UR - https://www.ssrn.com/abstract=3663098 Y2 - 2021/11/18/13:48:55 ER - TY - JOUR TI - Practical Lessons for Phone-Based Assessments of Learning AU - Angrist, Noam AU - Bergman, Peter AU - Evans, David K. AU - Hares, Susannah AU - Jukes, Matthew C. H. AU - Letsomo, Thato T2 - BMJ Global Health AB - School closures affecting more than 1.5 billion children are designed to prevent the spread of current public health risks from the COVID-19 pandemic, but they simultaneously introduce education risks as well as new, longer run health risks, via lost education. While some studies measure student involvement in educational activities during the crisis through phone-based surveys, the literature on assessing learning by phone is almost nonexistent, despite the fact that learning loss has major implications for school dropout and rising inequality. In this article, we draw on our pilot testing of phone-based assessments in Botswana, along with the existing literature on oral testing of reading and mathematics, to propose a series of preliminary principles to guide researchers and service providers as they try phone-based learning assessments. We provide guidance to help teams (1) ensure that children are not put at risk, (2) test the reliability and validity of phone-based measures, (3) use simple instructions and practice items to ensure the assessment is focused on the target skill, not general language and test-taking skills, (4) adapt the items from oral assessments that will be most effective in phone-based assessments, (5) keep assessments brief while still gathering meaningful learning data, (6) learn from the speed and confidence of responses, (7) use effective strategies to encourage respondents to pick up the phone, and (8) build rapport with adult caregivers and youth respondents. DA - 2020/07// PY - 2020 DO - 10.1136/bmjgh-2020-003030 DP - Zotero VL - 5 IS - 7 SP - 16 J2 - BMJ Glob Health LA - en SN - 2059-7908 UR - https://www.cgdev.org/sites/default/files/practical-lessons-phone-based-assessments-learning-revised-jul2020.pdf KW - ___working_potential_duplicate KW - health economics KW - health services research KW - public health KW - ⛔ No DOI found ER - TY - RPRT TI - Practical Lessons for Phone-Based Assessments of Learning AU - Angrist, Noam AU - Bergman, Peter AU - Evans, David K. AU - Hares, Susannah AU - Jukes, Matthew C. H. AU - Letsomo, Thato CY - Washington D.C. DA - 2020/07// PY - 2020 SP - 16 LA - en PB - Center for Global Development ER - TY - ELEC TI - Limiting Learning Loss using Phone-based Programming during Covid-19 in Botswana AU - Angrist, Noam AU - Bergman, Peter AU - Brewstar, Caton AU - Matsheng, Moitshepi T2 - The Abdul Latif Jameel Poverty Action Lab (J-PAL) AB - Working in Botswana, researchers rapidly evaluated a phone-based remote learning program aimed at keeping children engaged with math during the Covid-19 pandemic. Students who received weekly SMS messages and phone calls to review math exercises increased their math skills after twelve weeks, while students who received only SMS messages did not. DA - 2020/// PY - 2020 LA - en UR - https://www.povertyactionlab.org/evaluation/limiting-learning-loss-using-phone-based-programming-during-covid-19-botswana Y2 - 2022/06/25/21:49:32 ER - TY - JOUR TI - Stemming Learning Loss During the Pandemic: A Rapid Randomized Trial of a Low-Tech Intervention in Botswana AU - Angrist, Noam AU - Bergman, Peter AU - Brewster, Caton AU - Matsheng, Moitshepi T2 - SSRN Electronic Journal AB - The COVID-19 pandemic has closed schools for over 1.6 billion children, with potentially long-term consequences. This paper provides some of the first experimental evidence on strategies to minimize the fallout of the pandemic on education outcomes. We evaluate two low-technology interventions to substitute schooling during this period: SMS text messages and direct phone calls. We conduct a rapid trial in Botswana to inform real-time policy responses collecting data at four- to six-week intervals. We present results from the first wave. We find early evidence that both interventions result in cost-effective learning gains of 0.16 to 0.29 standard deviations. This translates to a reduction in innumeracy of up to 52 percent. We find increased parental engagement in their child’s education and more accurate parent perceptions of their child’s learning. In a second wave of the trial, we provide targeted instruction, customizing text messages to the child's learning level using data from the first wave. The low-tech interventions tested have immediate policy relevance and could have long-run implications for the role of technology and parents as substitutes or complements to the traditional education system. DA - 2020/// PY - 2020 DO - 10.2139/ssrn.3663098 DP - DOI.org (Crossref) J2 - SSRN Journal LA - en SN - 1556-5068 ST - Stemming Learning Loss During the Pandemic UR - https://www.ssrn.com/abstract=3663098 Y2 - 2020/08/12/15:36:02 KW - Education KW - Human Capital KW - Technology KW - _COVID_DEAA-List KW - __C:filed:1 KW - ___working_potential_duplicate ER - TY - JOUR TI - How behavioral science can empower parents to improve children's educational outcomes AU - Bergman, Peter T2 - Behavioral Science & Policy AB - Parents powerfully influence their children’s educational outcomes. Yet psychological and informational barriers impede parents’ ability to engage with their children in ways that improve outcomes: parents tend to have inflated perceptions of their children’s performance, which can deter them from taking helpful steps to effectively support their learning, and parenting is complex. Limited cognitive bandwidth for coping with complexities can steer parents’ attention away from actions that have long-term benefits for their children and toward actions yielding immediate returns. Poor school-to-parent communication and poverty exacerbate all of these problems. In this article, the author demonstrates how providing timely, actionable information to parents can lower these barriers and help parents engage with their children more productively from kindergarten through high school. Moreover, providing this information can improve educational outcomes at low cost. DA - 2019/// PY - 2019 DO - 10.1353/bsp.2019.0004 DP - DOI.org (Crossref) VL - 5 IS - 1 SP - 52 EP - 67 J2 - Behavioral Science & Policy LA - en SN - 2379-4615 UR - https://muse.jhu.edu/article/748230 Y2 - 2022/03/04/14:16:54 ER - TY - JOUR TI - Technology adoption in education: Usage, spillovers and student achievement AU - Bergman, Peter DA - 2016/// PY - 2016 DP - Google Scholar ST - Technology adoption in education KW - ⛔ No DOI found ER - TY - JOUR TI - Parent-Child Information Frictions and Human Capital Investment: Evidence from a Field Experiment AU - Bergman, Peter VL - 129 IS - 1 UR - https://www.journals.uchicago.edu/doi/abs/10.1086/711410?af=R Y2 - 2022/01/11/19:20:50 ER -