Item history (bjohas, Sat Aug 08 2020 21:56:31 GMT+0100 (BST))
TY  - JOUR 
TI  - Identifying the essential ingredients to literacy and numeracy improvement: Teacher professional development and coaching, student textbooks, and structured teachers’ guides 
AU  - Piper, Benjamin 
AU  - Simmons Zuilkowski, Stephanie 
AU  - Dubeck, Margaret 
AU  - Jepkemei, Evelyn 
AU  - King, Simon J. 
T2  - World Development 
AB  - Several rigorously evaluated programs have recently shown positive effects on early literacy and numeracy outcomes in developing countries. However, these programs have not been designed to evaluate which ingredients of the interventions are most essential to improve literacy outcomes. Policy makers therefore lack evidence as to whether program ingredients such as teacher professional development (PD), instructional coaching, learner materials, teachers’ guides, community support, or technology are required for program impact. The Kenya Primary Math and Reading Initiative was a randomized controlled trial that compared three treatment groups with speciﬁc ingredients and a control group. Using literacy and numeracy outcome measures for grades 1 and 2, we evaluated the beneﬁts of the following ingredients: (1) teacher PD and teacher instructional support and coaching; (2) revised student books in literacy and numeracy, at a 1:1 ratio, added to PD and instructional support; and (3) structured teacher lesson plans added to student books, PD, and instructional support. We found that two of the three combinations of ingredients had statistically signiﬁcant positive impacts on learning outcomes. The results showed that the third combination—PD, teacher instructional support and coaching, 1:1 student books, and structured teacher lesson plans—was most effective. A cost-effectiveness analysis on the ingredients showed that the option of PD and instructional support, 1:1 revised books, and teachers’ guides was the most expensive, but that the additional impact on learning made this the most cost-effective intervention. This study rigorously analyzes which ingredients for literacy and numeracy improvement would be most effective for overall impact, and suggests to policy makers that careful decisions regarding program ingredients will lead to more effectively designed and implemented interventions to improve learning in developing countries. 
DA  - 2018/06// 
PY  - 2018 
DO  - 10.1016/j.worlddev.2018.01.018 
DP  - DOI.org (Crossref) 
VL  - 106 
SP  - 324 
EP  - 336 
J2  - World Development 
LA  - en 
SN  - 0305750X 
ST  - Identifying the essential ingredients to literacy and numeracy improvement 
UR  - https://linkinghub.elsevier.com/retrieve/pii/S0305750X18300287 
Y2  - 2019/10/03/00:32:16 
N1  - <p><strong>Extracted Annotations (14/02/2020, 09:00:27)</strong></p> 
<p>"3.2. Sample and procedures" (<a href="zotero://open-pdf/groups/2405685/items/FWHAB347?page=5">Piper et al 2018:328</a>)</p> 
<p>"The impact evaluation of PRIMR used a three-stage stratified cluster design. Twenty-two school zones each within Bungoma and Machakos counties (i.e., 44 zones out of 72 total zones) were randomly selected and assigned to one of the three treatment groups or the control group. Sampling was done at the zone level because the government's Curriculum Support Officers are assigned to support whole zones of typically 12 to 20 schools, and the CSOs constituted the base of the instructional support system. Through a stepped implementation process, all of the zones— treatment and control—were receiving the full PRIMR intervention by 2015. With respect to the complex survey design, the zones were clusters of schools." (<a href="zotero://open-pdf/groups/2405685/items/FWHAB347?page=5">Piper et al 2018:328</a>)</p> 
<p>"The baseline assessment for the DFID PRIMR study sampled between 4 and 10 schools randomly from each of the 44 selected zones for a total of 171 randomly selected schools. The final sampling stage was selection of students within schools; students were stratified by grade and gender and five students per strata were randomly selected for each school. If less than five students were available, they were all selected. A total of 3309 grade 1 and 2 students were assessed in the March 2013 baseline study. Assessments were completed by a trained team of local assessors who had been conducting literacy and numeracy assessments with the same types of instruments (see below for further information) since 2007. Interrater reliability tests during training showed very high average rates of reliability—98.0% for mathematics, 95.5% for English, and 95.0% for Kiswahili. The three treatment groups received the DFID PRIMR program intervention between May 2013 and October 2014. Endline data were collected from 230 randomly selected schools (4566 students) in October 2014. The final analyses presented here include the rural zones in the research design, removing the two peri-urban zones in the two counties." (<a href="zotero://open-pdf/groups/2405685/items/FWHAB347?page=5">Piper et al 2018:328</a>)</p> 
<p>"3.3. Measures" (<a href="zotero://open-pdf/groups/2405685/items/FWHAB347?page=5">Piper et al 2018:328</a>)</p> 
<p>"The measures used in these analyses were drawn from the results of two instruments administered one-on-one to pupils— the Early Grade Reading Assessment, or EGRA (Gove &amp; Wetterberg, 2011) and the Early Grade Mathematics Assessment, or EGMA (Platas, Ketterlin-Gellar, Brombacher, &amp; Sitabkhan, 2014)—adapted locally. All assessments were piloted prior to use." (<a href="zotero://open-pdf/groups/2405685/items/FWHAB347?page=5">Piper et al 2018:328</a>)</p> 
<p>"At baseline (March 2013) and endline (October 2014), the EGRA, administered in English and Kiswahili, included measures of letter sound fluency, decoding fluency (nonwords), oral reading fluency, reading comprehension, and the proportion of children at the MoE benchmark; listening comprehension was tested in Kiswahili only (RTI International, 2015). The subcomponent and measure descriptions for English, Kiswahili, and mathematics can be found Table 1 below. The literacy assessments were equated across the data collection points (Albano &amp; Rodriguez, 2012). In the English assessment, the Cronbach's alphas averaged 0.90. The reliabilities were similar for Kiswahili, with an average of 0.92 and a range from 0.89 (nonword fluency and oral reading fluency) to 0.94 (listening comprehension). In the analyses presented in this paper, we use the average effect size in standard deviations across the instrument subcomponents for each language to present the overall effect of PRIMR on literacy in English and Kiswahili. The EGMA tool measured both conceptual and procedural numeracy, as described in Table 1. Specific areas included number identification, quantity discrimination, missing number patterns, word problems, addition fluency, and subtraction fluency. The overall Cronbach's alpha for the mathematics assessment was 0.90, with all subtasks having alphas of 0.89 or 0.90. As with literacy, we averaged the effect sizes in standard deviations across the six subcomponents of the numeracy instrument to obtain an overall measure of numeracy." (<a href="zotero://open-pdf/groups/2405685/items/FWHAB347?page=5">Piper et al 2018:328</a>)</p> 
<p>"3.4. Data analysis" (<a href="zotero://open-pdf/groups/2405685/items/FWHAB347?page=5">Piper et al 2018:328</a>)</p> 
<p>"In other USAID-funded PRIMR studies in literacy and numeracy (Piper, Ralaingita et al., 2016; Piper, Zuilkowski, Kwayumba, et 2016), differences-in-differences (DID) analytic methods were used to account for the fact that, while random selection and assignment were utilized in assigning zones to treatment, baseline equivalence among the various treatment and control groups was not always achieved on the key outcomes. In order to determine which identification strategy was required in this study, we compared student assessment results at the baseline." (<a href="zotero://open-pdf/groups/2405685/items/FWHAB347?page=5">Piper et al 2018:328</a>)</p> 
<p>"We utilized the differences-in-differences identification strategy to identify causal effects. The DID model differentiated among four sets of children, separating those children into randomly assigned treatment and control schools and by the assessment time, either the baseline in March 2013 or endline in October 2014. The models then separately compared each of the three treatment groups against the control. Given that previous results from DFID PRIMR showed that including control variables made little difference in the overall PRIMR impact estimates (Piper, Ralaingita et al., 2016; Piper, Zuilkowski, &amp; Ong'ele, 2016), we used a parsimonious model to estimate the effect of PRIMR using the svy commands in Stata. Sampling weights were applied to estimates and standard errors were adjusted to account for the three-stage stratified cluster sample design." (<a href="zotero://open-pdf/groups/2405685/items/FWHAB347?page=6">Piper et al 2018:329</a>)</p> 
<p>"PRIMR used a difference-in-difference method where the outcomes are measured for the control and treatment groups across time points." (<a href="zotero://open-pdf/groups/2405685/items/FWHAB347?page=6">Piper et al 2018:329</a>)</p> 
N1  - <p><strong>Extracted Annotations (14/02/2020, 09:00:27)</strong></p> 
<p>"3.2. Sample and procedures" (<a href="zotero://open-pdf/groups/2405685/items/FWHAB347?page=5">Piper et al 2018:328</a>)</p> 
<p>"The impact evaluation of PRIMR used a three-stage stratified cluster design. Twenty-two school zones each within Bungoma and Machakos counties (i.e., 44 zones out of 72 total zones) were randomly selected and assigned to one of the three treatment groups or the control group. Sampling was done at the zone level because the government's Curriculum Support Officers are assigned to support whole zones of typically 12 to 20 schools, and the CSOs constituted the base of the instructional support system. Through a stepped implementation process, all of the zones— treatment and control—were receiving the full PRIMR intervention by 2015. With respect to the complex survey design, the zones were clusters of schools." (<a href="zotero://open-pdf/groups/2405685/items/FWHAB347?page=5">Piper et al 2018:328</a>)</p> 
<p>"The baseline assessment for the DFID PRIMR study sampled between 4 and 10 schools randomly from each of the 44 selected zones for a total of 171 randomly selected schools. The final sampling stage was selection of students within schools; students were stratified by grade and gender and five students per strata were randomly selected for each school. If less than five students were available, they were all selected. A total of 3309 grade 1 and 2 students were assessed in the March 2013 baseline study. Assessments were completed by a trained team of local assessors who had been conducting literacy and numeracy assessments with the same types of instruments (see below for further information) since 2007. Interrater reliability tests during training showed very high average rates of reliability—98.0% for mathematics, 95.5% for English, and 95.0% for Kiswahili. The three treatment groups received the DFID PRIMR program intervention between May 2013 and October 2014. Endline data were collected from 230 randomly selected schools (4566 students) in October 2014. The final analyses presented here include the rural zones in the research design, removing the two peri-urban zones in the two counties." (<a href="zotero://open-pdf/groups/2405685/items/FWHAB347?page=5">Piper et al 2018:328</a>)</p> 
<p>"3.3. Measures" (<a href="zotero://open-pdf/groups/2405685/items/FWHAB347?page=5">Piper et al 2018:328</a>)</p> 
<p>"The measures used in these analyses were drawn from the results of two instruments administered one-on-one to pupils— the Early Grade Reading Assessment, or EGRA (Gove &amp; Wetterberg, 2011) and the Early Grade Mathematics Assessment, or EGMA (Platas, Ketterlin-Gellar, Brombacher, &amp; Sitabkhan, 2014)—adapted locally. All assessments were piloted prior to use." (<a href="zotero://open-pdf/groups/2405685/items/FWHAB347?page=5">Piper et al 2018:328</a>)</p> 
<p>"At baseline (March 2013) and endline (October 2014), the EGRA, administered in English and Kiswahili, included measures of letter sound fluency, decoding fluency (nonwords), oral reading fluency, reading comprehension, and the proportion of children at the MoE benchmark; listening comprehension was tested in Kiswahili only (RTI International, 2015). The subcomponent and measure descriptions for English, Kiswahili, and mathematics can be found Table 1 below. The literacy assessments were equated across the data collection points (Albano &amp; Rodriguez, 2012). In the English assessment, the Cronbach's alphas averaged 0.90. The reliabilities were similar for Kiswahili, with an average of 0.92 and a range from 0.89 (nonword fluency and oral reading fluency) to 0.94 (listening comprehension). In the analyses presented in this paper, we use the average effect size in standard deviations across the instrument subcomponents for each language to present the overall effect of PRIMR on literacy in English and Kiswahili. The EGMA tool measured both conceptual and procedural numeracy, as described in Table 1. Specific areas included number identification, quantity discrimination, missing number patterns, word problems, addition fluency, and subtraction fluency. The overall Cronbach's alpha for the mathematics assessment was 0.90, with all subtasks having alphas of 0.89 or 0.90. As with literacy, we averaged the effect sizes in standard deviations across the six subcomponents of the numeracy instrument to obtain an overall measure of numeracy." (<a href="zotero://open-pdf/groups/2405685/items/FWHAB347?page=5">Piper et al 2018:328</a>)</p> 
<p>"3.4. Data analysis" (<a href="zotero://open-pdf/groups/2405685/items/FWHAB347?page=5">Piper et al 2018:328</a>)</p> 
<p>"In other USAID-funded PRIMR studies in literacy and numeracy (Piper, Ralaingita et al., 2016; Piper, Zuilkowski, Kwayumba, et 2016), differences-in-differences (DID) analytic methods were used to account for the fact that, while random selection and assignment were utilized in assigning zones to treatment, baseline equivalence among the various treatment and control groups was not always achieved on the key outcomes. In order to determine which identification strategy was required in this study, we compared student assessment results at the baseline." (<a href="zotero://open-pdf/groups/2405685/items/FWHAB347?page=5">Piper et al 2018:328</a>)</p> 
<p>"We utilized the differences-in-differences identification strategy to identify causal effects. The DID model differentiated among four sets of children, separating those children into randomly assigned treatment and control schools and by the assessment time, either the baseline in March 2013 or endline in October 2014. The models then separately compared each of the three treatment groups against the control. Given that previous results from DFID PRIMR showed that including control variables made little difference in the overall PRIMR impact estimates (Piper, Ralaingita et al., 2016; Piper, Zuilkowski, &amp; Ong'ele, 2016), we used a parsimonious model to estimate the effect of PRIMR using the svy commands in Stata. Sampling weights were applied to estimates and standard errors were adjusted to account for the three-stage stratified cluster sample design." (<a href="zotero://open-pdf/groups/2405685/items/FWHAB347?page=6">Piper et al 2018:329</a>)</p> 
<p>"PRIMR used a difference-in-difference method where the outcomes are measured for the control and treatment groups across time points." (<a href="zotero://open-pdf/groups/2405685/items/FWHAB347?page=6">Piper et al 2018:329</a>)</p> 
N1  - <div><b>Item history (bjohas, Wed Jul 29 2020 12:26:53 GMT+0100 (BST))</b></div> 
<pre><p>TY&nbsp; - JOUR <br/>TI&nbsp; - Identifying the essential ingredients to literacy and numeracy improvement: Teacher professional development and coaching, student textbooks, and structured teachers’ guides <br/>AU&nbsp; - Piper, Benjamin <br/>AU&nbsp; - Simmons Zuilkowski, Stephanie <br/>AU&nbsp; - Dubeck, Margaret <br/>AU&nbsp; - Jepkemei, Evelyn <br/>AU&nbsp; - King, Simon J. <br/>T2&nbsp; - World Development <br/>AB&nbsp; - Several rigorously evaluated programs have recently shown positive effects on early literacy and numeracy outcomes in developing countries. However, these programs have not been designed to evaluate which ingredients of the interventions are most essential to improve literacy outcomes. Policy makers therefore lack evidence as to whether program ingredients such as teacher professional development (PD), instructional coaching, learner materials, teachers’ guides, community support, or technology are required for program impact. The Kenya Primary Math and Reading Initiative was a randomized controlled trial that compared three treatment groups with speciﬁc ingredients and a control group. Using literacy and numeracy outcome measures for grades 1 and 2, we evaluated the beneﬁts of the following ingredients: (1) teacher PD and teacher instructional support and coaching; (2) revised student books in literacy and numeracy, at a 1:1 ratio, added to PD and instructional support; and (3) structured teacher lesson plans added to student books, PD, and instructional support. We found that two of the three combinations of ingredients had statistically signiﬁcant positive impacts on learning outcomes. The results showed that the third combination—PD, teacher instructional support and coaching, 1:1 student books, and structured teacher lesson plans—was most effective. A cost-effectiveness analysis on the ingredients showed that the option of PD and instructional support, 1:1 revised books, and teachers’ guides was the most expensive, but that the additional impact on learning made this the most cost-effective intervention. This study rigorously analyzes which ingredients for literacy and numeracy improvement would be most effective for overall impact, and suggests to policy makers that careful decisions regarding program ingredients will lead to more effectively designed and implemented interventions to improve learning in developing countries. <br/>DA&nbsp; - 2018/06// <br/>PY&nbsp; - 2018 <br/>DO&nbsp; - 10.1016/j.worlddev.2018.01.018 <br/>DP&nbsp; - DOI.org (Crossref) <br/>VL&nbsp; - 106 <br/>SP&nbsp; - 324 <br/>EP&nbsp; - 336 <br/>J2&nbsp; - World Development <br/>LA&nbsp; - en <br/>SN&nbsp; - 0305750X <br/>ST&nbsp; - Identifying the essential ingredients to literacy and numeracy improvement <br/>UR&nbsp; - https://linkinghub.elsevier.com/retrieve/pii/S0305750X18300287 <br/>Y2&nbsp; - 2019/10/03/00:32:16 <br/>N1&nbsp; - &lt;p&gt;&lt;strong&gt;Extracted Annotations (14/02/2020, 09:00:27)&lt;/strong&gt;&lt;/p&gt; <br/>&lt;p&gt;&quot;3.2. Sample and procedures&quot; (&lt;a href=&quot;zotero://open-pdf/groups/2405685/items/FWHAB347?page=5&quot;&gt;Piper et al 2018:328&lt;/a&gt;)&lt;/p&gt; <br/>&lt;p&gt;&quot;The impact evaluation of PRIMR used a three-stage stratified cluster design. Twenty-two school zones each within Bungoma and Machakos counties (i.e., 44 zones out of 72 total zones) were randomly selected and assigned to one of the three treatment groups or the control group. Sampling was done at the zone level because the government&apos;s Curriculum Support Officers are assigned to support whole zones of typically 12 to 20 schools, and the CSOs constituted the base of the instructional support system. Through a stepped implementation process, all of the zones— treatment and control—were receiving the full PRIMR intervention by 2015. With respect to the complex survey design, the zones were clusters of schools.&quot; (&lt;a href=&quot;zotero://open-pdf/groups/2405685/items/FWHAB347?page=5&quot;&gt;Piper et al 2018:328&lt;/a&gt;)&lt;/p&gt; <br/>&lt;p&gt;&quot;The baseline assessment for the DFID PRIMR study sampled between 4 and 10 schools randomly from each of the 44 selected zones for a total of 171 randomly selected schools. The final sampling stage was selection of students within schools; students were stratified by grade and gender and five students per strata were randomly selected for each school. If less than five students were available, they were all selected. A total of 3309 grade 1 and 2 students were assessed in the March 2013 baseline study. Assessments were completed by a trained team of local assessors who had been conducting literacy and numeracy assessments with the same types of instruments (see below for further information) since 2007. Interrater reliability tests during training showed very high average rates of reliability—98.0% for mathematics, 95.5% for English, and 95.0% for Kiswahili. The three treatment groups received the DFID PRIMR program intervention between May 2013 and October 2014. Endline data were collected from 230 randomly selected schools (4566 students) in October 2014. The final analyses presented here include the rural zones in the research design, removing the two peri-urban zones in the two counties.&quot; (&lt;a href=&quot;zotero://open-pdf/groups/2405685/items/FWHAB347?page=5&quot;&gt;Piper et al 2018:328&lt;/a&gt;)&lt;/p&gt; <br/>&lt;p&gt;&quot;3.3. Measures&quot; (&lt;a href=&quot;zotero://open-pdf/groups/2405685/items/FWHAB347?page=5&quot;&gt;Piper et al 2018:328&lt;/a&gt;)&lt;/p&gt; <br/>&lt;p&gt;&quot;The measures used in these analyses were drawn from the results of two instruments administered one-on-one to pupils— the Early Grade Reading Assessment, or EGRA (Gove &amp;amp; Wetterberg, 2011) and the Early Grade Mathematics Assessment, or EGMA (Platas, Ketterlin-Gellar, Brombacher, &amp;amp; Sitabkhan, 2014)—adapted locally. All assessments were piloted prior to use.&quot; (&lt;a href=&quot;zotero://open-pdf/groups/2405685/items/FWHAB347?page=5&quot;&gt;Piper et al 2018:328&lt;/a&gt;)&lt;/p&gt; <br/>&lt;p&gt;&quot;At baseline (March 2013) and endline (October 2014), the EGRA, administered in English and Kiswahili, included measures of letter sound fluency, decoding fluency (nonwords), oral reading fluency, reading comprehension, and the proportion of children at the MoE benchmark; listening comprehension was tested in Kiswahili only (RTI International, 2015). The subcomponent and measure descriptions for English, Kiswahili, and mathematics can be found Table 1 below. The literacy assessments were equated across the data collection points (Albano &amp;amp; Rodriguez, 2012). In the English assessment, the Cronbach&apos;s alphas averaged 0.90. The reliabilities were similar for Kiswahili, with an average of 0.92 and a range from 0.89 (nonword fluency and oral reading fluency) to 0.94 (listening comprehension). In the analyses presented in this paper, we use the average effect size in standard deviations across the instrument subcomponents for each language to present the overall effect of PRIMR on literacy in English and Kiswahili. The EGMA tool measured both conceptual and procedural numeracy, as described in Table 1. Specific areas included number identification, quantity discrimination, missing number patterns, word problems, addition fluency, and subtraction fluency. The overall Cronbach&apos;s alpha for the mathematics assessment was 0.90, with all subtasks having alphas of 0.89 or 0.90. As with literacy, we averaged the effect sizes in standard deviations across the six subcomponents of the numeracy instrument to obtain an overall measure of numeracy.&quot; (&lt;a href=&quot;zotero://open-pdf/groups/2405685/items/FWHAB347?page=5&quot;&gt;Piper et al 2018:328&lt;/a&gt;)&lt;/p&gt; <br/>&lt;p&gt;&quot;3.4. Data analysis&quot; (&lt;a href=&quot;zotero://open-pdf/groups/2405685/items/FWHAB347?page=5&quot;&gt;Piper et al 2018:328&lt;/a&gt;)&lt;/p&gt; <br/>&lt;p&gt;&quot;In other USAID-funded PRIMR studies in literacy and numeracy (Piper, Ralaingita et al., 2016; Piper, Zuilkowski, Kwayumba, et 2016), differences-in-differences (DID) analytic methods were used to account for the fact that, while random selection and assignment were utilized in assigning zones to treatment, baseline equivalence among the various treatment and control groups was not always achieved on the key outcomes. In order to determine which identification strategy was required in this study, we compared student assessment results at the baseline.&quot; (&lt;a href=&quot;zotero://open-pdf/groups/2405685/items/FWHAB347?page=5&quot;&gt;Piper et al 2018:328&lt;/a&gt;)&lt;/p&gt; <br/>&lt;p&gt;&quot;We utilized the differences-in-differences identification strategy to identify causal effects. The DID model differentiated among four sets of children, separating those children into randomly assigned treatment and control schools and by the assessment time, either the baseline in March 2013 or endline in October 2014. The models then separately compared each of the three treatment groups against the control. Given that previous results from DFID PRIMR showed that including control variables made little difference in the overall PRIMR impact estimates (Piper, Ralaingita et al., 2016; Piper, Zuilkowski, &amp;amp; Ong&apos;ele, 2016), we used a parsimonious model to estimate the effect of PRIMR using the svy commands in Stata. Sampling weights were applied to estimates and standard errors were adjusted to account for the three-stage stratified cluster sample design.&quot; (&lt;a href=&quot;zotero://open-pdf/groups/2405685/items/FWHAB347?page=6&quot;&gt;Piper et al 2018:329&lt;/a&gt;)&lt;/p&gt; <br/>&lt;p&gt;&quot;PRIMR used a difference-in-difference method where the outcomes are measured for the control and treatment groups across time points.&quot; (&lt;a href=&quot;zotero://open-pdf/groups/2405685/items/FWHAB347?page=6&quot;&gt;Piper et al 2018:329&lt;/a&gt;)&lt;/p&gt; <br/>KW&nbsp; - Africa <br/>KW&nbsp; - Literacy <br/>KW&nbsp; - numeracy <br/>KW&nbsp; - Program Evaluation <br/>KW&nbsp; - Quantitative <br/>KW&nbsp; - randomized control trial <br/>KW&nbsp; - Teacher Professional Development <br/>ER&nbsp; -  <br/> <br/>TY&nbsp; - JOUR <br/>TI&nbsp; - Identifying the essential ingredients to literacy and numeracy improvement: Teacher professional development and coaching, student textbooks, and structured teachers’ guides <br/>AU&nbsp; - Piper, Benjamin <br/>AU&nbsp; - Simmons Zuilkowski, Stephanie <br/>AU&nbsp; - Dubeck, Margaret <br/>AU&nbsp; - Jepkemei, Evelyn <br/>AU&nbsp; - King, Simon J. <br/>T2&nbsp; - World Development <br/>AB&nbsp; - Several rigorously evaluated programs have recently shown positive effects on early literacy and numeracy outcomes in developing countries. However, these programs have not been designed to evaluate which ingredients of the interventions are most essential to improve literacy outcomes. Policy makers therefore lack evidence as to whether program ingredients such as teacher professional development (PD), instructional coaching, learner materials, teachers’ guides, community support, or technology are required for program impact. The Kenya Primary Math and Reading Initiative was a randomized controlled trial that compared three treatment groups with speciﬁc ingredients and a control group. Using literacy and numeracy outcome measures for grades 1 and 2, we evaluated the beneﬁts of the following ingredients: (1) teacher PD and teacher instructional support and coaching; (2) revised student books in literacy and numeracy, at a 1:1 ratio, added to PD and instructional support; and (3) structured teacher lesson plans added to student books, PD, and instructional support. We found that two of the three combinations of ingredients had statistically signiﬁcant positive impacts on learning outcomes. The results showed that the third combination—PD, teacher instructional support and coaching, 1:1 student books, and structured teacher lesson plans—was most effective. A cost-effectiveness analysis on the ingredients showed that the option of PD and instructional support, 1:1 revised books, and teachers’ guides was the most expensive, but that the additional impact on learning made this the most cost-effective intervention. This study rigorously analyzes which ingredients for literacy and numeracy improvement would be most effective for overall impact, and suggests to policy makers that careful decisions regarding program ingredients will lead to more effectively designed and implemented interventions to improve learning in developing countries. <br/>DA&nbsp; - 2018/06// <br/>PY&nbsp; - 2018 <br/>DO&nbsp; - 10.1016/j.worlddev.2018.01.018 <br/>DP&nbsp; - DOI.org (Crossref) <br/>VL&nbsp; - 106 <br/>SP&nbsp; - 324 <br/>EP&nbsp; - 336 <br/>J2&nbsp; - World Development <br/>LA&nbsp; - en <br/>SN&nbsp; - 0305750X <br/>ST&nbsp; - Identifying the essential ingredients to literacy and numeracy improvement <br/>UR&nbsp; - https://linkinghub.elsevier.com/retrieve/pii/S0305750X18300287 <br/>Y2&nbsp; - 2019/10/03/00:32:16 <br/>N1&nbsp; - &lt;p&gt;&lt;strong&gt;Extracted Annotations (14/02/2020, 09:00:27)&lt;/strong&gt;&lt;/p&gt; <br/>&lt;p&gt;&quot;3.2. Sample and procedures&quot; (&lt;a href=&quot;zotero://open-pdf/groups/2405685/items/FWHAB347?page=5&quot;&gt;Piper et al 2018:328&lt;/a&gt;)&lt;/p&gt; <br/>&lt;p&gt;&quot;The impact evaluation of PRIMR used a three-stage stratified cluster design. Twenty-two school zones each within Bungoma and Machakos counties (i.e., 44 zones out of 72 total zones) were randomly selected and assigned to one of the three treatment groups or the control group. Sampling was done at the zone level because the government&apos;s Curriculum Support Officers are assigned to support whole zones of typically 12 to 20 schools, and the CSOs constituted the base of the instructional support system. Through a stepped implementation process, all of the zones— treatment and control—were receiving the full PRIMR intervention by 2015. With respect to the complex survey design, the zones were clusters of schools.&quot; (&lt;a href=&quot;zotero://open-pdf/groups/2405685/items/FWHAB347?page=5&quot;&gt;Piper et al 2018:328&lt;/a&gt;)&lt;/p&gt; <br/>&lt;p&gt;&quot;The baseline assessment for the DFID PRIMR study sampled between 4 and 10 schools randomly from each of the 44 selected zones for a total of 171 randomly selected schools. The final sampling stage was selection of students within schools; students were stratified by grade and gender and five students per strata were randomly selected for each school. If less than five students were available, they were all selected. A total of 3309 grade 1 and 2 students were assessed in the March 2013 baseline study. Assessments were completed by a trained team of local assessors who had been conducting literacy and numeracy assessments with the same types of instruments (see below for further information) since 2007. Interrater reliability tests during training showed very high average rates of reliability—98.0% for mathematics, 95.5% for English, and 95.0% for Kiswahili. The three treatment groups received the DFID PRIMR program intervention between May 2013 and October 2014. Endline data were collected from 230 randomly selected schools (4566 students) in October 2014. The final analyses presented here include the rural zones in the research design, removing the two peri-urban zones in the two counties.&quot; (&lt;a href=&quot;zotero://open-pdf/groups/2405685/items/FWHAB347?page=5&quot;&gt;Piper et al 2018:328&lt;/a&gt;)&lt;/p&gt; <br/>&lt;p&gt;&quot;3.3. Measures&quot; (&lt;a href=&quot;zotero://open-pdf/groups/2405685/items/FWHAB347?page=5&quot;&gt;Piper et al 2018:328&lt;/a&gt;)&lt;/p&gt; <br/>&lt;p&gt;&quot;The measures used in these analyses were drawn from the results of two instruments administered one-on-one to pupils— the Early Grade Reading Assessment, or EGRA (Gove &amp;amp; Wetterberg, 2011) and the Early Grade Mathematics Assessment, or EGMA (Platas, Ketterlin-Gellar, Brombacher, &amp;amp; Sitabkhan, 2014)—adapted locally. All assessments were piloted prior to use.&quot; (&lt;a href=&quot;zotero://open-pdf/groups/2405685/items/FWHAB347?page=5&quot;&gt;Piper et al 2018:328&lt;/a&gt;)&lt;/p&gt; <br/>&lt;p&gt;&quot;At baseline (March 2013) and endline (October 2014), the EGRA, administered in English and Kiswahili, included measures of letter sound fluency, decoding fluency (nonwords), oral reading fluency, reading comprehension, and the proportion of children at the MoE benchmark; listening comprehension was tested in Kiswahili only (RTI International, 2015). The subcomponent and measure descriptions for English, Kiswahili, and mathematics can be found Table 1 below. The literacy assessments were equated across the data collection points (Albano &amp;amp; Rodriguez, 2012). In the English assessment, the Cronbach&apos;s alphas averaged 0.90. The reliabilities were similar for Kiswahili, with an average of 0.92 and a range from 0.89 (nonword fluency and oral reading fluency) to 0.94 (listening comprehension). In the analyses presented in this paper, we use the average effect size in standard deviations across the instrument subcomponents for each language to present the overall effect of PRIMR on literacy in English and Kiswahili. The EGMA tool measured both conceptual and procedural numeracy, as described in Table 1. Specific areas included number identification, quantity discrimination, missing number patterns, word problems, addition fluency, and subtraction fluency. The overall Cronbach&apos;s alpha for the mathematics assessment was 0.90, with all subtasks having alphas of 0.89 or 0.90. As with literacy, we averaged the effect sizes in standard deviations across the six subcomponents of the numeracy instrument to obtain an overall measure of numeracy.&quot; (&lt;a href=&quot;zotero://open-pdf/groups/2405685/items/FWHAB347?page=5&quot;&gt;Piper et al 2018:328&lt;/a&gt;)&lt;/p&gt; <br/>&lt;p&gt;&quot;3.4. Data analysis&quot; (&lt;a href=&quot;zotero://open-pdf/groups/2405685/items/FWHAB347?page=5&quot;&gt;Piper et al 2018:328&lt;/a&gt;)&lt;/p&gt; <br/>&lt;p&gt;&quot;In other USAID-funded PRIMR studies in literacy and numeracy (Piper, Ralaingita et al., 2016; Piper, Zuilkowski, Kwayumba, et 2016), differences-in-differences (DID) analytic methods were used to account for the fact that, while random selection and assignment were utilized in assigning zones to treatment, baseline equivalence among the various treatment and control groups was not always achieved on the key outcomes. In order to determine which identification strategy was required in this study, we compared student assessment results at the baseline.&quot; (&lt;a href=&quot;zotero://open-pdf/groups/2405685/items/FWHAB347?page=5&quot;&gt;Piper et al 2018:328&lt;/a&gt;)&lt;/p&gt; <br/>&lt;p&gt;&quot;We utilized the differences-in-differences identification strategy to identify causal effects. The DID model differentiated among four sets of children, separating those children into randomly assigned treatment and control schools and by the assessment time, either the baseline in March 2013 or endline in October 2014. The models then separately compared each of the three treatment groups against the control. Given that previous results from DFID PRIMR showed that including control variables made little difference in the overall PRIMR impact estimates (Piper, Ralaingita et al., 2016; Piper, Zuilkowski, &amp;amp; Ong&apos;ele, 2016), we used a parsimonious model to estimate the effect of PRIMR using the svy commands in Stata. Sampling weights were applied to estimates and standard errors were adjusted to account for the three-stage stratified cluster sample design.&quot; (&lt;a href=&quot;zotero://open-pdf/groups/2405685/items/FWHAB347?page=6&quot;&gt;Piper et al 2018:329&lt;/a&gt;)&lt;/p&gt; <br/>&lt;p&gt;&quot;PRIMR used a difference-in-difference method where the outcomes are measured for the control and treatment groups across time points.&quot; (&lt;a href=&quot;zotero://open-pdf/groups/2405685/items/FWHAB347?page=6&quot;&gt;Piper et al 2018:329&lt;/a&gt;)&lt;/p&gt; <br/>KW&nbsp; - Literacy <br/>KW&nbsp; - Africa <br/>KW&nbsp; - Program Evaluation <br/>KW&nbsp; - Teacher Professional Development <br/>KW&nbsp; - randomized control trial <br/>KW&nbsp; - numeracy <br/>KW&nbsp; - Quantitative <br/>ER&nbsp; -  <br/> <br/></p></pre> 
<div> 
<p>group:</td><td>2405685</p> 
<p>itemKey:</td><td>5LADY8ET</p> 
<p>itemKeyOld:</td><td>L36BGXMK</p> 
</div> 
KW  - Africa 
KW  - Literacy 
KW  - numeracy 
KW  - Program Evaluation 
KW  - Quantitative 
KW  - randomized control trial 
KW  - Teacher Professional Development 
ER  -  
 
TY  - JOUR 
TI  - Identifying the essential ingredients to literacy and numeracy improvement: Teacher professional development and coaching, student textbooks, and structured teachers’ guides 
AU  - Piper, Benjamin 
AU  - Simmons Zuilkowski, Stephanie 
AU  - Dubeck, Margaret 
AU  - Jepkemei, Evelyn 
AU  - King, Simon J. 
T2  - World Development 
AB  - Several rigorously evaluated programs have recently shown positive effects on early literacy and numeracy outcomes in developing countries. However, these programs have not been designed to evaluate which ingredients of the interventions are most essential to improve literacy outcomes. Policy makers therefore lack evidence as to whether program ingredients such as teacher professional development (PD), instructional coaching, learner materials, teachers’ guides, community support, or technology are required for program impact. The Kenya Primary Math and Reading Initiative was a randomized controlled trial that compared three treatment groups with specific ingredients and a control group. Using literacy and numeracy outcome measures for grades 1 and 2, we evaluated the benefits of the following ingredients: (1) teacher PD and teacher instructional support and coaching; (2) revised student books in literacy and numeracy, at a 1:1 ratio, added to PD and instructional support; and (3) structured teacher lesson plans added to student books, PD, and instructional support. We found that two of the three combinations of ingredients had statistically significant positive impacts on learning outcomes. The results showed that the third combination—PD, teacher instructional support and coaching, 1:1 student books, and structured teacher lesson plans—was most effective. A cost-effectiveness analysis on the ingredients showed that the option of PD and instructional support, 1:1 revised books, and teachers’ guides was the most expensive, but that the additional impact on learning made this the most cost-effective intervention. This study rigorously analyzes which ingredients for literacy and numeracy improvement would be most effective for overall impact, and suggests to policy makers that careful decisions regarding program ingredients will lead to more effectively designed and implemented interventions to improve learning in developing countries. 
DA  - 2018/06/01/ 
PY  - 2018 
DO  - 10.1016/j.worlddev.2018.01.018 
DP  - ScienceDirect 
VL  - 106 
SP  - 324 
EP  - 336 
J2  - World Development 
LA  - en 
SN  - 0305-750X 
ST  - Identifying the essential ingredients to literacy and numeracy improvement 
UR  - http://www.sciencedirect.com/science/article/pii/S0305750X18300287 
Y2  - 2020/08/05/16:12:11 
L1  - https://pdf.sciencedirectassets.com/271773/1-s2.0-S0305750X18X00023/1-s2.0-S0305750X18300287/main.pdf?X-Amz-Security-Token=IQoJb3JpZ2luX2VjEPj%2F%2F%2F%2F%2F%2F%2F%2F%2F%2FwEaCXVzLWVhc3QtMSJGMEQCIBvrwlM8hC6pa%2FfCGPMXN4bVo7ktx%2F0qgMW2QGvXuGhEAiBPHFy7bgrw1MG51BQY7JS3sLfxC%2FzIf%2BlH39aM5L6dUiq9AwjB%2F%2F%2F%2F%2F%2F%2F%2F%2F%2F8BEAMaDDA1OTAwMzU0Njg2NSIMHCAsPZh5ZvMZgyHdKpEDiLG6boDaAzSeN0xt9y4ZdsGK%2B2Q4MsvJda1Vro9xk8A4Y237pwCh%2B%2BNmNbkPyfthnsOVG24jdsxdbXgg2ftIKe8iDsVxFrxfMXe5NC5XO2%2BNAojsDx%2Bji2xYy4LF4huoO%2F%2B8oVpApVBIBikYpzyrBxBYTX2fWtA4zO8n6XB0nSD%2F%2FxKKfpzz4CxC%2BUbbtgt17MWjO1TbGpn6x5%2BWO9aStkOyPxirvTZl0%2FC4XwjCSHZtkj7VDX0qb1gOtcjCQo%2F%2F%2Fel9vTyw81I5j10RfXpOIV%2FtFnUNlfIkaQQVNfOyVMPb1gqiJVThIeDv5gkYAljNo%2BMWcdO5BtzmCmeJhyV36WclxZn1LDkgcNpzUY861zrCPBC24OPadUbedxHlDq5N4Jea3ogP2Ur94ug9Vm13CTmHSQ3mAsuxsvd0T5DDWlqzIehZvp6JIx3jY%2BzsZg%2B1aqoUtBx7M3VYUWaD9U6AYTivXh%2Fl%2B9nnNpXMEz2VjtgVYZaqBP1N4jkXFG1tX1AG1XuORn7tZDkCxXgDep%2Fuq3cw96Or%2BQU67AEFA7ls%2Bp4FhOPtiFSOtUqL66OfBmEakT7%2BI2xP%2B1%2Bgje3zF3wvrW8%2B1zZHkj%2FZSD0uFEOjIz8Ke0uHrW7ayrKF3bHvwjHhuyqrkuJ8WHnLQE6AAOL0zOL%2BKYZCvwn6yYlc4XbADRmKku%2FCkJ6MiUV1ycT4oY8zuhJzGDzGPRHPdltHcpaFAtb6h9N4hYg%2FUUpTwRVOnK7hJWroksDEeDJiImSBtnCGKvdka3Pe5hPlw7VRMCXRoe8S2P3AYB38GFc5MIxuHRoG5b5CSS4dHxheSr2OjCjusY3HnabUzNdfHF8YSg5FPNk12eh2AQ%3D%3D&X-Amz-Algorithm=AWS4-HMAC-SHA256&X-Amz-Date=20200805T161211Z&X-Amz-SignedHeaders=host&X-Amz-Expires=300&X-Amz-Credential=ASIAQ3PHCVTY6T56NUWN%2F20200805%2Fus-east-1%2Fs3%2Faws4_request&X-Amz-Signature=25b9ee6ea76d5794bf626352b96659a27a2540cf8f76e31267b66f8a78d3f754&hash=1cfaebd1e29d06e730c8e3b2e6c6cb73f41a7b7c1cd5aa24c55f2c81aaea77ea&host=68042c943591013ac2b2430a89b270f6af2c76d8dfd086a07176afe7c76c2c61&pii=S0305750X18300287&tid=spdf-8ce2585b-9559-480a-b12f-8fb01d52771f&sid=40393ca1116a5548f36aa3464eb18886c4ddgxrqb&type=client 
L2  - https://www.sciencedirect.com/science/article/pii/S0305750X18300287 
KW  - Literacy 
KW  - Africa 
KW  - Reading 
KW  - Numeracy 
KW  - Program evaluation 
KW  - Randomized controlled trial 
ER  -
group:2405685
itemKey:5LADY8ET
itemKeyOld:37II7RPP