Full Text

Turn on search term navigation

Education Coordinating Group

Parental, Community, and Familial Support Interventions to Improve Childrens Literacy in Developing Countries

Elizabeth Spier, Pia Britto, Terri Pigott, Eugene Roehlkapartain, Michael McCarthy, Yael Kidron, Mengli Song, Peter Scales, Dan Wagner, Julia Lane, and Janis Glover

A Campbell Systematic Review 2016:04

First published: March 2016 Search executed: July 2013

The Campbell Library comprises: Systematic reviews (titles, protocols and reviews) Policies and Guidelines Methods Series

Go to the library to download these resources, at: www.campbellcollaboration.org/lib/

Better Evidence for a Better World

Colophon

Title Parental, Community, and Familial Support Interventions to Improve Childrens Literacy in Developing Countries: A Systematic Review

Authors Spier, Elizabeth Britto, Pia

Pigott, Terri Roehlkapartain, Eugene McCarthy, Michael Kidron, Yael

Song, Mengli

Scales, Peter

Wagner, Dan

Lane, Julia

Glover, Janis

DOI 10.4073/csr.2016.4

No. of pages 98

Citation Spier ET, Britto PR, Pigott T, Roehlkapartain E, McCarthy M, Kidron Y, Song M, Scales P, Wagner D, Lane J, Glover J. Parental, Community, and Familial Support Interventions to Improve Childrens Literacy in Developing Countries: A Systematic Review: A Systematic Review.

Campbell Systematic Reviews 2016:4

DOI: 10.4073/csr.2016.4

ISSN 1891-1803

Copyright Spier et al. This is an open-access article distributed under the terms of the Creative Commons Attribution License, which permits unrestricted use, distribution,and reproduction in any medium, provided the original author and source are credited.

Roles and responsibilities

The team was led by Principal Investigators Elizabeth Spier and Pia Britto. Eugene Roehlkepartain and Peter Scales provided expertise in parent, family, and community engagement to improve child outcomes. Terri Pigott, Yael Kidron, and Mengli Song guided the process of review and analysis. Janis Glover and Michael McCarthy developed and carried out the study search process. Dan Wagner provided expertise in literacy in multiple international contexts. Elizabeth Spier will be responsible for updating this review as additional evidence accumulates and as funding becomes available.

Editors for this review

Editor: Sandra Jo Wilson Managing Editor: Sean Grant

Sources of support This review was conducted under the auspices of 3ie and is funded by a grant provided by USAID. American Institutes for Research is also providing financial support to facilitate the completion of this work.

Declarations of interest

The authors have no vested interest in the outcomes of this review, nor any incentive to represent findings in a biased manner.

Corresponding author

Elizabeth Spier, PhD

American Institutes for Research 2800 Campus Drive

San Mateo, CA 94403

United States

E-mail: [email protected]

Campbell Systematic Reviews

Editor-in-Chief Julia Littell, Bryn Mawr College, USA

Editors

Crime and Justice David B. Wilson, George Mason University, USA

Education Sandra Wilson, Vanderbilt University, USA

International

Development

Peter Tugwell, University of Ottawa, Canada Hugh Waddington, 3ie, India

Methods Ian Shemilt, University of Cambridge, UK

Ariel Aloe, University of Iowa, USA

The Campbell Collaboration (C2) was founded on the principle that systematic reviews on the effects of interventions will inform and help improve policy and services. C2 offers editorial and methodological support to review authors throughout the process of producing a systematic review. A number of C2's editors, librarians, methodologists and external peer-reviewers contribute.

The Campbell Collaboration P.O. Box 7004 St. Olavs plass0130 Oslo, Norway http://www.campbellcollaboration.org/

Web End =www.campbellcollaboration.org

Birte Snilstveit, 3ie, UK Hugh Waddington, 3ie, UK

Social Welfare Nick Huband, Institute of Mental Health, University of Nottingham, UK Geraldine Macdonald, Queens University, UK & Cochrane Developmental, Psychosocial and Learning Problems Group

Methods Therese Pigott, Loyola University, USA Emily Tanner-Smith, Vanderbilt University, USA

Chief Executive Officer Howard White, The Campbell Collaboration

Managing Editor Karianne Thune Hammerstrm, The Campbell Collaboration

Co-Chairs

Crime and Justice David B. Wilson, George Mason University, USA Peter Neyroud, University of Cambridge, UK

Education Sarah Miller, Queen's University Belfast, UK Gary W. Ritter, University of Arkansas, USA

Social Welfare Brandy Maynard, Saint Louis University, USA Mairead Furlong, National University of Ireland, Maynooth, Ireland

International

Development

Table of Contents

TABLE OF CONTENTS 3

ABSTRACT 5 Background 5 Objectives 5 Search Methods 5 Selection Criteria 6 Data Collection and Analysis 6 Results 6 Authors Conclusions 7

PLAIN LANGUAGE SUMMARY 9

1 BACKGROUND 12 1.1 Description of the Problem 12 1.2 The Intervention 13 1.3 How the Intervention Might Work 13 1.4 Why It Is Important to Conduct This Review 14 Figure 1: Nonschool Contextual Pathways to Literacy Learning 15

2 OBJECTIVES 16

3 METHODS 17 3.1 TITLE REGISTRATION AND REVIEW PROTOCOL 17 3.2 Criteria for Considering Studies for This Review 17 3.3 Search Methods for Identification of Studies 19 3.4 Data Collection and Analysis 21

4 RESULTS 27 4.1 Results of the Search 27 4.2 Description of Studies 29 4.3 Risk of Bias in Included Studies 38 4.4 Synthesis of Results 39

5 DISCUSSION 52 5.1 Summary of Main Results 52 5.2 Overall Completeness and Applicability of Evidence 53 5.3 Quality of the Evidence 54

3 The Campbell Collaboration | www.campbellcollaboration.org

5.4 Limitations and Potential Biases in the Review Process 54 5.5 Agreements and Disagreements With Other Studies and Reviews 54

6 AUTHORS CONCLUSIONS 56 6.1 Implications for Practice and Policy 56 6.2 Implications for Research 57

7 REFERENCES 59 7.1 References to Included Studies 59 7.2 References to Excluded Studies 60 7.3 Additional References 63

8 INFORMATION ABOUT THIS REVIEW 65 8.1 Review Authors 65 8.2 Roles and Responsibilities 68 8.3 Sources of Support 69 8.4 Declaration of Interest 69 8.5 Plans for Updating the Review 69 8.6 Author Declaration 69

9 TABLES 71

9 DATA AND ANALYSES 86 9.1 Parent Support Interventions 86 9.2 Tutoring Interventions 90

4 The Campbell Collaboration | www.campbellcollaboration.org

Abstract

BACKGROUND

For a majority of the worlds children, despite substantial increases in primary school enrollment, academic learning is neither occurring at expected rates nor supplying the basic foundational skills necessary to succeed in the 21st century. The significant lag in academic achievement tells us that simply making formal education available does not fully meet childrens needs for literacy development. Globally, many interventions are used to support childrens literacy development through channels outside of the formal education system, in childrens homes or communities. However, there is a lack of information regarding the effectiveness of these interventions.

OBJECTIVES

The objective of this systematic review was to examine the effectiveness of parental, familial, and community support for childrens literacy development in developing countries. This review provides information about the contextual influences of parental, familial, and community support on childrens literacy development skills through the use of interventions that target those influences.

We explored the following questions:

1. What models of reading and literacy learning programs have been implemented in homes and communities in low- and middle-income countries (LMICs)?

2. What models of reading and literacy learning programs implemented in homes and communities in LMICs have empirical evidence regarding their level of effectiveness?

3. How effective are these models in improving childrens literacy outcomes?

SEARCH METHODS

5 The Campbell Collaboration | www.campbellcollaboration.org

relevant programming, and governmental agencies. For example, we searched the websites of UNICEF, UNESCO, 3ie, J-PAL, USAID and others. Project staff and advisory panel members identified literature from their own organizations, and reached out to their contacts to ask for grey literature. The search was conducted from May to July, 2013.

SELECTION CRITERIA

To be included in this review, studies had to have been published in 2003 or later and include a test of an intervention involving parents, families, or community members with the goal of improving childrens literacy development; children ages 3 to 12 years (or preprimary or primary school age); a comparison group; and they had to take place in an LMIC (according to 2012 World Bank classification). Studies that addressed educational radio were eliminated from consideration because a systematic review of the impact of educational radio already exists (Ho & Thukral, 2009).

DATA COLLECTION AND ANALYSIS

Mendeley software was used to manage citations, abstracts, and documents. Abstracts from each database were initially screened by a single reviewer, but in fact many studies were cited in multiple databases and in turn were screened by two or more reviewers. For the studies that passed the screening, two researchers then independently reviewed each to ensure that it met the criteria for inclusion in this review. Thirteen studies were identified for this review. Information was extracted regarding the study setting, process used to form the control or comparison group, independence of the evaluation, outcome measures, attrition, baseline equivalence on child pre-literacy or literacy learning, descriptions of the treatment and comparison conditions, characteristics of participants and implementers, and statistics required for meta-analysis where available. Ten studies were included in meta-analyses. Eight of the studies were cluster randomized or quasi-experimental trials, where the level of assignment was at the school or district level. For these studies, effect sizes were computed using Hedges (2007) 2

d effect size assuming

RESULTS

6 The Campbell Collaboration | www.campbellcollaboration.org

equal cluster sample sizes.

parents learn how to support their childrens school readiness, and tutoring interventions delivered by peers or other community members. Most of these studies involved interventions to improve school readiness.

The three areas of intervention were examined separately, and studies were combined for meta-analysis in cases where they used the same intervention approach, and had the required statistical information available.

Five studies of child-to-child tutoring were included in the meta-analysis. These studies all reported difficulties in data collection, raising questions about the quality of data included in the evaluation. For the total reading post-test, the effect sizes from four studies including a total of 1,779 children were heterogeneous, with a mean of 0.15, and a 95 percent confidence interval that included 0 [-0.27, 0.58]. For the beginning reading sub-test, four studies including 1,767 children were also heterogeneous, with a mean of -0.107 and a 95 percent confidence interval that included 0 [-0.40, 0.18]. For the letter identification posttest, effect sizes from five studies including 2,300 children were heterogeneous with a mean of 0.22, and a 95 percent confidence interval that included 0 [-0.13, 0.57]. For the writing post-test, five effect sizes including 1,993 children were heterogeneous, with a mean of 0.27 that was significantly different from 0 (95 percent confidence interval: [0.02, 0.51]). For the follow-up test of reading achievement, effect sizes from three studies including 1,407 children were heterogeneous, with a mean effect size of 0.07 and a 95 percent confidence interval that included 0 [-0.25, 0.39]. For the follow-up test of writing achievement, effect sizes from three studies including 1395 students were homogeneous with a mean of 0.033 that was not significantly different from zero (95 percent confidence interval: [-0.10, 0.17]. For the follow-up test of overall literacy, effect sizes from three studies including 1,397 children were homogeneous with a mean effect size of 0.06 that was not statistically different from zero (95 percent confidence interval: [-0.15, 0.26].

AUTHORS CONCLUSIONS

7 The Campbell Collaboration | www.campbellcollaboration.org

readiness, and tutoring (provided by older peers or community members). Educational radio has been addressed elsewhere (see Ho & Thukral, 2009), so it was not considered in this review. Educational television had a positive impact on young childrens literacy development if the child viewed the programming three to five times per week (but not at a lower dosage). Interventions intended to support parents ability to develop their childs school readiness were not found to be effective overall, although they did have some positive effects in some countries. Peer-led tutoring was found to improve childrens school readiness in writing, but not in other areas of literacy. However, this approach did have significant effects across multiple areas of literacy in some country contexts. A tutoring program led by community members resulted in increases in childrens literacy.

There were several limitations to this review based on the scarcity of empirical studies and their limited focus on just a few interventions. Numerous descriptions of interventions exist, but few contained a study of program effectiveness in reference to a comparison group. We found only one study that addressed an intervention for children ages 7 and older, and found no eligible studies from Latin America. Therefore, we are left with significant gaps in our understanding of what works in LMICs to improve childrens literacy outcomes using interventions outside of the fo

8 The Campbell Collaboration | www.campbellcollaboration.org

Plain language summary

The Campbell review in brief

There is a wide range of models for out-of-school interventions to improve childrens literacy. Most of these models have not been subject to rigorous evaluation. Support to parents and peers has been largely ineffective in improving literacy, though it has worked in some places. Educational TV has positive effects.

What did the review study?

For a majority of the worlds children academic learning is neither occurring at expected rates nor supplying the basic foundational skills necessary to succeed in the 21st century. This review examines the availability of evidence and its findings about the effectiveness of interventions to improve parental, familial, and community support for childrens literacy development in developing countries.

What is the aim of this review?

This Campbell systematic review assesses the effectiveness of parental, familial, and community support for childrens literacy development in developing countries. The review summarises findings from 13 studies, of which 10 were used for meta-analysis.

What studies are included in this review?

Included studies were published since 2003 with a test of an intervention involving parents, families, or community members with the goal of improving the literacy of children aged 3 to 12 years. The study design had to have a comparison group, and report literacy-related outcomes.

Thirteen studies are included in the review, covering educational television, interventions that help parents learn how to support their childrens school readiness, and tutoring interventions delivered by peers.

What are the main results in this review?

What models of reading and literacy learning programs have been implemented in homes and communities?

Many models are widely used in low- and middle-income countries. These include the provision of libraries (standing or mobile) in many countries including

9 The Campbell Collaboration | www.campbellcollaboration.org

Zimbabwe, Kenya, India and Venezuela; local-language publishing in, for example, Cambodia, Sri Lanka, and Zambia; literacy instruction outside schools including the teaching of literacy through religious instruction; the distribution e-readers in countries such as Ghana and Uganda; educational TV and radio; and supporting community members to educate children.

What models of reading and literacy learning programs implemented in homes and communities in LMICs have empirical evidence regarding their level of effectiveness?

There is no rigorous evidence of the effectiveness of most of the models being used by governments and NGOs around the world. The exceptions are educational TV and radio, and supporting community members to educate children.

How effective are these models in improving childrens literacy outcomes?

Overall, interventions for parent training and of child-to-child tutoring are not effective. Eight out of nine reported outcomes show no significant effects. However, there is considerable variation in the findings, so some approaches may be effective in some contexts.

Educational television appears to improve literacy with frequent viewing, i.e. three to five times a week, over several months.

What do the findings in this review mean?

There are serious gaps in our knowledge. Programs that have worked in some settings should be replicated elsewhere so the contextual factors for success can be identified and understood. There is no evidence for most models used by governments and NGOs, none from one Latin America, and just one study presenting evidence of effects on children aged over seven.

How up to date is this review?

The review authors searched for studies published until July 2013. This Campbell Systematic Review was published in March 2016.

What is the Campbell Collaboration?

The Campbell Collaboration is an international, voluntary, non-profit research network that publishes systematic reviews. We summarise and evaluate the quality of evidence about programs in social and behavioural sciences. Our aim is to help people make better choices and better policy decisions.

About this summary

This summary was prepared by Howard White (Campbell Collaboration) based on the Campbell Systematic Review 2016:4 Parental, Community, and Familial Support Interventions to Improve Childrens Literacy in Developing Countries: A

10 The Campbell Collaboration | www.campbellcollaboration.org

Systematic Review by ET Spier, PR Britto, T Pigott, E Roehlkapartain, M McCarthy, Y Kidron, M Song, P Scales, D Wagner, J Lane and J Glover. Anne Mellbye (R-BUP) designed the summary, which was edited and produced by Tanya Kristiansen (Campbell Collaboration).

11 The Campbell Collaboration | www.campbellcollaboration.org

1 Background

1.1 DESCRIPTION OF THE PROBLEM

For a majority of the worlds children, despite substantial increases in access to primary school, academic learning is neither occurring at expected rates nor supplying the basic foundational skills necessary to succeed in the 21st century. As of 2010, approximately 61 million primary schoolage children worldwide were not attending school. Among those attending school, academic learning is far from assured. For example, only 46 percent of children in Nicaragua achieve Grade 4 learning standards, a figure that drops to less than 5 percent in Malawi. In Ghana, as of 2008, four out of five young women who had completed Grade 6 were still illiterate or only partially literate (UNESCO, 2012). The significant lag in academic achievement tells us that schools alone do not fully meet childrens needs for literacy development. Many reasons exist for these challenges in providing adequate literacy instruction within the school context. For example, a World Bank study found an average 19 percent teacher absence rate across Bangladesh, Ecuador, India, Indonesia, Peru, and Uganda; and, many teachers who were physically present were not spending their time teaching (Chaudhury, Hammer, Kremer, Muralidharan, & Rogers, 2006). Even when both children and teachers are in the classroom, student learning can be significantly hampered by unfamiliarity with the language of instruction (Ball, Paris, & Govinda, 2014), large class sizes because of an insufficient number of teachers, and teacher assignment practices that disproportionately allocate the lowest-performing teachers to the communities with the highest needs (UNESCO, 2014). Despite efforts and innovations in many countries, these problems will not be solved quickly, and alternative approaches are needed to support childrens literacy development. There is also a general lack of empirical research in low- and middle-income country (LMIC) contexts (Wagner, 2014).

Numerous initiatives are underway globally to try to improve childrens literacy development, including interventions that work through parents, families, and communities. These initiatives are intended to supplement childrens school-based learning or provide alternatives for children who do not have access to pre-primary or primary education. Examples of such interventions include tutoring and peer-assisted learning, mobile libraries, programs to build parental knowledge of how to support childrens literacy, literacy instruction outside regular schools (e.g., in the context of

12 The Campbell Collaboration | www.campbellcollaboration.org

religious education), and the provision of educational media for use outside regular classroom instruction.

There are numerous such interventions in LMICs, but there is little information regarding which interventions have evidence for (or against) their effectiveness, and what that evidence reveals. Therefore, this review addresses evidence of what works to improve childrens literacy development in LMICs, with interventions that focus on children between 3 and 12 years old and work through parents, families, and communities.

1.2 THE INTERVENTION

We drew on two dimensions of learning: (1) contexts that support literacy learning and (2) learning outcomes in the areas of pre-literacy and literacy. Ecological models have demonstrated that the most proximal contextsparticularly school, home, and communityare among the strongest influences on learning (Christenson & Reschly, 2010; Dickinson & Neuman, 2006). Within these contexts, influences can be categorized as human (e.g., families) and nonhuman (e.g., print). There have been studies of human influences, such as parent and child shared book reading, peer-to-peer learning, and community volunteers (Britto, Brooks-Gunn, & Griffin, 2006; Britto, Oketch, & Weisner, 2014). Nonhuman influences include access to print, learning resources, and mobile libraries; and access to print and learning support through digital means, such as educational radio or television and other technologies (Doiron, 2011). Although nonhuman influences require some human involvement (such as turning on an educational television program), the primary mode of delivering the literacy support is through the nonhuman materials rather than the human actions. Human and nonhuman influences can intersect to support childrens learning. For example, nonhuman interventions may rely on community and parental engagement to support implementation (Lancy, Bock, & Gaskins, 2010), often with support from international organizations, nongovernmental organizations (NGOs), and community based-organizations (Hoppers, 2006).

1.3 HOW THE INTERVENTION MIGHT WORK

Because literacy skills are acquired progressively, this study used a developmental lens that frames literacy from emergent pre-literacy skills to reading and writing. Therefore, this review includes interventions that are intended to improve childrens literacy development at any point from the preprimary period through middle childhood (i.e., 3 to 12 years old). The conceptual framework for this study drew on the contextual pathways that are linked with literacy from this developmental perspective.

13 The Campbell Collaboration | www.campbellcollaboration.org

Four features characterize this model (see Figure 1):

Proximal contextual supports for literacy include the family and the community. The model differentiates family-level supports from community-level supports. These supports may supplement, complement, or compensate for more formal preschool- and school-based contextual influences.

Pathways between these supports and child literacy outcomes can be mediated by three dimensions: (1) attitudes, beliefs, and expectations of families and communities regarding childrens literacy learning; (2) availability of resources, such as knowledge and print materials; and (3) the nature, quality, and quantity of interactions and practices that families and communities engage in to promote literacy.

Community members or organizations can affect child literacy outcomes by engaging with children directly or acting on families (who in turn engage with children).

Given the evidence that early learning is one of the strongest predictors of later literacy skills, from a developmental perspective, the model considers outcomes for children between 3 and 12 years old.

In some family or community contexts, one or more of the pathways shown in Figure 1 may be weak or nonexistent, reducing the likelihood that a child will reach his or her full potential with regard to literacy development. The interventions that were considered for the current review were expected to act on one or more weak or missing pathways, leading to improvements in childrens literacy development.

1.4 WHY IT IS IMPORTANT TO CONDUCT THIS REVIEW

Policy makers and practitioners at the country level and in multinational organizations increasingly want to select interventions that have documented, reliable evidence of their effectiveness. As discussed previously, poor literacy development is a persistent and significant concern in many countries. However, the published research literature available outside industrialized nations is quite limited, albeit growing, and there is a dearth of high-quality, quantitative studies. There also is a lack of balance in the quality and the quantity of evidence for the effectiveness of interventions across different contexts (e.g., more literature available from some countries or regions than others).

The field will benefit from knowing the types of interventions that have been subject to rigorous evaluation, as well as the evidence produced by those evaluations. We found significant gaps in the availability of evidence for what works to improve childrens literacy development in LMICs outside the formal education system. By highlighting the availability of evidence, our review may inform the effective allocation of evaluation resources.

14 The Campbell Collaboration | www.campbellcollaboration.org

FIGURE 1: NONSCHOOL CONTEXTUAL PATHWAYS TO LITERACY LEARNING

Actors

Pathways Outcomes

Motives Resources Practices Pre-literacy Literacy

Parents and families

Beliefs, attitudes, expectations

Interactions with child: oral language, engagement with print, book reading

Knowledge, print material

Vocabulary, phonemic awareness, print concepts, decoding skills, other skills

Reading, writing

Communities

Knowledge, print material

15 The Campbell Collaboration | www.campbellcollaboration.org

2 Objectives

The objective of this systematic review was to examine the availability of evidence and what that evidence says about the effectiveness of interventions to improve parental, familial, and community support for childrens literacy development in developing countries. This review provides information about the contextual influences of parental, family, and community support on childrens literacy development skills by using interventions that target those influences.

We explored the following questions:

1. What models of reading and literacy learning programs have been implemented in homes and communities in low- and middle-income countries (LMICs)?

2. What models of reading and literacy learning programs implemented in homes and communities in LMICs have empirical evidence regarding their level of effectiveness?

3. How effective are these models in improving childrens literacy outcomes?

The overarching goals of this review are to (1) increase the availability of information for evidence-based decision making for international agencies, NGOs, and government policy makers who select programming for children, and (2) identify evidence gaps regarding the effectiveness of interventions currently in use.

16 The Campbell Collaboration | www.campbellcollaboration.org

3 Methods

3.1 TITLE REGISTRATION AND REVIEW PROTOCOL

The title for this systematic review was registered on September 2, 2013. The systematic review protocol was approved on September 1, 2014. Both the title registration and the protocol are available in the Campbell Library at: www.campbellcollaboration.org/lib/

3.2 CRITERIA FOR CONSIDERING STUDIES FOR THIS REVIEW

3.2.1 Eligible designs

Eligible studies were required to include both a treatment group and a comparison group. Eligible designs included randomized control trials (RCTs) and regression discontinuity designs. We also included quasi-experimental studies, provided that there was a valid comparison group. In the registered review protocol, we specified that to be valid, the comparison group (1) must be drawn from the same population as the intervention group, and (2) must have baseline data available that demonstrates no pre-existing differences on outcomes of interest. However, given the small number of studies available, we did not exclude quasi-experimental studies that had unequal baseline scores on outcomes of interest (a deviation from the protocol). We excluded single-group, pre-post designs because of their weak internal validity. All other quasi-experimental study designs were eligible to the extent that methods existed for computing an appropriate measure of effect size. Purely descriptive studies were not included.

3.2.2 Types of participants

The target population was children between 3 and 12 years old living in low- and middle-income countries (LMICs), as defined by the World Banks country income classification. In some countries, many childrens births are not registered, and exact ages may be unknown. Therefore, in the absence of information regarding exact ages, we accepted studies with children described as being of preprimary or primary school age. Studies that focused on children with disabilities were eligible for inclusion, even though disabilities are not a distinct topic of interest here.

17 The Campbell Collaboration | www.campbellcollaboration.org

3.2.3 Types of interventions

Included studies were primary studies of interventions, not literature reviews or meta-analyses. Each intervention addressed literacy or pre-literacy skills and was delivered through family or community members. Interventions delivered within a school setting were eligible for inclusion only if the delivery mechanism was the family or community members. In addition, different types of interventions (e.g., cash transfers, vouchers, libraries) were included if their purpose was to address literacy outcomes and they were not delivered in formal schooling. The intervention could be a program, a product, a policy, or a practice; however, the primary focus of the study must have been aligned with the topic area of literacy.

3.2.4 Types of outcome measures

Eligible pre-literacy and literacy outcomes included a full range of skills, including phonemic awareness, listening, vocabulary, speaking, pronunciation, print concepts, knowledge of the alphabet, reading (comprehension, fluency), sight reading of words, writing, spelling, and narrative development. Pre-literacy and literacy outcomes were required to be assessed with standardized measures, country-specific or locally used assessments, or assessments developed for the evaluation (to the extent that they were not over-aligned with the contents of the intervention).

3.2.5 Types of literature

Documents were included if they were published in 2003 or later. Studies from more than 10 years ago have a high likelihood of assessing interventions that are no longer in use or are no longer being implemented in the same context (e.g., childrens access to primary education, parental literacy, and the use of first-language instruction have all increased in many LMICs during the last decade).

We searched for studies in English, Spanish and French. We also identified some studies with English language abstracts but text in Turkish, and these studies were screened by a native Turkish speaker with expertise in education research.

Unpublished studies of eligible interventions such as dissertations or research reports from government agencies and NGOs were included. Documents such as PowerPoint presentations, internal agency memos, editorials and notes, student term papers, advertisements or promotional materials, editorials, letters, case series, and personal communication notes were excluded.

18 The Campbell Collaboration | www.campbellcollaboration.org

3.3 SEARCH METHODS FOR IDENTIFICATION OF STUDIES

3.3.1 Electronic searches

Studies included in this systematic review were obtained from electronic academic literature, gray literature, and key informant solicitation. Searches for academic literature were conducted in online databases from across diverse disciplines (shown in Table 1).

Table 1 Online Databases Searched Discipline Resource(s)

Anthropology Africa-Wide NiPAD

Economics EconLit

Education Education Research Complete (EBSCO); Education

Research Information Center (ERIC)

Interdisciplinary Arts and Humanities Index; Web of Science;

FRANCIS

International Relations Public Affairs Information Service (PAIS)

Political Science Worldwide Political Science Abstracts

Psychology PsycINFO

Social Sciences Academic Search Premiere; Campbell Collaboration;

Social Science Citation Index; Social Sciences Full

Text (H. W. Wilson); Sociological Abstracts

A list of search terms was used to search the electronic databases (Table 2). To ensure that the searches are reliable across the three search strategies, the exact same conceptsphases of human development (early and middle childhood), and home and community-based learningwere searched. Because each of the electronic databases selected for the academic literature search uses different vocabularies to index its subjects and topics, the search terms needed to be adapted for each database, although the concepts of the phases of human development (early and middle childhood) and home and community-based learning remained constant. We also used a core set of search terms common to all databases, such as read* and lit*.

19 The Campbell Collaboration | www.campbellcollaboration.org

Table 2 Search Strings

Search String

"child*" or youth* or pre-reader* or low-readiness reader* or girl* or gender or boy

AND "READING achievement" OR "READING comprehension" OR "LITERACY education" OR "FAMILY literacy programs" OR "COMMUNITY education" OR "PARENT participation in children's reading" or "READING intervention" OR "LITERACY programs" or read* or liter*

AND assessment* or effect* or evaluat* or impact* or outcome* or interven* or program* or trial* or deliver* or service*

AND family literacy or community involvement or community support or collaborative learning or

facilitator* or learning resources or community centers or community organizations or community-based education or community-based or home-based or parent* or famil* or caregiv* or center or centre or home* or communit* or librar*

OR READING achievement or READING comprehension or "alphabet or basic skills or coaching or

cognitive skills or collaborative learning or comprehension or ECD program* or educat* or fluency or language or learn* or lexical or lexicon or linguistic or listening or narrative or morphem* or non-formal or informal or non-formal education or informal education or parental speech or phonem* or phonological or print or pronunciation or read* or sentence or sight words or spell* or stor* or storybook or syllable* or syntax or text or vocabulary or write or writing or written language or written text or word

AND Afghanistan or Angola or Armenia or Asia Pacific Region or Bangladesh or Belize or Benin or

Bhutan or Bolivia or Bosnia or Botswana or Brazil or Bulgaria or Burkina Faso or Burundi or Cambodia or Cameroon or Cape Verde or Caribbean or Central Africa or Central African Republic or Central America or Central Asia or Chad or Chile or China or Colombia or Comoros or Congo or Costa Rica or Cte d'Ivoire or Cuba or Developing countr* or Developing world or Djibouti or Dominica or Dominican Republic or East Africa or East Asia or Ecuador or Egypt or El Salvador or Eritrea or Ethiopia or Fiji or Francophone Africa or Gabon or Gambia or Gaza or Georgia or Ghana or Grenada or Guatemala or Guinea-Bissau or Guinea or Guyana or Haiti or Herzegovina or Himalayas or Honduras or Horn of Africa or India or Indonesia or Iran or Iraq or Jamaica or Jordan or Kazakhstan or Kenya or Kiribati or Korea or Kosovo or Kyrgyz or LAMIC Lao or Latin America or Latvia or Lebanon or Lesotho or Less developed countr* or Liberia or Libya or Lithuania or Low and middle income countr* or Low income countr* or Lusophone Africa or Macedonia or Madagascar or Malawi or Malaysia or Maldives or Mali or Marshall Islands or Mauritania or Mauritius or Mayotte or Mexico or Micronesia or Middle income countr* or Moldova or Mongolia or Montenegro or Morocco or Mozambique or Myanmar or Namibia or Nepal or Nicaragua or Niger or Nigeria or North Africa or Northeast Asia or Pakistan or Palau or Panama or Papua New Guinea or Paraguay or Peru or Philippines or Poor countr* or Poor region* or Romania or Russia or Russian Federation or Rwanda or Sahara or Sahel or Samoa or So Tom and Principe or Senegal or Serbia or Seychelles or Sierra Leone or Solomon Islands or Somalia or South Africa or South America or South Asia or Southeast Asia or Southern Africa or Sri Lanka or St. Kitts and Nevis or St. Lucia or St. Vincent and the Grenadines or Sub-Saharan Africa or Sudan or Suriname or Swaziland or Syria or Syrian Arab Republic or Tajikistan or Tanzania or Thailand or Timor-Leste or Togo or Tonga or Tunisia or Turkey or Turkmenistan or Tuvalu or Uganda or Ukraine or Under-developed countr* or Uruguay or Uzbekistan or Vanuatu or Venezuela or Vietnam or West Africa or West Bank or Yemen or Zambia or Zimbabwe

3.3.2 Searching other resources

To capture gray literature, we searched websites of nongovernmental and inter-governmental agencies, think tanks, and international research centers. Agency websites searched for gray literature included those of United Nations agencies, international

20 The Campbell Collaboration | www.campbellcollaboration.org

development banks, and aid groups; nongovernmental organizations (NGOs) and foundations; and international research institutes and centers of expertise. We also worked with our international advisory panel and members networks to identify relevant literature, and we reached out directly to our colleagues in the field.

3.4 DATA COLLECTION AND ANALYSIS

3.4.1 Selection of studies

Studies had to meet the following criteria to move on to the next stage of the review:

Published in 2003 or later; Included a test of an intervention; Addressed the topic of literacy (defined broadly); Included children ages 3 to 12 years, or primary schoolage children (overlap with other age groups was acceptable, as long as children in the target range were also included);

Included a comparison or control group drawn from the same population as the treatment group; and

Took place in an LMIC (according to 2012 World Bank categories).

Title and abstract screening was carried out by a team of researchers and research assistants. Studies identified for retrieval were reviewed by qualified researchers (masters degree or higher in a relevant field, with expertise in research methods). Some team members had been involved in carrying out or reporting on some of the studies slated for review. In those cases, the studies were assigned to reviewers from an organization that had not been involved in carrying out the research. In cases where the two reviewers did not initially agree regarding whether a study met the inclusion criteria, those reviewers then discussed the study and in all cases consensus was easily reached.

The studies were reviewed to determine whether they met the following criteria:

The intervention included parents, families, or community members. Interventions that took place in schools were acceptable as long as there was involvement of parents, families, or community members (including peers). Interventions that were solely delivered by a teacher, other school staff, or researcher were excluded.

The intervention group had a valid comparison group. There were one or more valid outcome measures that assessed literacy or pre-literacy skills (such as reading, vocabulary, writing, letter recognition, decoding skills, or print awareness). These measures could include standardized assessments, academic tests or scores, or researcher-developed instruments (as long as they were not over-aligned with the intervention).

The report contained adequate information about the evaluation to assess the above. For example, several documents were removed from consideration because they were

21 The Campbell Collaboration | www.campbellcollaboration.org

short summaries describing programming and did not include the kind of information required for the current review (such as a description of how the control group was formed).

In the protocol for this review, we indicated that we would exclude quasi-experimental studies that reported pre-test differences on measures of literacy or pre-literacy. All four studies of the OSI program, Getting ready for school, reported evidence of pre-existing differences between the treatment and control groups (greater than 0.25 standard deviations). Instead of excluding these studies from the review, effect sizes were computed and reported for the studies while noting that the overall effect size may be biased.

We also excluded studies that addressed the topic of educational radio, because a systematic review of the impact of educational radio was recently completed (Ho & Thukral, 2009). Therefore, we felt it was appropriate to reference but not duplicate the existing review.

3.4.2 Data extraction and management

Mendeley software was used to manage citations, abstracts, and documents. Citations were exported to active Excel worksheets for title and abstract screening. Two researchers then independently reviewed each article to ensure that it met the criteria for inclusion, noting the study characteristics in an active Excel worksheet. Studies that had been carried out by researchers involved in this review were identified for screening and reviewed by researchers who were entirely independent of the study.

Data extracted included: Information about the study setting (which country, urban or rural region, etc.); Group formation process (how treatment and comparison groups were formed, any concerns about comparability of non-randomized comparison group);

Independence of the evaluation; Outcome measures (what they are, any issues of over-alignment, etc.); Attrition; For quasi-experimental designs, whether adequate information was provided to assess baseline equivalence, and whether there were any concerns about baseline equivalence;

Descriptions of the intervention and comparison conditions; Characteristics of the participants and the implementers; and Statistics required for meta-analysis (where available)

For the 13 studies that met the criteria for inclusion in this review, data were extracted for inclusion in meta-analyses, where possible, and placed in an Excel worksheet. In one case, a studys author was contacted directly to request needed data for computing effect sizes that were not included in the article. Meta-analyses were carried out with Comprehensive Meta-Analysis (CMA) software.

22 The Campbell Collaboration | www.campbellcollaboration.org

3.4.3 Assessment of risk of bias in included studies

The following aspects of the eligible studies were coded and used as indicators of study quality: (a) formation of the control group, (b) information about attrition, and (c) baseline equivalence. For each study, the procedure for forming the control group was noted. These categories included random assignment, wait-list control, and pre-existing group. The number of participants that were missing from any of the outcome assessments from the treatment and control groups was also recorded for each study. For baseline equivalence, information was recorded for any included pre-tests including the summary statistics for reported baseline measures, the value of any statistical test of these baseline measures, and the results of those statistical tests. If no statistical tests were reported, a description of the difference between the baseline measures as discussed in the study was reported.

In addition, information about any difficulties in the evaluation of the interventions was recorded. The evaluation reports for the UNICEF Child-to-Child studies included descriptions of data collection problems that could have led to potential bias in the reporting of outcomes of the intervention.

3.4.4 Measures of treatment effects

The standardized mean effect size was used as the measure of treatment effect given that the outcome measures were all measured on a continuous scale. The standardized mean effect size for a nonclustered, experimental study is given by Hedges (1981) as

,E C

X X

d s

= (1)

where E

X and C

X are the experimental and control group means, respectively, and

2 p

s is the

pooled sample standard deviation given by

2 2

( 1) ( 1) E ( 1) ( 1)

n s n s

E C C

n n

+ (1)

E C

where

2 E

s and

s are the experimental and control group standard deviations, respectively. The experimental and control group sample sizes are given by E

2 C

n and .

n Effect sizes were also corrected for small sample bias using Hedges (1981) correction.

3.4.5 Unit of analysis issues

Several of the studies were cluster randomized or quasi-experimental trials, where the level of assignment was at the school or district level. For these studies, effect sizes were computed using Hedges (2007) 2

d effect size assuming equal cluster sample sizes. The

effect size is given by

23 The Campbell Collaboration | www.campbellcollaboration.org

X X

E C

d S

2( 1)

1 .

(1)

The overall means for the experimental and control groups (averaged across clusters and groups) are given by

S , estimated from the pooled sample standard deviation across both experimental and control groups. The intraclass correlation is . The total sample size for the trial is N, and the cluster sample sizes are given by n. We assumed equal cluster sizes; when the cluster sample sizes were not equal, we used the smallest cluster sample size in the computations. The effect sizes for unequal cluster sample sizes require the actual sample sizes for each cluster in a trial; the studies in our sample reported the cluster sizes on average across the experimental and control groups. Hedges (2007) indicates that the effect sizes computed for equal cluster sizes are not substantially different from those assuming unequal cluster sizes and can be used in place of the more complex formulas for unequal cluster sizes. The variance of the effect size

X and

X . The total sample standard deviation is given by T

d is given by

{ }

E C

T E C

N N

V d n

N NN n N n N n

d N N n

= + +

+ +

(1)

(1 ( 1) )

( 2)(1 ) ( 2 ) 2( 2 ) (1 )2( 2) ( 2) 2( 1)

2 2

[ ]

N are the experimental and control group sample sizes summed across clusters. For the clustered randomized trials that were included in the meta-analysis, the original analysis was available. The intraclass correlation was computed from the original data for these studies to obtain the effect size.

3.4.6 Dealing with missing data

As indicated above, the intraclass correlation coefficient was required for the computation of effect sizes from clustered trials. For UNICEFs Getting Ready for School: A Child-to-Child Approach studies and for the Open Society Institute Getting Ready for School program, the study authors were contacted to provide the intraclass correlation coefficient. Authors of the educational television studies and the Read India study were also contacted for information needed to compute effect sizes. The authors of the Read India study did provide more information, but summary statistics needed to compute the clustered effect sizes were unavailable. The authors of the educational television studies did not respond to requests for data, so additional analyses could not be conducted for those studies.

3.4.7 Assessment of heterogeneity

For studies that are included in a meta-analysis, heterogeneity was assessed using both

where

N and

and the test of the statistical significance of the random variance component,

. Given the

24 The Campbell Collaboration | www.campbellcollaboration.org

small numbers of studies included in the meta-analysis, we did not conduct moderator analyses.

3.4.8 Assessment of reporting biases

The studies were examined for evidence of reporting biases. For the UNICEF and Open Society Institute studies, we obtained complete reports of the studies and their protocols. We found no evidence of reporting biases in the remaining reports used in the meta-analysis. As mentioned above, the studies on educational television and the Read India reports did not provide enough information to compute an effect size.

3.4.9 Data synthesis

A random effects model was used for the synthesis of the studies. We chose a random effects model because the context and implementation of the trials differed across studies. In a random effects model, the variance of the effect sizes is given by

+ (1)

is the random effects variance component. The random effects variance component was computed using the method of moments through the program Comprehensive Meta-analysis 2.2.064 (Biostat, 2011). We planned to analyze studies with high risk of bias (high attrition rates or non-equivalent baselines) separately from studies with low risk of bias. We also planned to conduct separate meta-analyses for each outcome measure included in the studies to guard against dependency issues. In addition, we planned to analyze outcomes by follow-up year, separately analyzing year 1 follow-up from year 2 follow-up results. As will be seen in the results, we synthesized the results from two sets of programs, child-to-child peer tutoring, and parent education programs. A separate meta-analysis was conducted for the immediate posttest and for the follow-up posttest for each conceptually similar outcome. For example, the child-to-child peer tutoring studies each measured the following for the immediate posttest: Reading total, and Subtests for Beginning Reading, Letter Identification, Writing. For the follow-up, Year 2 outcomes, each study measured Reading, Writing and Overall Literacy. The parent education studies collected measures of overall literacy immediately after the treatment and then in Year 2. A separate meta-analysis was conducted for each outcome and time point (immediate posttest and follow-up).

3.4.10 Subgroup analysis and investigation of heterogeneity

We conducted two separate meta-analyses, one for programs based on child-to-child peer tutoring and one based on parent education programs. For each meta-analysis, there was an insufficient number of studies to conduct subgroup analyses or moderator analyses.

25 The Campbell Collaboration | www.campbellcollaboration.org

Var d

( )

N N d

= N N N N

E C

E C E C

+ + +

2( )

where

3.4.11 Sensitivity analysis

None of the effect sizes were outlying observations, so there was no need to examine the impact of effect sizes that would have been considered outliers.

26 The Campbell Collaboration | www.campbellcollaboration.org

4 Results

4.1 RESULTS OF THE SEARCH

Title and abstract screening resulted in the elimination of 10,357 studies that failed to meet one or more of the preceding criteria, leaving 73 studies identified for retrieval. After 21 duplicate citations were removed, we retrieved the remaining 52 studies. We then eliminated 3 studies for addressing the topic of educational radio, and 36 for one or more of the reasons listed above in Section 3.4.1. This left us with the 13 studies that were included in the review. See Figure 2.

Figure 2 Study Selection Process

Potentially relevant studies identified and screened for retrieval (n = 10,430)

Documents excluded for not meeting screening criteria (n = 10,357)

Studies identified for retrieval (n = 73)

Duplicate studies removed (n = 21)

Studies removed for not meeting inclusion criteria (n = 36)

Studies removed because topic is educational radio (n = 3)

Studies retrieved (n = 52)

Studies to be included in review (n = 13)

Studies included in meta-analysis (n = 10)

Studies included in review but not in meta-analysis (n = 3)

27 The Campbell Collaboration | www.campbellcollaboration.org

The list of included studies is shown in Table 3.

Table 3 Included Studies

Study Type of Intervention

Television Will You Play With Me? Turkey

Byktakapu (2012) Parent Instruction Family-Supported Pre-Reading Program

American Institutes for Research (2012a) Parent Instruction OSIs Getting Ready for School Armenia

American Institutes for Research (2012b) Parent Instruction OSIs Getting Ready for School Bosnia and

Herzegovina

American Institutes for Research (2012c) Parent Instruction OSIs Getting Ready for School Kazakhstan

American Institutes for Research (2012d) Parent Instruction OSIs Getting Ready for School Tajikistan

Banerjee, Banerji, Duflo, Glennerster, & Khemani (2008)

Bangladesh DR Congo Ethiopia Tajikistan Yemen

The results of our search are very illuminating in their own right. We found that there are many types of interventions occurring in LMICs that are intended to improve childrens literacy outcomes take place outside of the formal education system.

The interventions that did have evidence of effectiveness were: 1. Educational television; 2. Educational radio; 3. Programs that show parents how to support their childrens school readiness (including in the area of literacy); and

4. Programs that use a tutoring approach, with peers or adult community members teaching children literacy skills.

A large number of intervention approaches in use globally by both NGOs and government organizations did not appear to have empirical evidence for (or against) their effectiveness in improving childrens literacy. Many of these interventions are widely used in LMICs, such as the provision of libraries (standing or mobile) in many countries such as Zimbabwe, Kenya, India and Venezuela; local-language publishing in countries such as Cambodia, Sri Lanka, and Zambia; the teaching of literacy through religious instruction, such as in madrasas around the world; and the distribution e-readers in countries such as Ghana and Uganda. This issue is illustrated in Figure 3.

28 The Campbell Collaboration | www.campbellcollaboration.org

Name of Intervention Country

Borzekowski & Henry (2011) Television Jalan Sesama Indonesia

Baydar, Kaitibai, Kntay, & Gken (2008)

Turkey

Tutoring READ India India

UNICEF (2013) Tutoring Getting Ready for School: A

Child-to-Child Approach

Figure 3 Areas With and Without Rigorous Evidence

Intervention areas with rigorous evidence

Teaching parents how to

support childrens learning

Tutoring by volunteers

Educational television

Educational radio

Community libraries Distribution of print material

Local language publishing

E-readers & laptops used outside of school

Instruction in religious organizations

Community mobilization

Mobile libraries

Reading clubs

4.2 DESCRIPTION OF STUDIES

4.2.1 Topic areas of studies identified

The studies that fit our criteria for inclusion in this review were:

Two studies of educational television used outside of schools; Five studies that tested the effectiveness of programs designed to improve parents support for their childrens school readiness (including in the area of literacy); and

Six studies that assessed the effectiveness of tutoring interventions intended to improve childrens literacy; five using peer-assisted learning, and one using adult community members.

29 The Campbell Collaboration | www.campbellcollaboration.org

Intervention areas without rigorous evidence

Two of the interventions have similar names (the Getting Ready for School program, operated by the Open Society Institute, and Getting Ready for School: A Child-to-Child Approach, operated by UNICEF). These are distinct interventions.

The results of UNICEFs Getting Ready for School: A Child-to-Child Approach initiative were published in a single report that includes results for five countries (2013). Different data collection teams conducted the studies using different sampling approaches across the five countries; thus, each countrys results are treated as a separate study for this review.

4.2.2 Locations of studies identified

The studies identified for inclusion in this review cover a variety of LMICs:

Armenia Bangladesh Bosnia and Herzegovina Democratic Republic of Congo Ethiopia India Indonesia Kazakhstan Tajikistan Turkey Yemen

Two studies were conducted in Tajikistan, two in Turkey, and one in each of the other countries listed above. The list contains low-income countries (Bangladesh, Democratic Republic of Congo, Ethiopia, and Tajikistan), lower-middle-income countries (Armenia, India, Indonesia, and Yemen) and upper-middle-income countries (Bosnia and Herzegovina, Kazakhstan, and Turkey).

Latin American countries are noticeably absent from this list, despite our efforts to access both academic and gray literature in English and Spanish. Studies from Latin America were identified in the initial search, but none met our criteria for inclusion in this review.

4.2.3 Description of the interventions

The 13 studies identified for this review covered six different interventions: Jalan Sesama and Will You Play With Me? in the area of educational television; the Family Supported Pre-Reading Program and Open Society Institute (OSI) Getting Ready for School in the area of parent support programs; and UNICEFs Getting Ready for School: A Child-to-Child Approach and Read India in the area of tutoring.

30 The Campbell Collaboration | www.campbellcollaboration.org

Jalan Sesama

Jalan Sesama is a version of Sesame Street that was created to support the developmental needs of children in Indonesia. In many Indonesian communities, children and their families face significant challenges that can negatively impact educational attainment, such as economic hardship, natural disasters, and civil unrest. This intervention is based on evidence from other contexts that educational television can have a positive effect on childrens learning (including literacy) and social development (Mares & Pan, 2013). Jalan Sesama is available to the public on Indonesian television. The program consists of 52 weekly episodes and addresses early cognitive skills, literacy, mathematics, health, safety, social development, environmental awareness, and cultural awareness. However, not all communities have access to television.

The intervention reviewed here involved children viewing DVD recordings of Jalan Sesama at a local school outside of the normal school day. Teachers did not provide any additional support or instruction to the children. A high-exposure group viewed three or four Jalan Sesama episodes per week over 14 weeks (they saw each of the 52 episodes once). A low-exposure group viewed one episode of Jalan Sesama per week over 14 weeks (the same first 14 episodes viewed by the high-exposure group).

Will You Play With Me?

Will You Play With Me? (Benimle Oynar Msn?) is a Turkish educational television program developed by the Mother-Child Education Foundation. The goal of this program is to improve school readiness among Turkish children; it targeted children ages 4 to 6 years from lower socio-economic backgrounds who lacked access to preprimary education. The program is similar to Sesame Street, but with additional components intended to involve parents in their childrens learning. Its topic areas include cognitive development, family relationships, social and emotional development, health, and environmental awareness. The program consists of 65 half-hour segments that are aired twice per day on Turkish television.

The intervention reviewed here involved asking parents of children ages 4 to 7 years to have their children watch Will You Play With Me? on television in their home every weekday for 13 weeks, starting with the first program of the series (so they watched all 65 episodes by the end of the 13 weeks).

Family Supported Pre-Reading Program

The Family Supported Pre-Reading Program was developed by researchers at Mevlana University in Turkey. The program was developed for use with children age 6 who attend preprimary education programs and was intended to build childrens literacy skills by teaching parents how to become active partners in supporting their childrens literacy development. The logic model for this intervention states that parental support for childrens development (in addition to formal preprimary education) can provide children with more

31 The Campbell Collaboration | www.campbellcollaboration.org

school readiness in the area of literacy than they would get from formal preprimary education alone. The Family Supported Pre-Reading Program is specific to Turkey (not currently offered in other countries).

For the study reviewed here, parents attended weekly meetings for 13 weeks. Each week they were given 18 specific activities to do with their children to promote literacy (phonological awareness, letter recognition, storytelling, reading concepts, and chronology). Parents then carried out these activities with their children at home between sessions, using 13 workbooks provided by the program.

OSIs Getting Ready for School

OSI is a non-governmental organization dedicated to promoting a range of educational initiatives. OSIs Getting Ready for School program was developed in partnership with the International Step-by-Step Association for use in disadvantaged communities in Eurasia where preprimary education was unavailable to most children. The program was designed for contexts where parents had at least a basic educational level (i.e., basic literacy). The goal of the intervention is to improve childrens school readiness across domains (including literacy) and teach parents how to become active partners in supporting their childrens education. The logic model for this intervention states that, where preprimary education is unavailable, parents or other adult family members (such as grandparents) can be taught how to develop childrens school-readiness skills. This program is currently in use in multiple countries in Eastern Europe and Central Asia.

For the study reviewed here, parent educators were trained to provide guidance and supervision to parents of children who were 1 year away from expected on-time Grade 1 entry at the start of the program. The trainers met with parents in a group for nine monthly sessions that were typically held at a school (outside of regular school hours). During the sessions, parents were provided with information about child development and how to support childrens learning, and were taught how to do specific activities with their children to promote academic learning, cognitive skills, and social and behavioral skills. Parents then carried out these activities with their children at home between sessions. These activities were designed to support child development through play, addressing literacy (such as vocabulary, letter recognition, sight reading, storytelling, and beginning writing) as well as mathematics and other areas of development. For example, for one activity, parents were shown how to create a board game with their child for use at home. This activity was intended to build parent-child collaboration, plus build childrens skills in areas such as counting, taking turns, and following instructions.

UNICEFs Getting Ready for School: A Child-to-Child Approach

UNICEFs Getting Ready for School: A Child-to-Child Approach was developed in collaboration with the Child-to-Child Trust for use in disadvantaged communities where preprimary education is unavailable to most children (and where parents may not be able to

32 The Campbell Collaboration | www.campbellcollaboration.org

support their childrens learning well because of their own lack of education). The program was designed to be appropriate across cultural contexts. The goal of the intervention is to improve childrens school readiness across domains (including literacy), and increase on-time enrollment in primary school. The logic model for this intervention states that young children are influenced by older children (siblings, playmates) in their homes and communities. By teaching older children how to support younger childrens learning, and providing a structured way for them to do so, the younger children will develop better school readiness. And when young children become enthusiastic about learning and are well prepared to succeed in primary school, parents are more likely to enroll them on time and keep them enrolled. This program is currently in use across multiple countries in Sub-Saharan Africa, the Middle East, Central Asia, Southeast Asia, and South America.

For the study reviewed here, teachers were trained to provide guidance and supervision to students in Grades 48 who acted as Young Facilitators. The Young Facilitators were matched with two or more young children in the community who were one year away from on-time Grade 1 enrollment. Young Facilitators and young children met for 35 sessions that were typically held twice weekly at a school (outside of regular school hours). In some countries, Young Facilitators and young children also met in the community for additional sessions. Young Facilitators worked through a series of planned activities with the young children. These activities were designed to support child development through play; they addressed literacy, mathematics, cognitive skills, and social and behavioral skills.

Read India

Read India is a widely used intervention developed and implemented by the Pratham Mumbai Education Initiative. It is available in just over half of all villages in India. Read India is based on a logic model that says that engaging individual community members can help improve childrens learning outcomes, even in the absence of the involvement of the formal education system or wider community, and that community ownership is the key to achieving change. The goals of the program are to ensure that all children in Standard 1 know their letters and numbers, all children in Standard 2 can read words and perform simple arithmetic, and that all children in Standards 3 to 5 can read simple texts fluently and solve mathematical problems.

The program evaluated here involved training local volunteers to provide literacy education through camps held outside of school hours. The villages involved in the evaluation were located in a state with poor basic literacy achievement. Villages had as many as 16 literacy camps, staffed by different volunteers. The classes were open to children in the villages assigned to the treatment condition, but specific children were not targeted for participation or assigned to receive the program. Across all participating villages, approximately 8 percent of the children participated in programming.

33 The Campbell Collaboration | www.campbellcollaboration.org

4.2.4 Description of the included studies

The included studies employed randomized controlled trials (RCTs) and quasi-experimental designs to evaluate the impacts of the interventions. In this section, the study design is presented for each intervention described above. Some interventions were studied in multiple countries with multiple evaluation designs. In such cases, the design for each separate study is specified (Table 4). See Tables 15 through 27 (Appendix) for additional details for each study, including specific location(s), formation of treatment and control groups, participant characteristics, the treatment and comparison conditions, outcomes assessed, and attrition.

Table 4 Summary of Included Studies

Intervention Location Design Approach

Jalan Sesama Indonesia RCT Educational television

Will You Play With Me? Turkey RCT Educational television

Family Supported Pre-Reading Turkey QED Parent instruction

OSIs Getting Ready for School Armenia RCT Parent instruction

OSIs Getting Ready for School Bosnia-Herzegovina QED Parent instruction

OSIs Getting Ready for School Kazakhstan QED Parent instruction

OSIs Getting Ready for School Tajikistan QED Parent instruction

UNICEF Child-to-Child Bangladesh RCT Peer tutoring

UNICEF Child-to-Child Democratic Republic of Congo QED Peer tutoring

UNICEF Child-to-Child Ethiopia QED Peer tutoring

UNICEF Child-to-Child Tajikistan RCT Peer tutoring

UNICEF Child-to-Child Yemen RCT Peer tutoring

Read India India RCT Peer tutoring

Jalan Sesama

The evaluation of Jalan Sesama was conducted with a single RCT conducted in Indonesia. Children ages 3.5 to 6 years in low-income communities were randomly assigned to high exposure to an educational television program, low-exposure, or a no-exposure control group that watched a non-educational television program. Outcomes assessed included letter recognition, phonemic awareness, and writing. There was zero attrition. See Table 15 (Appendix) for more information regarding this study.

Will You Play With Me?

The evaluation of Will You Play With Me? was assessed with a single RCT conducted in the largest metropolitan area of Turkey. Children aged 5 years, 3 months on average were randomly assigned to an intervention group (watched the program regularly), control group (did not watch the program), or a natural observation group (parents were informed about

34 The Campbell Collaboration | www.campbellcollaboration.org

the program and were free to have their children watch or not watch it). Outcomes assessed included syllabification and vocabulary. There was 5 percent attrition in the intervention group, 9 percent in the control group, and 29 percent in the natural observation group. See Table 16 (Appendix) for more information regarding this study.

Family Supported Pre-Reading Program

The evaluation of the Family Supported Pre-Reading Program was a small quasi-experimental study carried out with families whose children attended a preschool program in Konya, Turkey. The comparison group was made up of children who had attended another preschool in Konya during the same time period (it was unclear whether the comparison children were drawn from one other preschool or multiple preschools). Demographic information (such as child age or family socio-economic status) was not available. Outcomes assessed included reading comprehension, mechanical reading skills, and writing. There was zero attrition. See Table 17 (Appendix) for more information regarding this study.

OSIs Getting Ready for School

Four separate evaluations were carried out to assess the effects of OSIs Getting Ready for School interventionone each in Armenia, Bosnia and Herzegovina, Kazakhstan, and Tajikistan. All were carried out in communities where children lacked access to preprimary education. The program was provided to children one year before on-time Grade 1 enrollment. In the treatment condition, families participated in nine monthly sessions where they were instructed how to support their childrens school readiness.

The study in Armenia was an RCT. Communities across five provinces were randomly assigned to the treatment (program) or control (no program) group. Nearly all children were age 5 at baseline. At outcome, children were assessed in the areas of letter identification, phonics, print concepts, reading, and writing. At the end of Grade 1, children were assessed in the areas of phonics, reading comprehension, rhyming, writing, and teacher ratings for overall literacy development. By the end of the study (end of Grade 1), 7 percent of the intervention group and 11 percent of the control group had been lost to attrition. See Table 18 (Appendix) for more information regarding the study in Armenia.

The study in Bosnia and Herzegovina consisted of a quasi-experimental design, with four communities identified for the intervention, then four similar communities identified to serve as a no-treatment comparison group. Children averaged age 5 at baseline. At outcome, children were assessed in the areas of letter identification, phonics, print concepts, reading, and writing. At the end of Grade 1, children were assessed using teacher ratings for overall literacy development. By the end of Grade 1, 26 percent of the intervention group and 18 percent of the control group had been lost to attrition. See Table 19 (Appendix) for information regarding this program evaluation in Bosnia and Herzegovina.

In Kazakhstan, four communities were first selected for the intervention, and then four similar communities were identified to serve as a no-treatment comparison group. Then

35 The Campbell Collaboration | www.campbellcollaboration.org

individual children in the comparison group communities were identified based on a demographic match with individual children in the intervention group communities. Children in both groups averaged 5.5 years old at baseline. At outcome, children were assessed in the areas of letter identification, phonics, print concepts, reading, and writing. At the end of Grade 1, children were assessed using teacher ratings for overall literacy development. By the end of Grade 1, 5 percent of the intervention group and 3 percent of the control group had been lost to attrition. See Table 20 (Appendix) for more information regarding the program evaluation in Kazakhstan.

In Tajikistan, four communities were first selected for the intervention, and then four similar communities were identified to serve as a no-treatment comparison group. Then individual children in the comparison group communities were identified based on a demographic match with individual children in the intervention group communities. Children in both groups averaged 6.5 years old at baseline. At outcome, children were assessed in the areas of letter identification, phonics, print concepts, reading, and writing. At the end of Grade 1, children were assessed using teacher ratings for overall literacy development. By the end of Grade 1, 4 percent of the intervention group and 5 percent of the control group had been lost to attrition. See Table 21 for more information regarding this program evaluation in Tajikistan.

UNICEFs Getting Ready for School: A Child-to-Child Approach

Five separate evaluations were carried out to assess the effects of UNICEFs Getting Ready for School: A Child-to-Child Approach interventionone each in Bangladesh, Democratic Republic of Congo, Ethiopia, Yemen, and Tajikistan. All were carried out in communities where children lacked access to preprimary education. In all countries except Tajikistan, parents typically had a low level of education, and on-time enrollment of children in primary school was low in the participating communities. The program was provided to children one year before on-time Grade 1 enrollment. The program consisted of peer tutoring sessions where older students were trained by teachers to support young childrens school readiness across a range of academic areas. The older and younger children met together in the school approximately weekly during the school year, and met for additional sessions in the community in all countries except Tajikistan.

The study of UNICEFs Getting Ready for School: A Child-to-Child Approach in Bangladesh was an RCT. One district and two upazilas (subdistricts) from the district within each of the six administrative divisions of the country were selected based on high drop-out and low primary school completion rates. Then the pair of upazilas within each district was randomly assigned, with one each in the intervention group and the control group. Then five schools from each upazila in each group were randomly selected to participate in the program (if in the intervention group) and the evaluation. Childrens ages were not available (birth registration is not universal in Bangladesh, and many parents do not know their childs date of birth). Outcomes assessed at the conclusion of the program included letter identification, reading, and writing. Teachers were also asked to assess childrens literacy development at

36 The Campbell Collaboration | www.campbellcollaboration.org

and of grade one. At the first outcome assessment (end of program year), attrition was 12 percent in the intervention group and 11 percent in the control group. Grade 1 outcome data was unavailable for 47 percent of the intervention group and 5 percent of the control group (these figures includes children who had not yet enrolled in Grade 1 so they had no teachers to survey). See Table 22 (Appendix) for more information about this study in Bangladesh.

The study of UNICEFs Getting Ready for School: A Child-to-Child Approach in the Democratic Republic of Congo consisted of a quasi-experimental design. Fifteen schools in Kinshasa and 10 in Mbandaka were identified for the intervention. Then 15 matched schools in Kinshasa and 10 in Mbandaka were identified to serve as the control group. Children averaged age 5 at baseline. Outcomes assessed included letter identification, reading and writing. There were significant difficulties with data collection caused by impassable roads and incursions of the civil conflict, resulting in missing data from Mbandaka. After one year, follow-up data was available for only 38 percent of the intervention group and 35 percent of the control group. See Table 23 (Appendix) for more information about this study in the Democratic Republic of Congo.

The study of UNICEFs Getting Ready for School: A Child-to-Child Approach in Ethiopia consisted of a quasi-experimental design. A total of 20 schools from Harar, Oromia, and Tigray were identified for the intervention. Then 20 matched schools from the same school clusters as the intervention schools were identified to serve as the control group. Children averaged age 6 at baseline. Outcomes assessed included letter identification, reading and writing. This study had many issues with missing data, and attrition was very high (56 percent for intervention group and 30 percent for control group). See Table 24 (Appendix) for more information about the study in Ethiopia.

The study of UNICEFs Getting Ready for School: A Child-to-Child Approach in Tajikistan consisted of an RCT. Forty schools took part in the evaluation: 20 from Rumi and 20 from Bokhtar. Within each region, half of the schools were randomly assigned to the intervention group and half to the control group. Children averaged age 7 at baseline. Outcomes assessed at the conclusion of the program included letter identification, reading, and writing. Teachers were also asked to assess childrens literacy development at the end of Grade 1. At the first outcome assessment (end of program year), attrition was less than 1 percent in the intervention group and less than 1 percent in the control group. At the end of Grade 1, attrition was 9 percent in the intervention group and 8 percent in the control group. See Table 25 (Appendix) for more information about this study in Tajikistan.

The study of UNICEFs Getting Ready for School: A Child-to-Child Approach in Yemen was an RCT. Thirty schools in the Taiz governorate took part in the evaluation, with 10 each from Haifan, Al-Makha, and Mawza. Within each of the three regions, half of the schools were randomly assigned to the intervention group and half to the control group. Children averaged just under 6 years of age at baseline. Outcomes assessed at the conclusion of the program included letter identification, reading, and writing. Teachers were also asked to

37 The Campbell Collaboration | www.campbellcollaboration.org

assess childrens literacy development at the end of Grade 1. At the first outcome assessment (end of program year), attrition was 12 percent in the intervention group and 15 percent in the control group. At the time of the Grade 1 assessment, 19 percent of the intervention group and 38 percent of the control group did not have teacher surveys available. These figures reflect the fact that not all children enrolled in Grade 1 (especially in the control group). See Table 26 (Appendix) for more information about this study in Yemen.

Read India

The evaluation of Read India was a large RCT carried out in Jaunpur District, in the state of Uttar Pradesh, India. From a pool of 280 villages that participated in a baseline assessment, 65 were randomly assigned to the intervention group, and 85 to a control group (the remaining participated in other initiatives not part of the current review). In the intervention group villages, Read India program was made available to children ages 7 to 14 from those villages. The control group villages did not have Read India programming available. Outcomes assessed included letter recognition, single word reading, and text reading. The reported study sample only included children who participated in both the pretest and the posttest. Original sample size is not available for the group of children involved in this specific intervention. See Table 27 (Appendix) for more information about this study.

4.3 RISK OF BIAS IN INCLUDED STUDIES

4.3.1 Interventions to help parents support their childrens school readiness

Certain limitations of the studies included in this analysis pose potential risks of bias to the results. For the evaluation of OSIs Getting Ready for School initiative, the studies conducted in Armenia, Kazakhstan, and Tajikistan had substantial imbalance in childrens literacy scores at baseline, with treatment children outperforming comparison children by 0.310.50 standard deviations across the three countries, which was above our planned cut point and may have biased the impact findings even with adjustment for baseline covariates. It is also worth noting that in the Tajikistan study, the effect size for the second-year impact on childrens literacy skills was unusually large for educational interventions (1.74), which is particularly perplexing, given that the intervention had a substantial negative impact in the first year (effect size = -0.32). Therefore, findings from the Tajikistan study, particularly the second-year findings, may have biased the results of the meta-analysis. No significant risks of bias were evident in the study of OSIs Getting Ready for School initiative in Bosnia and Herzegovina, nor in the study of the Family Supported Pre-Reading Program.

4.3.2 Tutoring interventions

The studies included in this analysis each had a number of potential risks of bias to the results. All studies on UNICEFs Getting Ready for School: A Child-to-Child Approach were included in a single report and included information about difficulties in collecting data. For

38 The Campbell Collaboration | www.campbellcollaboration.org

example, in the Democratic Republic of Congo, flooding and conflict incursions hindered data collection in almost half of the matched pairs of schools, resulting in a loss of over half the sample for follow-up. In Ethiopia, poor implementation of data collection resulted in missing data. In Bangladesh, there were substantial differences between the treatment and control groups in their participation in other early childhood programs, with 69 percent of the control group compared to 11 percent of the intervention group attending other early childhood programs. In Tajikistan, the program was suspended during the winter for two months, thus impacting the implementation of all 35 planned sessions. The report did not provide information about the exact number of planned participants lost from the evaluation due to data collection difficulties; only the numbers of participants in each countrys evaluation are included. No significant risks of bias were evident in the trial in Yemen.

4.4 SYNTHESIS OF RESULTS

4.4.1 Effects of educational television

Two educational television studies, Baydar et al. (2008) and Borzekowski and Henry (2011), met the criteria for inclusion in the review. However, neither study provided the summary statistics needed to compute effect sizes. Borzekowski and Henry (2011) randomly assigned children to three levels of exposure to Jalan Sesama: control, low, and high. The high-exposure group watched three to four episodes of the target program over a 14 week period, while the low-exposure group watched one episode a week over 14 weeks. Childrens performance was assessed in letter recognition, phonemic awareness, and writing. The data were analyzed using regression to control for baseline scores, gender, age, parent education, and exposure. Children with the most exposure to the target intervention had the largest increases in test scores from baseline to post-treatment.

Baydar et al. (2008) randomly assigned low-income mothers and their preschool children to one of three groups: an experimental group that was asked to watch the intervention program for 13 weeks, a control group that was asked to watch an alternative program for 13 weeks, and a natural observation group that was informed of the intervention program but not required to watch any television during the study. The study reported on a regression analysis using self-reported exposure to the target intervention program and pretest assessments to compare the groups on syllabification and vocabulary. The effects of the target intervention were related to the level of exposure, with preschool children with the highest exposure to the target program performing higher than the children in the control group or the natural observation group.

4.4.2 Effects of interventions to help parents support their childrens school readiness

There were five studies in this category. Four of the studies were of the Open Society Institutes Getting Ready for School intervention and the fifth was the evaluation of the Pre-

39 The Campbell Collaboration | www.campbellcollaboration.org

Reading Training Program. All five of these studies were included in a meta-analysis. The effect sizes for the Open Society Institute studies were computed using Hedges (2007) dT2

estimate, which requires the use of the intraclass correlation coefficient, , and an estimate of ST, the total variance for the outcome. The Family Supported Pre-Reading Program study used a non-clustered trial design, and the usual standardized mean difference as given in equation (1) was used to compute the effect size. For the meta-analysis, only the measure of basic literacy was used as this measure was closest conceptually to those in the other four studies. Because basic literacy was only measured at post-test, this study appears only in the analysis for the immediate post-test. Table 28 and Table 29 in the Data and Analyses section provide the summary statistics used to compute the effect sizes for the two sets of studies. See Table 5 and Table 6 (below) for results of the meta-analysis.

OSIs Getting Ready for School initiative measured childrens beginning literacy in the areas of print concepts, letter sounds, letter identification, and beginning writing, given as an immediate posttest and as a follow-up in year 2. The Family Supported Pre-Reading Program study measured basic reading and writing skillsincluding letter identification and reading and writing simple words and sentencesgiven as an immediate posttest. The estimates of the effect size for the immediate literacy assessment were heterogeneous, with a variance component of 2 equal to 0.19 that is significantly different from zero (2= 24.47, df=4, p=0.00). The value of I2 is 83.65. The overall mean effect size is 0.35 with a 95 percent confidence interval that includes zero (-0.074, 0.77). Given the significant heterogeneity, and the potential bias for non-equivalent groups in Armenia, Kazakhstan and Tajikistan, the mean effect size should be interpreted with caution. We did not conduct a sensitivity analysis excluding these three studies as there are only five studies in the review. The year 2 literacy assessment is also heterogeneous, with the study from Tajikistan contributing a large effect size of 1.74. The variance component of 2 is equal to 0.69, and is significantly different from zero (2= 71.19, df=3, p=0.00). The value of I2 is 95.79. The overall effect size mean is 0.48 with a 95 percent confidence interval that covers zero (-0.36, 1.31). The mean effect sizes for beginning literacy both in the immediate posttest and in the year 2 follow-up are not different from zero. The significant heterogeneity, and potential problems with non-equivalent groups in Armenia, Kazakhstan and Tajikistan require caution in the interpretation of the mean effect size. We did not conduct a sensitivity analysis excluding these three studies as there are only five studies in the review.

40 The Campbell Collaboration | www.campbellcollaboration.org

Table 5 Parent Education Studies: Overall Literacy of Year 1

Overall Literacy Year 1

Studyname Statistics for each study Hedges's g and 95% CIHedges's Standard Lower Upperg error Variance limit limit Z-Value p-Value

Armenia 0.630 0.224 0.050 0.192 1.068 2.817 0.005 Bosnia 0.340 0.245 0.060 -0.140 0.820 1.388 0.165 Kazakhstan 0.440 0.141 0.020 0.163 0.717 3.111 0.002 Tajikistan -0.320 0.141 0.020 -0.597 -0.043 -2.263 0.024 Buyuktaskapu 0.810 0.300 0.090 0.222 1.398 2.700 0.0070.350 0.216 0.047 -0.074 0.774 1.620 0.105

-1.00 -0.50 0.00 0.50 1.00

Favours Control Favours Treatment

Cluster-adjusted Effect Sizes

41 The Campbell Collaboration | www.campbellcollaboration.org

Table 6 Parent Education Studies: Overall Literacy of Year 2 (Grade 1)

Overall Literacy Year 2

Studyname Statistics for each study Hedges's g and 95% CI

Hedges's Standard Lower Upperg error Variance limit limit Z-Value p-Value

Armenia 0.010 0.173 0.030 -0.329 0.349 0.058 0.954 Bosnia 0.050 0.224 0.050 -0.388 0.488 0.224 0.823 Kazakhstan 0.100 0.141 0.020 -0.177 0.377 0.707 0.480 Tajikistan 1.740 0.173 0.030 1.401 2.079 10.046 0.0000.477 0.424 0.180 -0.355 1.308 1.123 0.261

-1.00 -0.50 0.00 0.50 1.00

Favours Control Favours Treatment

Cluster-adjusted Effect Sizes

42 The Campbell Collaboration | www.campbellcollaboration.org

4.4.3 Effects of tutoring interventions

As described above, the Read India and the Getting Ready for School: A Child-to-Child Approach interventions used different mechanisms for delivering tutoring support. We were unable to include Read India in the meta-analysis because the study did not report on the tutoring separately, and thus was not an intervention that could be compared to the other tutoring interventions. Therefore, the five studies of Getting Ready for School: A Child-to-Child Approach were combined for meta-analysis, but the effects of Read India are reported separately.

Getting Ready for School: A Child-to-Child Approach

For the Getting Ready for School: A Child-to-Child Approach studies, schools or districts were randomly assigned to treatment and control groups. Effect sizes for these studies were computed using Hedges (2007) dT2 estimate, which requires the use of the intraclass correlation coefficient, , and an estimate of ST, the total variance for the outcome. Table 30 and Table 31 in the Data and Analyses section provide the summary statistics used to compute the effect sizes. The trial in Bangladesh differed from the other studies in that the treatment was randomly assigned to subdistricts, and thus the third-level intraclass correlation was used for the computation of the effect sizes at the level of the subdistrict. See Tables 7 through 13 (below) for results of the meta-analysis.

Tables 24 through 30 present the Forest plots for the literacy outcomes in years 1 and 2 for the Getting Ready for School: A Child-to-Child Approach intervention. For the Reading Total score in year 1, there is significant heterogeneity among the studies. The estimate of the variance component, 2, is equal to 0.162, and is significantly different from zero (2= 26.37, df= 3, p=0.00). The value of I2 is 88.62, also indicating significant variation among studies. The overall mean effect size is 0.15, but the 95 percent confidence interval covers zero (-0.27, 0.58). There is also significant heterogeneity among studies on the subscale scores. The Beginning Reading subscale has a variance component, 2, equal to 0.061, and is significantly different from zero (2= 10.60, df=3, p=0.014). The value of I2 is 71.68. On the Letter Identification subscale, the variance component, 2, is equal to 0.13, and is significantly different from zero (2 = 24.35, df=4, p=0.00). The value of I2 is 83.57. Neither of the mean effect sizes is significantly different from zero.

There is also significant variation among studies on the Writing scale in year 1. The variance component, 2, is equal to 0.046, and is significantly different from zero (2= 10.28, df=4, p=0.036). The value of I2 is 61.08. Unlike the other outcomes in year 1, the mean effect size for Writing is significantly different from zero, with a value of 0.265 and a 95 percent confidence interval of (0.018, 0.51).

The year 2 outcomes included Reading, Writing, and Overall Literacy. There is significant variation among the three studies that report the Reading follow-up, with a variance component, 2, equal to 0.058, significantly different from zero (2= 7.8, df=2, p=0.020).

43 The Campbell Collaboration | www.campbellcollaboration.org

The value of I2 is 74.36. The mean effect size for Reading is not significantly different from zero. The three studies that report a Writing follow-up assessment are homogeneous, with a variance component of 2 equal to 0.00. The value of I2 is also 0.0. The overall mean effect size is 0.03, with a 95 percent confidence interval that covers zero (-0.10, 0.17). The three studies that report the Overall Literacy measure are also homogeneous, with a variance component of 2 equal to 0.031 and an I2 of 38.05. The mean effect size is 0.055, with a 95 percent confidence interval that covers zero (-0.15, 0.26).

As discussed earlier, three of the five studies included in this meta-analysis were RCTs (Bangladesh, Yemen and Tajikistan). A sensitivity analysis excluding quasi-experimental designs would be based on only three studies. We prefer to use caution in interpreting the results given the small number of studies, all conducted under the same program.

Read India

Banerjee et al. (2008) met the study inclusion criteria, but the intervention included both tutoring support and library visits. The study did not report on the results of the tutoring intervention separately, and thus the study was not included in the meta-analysis. Banerjee et al. (2008) report on a randomized experiment to increase community members awareness of and participation in local schools functioning and childrens educational outcomes. One of the three interventions in the study included training villagers to teach children reading skills using the Read India program. Children exposed to the Read India intervention increased their reading skills compared to children in a control group, with children with the lowest pretest scores making the most improvement (Table 32 in Data and Analyses section).

44 The Campbell Collaboration | www.campbellcollaboration.org

Table 7 UNICEF Child-to-Child Studies: Reading Total Score of Year 1

Model Studyname Statistics for each study Hedges's g and 95% CI

Hedges's Standard Lower Upperg error Variance limit limit Z-Value p-Value

DRC 0.685 0.174 0.030 0.343 1.027 3.929 0.000 Ethiopia 0.185 0.213 0.045 -0.233 0.603 0.868 0.385 Tajikistan 0.041 0.164 0.027 -0.281 0.362 0.248 0.804 Bangladesh -0.243 0.072 0.005 -0.384 -0.101 -3.366 0.001 Random 0.151 0.217 0.047 -0.274 0.576 0.696 0.486

-1.00 -0.50 0.00 0.50 1.00

Favors Control Favours Treatment

Cluster-adjusted Effect Sizes

Reading Total Post-test

45 The Campbell Collaboration | www.campbellcollaboration.org

Table 8 UNICEF Child-to-Child Studies: Beginning Reading Subtest of Year 1

Beginining Reading Subscore Post-test

Study name Statistics for each study Hedges's g and 95% CIHedges's Standard Lower Upperg error Variance limit limit Z-Value p-Value

DRC 0.240 0.173 0.030 -0.099 0.579 1.386 0.166 Ethiopia -0.300 0.200 0.040 -0.692 0.092 -1.500 0.134 Tajikistan 0.030 0.173 0.030 -0.309 0.369 0.173 0.862 Bangladesh -0.350 0.100 0.010 -0.546 -0.154 -3.500 0.000

-0.107 0.147 0.022 -0.396 0.182 -0.728 0.467

-1.00 -0.50 0.00 0.50 1.00

Favors Control Favours Treatment

Cluster-adjusted Effect Sizes

46 The Campbell Collaboration | www.campbellcollaboration.org

Table 9 UNICEF Child-to-Child Studies: Letter Identification Subtest of Year 1

Studyname Statistics for each study Hedges's g and 95% CIHedges's Standard Lower Upper

g error Variance limit limit Z-Value p-Value

DRC 0.710 0.173 0.030 0.371 1.049 4.099 0.000 Ethiopia 0.240 0.224 0.050 -0.198 0.678 1.073 0.283 Tajikistan 0.040 0.173 0.030 -0.299 0.379 0.231 0.817 Yemen 0.370 0.173 0.030 0.031 0.709 2.136 0.033 Bangladesh -0.200 0.100 0.010 -0.396 -0.004 -2.000 0.0460.219 0.178 0.032 -0.129 0.568 1.234 0.217

-1.00 -0.50 0.00 0.50 1.00

Favours Control Favours Treatment

Cluster-adjusted Effect Sizes

Letter Identification Posttest

47 The Campbell Collaboration | www.campbellcollaboration.org

Table 10 UNICEF Child-to-Child Studies: Writing Subtest of Year 1

Writing Post-test

Study name Statistics for each study Hedges's g and 95% CIHedges's Standard Lower Upper

g error Variance limit limit Z-Value p-Value

DRC 0.730 0.173 0.030 0.391 1.069 4.215 0.000 Ethiopia 0.040 0.200 0.040 -0.352 0.432 0.200 0.841 Tajikistan 0.040 0.173 0.030 -0.299 0.379 0.231 0.817 Yemen 0.150 0.316 0.100 -0.470 0.770 0.474 0.635 Bangladesh 0.280 0.100 0.010 0.084 0.476 2.800 0.0050.265 0.126 0.016 0.018 0.512 2.105 0.035

-1.00 -0.50 0.00 0.50 1.00Favours Control Favours Treatment

Cluster-adjusted Effect Sizes

48 The Campbell Collaboration | www.campbellcollaboration.org

Table 11 UNICEF Child-to-Child Studies: Reading Achievement of Year 2 (Grade 1)

Reading Achievement Follow-up

Study name Statistics for each study Hedges's g and 95% CIHedges's Standard Lower Upper

g error Variance limit limit Z-Value p-Value

Tajikistan 0.020 0.141 0.020 -0.257 0.297 0.141 0.888 Yemen 0.460 0.200 0.040 0.068 0.852 2.300 0.021 Bangladesh -0.160 0.100 0.010 -0.356 0.036 -1.600 0.1100.069 0.163 0.027 -0.250 0.388 0.423 0.672

-1.00 -0.50 0.00 0.50 1.00

Favours Control Favours Treatment

Cluster-adjusted Effect Sizes

49 The Campbell Collaboration | www.campbellcollaboration.org

Table 12 UNICEF Child-to-Child Studies: Writing Achievement of Year 2 (Grade 1)

Writing Achievement Follow-up

Study name Statistics for each study Hedges's g and 95% CIHedges's Standard Lower Upper

g error Variance limit limit Z-Value p-Value

Tajikistan 0.060 0.100 0.010 -0.136 0.256 0.600 0.549 Yemen 0.240 0.283 0.080 -0.314 0.794 0.849 0.396 Bangladesh -0.020 0.100 0.010 -0.216 0.176 -0.200 0.8410.033 0.069 0.005 -0.102 0.167 0.480 0.631

-1.00 -0.50 0.00 0.50 1.00

Favours Control Favours Treatment

Cluster-adjusted Effect Sizes

50 The Campbell Collaboration | www.campbellcollaboration.org

Table 13 UNICEF Child-to-Child Studies: Overall Literacy of Year 2 (Grade 1)

Overall Literacy Grade 1

Studyname Statistics for each study Hedges's g and 95% CIHedges's Standard Lower Upper

g error Variance limit limit Z-Value p-Value

Tajikistan 0.030 0.141 0.020 -0.247 0.307 0.212 0.832 Yemen 0.390 0.224 0.050 -0.048 0.828 1.744 0.081 Bangladesh -0.050 0.100 0.010 -0.246 0.146 -0.500 0.6170.055 0.106 0.011 -0.152 0.262 0.520 0.603

-1.00 -0.50 0.00 0.50 1.00

Favours Control Favours Treatment

Cluster-adjusted Effect Sizes

51 The Campbell Collaboration | www.campbellcollaboration.org

5 Discussion

5.1 SUMMARY OF MAIN RESULTS

We included studies in three topic areas: educational television, interventions that teach parents how to support their childrens school readiness, and tutoring by older peers or community members.

In the area of educational television, two studies showed that children who received high levels of exposure to educational programming (at least 3 to 5 times per week) outperform the low-exposure and control groups in literacy development (such as vocabulary), although the exact size of this effect was unclear. It is important to note that these results were obtained in contexts where children do not have heavy exposure to screen time otherwise.

There were five studies that assessed the effects of interventions intended to support parents ability to help their children develop school readiness (including in the area of literacy). Across studies, we found no significant effect of this approach on childrens literacy development. However, there were positive impacts for individual studies. The low number of studies does not allow us to draw conclusions about where, how, or with whom these interventions must be implemented to achieve an effect on childrens literacy.

The Read India and the Getting Ready for School: A Child-to-Child Approach interventions used different mechanisms for delivering tutoring support. The Getting Ready for School intervention used peer-assisted learning, and had a significant effect on childrens early writing across countries, and significant effects on other aspects of childrens literacy development in some countries. These effects were sustained over time in some but not all cases. The Read India intervention used community members to help children increase their reading skills, with children with the lowest pretest performance making the most improvement.

All of these interventions relate to the logic model presented in Figure 1. They showed that at least in some contexts, providing education and support to parents, families and community members can lead to behavior change among those groups (such as spending time showing children how to write letters), which in turn leads to improved literacy outcomes. In other cases, adults create programming delivered

52 The Campbell Collaboration | www.campbellcollaboration.org

through mechanisms such as educational television and radio, and in this way influence childrens literacy development.

5.2 OVERALL COMPLETENESS AND APPLICABILITY OF EVIDENCE

Our screening of 10,430 citations from across academic and grey literature yielded just 13 studies that met our criteria, from across three topic areas (plus studies of educational radio that were excluded from the current review because a review of its effects already existed). The comprehensiveness of our search and ability to obtain all articles identified for review makes us confident that we identified the literature that was available. But these results also tell us that there are serious limitations in the availability of evidence.

We found that there are many types of interventions occurring in LMICs that are intended to improve childrens literacy outcomes and take place outside of the formal education system. However, only a fraction of these types of interventions offer any empirical evidence regarding their effectiveness. This leaves a large number of intervention approaches that do not appear to have empirical evidence for (or against) their effectiveness in improving childrens literacy development. Many of these interventions are widely used in LMICs and receive significant investment, such as the provision of libraries (standing or mobile), distribution of books and print materials (including local language publishing), the teaching of literacy through religious instruction, and the distribution of laptops and e-readers. For example, a recent comprehensive review found a widespread use of mobile technology in LMICs as part of the effort to improve childrens literacy but found almost no empirical evidence regarding the effectiveness of these efforts and investments (Wagner, Castillo, Murphy, Tuz Zahra, Crofton, & Phelan, 2014).

There are also significant gaps in the literature with regard to the age of children receiving the intervention. Only one of the studies we identified (Read India) involved children past age 7. The remainder focused on children ages 3 through 6. Early childhood is a critical time for development, but we also know that many children continue to need additional support to become fluent readers as they move through primary school.

The available literature does not allow us to draw any conclusions regarding the relative effects of interventions by context (for example, with children whose parents are literate versus illiterate, or with children who are monolingual versus dual language learners).

And finally, Latin American countries are noticeably absent from this list, despite our efforts to access both academic and grey literature in English, French, and Spanish. Some studies from Latin America were identified in the initial search, but none met our criteria for inclusion in this review.

53 The Campbell Collaboration | www.campbellcollaboration.org

5.3 QUALITY OF THE EVIDENCE

Among studies of a relevant intervention screened for this review, they quite clearly fell into two categories: studies that used a rigorous evaluation design and studies that did not. For studies that were excluded on methodological grounds, in all cases the issue was either the lack of a control group or the use of a quasi-experimental design without a baseline. As noted in the section on Selection of Studies, we did include in the review the four OSI studies that reported differences in baseline measures of literacy and pre-literacy. These four studies all evaluated the same intervention in different countries, and provided some tentative evidence for the treatments effectiveness. There were no studies that almost made it into the review.

Even though included studies used a rigorous evaluation design, they all suffered from potential bias either from differences in the control and treatment groups at baseline, or from reported problems with data collection in difficult regions. Thus, the meta-analysis results must be interpreted with caution.

We were able to calculate effect sizes for 11 of the 13 studies included in this review. The studies overlapped substantially in the types of outcomes assessed and the age groups of children studied, allowing us to combine studies for meta-analysis across two topics.

5.4 LIMITATIONS AND POTENTIAL BIASES IN THE REVIEW PROCESS

There were several limitations to this review. First, we had hoped to obtain information that would allow us to identify the relative effectiveness of different interventions in the same context. In other words, we were hoping to be able to provide information for the field regarding the effectiveness of interventions with the same conditions and those conducted for different populations within the same general context. The scarcity of empirical studies and their limited focus on just a few interventions prevents us from being able to provide this information. Numerous descriptions of interventions exist, but few contained a study of program effectiveness in reference to a comparison group. Second, we found only one study that addressed an intervention for children ages 7 and older, and found no studies from Latin America. Therefore, we are left with significant gaps in our understanding of what works in LMICs to improve childrens literacy outcomes using interventions outside of the formal education system.

5.5 AGREEMENTS AND DISAGREEMENTS WITH OTHER STUDIES AND REVIEWS

The current review complements a review of the effectiveness of educational radio in improving childrens literacy outcomes in Sub-Saharan Africa (Ho & Thukral, 2009).

54 The Campbell Collaboration | www.campbellcollaboration.org

This reviews authors are not aware of any other studies or systematic reviews in LMICs that assess the effectiveness of interventions outside of the formal education system for improving childrens literacy development.

55 The Campbell Collaboration | www.campbellcollaboration.org

6 Authors Conclusions

6.1 IMPLICATIONS FOR PRACTICE AND POLICY

Educational Television

Educational television approaches to child literacy in developing countries use onair television broadcasts or DVDs of TV programs to promote a variety of literacy and other developmental outcomes in preschool children, from letter recognition, pattern grouping, and basic counting, to health, social development, and cultural awareness. Most educational efforts are intended to be compensatory, provided in countries where formal preschool programs are either not widely available or not widely affordable, and target children and families from lower socio-economic backgrounds. Few are intended primarily to supplement school- or center-based programs already accessed by children.

There has been extensive research on the extent of general television viewing by preschool children, and a broad range of effects, both positive and negative, have been reported. Negative effects generally are associated with entertainment viewing, and positive effects generally associated with viewing educational television. Most studies of educational television have examined the effects of children watching local translations or adaptations of Sesame Street, which is available in more than 120 countries. However, most studies are correlational, and few reports use even quasi-experimental designs, much less true experimental designs that use a randomized controlled trial approach. Thus, although the majority of the research points to positive child literacy outcomes associated with watching educational programs, confidence in that conclusion is reduced by the nonexperimental approaches used in the great majority of the studies. The studies reviewed here did show that educational television (whether viewed at home or outside of the home) has a positive effect on childrens early literacy development when children view these programs three to five days per week over several months. Occasional viewing did not produce these effects.

Parent Education and Training Programs

Among the most common out-of-school approaches to supporting childrens early academic learning are programs that aim to help parents be better able to support their childs learning. In the developing world, these approaches are typically used in settings of limited formal preschool opportunities but high rates of parental literacy.

56 The Campbell Collaboration | www.campbellcollaboration.org

Programs are quite varied in structure, duration and intensity; with take-home assignments for parents and children to work on between sessions. This approach requires parents or other adult caregivers to have the time available to attend sessions and to engage in these activities with their children.

The results of the current review tell us that, though these approaches may work well in some contexts, there is no evidence that they work universally. However, the limited number of studies available makes it difficult to draw any valid conclusions regarding the kind of context and/or intervention required for this approach to have a positive effect.

Tutoring Approaches

Child-to-child approaches to literacy in developing countries use older children in primary grades to help preschool children develop literacy skills. There is a vast body of literature in general on peer helping or mentoring in developed countries, in particular on peer tutoring. These studies tend to show mostly positive, but sometimes null, results for the mentored or tutored students and generally positive outcomes for the older students who are the peer helpers, mentors, or tutors. However, these studies typically focus on school-age children and youth helping other school-age children and youth, not on helping preprimary-aged children. They also tend to be framed as supplemental approaches to enrich school-based content learning, not as a primary approach for learning broadly foundational literacy skills. The Read India intervention provides a similar model, using community volunteers. These few studies suggest that the effects of tutoring vary widely from country to country.

6.2 IMPLICATIONS FOR RESEARCH

The results of this review have substantial implications for future research. There were many practices widely used in LMICs that work outside of formal education systems with a goal of improving childrens learning outcomes, but that very few have any evidence for (or against) their effectiveness. The evidence that is available is almost entirely focused on children ages 5 to 7 years, and on a very limited selection of intervention strategies (Figure 3). Policymakers and practitioners implement programming that they believe will be effective, but in most cases, they do not have adequate information available for evidence-based decision making. Empirical evidence is urgently required regarding the effectiveness of interventions that are currently receiving significant investment of scarce resources, such as technology-based supports.

Too few studies were available for us to come to any conclusions regarding patterns of effectiveness. And the studies that we did find showed that similar programming can have a positive effect on childrens literacy development in some contexts and none in others.

57 The Campbell Collaboration | www.campbellcollaboration.org

Based on this review, we make the following recommendations to improve the evidence for what works to improve childrens literacy outcomes outside of formal education system: First, the field should prioritize studying interventions that are already in widespread use, but lack evidence of their effectiveness. Otherwise, there is no way for stakeholders (funders, program implementers, families, etc.) to know to what extent the intervention is a good use of their time and other resources (versus something else that may be more effective or may achieve the same or better outcomes with fewer resources).

Second, for interventions that have a positive impact at least some contexts (but maybe not others), investment should be made in replication studies to determine which children will benefit from these interventions, and under what conditions. The intervention may be effective or more effective specifically for children with certain characteristics (e.g., dual language learners, 7-year-olds, rural, etc.). The conditions of implementation include quality of implementation, dosage, delivery setting, and so on.

In carrying out these first and second recommendations, the field will begin to address our third recommendation, which is expanding the evidence base to include under-studied populations (such as children over age 7) and regions (such as Latin America).

When policymakers and practitioners select programming, they need to know what will be effective in their particular context, for the population they wish to serve. Carrying out these recommendations would greatly improve the evidence base available to stakeholders who wish to make wise decisions about what will help children in their country or community improve their literacy outcomes through support outside of formal education systems.

58 The Campbell Collaboration | www.campbellcollaboration.org

7 References

7.1 REFERENCES TO INCLUDED STUDIES

American Institutes for Research. (2012a). Evaluation of the Open Society

Foundation Getting Ready for School Initiative in Armenia. Washington, DC: Author.

American Institutes for Research. (2012b). Evaluation of the Open Society

Foundation Getting Ready for School Initiative in Bosnia and Herzegovina. Washington, DC: Author.

American Institutes for Research. (2012c). Evaluation of the Open Society

Foundation Getting Ready for School Initiative in Kazakhstan. Washington, DC: Author.

American Institutes for Research. (2012d). Evaluation of the Open Society

Foundation Getting Ready for School Initiative in Tajikistan. Washington, DC: Author.

Banerjee, A. V., Banerji. R., Duflo, E., Glennerster, R., & Khemani, S. (2008).

Pitfalls of participatory programs: Evidence from a randomized evaluation in education in India. Policy Research Working Paper 4584. Washington, DC: World Bank.

Baydar, N., Kaitibai, ., Kntay, A. C., & Gken, F. ( 2008). Effects of an educational television program on preschoolers: Variability in benefits. Journal of Applied Developmental Psychology, 29, 349360.

Borzekowski, D. L. G., & Henry, H. K. (2011). The impact of Jalan Sesama on the educational and healthy development of Indonesian preschool children: An experimental study. International Journal of Behavioral Development, 35(2), 169-179.

Byktakapu, S. (2012). Effect of family supported pre-reading training program given to children in preschool education period on reading success in primary school. Educational Sciences: Theory and Practice, 12, 309316.

59 The Campbell Collaboration | www.campbellcollaboration.org

UNICEF. (2013). Getting ready for school: A child-to-child approach, program evaluation report. New York, NY: Author.

7.2 REFERENCES TO EXCLUDED STUDIES

Aboud, F. E., & Hossain, K. (2011). The impact of preprimary school on primary school achievement in Bangladesh. Early Childhood Research Quarterly, 26(2), 237246.

Aboud, F. E., Hossain, K., & OGara, C. (2008). The Succeed Project: Challenging early school failure in Bangladesh. Research in Comparative and International Education, 3(3), 295307.

Aga Khan Development Network (2012). Learning support centres. New Delhi:

Author.

Adnams, C. M., Sorour, P., Kalberg, W. O., Kodituwakku, P., Perold, M. D., Kotze, A., et al. (2007). Language and literacy outcomes from a pilot intervention study for children with fetal alcohol spectrum disorders in South Africa. Alcohol, 41(6), 403414.

Begeny, J., Yeager, A., & Martnez, R. (2012). Effects of Small-Group and One-on-

One Reading Fluency Interventions with Second Grade, Low-Performing Spanish Readers. Journal of Behavioral Education, 21(1), 5879.

Bekman, S. (2004). Early Home Intervention to Promote School Readiness: A

Turkish Experience. NHSA Dialog, 7(1), 1634.

Bekman, S., Aksu-Ko, A., & Erguvanl-Taylan, E. (2011). Effectiveness of an intervention program for six-year-olds: a summer-school model. European Early Childhood Education Research Journal, 19(4), 409431.

Bernbaum, M., Rivero Herrera, J., & Schiefelbein, E. (2010). Evaluation of

USAID/Perus Education Program: APRENDES and CETT-ANDINO. Washington, DC: Mitchell Group.

Borzekowski, D. L. G., & Macha, J. E. (2010). The role of Kilimani Sesame in the healthy development of Tanzanian preschool children. Journal of Applied Developmental Psychology, 31, 298305.

Brooker, S., Okello, G., Njagi, K., Dubeck, M. M., Halliday, K. E., Inyega, H., &

Jukes, M. C. H. (2010). Improving educational achievement and anaemia of school children: design of a cluster randomised trial of school-based malaria prevention and enhanced literacy instruction in Kenya. TRIALS, 11.

60 The Campbell Collaboration | www.campbellcollaboration.org

Cardoso-Martins, C., Lara Mesquita, T. C., & Ehri, L. (2011). Letter names and phonological awareness help children to learn lettersound relations. Journal of Experimental Child Psychology, 109(1), 2538.

Cristia, J. P., Ibarrarn, P., Cueto, S., Santiago, A., & Severn, E. (2012). Technology and Child Development: Evidence from the One Laptop per Child Program. IZA Discussion Paper 6401. Available at http://ftp.iza.org/dp6401.pdf.

De Souza, D. G., de Rose, J. C., Faleiros, T. C., Bortoloti, R., Hanna, E. S., &

McIlvane, W. J. (2009). Teaching Generative Reading Via Recombination of Minimal Textual Units: A Legacy of Verbal Behavior to Children in Brazil. International Journal of Psychology & Psychological Therapy, 9(1), 1944.

Dixon, P., Schagen, I., & Seedhouse, P. (2011). The impact of an intervention on childrens reading and spelling ability in low-income schools in India. School Effectiveness & School Improvement, 22(4), 461482.

Dowd, A. J., & Advisor, S. (2011). Literacy Boost: International Literacy Day 2011.

Reading.

Educational Development Center (2012). Southern Sudan Interactive Radio

Instruction. Washington, DC: Author.

Educational Development Center (2009). Radio Instruction to Strengthen

Education (RISE) in Zanzibar. Washington, DC: Author.

Educational Development Center (2007). Somali Interactive Radio Instruction

Program: Grade One Evaluation 2006/2007. Washington, DC: Author.

Ho, J., & Thukral, H. (2009). Tuned in to student success: Assessing the impact of interactive radio instruction for the hardest-to-reach. Washington, DC: Educational Development Center.

Iikdoan, N., & Kargin, T. (2010). Investigation of the Effectiveness of the Story-

Map Method on Reading Comprehension Skills among Students with Mental Retardation. Educational Sciences Theory and Practice, 10(3), 14891531.

Lucas, A. M., McEwan, P. J., Ngware, M., & Oketch, M. (2013). Improving Early

Grade Literacy in East Africa: Experimental Evidence from Kenya and Uganda. Working paper #2013-3. Newark, DE: University of Delaware.

Martinez, S., Naudeu, S., & Pereira, V. (2013). The promise of preschool in Africa: a randomized impact evaluation of early childhood development in rural Mozambique. Washington DC: World Bank.

Mills-Tettey, G. A., Mostow, J., Dias, M. B., Sweet, T. M., Belousov, S. M., Dias, M. F., & Gong, H. (2009). Improving child literacy in Africa: Experiments with an

61 The Campbell Collaboration | www.campbellcollaboration.org

automated reading tutor. Information Technologies and International Development, 6, 119.

Mishra, V., & Lal, R. (2006). Cognitive behavioral intervention for remediation of dyslexic children. Indian Journal of Community Psychology, 2(2), 1625.

Mithani, S., Alam, I., Babar, J. A., Dowd, J. A., Hanson, J., & Ochoa, C. (2011).

Literacy Boost Pakistan Year One Report. Westport, CT: Save the Children.

Neugebauer, S. R., & Currie-Rubin, R. (2009). Read-Alouds in Calca, Peru: A

Bilingual Indigenous Context. Reading Teacher, 62, 396405.

Ntuli, D., & Pretorius, E. J. (2005). Laying foundations for academic language competence: the effects of storybook reading on Zulu language, literacy and discourse development. Southern African Linguistics & Applied Language Studies, 23, 91-109.

Olivier, A. M., Anthonissen, C., & Southwood, F. (2010). Literacy development of

English language learners: the outcomes of an intervention programme in grade R. South African journal of Communication Disorders, 57, 5865.

Opel, A., Ameer, S. S., & Aboud, F. E. (2009). The effect of preschool dialogic reading on vocabulary among rural Bangladeshi children. International Journal of Educational Research, 48, 1220.

Piper, B. & Korda, M. (2011). EGRA Plus: Liberia Program Evaluation Report.

Research Triangle Park, SC: RTI.

Pretorius, E. J., & Machet, M. P. (2008). The impact of storybook reading on emergent literacy: evidence from poor rural areas in KwaZulu-Natal, South Africa. Mousaion, 26, 206234.

Rolla San Francisco, A., Arias, M., Villers, R., & Snow, C. (2006). Evaluating the impact of different early literacy interventions on low-income Costa Rican kindergarteners. International Journal of Educational Research, 45(3), 188 201.

Sailors, M., Hoffman, J. V., Pearson, D. P., Beretvas, S. M., & Metthee, B. (2010).

The effects of first- and second-language instruction in rural South African schools. Bilingual Research Journal 33, 2141.

Save the Children (2007). Save the Childrens Literacy Boost Results: Nepal.

Westport, CT: Author.

imek etin, ., & Alisinanolu, F. (2013). Investigation of the effects of writing and reading readiness studies on the print awareness of pre-school children. International Online Journal of Educational Sciences 5(1), 160161.

62 The Campbell Collaboration | www.campbellcollaboration.org

USAID Malawi (2009). Primary School Support Program: A School Fees Pilot

(PSSP:SFP) Final Report January 2006 February 2009. Retrieved from http://pdf.usaid.gov/pdf_docs/PDACO621.pdf

Web End =http://pdf.usaid.gov/pdf_docs/PDACO621.pdf

Van der Bijl, C., Alant, E., & Lloyd, L. (2006). A comparison of two strategies of sight word instruction in children with mental disability. Research in Developmental Disabilities 27, 4355.

Visser, M. J., & Chamberlain, A. (2004). Cognitive stimulation in pre-school facilities in a disadvantaged community. Journal of Psychology in Africa, 14, 1727.

Walter, S. L., & Dekker, D. E. (2011). Mother tongue instruction in Lubuagan: A case study from the Philippines. International Review of Education, 57(56), 667 683.

Wu, X., Anderson, R., Li, W., Wu, X., Li, H., Zhang, J., Zheng, Q., Zhu, J., Shu, H.,

Jiang, W., Chen, X., Wang, Q., Yin, L., He, Y., Packard, J., & Gaffney,. J. S. (2009). Morphological awareness and Chinese childrens literacy development: An intervention study. Scientific Studies of Reading, 13, 2652.

7.3 ADDITIONAL REFERENCES

Ball, J., Paris, S. G., & Govinda, R. (2014). Literacy and numeracy skills among children in developing countries. In D.A. Wagner (Ed.) Learning and education in developing countries: Research and policy for the post-2015 UN development goals (pp. 20-40). New York: Palgrave Macmillan.

Biostat. (2011). Comprehensive Meta-Analysis version 2.2.064. Retrieved from http://www.meta-analysis.com/

Web End =www.Meta-Analysis.com

Britto, P. R., Brooks-Gunn, J., & Griffin, T. (2006). Maternal reading and teaching patterns: Associations with school readiness in low-income, African-American families. Reading Research Quarterly, 41, 6889.

Britto, P. R., Oketch, M., & Weisner, T. S. (2014). Non-formal education and learning. In D. A. Wagner (Ed.), Learning and education in developing countries: Research and policy for the Post-2015 UN Development Goals. New York, NY: Palgrave Macmillan.

Chaudhury, N., Hammer, J. S., Kremer, M., Muralidharan, K. & Rogers, F. H.

(2006). Missing in action: Teacher and health worker absence in developing countries." Journal of Economic Perspectives, 20, 91-116.

Christenson, S. L. & Reschly, A. L. (Eds). (2010). Handbook of School-Family

Partnerships. New York: Routledge, Taylor and Francis.

63 The Campbell Collaboration | www.campbellcollaboration.org

Dickinson, D., & Neuman, S. B. (2006). Handbook of Early Literacy Research, Vol.

2. New York: Guilford.

Doiron, R. (2011). Camels, burros, elephants, boats, and trucks: Bringing books and literacy to children in remote communities. In L. Marquart & D. Oberg (Eds.), Global perspectives on school libraries: Projects and practices (pp. 159167). The Hague, Netherlands: International Federation of Library Associations and Institutions.

Hedges, L. V. (1981). Distribution theory for Glass estimator of effect size and related estimators. Journal of Educational Statistics, 6, 107-128.

Hedges, L. V. (2007). Effect sizes in cluster-randomized trials. Journal of

Educational and Behavioral Statistics, 32, 341-370.

Hoppers, W. (2006). Non-formal education and basic education reform: A conceptual review. Paris, France: UNESCO.

Lancy, D. F., Bock, J., & Gaskins, S. (Eds.) (2010). The anthropology of learning in childhood. Lanham, MD: Alta Mira.

Mares, M.-L., & Pan, Z. (2013). Effects of Sesame Street: A meta-analysis of childrens learning in 15 countries. Journal of Applied Developmental Psychology, 34, 140151.

UNESCO (2014). Teaching and learning: Achieving quality for all. Education for All

Global Monitoring Report. Paris, France: Author.

UNESCO (2012). The state of education in developing countries. Education for All

Global Monitoring Report. Paris, France: Author.

UNICEF. (2012). Inequities in early childhood development: What the data say evidence from the Multiple Indicator Cluster Survey. New York, NY: Author.

Wagner, D. A. (Ed.). (2014). Learning and education in developing countries:

Research and policy for the Post-2015 UN Development Goals. New York, NY: Palgrave Macmillan.

Wagner, D. A., Castillo, N., Murphy, K. M., Tuz Zahra, F., Crofton, M. & Phelan, H.

(2014). Mobiles for reading: A landscape research review. Washington, DC: USAID.

64 The Campbell Collaboration | www.campbellcollaboration.org

8 Information About This Review

8.1 REVIEW AUTHORS

Lead review author:

The lead author is the person who develops and coordinates the review team, discusses and assigns roles for individual members of the review team, liaises with the editorial base, and takes responsibility for the on-going updates of the review.

Name: Elizabeth Spier, PhD

Title: Senior Research Analyst

Affiliation: American Institutes for Research

Address: 2800 Campus Drive

City, State, Province or County: San Mateo, CA

Postal Code: 94403

Country: United States

Phone: (+01) 650-843-8226

Mobile: (+01) 202-285-8380

Email: [email protected]

Co-author(s):

Name: Pia Britto, PhD

Title: Senior Advisor

Affiliation: UNICEF

Address: 3 UN Plaza

City, State, Province or County: New York, NY

Postal Code: 10017

Country: United States

Phone: (+01) 212-303-7955

Email: [email protected]

65 The Campbell Collaboration | www.campbellcollaboration.org

Name: Terri Pigott, PhD

Title: Interim Dean

& Professor of Research Methodology

Affiliation: Loyola University Chicago

Address: 820 North Michigan Avenue

City, State, Province or County: Chicago, IL

Postal Code: 60611

Country: United States

Phone: (+01) 312-915-6245

Email: [email protected]

Name: Eugene Roehlkepartain

Title: Vice President, Research and Development

Affiliation: Search Institute

Address: 615 First Avenue, NE City, State, Province or County: Minneapolis, MN

Postal Code: 55413

Country: United States

Phone: (+01) 800-888-7828

Email: [email protected]

Name: Michael McCarthy

Title: Research Associate

Affiliation: Yale University

Address: 230 South Frontage Road

City, State, Province or County: New Haven, CT

Postal Code: 06520

Country: United States

Mobile: (+01) 203-257-2062

Email: [email protected]

66 The Campbell Collaboration | www.campbellcollaboration.org

Name: Yael Kidron, PhD

Title: Senior Researcher

Affiliation: American Institutes for Research

Address: 2800 Campus Drive

City, State, Province or County: San Mateo, CA

Postal Code: 94403

Country: United States

Phone: (+01) 650-843-8122

Email: [email protected]

Name: Janis Glover, MLS

Title: Senior Reference Librarian

Affiliation: Yale University

Address: P.O. Box 208014 City, State, Province or County: New Haven, CT

Postal Code: 06520

Country: United States

Phone: (+01) 203-737-2962

Email: [email protected]

Name: Daniel Wagner

Title: Professor, UNESCO Chair in Learning and

Literacy; Chair, International Literacy Institute and National Center on Adult Literacy; Head, Program in International Educational Development

Affiliation: University of Pennsylvania Address: 3700 Walnut Street

City, State, Province or County: Philadelphia, PA

Postal Code: 19104

Country: United States

Phone: (+01) 215-898-9803

Email: [email protected]

67 The Campbell Collaboration | www.campbellcollaboration.org

Name: Julia Lane, PhD

Title: Managing Economist

Affiliation: American Institutes for Research

Address: 1000 Thomas Jefferson Street, NW

City, State, Province or County: Washington, DC

Postal Code: 20007

Country: United States

Phone: (+01) 202-403-5867

Email: [email protected]

Name: Mengli Song, PhD

Title: Principal Researcher

Affiliation: American Institutes for Research

Address: 1000 Thomas Jefferson Street, NW City, State, Province or County: Washington, DC

Postal Code: 20007

Country: United States

Phone: (+01) 202-403-5267

Email: [email protected]

8.2 ROLES AND RESPONSIBILITIES

The work was led by AIR, in collaboration with Search Institute and Yale University, as well as with expert consultants.

The team was led by Principal Investigator Elizabeth Spier, who has extensive experience in conducting international education evaluations. Team members included Pia Britto, an expert in child development and education; Eugene Roehlkepartain, an expert in parent, family, and community engagement to improve child outcomes; and Terri Pigott, Yael Kidron, and Mengli Song, experts in systematic review methods. Janis Glover, Yale Senior Reference Librarian, provided her extensive expertise in information retrieval, and she was assisted by Michael McCarthy. Dan Wagner, an expert of literacy in multiple international contexts, served as senior advisor.

An advisory board, comprised of academic researchers from developing countries, provided input about literatures and contexts, and helped with dissemination of review results through their networks.

Quality assurance was provided by Julia Lane, who brings skills in statistics in the education context, and by AIRs professional editors.

68 The Campbell Collaboration | www.campbellcollaboration.org

To summarize:

Content: Pia Britto, Elizabeth Spier, Dan Wagner, Eugene Roehlkepartain Systematic review methods: Terri Pigott, Yael Kidron Statistical analysis: Terri Pigott, Mengli Song Information retrieval: Michael McCarthy, Janis Glover

8.3 SOURCES OF SUPPORT

This review is being conducted under the auspices of 3ie and is funded by an USAID grant. AIR is also providing financial support to facilitate the completion of this work.

8.4 DECLARATION OF INTEREST

The authors have no vested interest in the outcomes of this review nor have any incentive to represent findings in a biased manner.

8.5 PLANS FOR UPDATING THE REVIEW

Given the rapid increase in the use of rigorous evaluations in the field, it would be highly beneficial to update this review within the next few years. The authors of this review would be pleased to complete such an update, should funding become available.

8.6 AUTHOR DECLARATION

Authors Responsibilities

By completing this form, you accept responsibility for maintaining the review in light of new evidence, comments and criticisms, and other developments, and updating the review at least once every five years, or, if requested, transferring responsibility for maintaining the review to others as agreed with the Coordinating Group. If an update is not submitted according to agreed plans, or if we are unable to contact you for an extended period, the relevant Coordinating Group has the right to propose the update to alternative authors.

Publication in the Campbell Library

The Campbell Collaboration places no restrictions on publication of the findings of a Campbell systematic review in a more abbreviated form as a journal article either before or after the publication of the monograph version in Campbell Systematic Reviews. Some journals, however, have restrictions that preclude publication of findings that have been, or will be, reported elsewhere, and authors considering publication in such a journal should be aware of possible conflict with publication of the monograph version in Campbell Systematic Reviews. Publication in a journal

69 The Campbell Collaboration | www.campbellcollaboration.org

after publication or in press status in Campbell Systematic Reviews should acknowledge the Campbell version and include a citation to it. Note that systematic reviews published in Campbell Systematic Reviews and co-registered with the Cochrane Collaboration may have additional requirements or restrictions for co-publication. Review authors accept responsibility for meeting any co-publication requirements.

I understand the commitment required to update a Campbell review, and agree to publish in the Campbell Library. Signed on behalf of the authors:

Form completed by: Date:

70 The Campbell Collaboration | www.campbellcollaboration.org

9 Tables

Table 14 List of Excluded Studies

Study Reason for Exclusion

Aboud & Hossain (2011) Intervention not delivered through parents, family members, or community members

Aboud, Hossain, & OGara (2008) Lack of a valid comparison group, no valid baseline

Adnams, Sorour, Kalberg, Kodituwakku, Perold, Kotze, September, et al. (2007)

Intervention not delivered through parents, family members, or community members

Aga Khan Development Network (2012)

Program information summary, not an evaluation report

Begeny, Yeager, & Martnez (2012) Intervention not delivered through parents, family members, or community members

Bekman (2004) QED with no valid baseline

Bekman, Aksu-Ko, & Erguvanl-Taylan (2011)

Intervention not delivered through parents, family members, or community members

Bernbaum, Rivero Herrera, & Schiefelbein (2010)

Intervention not delivered through parents, family members, or community members

Borzekowski & Macha (2010) Intervention not delivered through parents, family members, or community members

Brooker, Okello, Njagi, Dubeck, Halliday, Inyega, & Jukes (2010)

Description of a future study, not a completed study

Cardoso-Martins, Lara Mesquita, & Ehri (2011)

Intervention not delivered through parents, family members, or community members

de Souza, de Rose, Faleiros, Bortoloti, Hanna, & McIlvane (2009)

Intervention not delivered through parents, family members, or community members

Cristia, Ibarrarn, Cueto, Santiago, & Severn (2012)

This intervention was intended to include a home component, with children using laptops at home, but few children took their laptops outside of school, and the intervention was not really delivered through parents, families or community members.

Dixon, Schagen, & Seedhouse (2011) Intervention not delivered through parents, family members, or community members

Dowd & Advisor (2011) Program information summary, not an evaluation report

Educational Development Center (2012)

Topic area is educational radio

Educational Development Center (2009)

Topic area is educational radio

71 The Campbell Collaboration | www.campbellcollaboration.org

Study Reason for Exclusion

Educational Development Center (2007)

Intervention not delivered through parents, family members, or community members

Martinez, Naudeu, & Pereira (2013) Intervention not delivered through parents, family members, or community members

Mills-Tettey, Mostow, Dias, Sweet, Belousov, Dias, & Gong (2009)

Intervention is primarily delivered through teachers at schools. There is a parent involvement component, but its effects have not been assessed independently of the teacher-delivered component.

Neugebauer & Currie-Rubin (2009) Intervention not delivered through parents, family members, or community members

Ntuli & Pretorius (2005) Lack of a valid comparison group, no valid baseline

Olivier, Anthonissen, & Southwood (2010)

Lack of a valid comparison group, no valid baseline

Intervention not delivered through parents, family members, or community members

Save the Children (2007) Program information summary, not an evaluation report

imek etin & Alisinanolu (2013) Intervention not delivered through parents, family members, or community members

USAID Malawi (2009) Intervention not delivered through parents, family members, or community members

Van der Bijl, Alant, & Lloyd (2006) Intervention not delivered through parents, family members, or community members

Visser & Chamberlain (2004) Intervention not delivered through parents, family members, or community members

Walter & Dekker (2011) Intervention not delivered through parents, family members, or community members

Wu, Anderson, Li, Wu, Li, Zhang, et. al. (2009)

72 The Campbell Collaboration | www.campbellcollaboration.org

Topic area is educational radio

Iikdoan & Kargin (2010) Intervention not delivered through parents, family members, or community members

Lucas, McEwan, Ngware, & Oketch (2013)

Intervention not delivered through parents, family members, or community members

Mishra & Lal (2006) Intervention not delivered through parents, family members, or community members

Mithani, Alam, Babar, Dowd, Hanson, & Ochoa (2011)

Intervention not delivered through parents, family members, or community members

Opel, Ameer, & Aboud (2009) Intervention not delivered through parents, family members, or community members

Piper & Korda (2011) Intervention not delivered through parents, family members, or community members

Pretorius & Machet (2008) It is unclear whether there is a valid comparison group, and there is no valid baseline

Rolla San Francisco, Arias, Villers, &

Snow (2006)

Sailors, Hoffman, Pearson, Beretvas, & Metthee (2010)

Intervention not delivered through parents, family members, or community members

Table 15 Attributes of the Study of Jalan Sesama

Location Three villages (Munjul, Kota Dukuh, and Gunung Batu) in the Pandeglang

District, Banten Province, Indonesia

Design Randomized controlled trial

Group Formation

At outcome only: Letter recognition Reading (phonemic awareness) Writing

Attrition There was zero attrition across all three groups.

The children were randomly assigned to one of three conditions: 1. High-exposure group 2. Low-exposure group 3. Control group Participants High-exposure group = 58 children

Low-exposure group = 48 children Control group = 54 children

Additional demographics: Average age was 4.9, with an age range of 3.5 to 6 years old 83 girls (52%) and 77 boys (48%) Many children attended school (43% of high-exposure group, 60% of low-exposure group, 85% of control group)

Conditions High-exposure group: Invited to watch 3 to 4 episodes of Jalan Sesama per week for 14 weeks (52 episodes total)

Low-exposure group: Invited to watch 1 episode of Jalan Sesama per week for 14 weeks (14 episodes total)

Control group: Invited to watch 1 episode of another popular childrens television program (such as Dora the Explorer, Tom and Jerry, or Sponge Bob Square Pants) per week for 14 weeks (14 episodes total)

Literacy Outcomes Assessed

73 The Campbell Collaboration | www.campbellcollaboration.org

Table 16 Attributes of the Study of Will You Play With Me? Location Largest metropolitan area in Turkey

Design Randomized controlled trial

Group Formation

At outcome only: Syllabification Vocabulary

Attrition There was 5% attrition in the intervention group, 9% attrition in the control group, and 29% in the natural observation group.

The children were randomly assigned to one of three groups: 1. Intervention group 2. Control group 3. Natural observation group

Participants Intervention group = 139 children and their families

Control group = 127 children and their families

Natural observation group = 133 children and their families

Additional demographics: Average age 5 years 3 months, and ranged from 4 years 7 months to 7 years 3 months

The intervention group consisted of 50.4% boys, the control group 52.0% boys, and the natural observation group 58.1% boys

Mothers averaged 5.5 years of education in the intervention group, 5.1 years in the control group, and 4.9 years in the natural observation group

Conditions Intervention group: Parents were instructed to have their children watch Will You Play With Me? daily for 13 weeks (65 episodes total)

Control group: Parents were instructed to have their children watch another television program (that was on opposite Will You Play With Me?) daily for 13 weeks (65 episodes total)

Natural observation group: Parents were informed about Will You Play With Me? and its potential benefits but were not given further instructions (they were free to have their children watch the program or not, as they saw fit)

Literacy Outcomes Assessed

74 The Campbell Collaboration | www.campbellcollaboration.org

Table 17 Attributes of the Study of the Family Supported Reading Program Location Konya, Turkey

Design Quasi-experimental design

Group Formation

At outcome only: Reading comprehension Mechanical reading skills Writing

Attrition There was zero attrition across both study groups.

Intervention group children were drawn from one nursery school program, and control group students attended other nursery school programs (but not the same nursery school as the intervention group).

Participants Intervention group = 25 children and their families Control group = 25 children and their families

Additional demographics: The intervention group consisted of children attending a specific pre-primary class in 2009-2010 and enrolled in Grade 1 in 2010-2011

The control group consisted of children who had attended other pre-primary programs in 2009-2010 and enrolled in Grade 1 in 2010-2011. It is unclear whether the control group was drawn from a single other pre-primary class, or multiple classes

Conditions Intervention group: Parents were invited to participate in 13 weekly program sessions

Control group: Parents were not invited to any program sessions

Literacy Outcomes Assessed

75 The Campbell Collaboration | www.campbellcollaboration.org

Table 18 Attributes of the Study in Armenia of OSIs Getting Ready for School Location Provinces of Kotayk, Yeghegnadzor, Yerevan, Gegharkunik, and Tavush,

Armenia

Design Experimental design

Group Formation

At baseline and outcome: Letter identification Phonics Print concepts Reading (words) Writing

At last follow-up (end of Grade 1): Phonics Reading comprehension Rhyming Teacher ratings of childs literacy development Writing

Attrition By the end of Grade 1 (last follow-up), 7% of the intervention group and 11% of the control group had been lost to attrition.

76 The Campbell Collaboration | www.campbellcollaboration.org

The families were divided into one of two groups: 1. Intervention group 2. Control group

Communities were stratified by province then randomly assigned to the intervention group or the control group.

The intervention group consisted of communities in which the program was made available to all parents with children who were 1 year away from on-time school entry.

The control group consisted of communities in which the program was not made available.

Participants Intervention group = 17 communities (123 families)

Control group = 15 communities (120 families)(In communities with more than 25 eligible families, 25 families were randomly selected for the study; in communities with fewer than 25 eligible families, all eligible families were included in the study sample.)

Additional demographics: Most children were 5 years old at the time of baseline (93% of intervention group and 93% of control group).

Intervention group was 44% female, and control group was 38% female.

Conditions Intervention group: Parents were invited to participate in nine monthly program sessions.

Control group: Parents were not invited to any program sessions, and the program was not available in their community.

Literacy Outcomes Assessed

Table 19 Attributes of the Study in Bosnia and Herzegovina of OSIs Getting Ready for School

Location Communities of Jablanica, Kiseljak, Tuzla (Sjenjak), and Prijedor (Kokin Grad) in Bosnia and Herzegovina

Design Quasi-experimental design

Group Formation

At baseline and outcome: Letter identification Phonics Print concepts Reading (words) Writing

At last follow-up (end of Grade 1): Teacher ratings of childs literacy development

Attrition By the end of Grade 1 (last follow-up), 26% of the intervention group and 18%

of the control group had been lost to attrition.

77 The Campbell Collaboration | www.campbellcollaboration.org

The communities were divided into one of two groups: 1. Intervention group 2. Control group

Communities were selected to receive the intervention. Then, one community that matched requirements was identified for each intervention village. Matching was based on demographic information, such as community size, region, ethnic characteristics, and whether it was urban or rural.

The intervention group consisted of communities in which the program was made available to all parents with children who were 1 year away from on-time school entry.

The control group consisted of communities in which the program was not made available.

Participants Intervention group = 4 communities (101 families)

Control group = 4 communities (110 families)

Additional demographics: Children in both groups averaged 5 years old at the time of baseline. Conditions Intervention group: Parents were invited to participate in nine monthly program sessions.

Control group: Parents were not invited to any program sessions, and the program was not available in their community.

Literacy Outcomes Assessed

Table 20 Attributes of the Study in Kazakhstan of OSIs Getting Ready for School Location Communities of Melliorator, Zapadnii, Kirgauldi village, and Irgeli village,

Kazakhstan

Design Quasi-experimental design

Group Formation

At baseline and outcome: Letter identification Phonics Print concepts Reading (words) Writing

At last follow-up (end of Grade 1): Teacher ratings of childs literacy development

Attrition By the end of Grade 1 (last follow-up), 5% of the intervention group and 3% of the control group had been lost to attrition.

78 The Campbell Collaboration | www.campbellcollaboration.org

The communities were divided into one of two groups: 1. Intervention group 2. Control group

Individual control group families were matched with individual intervention group families based on demographic characteristics, such as child age and child gender.

Participants Intervention group = 4 communities (110 families)

Control group = 4 communities (110 families)

Additional demographics: Children in both groups averaged age 5.5 at the time of baseline. Conditions Intervention group: Parents were invited to participate in nine monthly program sessions.

Control group: Parents were not invited to any program sessions, and the program was not available in their community.

Literacy Outcomes Assessed

Table 21 Attributes of the Study in Tajikistan of OSIs Getting Ready for School Location Communities of Bokhtar, Kulob, Khujand, and Jamoat Vodnin, from Khatlon and Sughd provinces, Tajikstan

Design Quasi-experimental design

Group Formation

At baseline and outcome: Letter identification Phonics Print concepts Reading (words) Writing

At last follow-up (end of Grade 1): Teacher ratings of childs literacy development

Attrition By the end of Grade 1 (last follow-up), 4% of the intervention group and 5% of the control group had been lost to attrition.

79 The Campbell Collaboration | www.campbellcollaboration.org

The communities were divided into one of two groups: 1. Intervention group 2. Control group

Individual control group families were matched with individual intervention group families based on demographic characteristics, such as child age, child gender, and home language.

Participants Intervention group = 4 communities (100 families)

Control group = 4 communities (100 families)

Additional demographics: Children in both groups averaged age 6.5 at the time of baseline. Conditions Intervention group: Parents were invited to participate in program sessions. The number and timing of program sessions varied by community.

Control group: Parents were not invited to any program sessions, and the program was not available in their community.

Literacy Outcomes Assessed

Table 22 Attributes of the Evaluation in Bangladesh of UNICEFs Getting Ready for School: A Child-to-Child Approach

Location Thirty communities across Bangladesh

Design Randomized controlled trial

Group Formation

At baseline and outcome: Letter identification Reading (words) Writing

At last follow-up (end of Grade 1): Teacher ratings of childs literacy development

Attrition At the first outcome assessment (end of program year), attrition was 12% in the intervention group and 11% in the control group. Grade 1 outcome data was unavailable for 47% of the intervention group and 5% of the control group, but this figure includes children who had not yet enrolled in Grade 1 (therefore their teachers could not be surveyed).

80 The Campbell Collaboration | www.campbellcollaboration.org

The children were divided into one of two groups: 1. Intervention group 2. Control group

From each of the six administrative divisions of the country, one district and its two upazilas (subdistricts) were selected based on high drop-out and low primary school completion rates. The districts chosen were geographically representative of the country. The two upazilas in each district were randomly assigned either an intervention group or a control group. In each groups upazila, five schools were randomly selected to participate in the program (if in the intervention group) and the evaluation.

Participants Intervention group = 30 schools, with 432 children randomly selected from within those communities to participate in the study (approximately 900 children took part in the intervention)

Control group = with 451 children randomly selected from within those communities to participate in the study

Additional demographics:

All children were judged to be 1 year away from on-time primary school enrollment in their communities, although exact age information was unavailable.

Conditions Intervention group: All young children in the community who were 1 year away from on-time primary school enrollment were invited to participate.

Control group: Children were not invited to any program sessions, and the program was not available in their community.

Literacy Outcomes Assessed

Table 23 Attributes of the Evaluation in Democratic Republic of Congo of UNICEFs Getting Ready for School: A Child-to-Child Approach

Location Fifteen communities in Kinshasa and 10 in Mbandaka, Democratic Republic of Congo

Design Quasi-experimental design

Group Formation

At baseline and outcome: Letter identification Reading (words) Writing

Attrition There were significant difficulties with data collection (especially in Mbandaka) due to both impassable roads and incursions of the civil conflict. Therefore, after 1 year, follow-up data was available for only 38% of the intervention group and 35% of the control group.

81 The Campbell Collaboration | www.campbellcollaboration.org

The children were divided into one of two groups: 1. Intervention group 2. Control group

Fifteen schools in Kinshasa and 10 schools in Mbandaka were identified for the study. Then, an equal number of similar comparison schools were identified in each location.

Participants Intervention group = 25 schools, with 375 children randomly selected (out of approximately 1,000 to 1,500 children in those communities who took part in the intervention) to participate in the study

Control group = 25 schools, with 373 children randomly selected from within those communities to participate in the study

Additional demographics: Children in both the intervention and control groups averaged 5 years old.

The intervention group was 51% boys, and the control group was 46% boys.

Conditions Intervention group: All children in the communities who were 1 year away from on-time primary school enrollment were invited to participate.

Control group: Children were not invited to any program sessions, and the program was not available in their community.

Literacy Outcomes Assessed

Table 24 Attributes of the Evaluation in Ethiopia of UNICEFs Getting Ready for School: A Child-to-Child Approach

Location Twenty communities from the regions of Harar, Oromia, and Tigray, Ethiopia

Design Quasi-experimental design

Group Formation

The children were divided into one of two groups: 1. Intervention group 2. Control group

School clusters were small groups of schools (typically five to seven schools in each cluster) in relative proximity, linked through one school that acted as the cluster resource center. All 20 schools participating in the program were in rural areas and were selected to take part on the basis of good working relations among the school clusters and the willingness of the headmasters. Within each cluster, 17 schools that were geographically close to and shared similar community characteristics with the intervention schools were selected to serve as comparison schools.

Participants Intervention group = 20 schools, with 117 children randomly selected (out of approximately 2,000 children in those communities took part in the intervention) to participate in the study

Control group = 17 schools, with 114 children randomly selected from within those communities to participate in the study

Additional demographics: Children in both the intervention and control groups averaged 6 years old.

The intervention group was 51% boys, and the control group was also 51% boys.

Conditions Intervention group: All young children in the community who were 1 year away from on-time primary school enrollment were invited to participate.

Control group: Children were not invited to any program sessions, and the program was not available in their community.

Literacy Outcomes Assessed

82 The Campbell Collaboration | www.campbellcollaboration.org

At baseline and outcome: Letter identification Reading (words) Writing

Attrition Posttest data were available only for 44% of the intervention group children and

70% of the control group children.

Table 25 Attributes of the Evaluation in Tajikistan of UNICEFs Getting Ready for School: A Child-to-Child Approach

Location Rumi and Bokhtar, Tajikistan

Design Randomized controlled trial

Group Formation

At baseline and outcome: Letter identification Reading (words) Writing

At last follow-up (end of Grade 1): Teacher ratings of literacy development

Attrition Less than 1% of the intervention group and less than 1% of the control group was lost to attrition at the time of the posttest. At the time of the Grade 1 follow-up, 9% of the intervention group and 8% of the control group had been lost to attrition.

83 The Campbell Collaboration | www.campbellcollaboration.org

The children were divided into one of two groups: 1. Intervention group 2. Control group

Forty schools participated in the evaluation, with 20 each from the Rumi and Bokhtar districts. Within each district, half of the schools were randomly assigned to the intervention group and half to the control group.

Participants Intervention group = 20 schools, with 300 children randomly selected (out of approximately 2,500 children in those communities who took part in the intervention) to participate in the study

Control group = 20 schools, with 300 children randomly selected from within those communities to participate in the study

Additional demographics: Children in both the intervention and control groups averaged 7 years old.

Parental literacy was high (94%) in these communities.

Conditions Intervention group: All young children in the community who were 1 year away from on-time primary school enrollment were invited to participate.

Control group: Children were not invited to any program sessions, and the program was not available in their community.

Literacy Outcomes Assessed

Table 26 Attributes of the Evaluation in Yemen of UNICEFs Getting Ready for School: A Child-to-Child Approach

Location Districts of Haifan, Al-Makha, and Mawza , Taiz Governorate, Yemen

Design Randomized controlled trial

Group Formation

At baseline and outcome: Letter identification Reading (words) Writing

At last follow-up (end of Grade 1): Teacher ratings of literacy development

Attrition At the time of the posttest, the attrition rate was 12% for the intervention group and 15% for the control group. At the time of the Grade 1 assessment, 19% of the intervention group children and 38% of the control group children did not have teacher surveys available. These figures reflect the fact that not all children enrolled in Grade 1 (especially in the control group).

84 The Campbell Collaboration | www.campbellcollaboration.org

The children were divided into one of two groups: 1. Intervention group 2. Control group

Thirty schools took part in the evaluation, with 10 each from Haifan, Al-Makha, and Mawza. Within each of the three regions, half of the schools were randomly assigned to the intervention group and half to the control group.

Participants Intervention group = 15 schools, with 301 children randomly selected (out of approximately 700 to 1,000 children in those communities who took part in the intervention) to participate in the study

Control group = 15 schools, with 300 children randomly selected from within those communities to participate in the study

Additional demographics: Children in both the intervention and control groups averaged age 5.5 years

Conditions Intervention group: All young children in the community who were 1 year away from on-time primary school enrollment were invited to participate.

Control group: Children were not invited to any program sessions, and the program was not available in their community.

Literacy Outcomes Assessed

Table 27 Attributes of the Evaluation of Read India

Location Villages in Jaunpur District, State of Uttar Pradesh, India

Design Quasi-experimental design

Group Formation

At outcome only: Letter recognition Reading (words and text)

Attrition The analytic sample only included children who participated in both the pretest and the posttest. Original sample size is not available for the group of children involved in this specific intervention.

The villages were divided into one of two groups: 1. Intervention group 2. Control group

Intervention group villages were randomly selected out of a pool of 280 villages that had participated in the baseline.

Control group villages were selected (further details not available).

Participants Intervention group = 65 villages, with 3,671 children from households randomly selected (out of 7,453 children in those communities who took part in the intervention) to participate in the study

Comparison group = 85 villages with 4,730 children from households randomly within those communities to participate in the study

Additional demographics: The program was available to children 7 to 14 years old.

Conditions Intervention group communities had the Read India program available to children. Control group villages did not have the Read India program available.

Literacy Outcomes Assessed

85 The Campbell Collaboration | www.campbellcollaboration.org

9 Data and Analyses

9.1 PARENT SUPPORT INTERVENTIONS

Table 28 Summary Statistics for Parent Support Interventions, Part I Intervention Group Analysis Sample

Control Group Analysis Sample

Study RCT/ QED

Unit of Assignment

Outcome Timing N

Children

N Clusters

N Children

N Clusters

OSI Armenia Cluster RTC

Community Literacy (Baseline) Baseline 123 5 119 5

Literacy (Year 1) Posttest 119 5 113 5

Literacy (Year 2) Follow-up (Grade 1) 108 5 111 5

OSI Bosnia & Herzegovina

QED Community Literacy (Baseline) Baseline 101 4 110 4

Literacy (Year 1) Posttest 84 4 95 4

Literacy (Year 2) Follow-up

(Grade 1) 75 4 90 4

OSI Kazakhstan

QED Family/child Literacy (Baseline) Baseline 110 n/a 110 n/a

Literacy (Year 1) Posttest 107 n/a 104 n/a

Literacy (Year 2) Follow-up

(Grade 1) 106 n/a 100 n/a

OSI Tajikistan

QED Family/child Literacy (Baseline) Baseline 100 n/a 100 n/a

Literacy (Year 1) Posttest 95 n/a 97 n/a

Literacy (Year 2) Follow-up

(Grade 1) 95 n/a 97 n/a

Pre-Reading Family Support

QED Family/child Basic literacy

(reading & writing)

Baseline 25 n/a 25 n/a

Basic literacy (reading & writing)

Posttest 25 n/a 25 n/a

Reading comprehension

Baseline 25 n/a 25 n/a

Reading comprehension

Posttest 25 n/a 25 n/a

86 The Campbell Collaboration | www.campbellcollaboration.org

Intervention Group Analysis Sample

Control Group Analysis Sample

Study RCT/ QED

Unit of Assignment

Outcome Timing N

Children

N Clusters

N Children

N Clusters

Reading comprehension

Follow-up 25 n/a 25 n/a

Mechanical reading skills

Baseline 25 n/a 25 n/a

Mechanical reading skills

Posttest 25 n/a 25 n/a

Mechanical reading skills

Follow-up 25 n/a 25 n/a

Writing skills Baseline 25 n/a 25 n/a Writing skills Posttest 25 n/a 25 n/a Writing skills Follow-up 25 n/a 25 n/a

87 The Campbell Collaboration | www.campbellcollaboration.org

Table 29 Summary Statistics for Parent Support Interventions, Part II Study Outcome Intervention

Group Mean

Control Group Mean

Statistical Test Effect size SE ICC L2

OSI Armenia Literacy (Baseline) 0.15 -0.15 No 1.050 0.925 HLM (children/ communities) n/a n/a 0.10

Literacy (Year 1) 0.29 -0.31 No 0.977 0.935 HLM (children/ communities) 0.63 0.22 0.08 Literacy (Year 2) 0.00 -0.01 No 1.052 0.956 HLM (children/ communities) 0.01 0.17 0.03

OSI Bosnia Literacy (Baseline) 0.09 -0.08 No 1.056 0.943 HLM (children/ communities) n/a n/a 0.16 and

Herzegovina

Literacy (Year 1) 0.20 -0.14 No 0.964 0.998 HLM (children/ communities) 0.34 0.24 0.08 Literacy (Year 2) 0.03 -0.02 No 0.958 1.032 HLM (children/ communities) 0.05 0.22 0.07

OSI Kazakhstan

Literacy (Baseline) 0.24 -0.24 No 1.034 0.91 Multiple regression n/a n/a n/a Literacy (Year 1) 0.22 -0.21 No 0.863 1.106 Multiple regression 0.44 0.14 n/a Literacy (Year 2) 0.05 -0.05 No 1.027 0.986 Multiple regression 0.10 0.14 n/a

OSI Tajikistan Literacy (Baseline) 0.25 -0.25 No 1.103 0.816 Multiple regression n/a n/r n/a

Literacy (Year 1) -0.16 0.15 No 0.708 1.174 Multiple regression -0.32 0.14 n/a Literacy (Year 2) 0.66 -0.66 No 0.482 0.961 Multiple regression 1.74 0.17 n/a

Means Adjusted?

Treatment Group SD

Control Group SD

88 The Campbell Collaboration | www.campbellcollaboration.org

Table 29, continued

Study Outcome Intervention Group Mean

Control Group Mean

Means Adjusted?

Treatment Group SD

Control Group SD

Statistical Test Effect size SE ICC L2

Pre-Reading Support

Basic literacy (reading-writing) (Baseline) 62.30 60.05

No 12.15 11.09 Mann Whitney U test n/a n/r n/a

Basic literacy (reading-writing) (Posttest) 92.85 74.25

No 13.53 29.39 Mann Whitney U test 0.81 0.30 n/a

Reading comprehension (Baseline)

3.95 3.55

1.84 2.18 Mann Whitney U test n/a n/r n/a

Reading comprehension (Posttest)

7.50 5.00

4.44 5.12 Mann Whitney U test 0.52 0.29 n/a

Reading comprehension (Follow-up)

7.65 5.50

2.03 2.13 Mann Whitney U test 1.03 0.30 n/a

Mechanical readingskills (Baseline) 3.85 3.25

No 2.53 2.57 Mann Whitney U test n/a n/r n/a

Mechanical readingskills (Posttest) 14.25 5.07

No 8.44 8.93 Mann Whitney U test 1.06 0.30 n/a

Mechanical readingskills (Follow-up) 19.40 16.65

No 1.04 4.55 Mann Whitney U test 0.83 0.29 n/a

Writing skills (Baseline) 4.80 4.30 No 2.54 2.77 Mann Whitney U test n/a n/r n/a Writing skills (Posttest) 17.65 15.45 No 4.22 5.90 Mann Whitney U test 0.43 0.29 n/a Writing skills (Follow-up) 19.00 17.85

No 0.00 4.29 Mann Whitney U test 0.38 0.29 n/a

89 The Campbell Collaboration | www.campbellcollaboration.org

9.2 TUTORING INTERVENTIONS

Table 30 Statistics for UNICEF Getting Ready for School: A Child-to-Child Approach Studies, Part I

Intervention Group Analysis Sample

Control Group Analysis Sample

Study RCT/ QED

Unit of Assignment

Outcome Timing N

Children

N Clusters

N Children

N Clusters

UNICEF Bangladesh

RCT Sub-district Reading total Baseline 382 6 400 6

Beginning reading subscale

Baseline 382 6 400 6

Letter identification subscale

Baseline 382 6 400 6

Writing total Baseline 382 6 400 6 Reading total Posttest 382 6 400 6 Beginning readingsubscale

Posttest 382 6 400 6

Letter identification subscale

Posttest 382 6 400 6

Writing total Posttest 382 6 400 6 Readingachievement

Grade 1 227 6 202 6

Writing achievement Grade 1 217 6 201 6 Overall literacyachievement

Grade 1 217 6 202 6

UNICEF D. R. Congo

QED School Reading total Baseline 143 25 130 25

Beginning reading subscale

Baseline 141 25 130 25

Letter identification subscale

Baseline 143 25 129 25

Writing total Baseline 141 25 130 25 Reading total Posttest 143 25 130 25 Beginning readingsubscale

Posttest 141 25 129 25

Letter identification subscale

Posttest 143 25 130 25

Writing total Posttest 141 25 127 25

90 The Campbell Collaboration | www.campbellcollaboration.org

Table 30, continued

Intervention Group Analysis Sample

Control Group Analysis Sample

Study RCT/ QED

Unit of Assignment

Outcome Timing N

Children

N Clusters

N Children

N Clusters

UNICEF Ethiopia

QED School Reading total Baseline 51 20 80 17

Beginning reading subscale

Baseline 49 20 73 17

Letter identification subscale

Baseline 51 20 80 17

Writing total Baseline 51 20 80 17 Reading total Posttest 49 20 73 17 Beginning readingsubscale

Posttest 51 20 80 17

Letter identification subscale

Posttest 50 20 78 17

Writing total Posttest 51 20 80 17

UNICEF Tajikistan

RCT School Reading total Baseline 295 20 298 20

Beginning reading subscale

Baseline 295 20 298 20

Letter identification subscale

Baseline 295 20 298 20

Writing total Baseline 294 20 298 20 Reading total Posttest 295 20 298 20 Beginning readingsubscale

Posttest 295 20 298 20

Letter identification subscale

Posttest 295 20 298 20

Writing total Posttest 294 20 298 20 Readingachievement

Grade 1 272 20 275 20

Writing achievement Grade 1 272 20 275 20 Overall literacyachievement

Grade 1 272 20 275 20

91 The Campbell Collaboration | www.campbellcollaboration.org

Table 30, continued

Intervention Group Analysis Sample

Control Group Analysis Sample

Study RCT/ QED

Unit of Assignment

Outcome Timing N

Children

N Clusters

N Children

N Clusters

UNICEF Yemen

RCT School Letter identification subscale

Baseline 265 15 256 15

Writing total Baseline 265 15 256 15 Letter identificationsubscale

Posttest

265 15 256 15 Writing total Posttest 265 15 256 15 Readingachievement

Grade 1

245 15 186 15 Writing achievement Grade 1 244 15 186 15 Overall literacyachievement

Grade 1

245 15 186 15

92 The Campbell Collaboration | www.campbellcollaboration.org

Table 31 Statistics for UNICEF Getting Ready for School: A Child-to-Child Approach Studies, Part II Study Outcome Intervention

Group Mean

Control Group Mean

Means Adjusted?

Treatment Group SD

Control Group SD

Statistical Test Effect size SE ICC L2

ICC L3

UNICEF Bangladesh

Reading total (Baseline) -0.12 0.07 No 1.00 1.00 3-level HLM

(students/schools/sub-districts) n/a -1.36 n/r

Beginning readingsubscale (Baseline) -0.13 0.07 No 1.00 1.00

3-level HLM(students/schools/sub-districts) -n/a -1.67 n/r

Letter identificationsubscale (Baseline) -0.12 0.07 No 1.00 1.00

3-level HLM (students/schools/sub-districts) n/a -1.46 n/r

Writing total (Baseline) -0.13 0.06 No 1.00 1.00 3-level HLM

(students/schools/sub-districts) n/a -1.43 n/r Reading total (Posttest) -0.13 0.11 No 0.87 1.09 3-level HLM

(students/schools/sub-districts) -0.24 0.07 -1.83 0.00

Beginning readingsubscale (Posttest) -0.18 0.16 No 0.81 1.12

3-level HLM (students/schools/sub-districts) -0.35 0.10 -3.00 0.00

Letter identificationsubscale (Posttest) -0.11 0.09 No 0.88 1.08

3-level HLM (students/schools/sub-districts) -0.20 0.10 -1.58 0.00

Writing total (Posttest) 0.14 -0.14 No 0.97 1.01 3-level HLM

(students/schools/sub-districts) 0.28 0.10 4.11 0.00

Reading achievement(Grade 1) -0.07 0.09 No 1.03 0.97

3-level HLM (students/schools/sub-districts) -0.16 0.10 -1.05 0.00

Writing achievement(Grade 1) 0.00 0.02 No 0.92 1.06

3-level HLM(students/schools/sub-districts) -0.02 0.10 0.00 0.00

Overall literacyachievement (Grade 1) -0.02 0.03 No 0.93 1.06

3-level HLM(students/schools/sub-districts) -0.05 0.10 -0.44 0.01

93 The Campbell Collaboration | www.campbellcollaboration.org

Table 31, continued

Study Outcome Intervention Group Mean

Control Group Mean

Means Adjusted?

Treatment Group SD

Control Group SD

Statistical Test Effect size SE ICC

ICC L3

UNICEF D. R. Congo

Reading total (Baseline) 0.10 0.10 No 1.00 1.00 2-level HLM

(students/schools) n/a 0.11 n/a

Beginning readingsubscale (Baseline) 0.17 0.17 No 1.00 1.00

2-level HLM (students/schools) n/a 0.55 n/a

Letter identificationsubscale (Baseline) 0.09 0.09 No 1.00 1.00

2-level HLM (students/schools) n/a 0.06 n/a

Writing total (Baseline) 0.00 0.00 No 1.00 1.00 2-level HLM

(students/schools) n/a 0.92 n/a Reading total (Posttest) 0.34 -0.32 No 1.05 0.85 2-level HLM

(students/schools) 0.68 0.17 2.61 n/a

Beginning readingsubscale (Posttest) 0.13 -0.12 No 1.03 1.07

2-level HLM (students/schools) 0.24 0.17 1.12 n/a

Letter identificationsubscale (Posttest) 0.35 -0.33 No 1.07 0.80

2-level HLM (students/schools) 0.71 0.17 2.67 n/a

Writing total (Posttest) 0.31 -0.37 No 1.10 0.69 2-level HLM

(students/schools) 0.73 0.17 3.44 n/a

94 The Campbell Collaboration | www.campbellcollaboration.org

Table 31, continued

Study Outcome Intervention Group Mean

Control Group Mean

Means Adjusted?

Treatment Group SD

Control Group SD

Statistical Test Coefficient SE ICC L2 ICC L3

UNICEF Ethiopia

Reading total (Baseline) 0.04 0.42 No 1.00 1.00 2-level HLM

(students/schools) n/a n/r n/a

Beginning readingsubscale (Baseline) 0.07 0.12 No 1.00 1.00

2-level HLM (students/schools) n/a n/r n/a

Letter identificationsubscale (Baseline) 0.04 0.39 No 1.00 1.00

2-level HLM (students/schools) n.a n/r n/a

Writing total (Baseline) -0.01 0.32 No 1.00 1.00 2-level HLM

(students/schools) n/a n/r n/a Reading total (Posttest) 0.15 -0.04 No 0.87 1.11 2-level HLM

(students/schools) 0.18 0.21 0.41 n/a

Beginning readingsubscale (Posttest) -0.23 0.08 No 0.71 1.20

2-level HLM (students/schools) -0.30 0.20 0.21 n/a

Letter identificationsubscale (Posttest) 0.19 -0.06 No 0.90 1.10

2-level HLM (students/schools) 0.24 0.22 0.44 n/a

Writing total (Posttest) 0.10 0.06 No 0.97 1.15 2-level HLM

(students/schools) 0.04 0.20 0.37 n/a

95 The Campbell Collaboration | www.campbellcollaboration.org

Table 31, continued

Study Outcome Intervention Group Mean

Control Group Mean

Means Adjusted?

Treatment Group SD

Control Group SD

Statistical Test Effect size SE ICC L2

ICC L3

UNICEF Tajikistan

Reading total (Baseline) 0.02 -0.01 No 1.00 1.00 2-level HLM

(students/schools) n/a n/r n/a

Beginning readingsubscale (Baseline) 0.03 -0.02 No 1.00 1.00

2-level HLM (students/schools) n/a n/r n/a

Letter identificationsubscale (Baseline) 0.02 -0.01 No 1.00 1.00

2-level HLM (students/schools) n/a n/r n/a

Writing total (Baseline) 0.01 -0.01 No 1.00 1.00 2-level HLM

(students/schools) n/a n/r n/a Reading total (Posttest) 0.04 0.00 No 1.00 0.96 2-level HLM

(students/schools) 0.04 0.16 0.23 n/a

Beginning readingsubscale (Posttest) 0.03 0.00 No 0.99 0.99

2-level HLM (students/schools) 0.03 0.17 0.21 n/a

Letter identificationsubscale (Posttest) 0.04 0.00 No 1.00 0.96

2-level HLM (students/schools) 0.04 0.17 0.22 n/a

Writing total (Posttest) 0.02 -0.02 No 0.99 1.01 2-level HLM

(students/schools) 0.04 0.17 0.23 n/a

Reading achievement(Grade 1) 0.01 -0.01 No 1.03 0.98

2-level HLM (students/schools) 0.02 0.14 0.09 n/a

Writing achievement(Grade 1) 0.03 -0.03 No 0.96 1.04

2-level HLM (students/schools) 0.06 0.10 0.00 n/a

Overall literacyachievement (Grade 1) 0.01 -0.02 No 1.02 0.99

2-level HLM (students/schools) 0.03 0.14 0.09 n/a

96 The Campbell Collaboration | www.campbellcollaboration.org

Table 31, continued

Study Outcome Intervention Group Mean

Control Group Mean

Means Adjusted?

Treatment Group SD

Control Group SD

Statistical Test Effect size SE ICC L2

ICC L3

UNICEF Yemen

Letter identificationsubscale (Baseline) 0.15 -0.12 No 1.00 1.00

2-level HLM (students/schools) n/a n/r n/a

Writing total (Baseline) 0.08 -0.04 No 1.00 1.00 2-level HLM

(students/schools) n/a n/r n/a

Letter identificationsubscale (Posttest) 0.25 -0.12 No 1.01 0.98

2-level HLM (students/schools) 0.37 0.17 0.22 n/a

Writing total (Posttest) 0.10 -0.06 No 1.05 0.97 2-level HLM

(students/schools) 0.15 0.32 0.77 n/a

Reading achievement(Grade 1) 0.22 -0.23 No 0.92 1.03

2-level HLM (students/schools) 0.46 0.120 0.31 n/a

Writing achievement(Grade 1) 0.13 -0.11 No 0.90 1.06

2-level HLM (students/schools) 0.24 0.28 0.67 n/a

Overall literacyachievement (Grade 1) 0.19 -0.19 No 0.91 1.04

2-level HLM (students/schools) 0.39 0.22 0.34 n/a

97 The Campbell Collaboration | www.campbellcollaboration.org

Table 32 Statistics for Read India

Intervention Group Analysis Sample

Control Group Analysis Sample

Study RTC/QED Outcome Timing N

Children

N Clusters

N Children

N Clusters

N Successful Intervention Group

N Successful Control Group

Proportion Successful Intervention Group

Proportion Successful Control Group

Read India Cluster

RCT

Reads at least letters Baseline 3,671 65 4,730 85 3,120 4,021 0.85 0.85 Reads at least letters Posttest 3,671 65 4,730 85 3,341 4,210 0.91 0.89 Reads at least wordsor paragraphs Baseline 3,671 65 4,730 85 2,313 2,980 0.63 0.63

Reads at least wordsor paragraphs Posttest 3,671 65 4,730 85 2,423 3,027 0.66 0.64

Reads stories Baseline 3,671 65 4,730 85 1,505 1,845 0.41 0.39 Reads stories Posttest 3,671 65 4,730 85 1,946 2,365 0.53 0.50

98 The Campbell Collaboration | www.campbellcollaboration.org

The Campbell Collaboration

[email protected] Phone: (+47) 23 25 50 00

Mailing address:P.O. Box 4004, Nydalen N-0403 Oslo, Norway

Visiting address:Pilestredet Park 7(Entrance from Stensberggata)

Website: www.campbellcollaboration.org

About this review

There is a wide range of models for out-of-school interventions to improve childrens literacy. Most of these models have not been subject to rigorous evaluation. Support to parents and peers has been largely ineective in improving literacy, though it has worked in some places. Educational TV has positive eects.

This Campbell systematic review assesses the eectiveness of parental, familial, and community support for childrens literacy development in developing countries. The review summarises ndings from 13 studies, of which 10 were used for meta-analysis.

The Campbell Collaboration

[email protected] Phone: (+47) 23 25 50 00

Mailing address:P.O. Box 4004, Nydalen N-0403 Oslo, Norway

Visiting address:Pilestredet Park 7(Entrance from Stensberggata)

Website: www.campbellcollaboration.org

Word count: 26584

Show less

Abstract

Translate

BACKGROUND

For a majority of the world's children, despite substantial increases in primary school enrollment, academic learning is neither occurring at expected rates nor supplying the basic foundational skills necessary to succeed in the 21st century. The significant lag in academic achievement tells us that simply making formal education available does not fully meet children's needs for literacy development. Globally, many interventions are used to support children's literacy development through channels outside of the formal education system, in children's homes or communities. However, there is a lack of information regarding the effectiveness of these interventions.

OBJECTIVES

The objective of this systematic review was to examine the effectiveness of parental, familial, and community support for children's literacy development in developing countries. This review was intended to provide information about the contextual influences of parental, familial, and community support on children's literacy development skills through the use of interventions that target those influences.

We explored the following questions:

1. What models of reading and literacy learning programs have been implemented in homes and communities in low- and middle-income countries (LMICs)?

2. What models of reading and literacy learning programs implemented in homes and communities in LMICs have empirical evidence regarding their level of effectiveness?

3. How effective are these models in improving children's literacy outcomes?

SEARCH METHODS

Searches for academic literature were conducted in 15 online databases from across the disciplines of anthropology, economics, education, international relations, political science, psychology, and sociology. To capture gray literature, we searched the websites of United Nations agencies, multinational organizations that provide relevant programming, and governmental agencies. For example, we searched the websites of UNICEF, UNESCO, 3ie, J-PAL, USAID and others. Project staff and advisory panel members identified literature from their own organizations, and reached out to their contacts to ask for grey literature. The search was conducted from May to July, 2013.

SELECTION CRITERIA

To be included in this review, studies had to have been published in 2003 or later and include a test of an intervention involving parents, families, or community members with the goal of improving children's literacy development; children ages 3 to 12 years (or "preprimary" or "primary school" age); a comparison group; and they had to take place in an LMIC (according to 2012 World Bank classification). Studies that addressed educational radio were eliminated from consideration because a systematic review of the impact of educational radio already exists (Ho & Thukral, 2009).

DATA COLLECTION AND ANALYSIS

Mendeley software was used to manage citations, abstracts, and documents. Abstracts from each database were initially screened by a single reviewer, but in fact many studies were cited in multiple databases and in turn were screened by two or more reviewers. For the studies that passed the screening, two researchers then independently reviewed each to ensure that it met the criteria for inclusion in this review. Thirteen studies were ultimately identified for this review. Information was extracted regarding the study setting, process used to form the control or comparison group, independence of the evaluation, outcome measures, attrition, baseline equivalence on child preliteracy or literacy learning, descriptions of the treatment and comparison conditions, characteristics of participants and implementers, and statistics required for meta-analysis (where available). Ten studies were included in meta-analyses. Eight of the studies were cluster randomized or quasi-experimental trials, where the level of assignment was at the school or district level. For these studies, effect sizes were computed using Hedges' (2007) effect size assuming equal cluster sample sizes.

RESULTS

The initial search of both the academic and gray literature yielded 10,430 study abstracts. Title and abstract screening resulted in the elimination of 10,357 studies, and 21 duplicate citations were removed. Of the remaining 52 studies, 3 were eliminated for addressing the topic of educational radio, and 36 for failure to meet our inclusion criteria. This left the 13 studies that were ultimately included in this review. These studies fell into three topic areas: educational television, interventions that help parents learn how to support their children's school readiness, and tutoring interventions delivered by peers or other community members. Most of these studies involved interventions to improve school readiness.

The three areas of intervention were examined separately, and studies were combined for meta-analysis in cases where they used the same intervention approach (and had the required statistical information available).

Five studies provided effect size estimates for interventions that help parents support their children learning. Three of the five studies reported significant differences in baseline literacy scores. For overall literacy immediately after the intervention, the effect sizes from five studies including a total of 864 children were heterogeneous, with a mean effect of 0.35 and a 95 percent confidence interval that included 0 [-0.07, 0.77]. Four studies including a total of 786 children provided information about overall literacy at one-year follow-up. These effect sizes were also heterogeneous, with a mean effect of 0.48 and a 95 percent confidence interval that included 0 [-0.35, 1.30]. Five studies of child-to-child tutoring were included in the meta-analysis. These studies all reported difficulties in data collection, raising questions about the quality of data included in the evaluation. For the total reading post-test, the effect sizes from four studies including a total of 1,779 children were heterogeneous, with a mean of 0.15, and a 95 percent confidence interval that included 0 [-0.27, 0.58]. For the beginning reading sub-test, four studies including 1,767 children were also heterogeneous, with a mean of -0.107 and a 95 percent confidence interval that included 0 [-0.40, 0.18]. For the letter identification posttest, effect sizes from five studies including 2,300 children were heterogeneous with a mean of 0.22, and a 95 percent confidence interval that included 0 [-0.13, 0.57]. For the writing post-test, five effect sizes including 1,993 children were heterogeneous, with a mean of 0.27 that was significantly different from 0 (95 percent confidence interval: [0.02, 0.51]). For the follow-up test of reading achievement, effect sizes from three studies including 1,407 children were heterogeneous, with a mean effect size of 0.07 and a 95 percent confidence interval that included 0 [-0.36, 0.04]. For the follow-up test of writing achievement, effect sizes from three studies including 1395 students were homogeneous with a mean of 0.033 that was not significantly different from zero (95 percent confidence interval: [-0.10, 0.17]. For the follow-up test of overall literacy, effect sizes from three studies including 1,397 children were homogeneous with a mean effect size of 0.06 that was not statistically different from zero (95 percent confidence interval: [-0.15, 0.26].

AUTHORS' CONCLUSIONS

This review identified four areas where evidence was available regarding the effectiveness of an intervention approach: educational television, educational radio, interventions intended to support parents' ability to develop their children's school readiness, and tutoring (provided by older peers or community members). Educational radio has been addressed elsewhere (see Ho & Thukral, 2009), so it was not considered in this review. Educational television had a positive impact on young children's literacy development if the child viewed the programming three to five times per week (but not at a lower dosage). Interventions intended to support parents' ability to develop their child's school readiness were not found to be effective overall, although they did have some positive effects in some countries. Peer-led tutoring was found to improve children's school readiness in writing, but not in other areas of literacy. However, this approach did have significant effects across multiple areas of literacy in some country contexts. A tutoring program led by community members resulted in increases in children's literacy.

There were several limitations to this review based on the scarcity of empirical studies and their limited focus on just a few interventions. Numerous descriptions of interventions exist, but few contained a study of program effectiveness in reference to a comparison group. We found only one study that addressed an intervention for children ages 7 and older, and found no eligible studies from Latin America. Therefore, we are left with significant gaps in our understanding of what works in LMICs to improve children's literacy outcomes using interventions outside of the formal education system.

Details

Title

Parental, Familial, and Community Support Interventions to Improve Children's Literacy in Developing Countries: A Systematic Review

Author

Spier, Elizabeth; Britto, Pia; Pigott, Terri; Kidron, Yael; Lane, Julia; Roehlkepartain, Eugene; Scales, Peter; Wagner, Dan; McCarthy, Michael; Song, Mengli; Glover, Janis

Pages

n/a

Publication year

2016

Publication date

2016

Publisher

John Wiley & Sons, Inc.

e-ISSN

18911803

Source type

Scholarly Journal

Language of publication

English

ProQuest document ID

1775789613

Parental, Familial, and Community Support Interventions to Improve Children's Literacy in Developing Countries: A Systematic Review

Jump to:

Full Text

Abstract

Details

Suggested sources