Introduction

Schooling is critically important for children and adolescents in sub-Saharan Africa (SSA). The benefits of education cut across domains of life, including improved behavioral and health outcomes (Hargreaves et al. 2008; Luke 2003; Magadi and Agwanda 2009; Taffa et al. 2003) and adult economic stability (Psacharopoulos and Patrinos 2004).

Achieving adequate schooling, however, is often a challenge for the 52 million orphans in SSA (children under age 18 who have lost one or both parents) (UNICEF 2014). Orphans are at increased risk for poor educational outcomes, with inadequate household funds for educational expenses being one of the most common reasons for school absence and dropout (Bicego et al. 2003; Case et al. 2004; Evans and Miguel 2007; Ssewamala et al. 2016). At the same time, school may be particularly important for orphans. In addition to the human capital acquired through education, schools provide orphans with a consistent environment, routine, and access to caring adults. UNICEF has called for cash transfers as one way to reduce barriers to education for orphans (Arnold et al. 2011; Greenberg 2007; World Bank 2015), and cash transfers have been increasingly utilized in SSA to address educational inequities (Pettifor 2012; Garcia and Moore 2012; Miller and Samson 2012; UNDP 2014).

Interventions promoting schooling among children and adolescents in SSA generally take two forms: cash transfers (either unconditional or conditional) and school support interventions. Unconditional cash transfer (UCT) programs provide funds without any condition of their receipt. Conditional cash transfer programs (CCTs) provide support only when certain conditions (such as attending school) have been met. Donors typically prefer conditional over unconditional cash transfers, as they allow for better control over how their funds are used (Schubert and Slater 2006). For CCTs aimed at improving educational outcomes, students must meet certain school enrollment or attendance criteria in order to receive the funding. This funding then typically is provided directly to the student or to his or her household (Baird et al. 2013, 2014) rather than being specifically earmarked for education.

School support interventions are a more targeted method that directly pays schools, rather than students or families, for the costs of students’ tuition, uniforms, supplies, and other educational expenses. Like more typical CCTs, direct school support interventions are conditional on enrollment criteria. A student must be enrolled in school, for example, in order to have school fees paid. Despite their promise, direct school support interventions and CCTs in general have only been recently adopted in SSA, where resource scarcity can make it more difficult to set up and administer such programs.

Baird et al. (2013, 2014) recently conducted a systematic review of published and unpublished evaluations of cash transfer programs with respect to schooling outcomes, and the findings suggest that such programs increase adolescents’ likelihood of staying in school. Below, we present findings of experimental evaluations of CCTs or school support interventions that are specific to educational outcomes in SSA. The most frequently examined educational outcomes in evaluations are school enrollment and school dropout. All studies except one found positive impacts on these outcomes. Students who received the interventions were more likely to be enrolled in school and less likely to drop out (Akresh, De Walque, and Kazianga, 2013; Baird et al. 2009; Duflo et al. 2015; Duflo et al. 2006; Hallfors et al. 2011,2015; Iritani et al. 2016). The exception to this finding was a CCT that provided cash incentives to both young women and their parent/guardian in rural South Africa, which found no effects on permanent school dropout. This null effect is explained by the universally high level of school attendance and existing social protection programs in the region (Pettifor et al. 2016). School attendance or absenteeism is another commonly examined outcome of educational interventions in SSA. Three of the five evaluated interventions improved attendance (Akresh et al. 2013; Crea et al. 2015; Hallfors et al. 2011,2015; Iritani et al. 2016; Robertson et al. 2013). The two programs that failed to improve attendance were the South African CCT (Pettifor et al. 2016) in which school attendance was universally high, and an intervention that only subsidized school uniforms for sixth graders in western Kenya (Duflo et al. 2015). Grade progression and test scores are two other outcomes that have been studied in evaluations of education intervention. Recipients of school support programs tend to experience more normal (advancing each year) grade progression for their age (Duflo et al. 2015; Iritani et al. 2016). Evidence for interventions’ effects on test scores has been more mixed. A hybrid program that provided both school fees and a cash transfer to students in Malawi was found to improve test scores (Baird et al. 2011), while a comprehensive school support intervention in Zimbabwe had no impact on test scores (Iritani et al. 2016). A third program, a CCT in Burkina Faso, had mixed results, with no impact on the majority of test scores but improved French reading scores (Akresh et al. 2013). Finally, a Zimbabwe study found that the intervention improved students’ educational expectations for the future (Hallfors et al. 2011). In summary, although some mixed study findings exist, overall, the evidence on school support programs in SSA appears promising, with various kinds of CCT and school support programs improving school attendance, preventing dropout, helping students progress without grade retention, and increasing educational expectations.

The original aim of the school support intervention in our study was to help orphaned adolescents stay in school and reduce risky sexual behaviors and sexually transmitted infections including HIV (Cho et al. n.d.). The cRCT analyses we report in the present paper test whether a direct school support intervention specifically improves educational outcomes over a 4-year period in western Kenya. Our study adds to existing literature in a number of ways. First, we focus on orphans, who, as previously noted, face additional significant barriers to school attendance. We are aware of only one other intervention for orphaned adolescents (Hallfors et al. 2011, 2015). Second, we include both male and female orphaned adolescents. Many experimental interventions focus exclusively on girls (Baird et al. 2011; Hallfors et al. 2011; Pettifor et al. 2016), though it is also important to evaluate whether the same interventions impact boys. Third, our educational outcome measures are comprehensive, including all of the outcomes reviewed above. Fourth, we use a direct school support intervention rather than more common methods of cash transfers. We expect that providing support for more substantial school costs such as tuition will improve educational outcomes among orphaned adolescents.

Methods

Participants, Setting, and Procedures

Our data are from a cluster randomized control trial (cRCT) conducted in Siaya County, Kenya. This area has the highest prevalence of orphanhood and HIV in the country (NASCOP 2009). In 2011, we selected 26 primary schools in which at least 20 orphans were enrolled and invited all orphans in grades seven or eight in these schools to participate in the study (n = 937 students). Eight hundred thirty-seven students met the eligibility criteria for the study. We estimated a sample of about 840 participants would have acceptable power to detect small effects on study outcomes. Using primary schools as the unit of random assignment, and total school enrollment size as a stratum, we conducted stratified randomization procedures which assigned 13 schools to receive the intervention (n = 411), and 13 to be control schools (n = 426). The random assignment was conducted with blinding of school names and adequate allocation concealment using a random number generator. Assignment to study condition was revealed to participants and schools after the baseline survey. In each of the three subsequent years (2012–2014), we administered the questionnaire using audio computer-assisted self-interview (ACASI) with participants listening, reading, and responding to questions on personal digital assistant (PDA) devices in either Luo or English. Study staff collected the school records data from participating schools about participants’ enrollment and test scores.

Intervention

Public education in Kenya follows an 8–4–4 system, with 8 years of fee-free primary school, followed by 4 years of secondary school, and 4 years of college or university. Students normally begin class 1 in primary school at age six and progress one grade each year (e.g., class 1–8 in primary school corresponding to ages 6–13 and form 1–4 in secondary school corresponding to ages 14–18). Students take the nationally administered Kenya Certificate of Primary Education (KCPE) exam and the Kenya Certificate of Secondary Education (KCSE) exam at the end of primary and secondary school, respectively. The study paid for the KCPE exam fee and provided a primary school uniform for participants in primary schools selected for the experimental/intervention condition (henceforth referred to as E students or E participants) versus the control condition (C students or C participants). If E students were promoted to secondary school, the study provided a school uniform for their first year of secondary school and paid their secondary school tuition fees. We also provided tuition for E students to attend craft or vocational schools. Contingent on their enrollment, the study continued to pay school tuition for E participants for the remainder of the study period (2011–2015). Nurse research staff visited schools that E students attended to pay school fees each term and to monitor E students’ school attendance and to help them address barriers to attending school. Participants from the original 26 primary schools matriculated into 114 secondary schools across Kenya. The average school fee per E participant was approximately $360 per year: half of the E students attended either day schools (range = $98–500 per year) or boarding schools ($300–1440) and 10% attended vocational/craft schools ($92–1341).

Measures

The survey questionnaire was developed from several validated instruments used in previous studies in SSA (Hallfors et al. 2011, 2015; Cho et al. 2011). Measures used in the present analyses are from the adolescents’ self-reported survey data unless otherwise indicated. School absence was measured by the survey item “During the last term, how often were you absent from school” (1 = never; 2 = once or twice; 3 = once a month or less; 4 = 2 or 3 days a month; 5 = more than 3 days a month). Respondents who reported being absent at least once during the last term were asked about their reasons for school absence and allowed to answer “yes” or “no” to indicate any of the following reasons: lacking a school uniform, being turned away from school for lack of fees, sickness, work or caring for someone at home, lacking school supplies, school being too far away, and menstrual period (for girls only).

A variable measuring school dropout at any point in the study was created from the self-reported annual survey and school records. Participants were asked at each survey wave if they were in school or not in school, and if so, what class they were in. We then created one variable with three mutually exclusive categories: (a) permanent dropout (dropped out during the study and never returned to school); (b) temporary dropout (dropped out of school for one or more years during the study but returned to either regular school or vocational/craft school); and (c) continuous enrollment (never dropped out). We also prepared a variety of measures of progress and performance in school, including number of years behind appropriate grade for age, which was calculated based on age-grade standards in Kenya. Transition to secondary school was measured by whether the participant had ever entered secondary school. Highest grade achieved was measured using the highest grade level that respondents reported in their last survey or the highest grade they completed if they had dropped out of school. KCPE and KCSE scores were collected from participating schools. Educational expectations were asked at each wave about the respondent’s perceived chances of completing secondary school and of graduating from college or university. Response categories ranged from 1 to 5 (1 = almost no chance; 5 = almost certain).

Statistical Analyses

We conducted intent-to-treat analysis, coding participants as being in the intervention condition regardless of whether or not they received school support and also regardless of how much school support they received. To assess whether the E and C groups were equivalent at baseline, we conducted significance tests on demographic and outcome variables using t tests and Rao-Scott chi-square tests, as appropriate. When estimating differential change from baseline to follow-ups for school absence, reasons for school absence, and educational expectations, we used generalized estimating equations (GEE). The models assessed condition (E vs. C), time (as a continuous variable), and a condition by time interaction. We were able to use all of the available observations from the four waves of longitudinal data. GEE is commonly used to analyze longitudinal data with missing data, performing weighted estimation assuming missing at random and the population-average effect is of interest. We conducted multinomial logistic regression for the three-category school dropout variable and also conducted either logistic regression (e.g., transition to secondary schools) or OLS regression analyses (e.g., highest grade achieved) based on the type of outcome. Some outcome variables (e.g., highest grade achieved, test score) are only available for the respondent who completed the final survey or took the national tests; thus, the sample size varies by outcome variable. The study is longitudinal with multiple measurements of the same orphans nested within schools, and intra-correlation coefficients (ICC) ranged from 0.03 to 0.06. We conducted GEE with identity link and robust variance to account for repeated measures on each participant, and we used survey procedures for regression analysis in SAS software (SAS Institute Inc.) to account for the clustering in primary schools (Liang and Zeger 1986). The participants’ age, sex, and a count index of household material assets to indicate socio-economic status (SES) were included as covariates.

Human Subject Protection

All study participation was voluntary. We obtained written informed permission from either a surviving parent or custodial guardian and written assent from all participants. Written informed consent was obtained from adolescents age 18 years or older. The institutional review boards of the Pacific Institute for Research and Evaluation (US) and Moi University (Kenya) reviewed and approved all study procedures. Participating schools in the control condition were provided cash incentives of $240 annually to use for school development projects. Individual participants received small incentives ($3) for completing surveys. This study was registered at the clinicaltrials.gov (ID: NCT01501864).

Results

Baseline Equivalence

Eight hundred thirty-seven students were enrolled in the study at baseline. One participant turned out to be ineligible for the study and one participant withdrew, leaving 835 baseline study participants. Participants in both the C and E groups had a mean age of 15 years (range = 11–20 years) at baseline, and males slightly outnumbered females (see Table 1). About half of the study participants were paternal orphans, 12% were maternal orphans, and another 35% were double orphans. Before the intervention began, the most commonly reported reason for participants’ school absences was sickness (87%), followed by being unable to pay school expenses (80%), and lacking school supplies (70%). On average, the study participants were 2.4 years behind the appropriate grade level for their age. There was no significant difference between C and E groups at baseline (p < 0.05). Overall retention was 97–98% at the first and second follow-ups, and 90% of the participants completed the final survey (see Fig. 1). Eighty-eight percent of the study participants (n = 733) completed all four surveys. These retention rates did not significantly differ by study condition. However, final survey respondents were more likely to be male, younger, and had a higher future expectation of graduating from high school at baseline compared to non-responders (data are not shown).

Table 1 Baseline equivalence between intervention and control groups, 2011 (N = 835)
Full size table
Fig. 1
figure 1

Study design flowchart

Full size image

Educational Outcomes

As seen in Table 2, the intervention had a significant effect on frequency of school absences. School absences remained stable for E group participants throughout the study period but increased in frequency for C group members. Reported reasons for school absences differed between groups as well. Among those who reported being absent from school, participants in the E group were less likely to report absences due to a lack of school fees and school supplies compared to the C group. Rates of absences due to obligations to work or care for others, distance from the school, or menstrual periods (for girls) were similar for both study groups.

Table 2 School absence and reasons for absence: results from GEE analysis using longitudinal data with four time points (N = 835 participants)a
Full size table

The results of multinomial logistic regression in Table 3 show that the intervention, gender, and age had significant impacts on school dropout while SES had no effect. The E group had 0.41 times lower odds than the C group of dropping out of school permanently (confidence interval [CI] = 0.23–0.75) or temporarily (CI = 0.21–0.80) compared to continuous enrollment. Girls had 2.4 times higher odds of dropping out of school permanently (CI = 1.53–3.81) instead of continuous enrollment compared to boys, though boys and girls did not differ significantly in their odds of dropping out temporarily (odds ratio = 1.66, CI = 0.92–3.02) compared to continuous enrollment. Each additional year of age increased the odds of permanent and temporary dropout compared to continuous enrollment.

Table 3 School dropout at time 4: bivariate and multinomial logistic regression analyses (N = 767)
Full size table

At the endline survey, students in the C group lagged one quarter of a year further behind grade for age than students in the E group (see Table 4). In other words, the E group made more progress to the next grade than did C students during the study. The intervention was also associated with greater odds of transitioning to secondary school and higher grade achievement. Being female and being older were both associated with lower odds of making that transition to secondary school. The intervention had no effect on KCPE and KCSE scores, whereas older age and being female had negative effects on these scores. GEE analyses displayed in Table 5 show that the E group maintained significantly higher expectations of completing secondary school and graduating from college or university than the C group, who tended to lower their expectations over time. Females had higher expectations about graduating from both secondary school and college compared to males.

Table 4 School progression, achievement, and performance: results of logistic/OLS regression analysis
Full size table
Table 5 Educational expectations: results from GEE analysis using longitudinal data with four time points (N = 835 participants)a
Full size table

Discussion

Our findings suggest that a direct school support intervention, involving payment of school-related expenses to the school when students are enrolled, can significantly improve educational outcomes in a sample of orphaned girls and boys. The intervention we tested reduced school absence (especially absence due to lack of school fees) as well as both permanent and temporary school dropouts. Intervention participants were more likely to stay on track in grade for age, matriculate into secondary school, and reach higher grade levels. The intervention also increased participants’ expectations of graduating from college in the future. However, we found no significant impact on primary and secondary school test scores.

School absenteeism is a strong precursor of school dropout (Sabates et al. 2010; Flisher et al. 2010). At baseline, both intervention and control group students reported that their major reasons for school absence were being sick, lack of school fees, and school supplies. Whereas school absence in the control group steadily increased over time, the intervention group had stable rates of absence. Furthermore, intervention participants were less likely to be absent from school due to lack of school fees and school supplies over time compared to the control group. It appears that the provision of school fees freed up household funds to pay for additional school-related expenses among intervention participants. However, regardless of study condition, 85% of the participants who reported being absent still reported school absence due to sickness. Siaya County is a highly AIDS- and malaria-affected rural area, vulnerable to flood disasters, and its residents have limited access to health care facilities or safe water. Despite the provision of school fees, 36% of the intervention participants (compared to 94% of the control group) reported that they were absent due to a lack of school fee. Although primary school is tuition-free and compulsory, there are additional expenses that families must cover such as registration and admission fees, additional uniforms, school building funds, PTAs, and extra tutorials (Kagotho 2016; Mungai 2012; Tooley et al. 2008). These costs are even larger in secondary school, and especially at boarding schools which require additional supplies and travel costs. The study participants may have been to referring these additional school-related costs when they were asked about the lack of “school fees.” This type of detail in interpretation of wording may be a limitation of self-reported data.

We found that intervention participants were, on average, less behind in appropriate grade due to grade repetition, more likely to transition into secondary school, and achieved a higher grade level on average by the end of the study. These findings are congruent with previous direct school support intervention studies in SSA (Hallfors et al. 2011, 2015; Iritani et al. 2016; Duflo et al. 2015). We did not find a significant program impact on test scores. The control group students who remained in school to take the exams might be, on average, better students than those who already had dropped out. Thus, selectivity of who stayed in school among control group participants could have made the average test scores across study conditions more similar. While some researchers have found significant impacts on test scores (Baird et al. 2011), the present finding is consistent with another orphan study using a standardized national test (Iritani et al. 2016). Orphaned youth may not be able to take full advantage of learning while attending school due to their lack of emotional support or even abuse by their caregivers and low educational expectations (Coneus et al. 2014; Evans 2002; Goldberg and Short 2012; Morantz et al. 2013; Ng’weshemi et al. 2002). In addition, unstable housing, caring for sick caretakers, taking on adult responsibilities at home, or other disadvantages that orphan youth face may contribute to their poor performance (Carroll and Boker 2003). It is also possible that the quality of schools and of teaching in rural Kenya needs improvement. One study reported that poor performance on the KCPE is explained by high teacher turnover and teacher absenteeism (Reche et al. 2012).

In the cRCT, we enrolled school-going orphaned youth attending the last two primary school grades (7 and 8) as participants in order to support their transition from primary to secondary school. These grades were chosen because school dropout substantially increases during this critical period, and the cost of secondary education is often prohibitive. In spite of providing full tuition support, we could not prevent some degree of school dropout—10% of the intervention group dropped out of school permanently. An additional 8% of the intervention group dropped out of school and then returned either to a regular school or to a craft/vocational school for technical training. Older age was negatively associated with most educational outcomes in this study, which is consistent with other research (Lewin and Sabates 2012; Iritani et al. 2016). The reasons for school dropout among vulnerable orphan youth are complex, since multiple risk factors may compound in their life over time. In spite of not performing significantly better than control group participants on tests, we found that intervention group participants maintained higher educational expectations to a greater degree than those in the control group. It is likely that providing school support to orphans makes them more confident in their ability to fulfill their aspirations, which, in turn, keeps them progressing further at a quicker pace. Our study provided school fees directly to the schools regardless of the type of school chosen, including boarding, craft, or vocational schools. The nurse research staff also helped the intervention students to solve barriers to attending school, such as by addressing health problems. Cost-effectiveness analysis (CEA) could be useful to learn the relative impact of each of these components on educational outcomes. CEA could also investigate the effects of additional monetary support or the type of school attended on longer-term educational gains. For example, CEA among female orphans in Zimbabwe found that school support intervention resulted in quality-life adjusted year (QALY) gains for participants, but that providing fees for boarding schools was not cost-effective (Miller et al. 2013).

There are limitations to this study. We primarily relied on self-reported outcome measures, and although ACASI reduces social desirability bias (Langhaug et al. 2010), the risk of bias due to self-reporting still exists (Baird and Ozler 2012; Cho et al. 2015). We were able to improve the reliability of the school dropout measure somewhat by checking self-reports with school administration records. Furthermore, education-related questions may not be particularly sensitive to many adolescents. For some outcomes, we limited our analysis to participants who completed the final survey. The generalizability of our study is also somewhat limited because our participants were mostly Luo orphaned youth in rural western Kenya. Nonetheless, overall rates of attrition were low over time, and we still believe that our findings are important for improving the educational experiences of orphan and vulnerable youth more broadly. Finally, we treated each of the educational outcomes entirely independently, although these outcomes are interrelated and one may moderate or mediate another. For example, school absenteeism may have explained being behind grade for one’s age. Possible mediation and moderation analyses are warranted in future research.

In summary, the direct school support provided in this cRCT did much to improve the chances of orphaned youth staying in school. It is clear from this study that paying school fees has immediate and direct impacts on a wide range of educational outcomes in this most vulnerable group. Nevertheless, progress in addressing other obstacles is still needed, as many orphaned adolescents delay their schooling or leave school without graduating. Addressing the health status of orphaned adolescents may help reduce school absenteeism so that they can complete school. Further efforts should be directed to strengthen academic assistance programs, such as homework assistance or tutoring programs to improve school performance. Subgroup analyses are needed to test whether the program’s impact varies by different age groups or gender. Longitudinal research is also needed to investigate how educational gains would further affect sexual and reproductive health, as well as economic and social well-being during the transition to adulthood. In conclusion, educational interventions should jointly collaborate with other health interventions in order to ensure that orphan youth can be AIDS-free and healthy in order to maximize educational benefits.