Family Income Has Only Weak Effects on Cognitive Scores in Australia: A Comment on Khanam and Nghiem

A recent Demography article by Khanam and Nghiem (2016), entitled “Family Income and Child Cognitive and Noncognitive Development in Australia: Does Money Matter?,” concluded that “family income is significantly associated with children’s cognitive skills” (Khanam and Nghiem 2016:597). For a 1 standard deviation difference in logged family income, the authors’ estimates were a 0.29 standard deviation increase in scores on the Peabody Picture Vocabulary Test (PPVT), a 0.26 standard deviation increase for Matrix Reasoning (MR) scores, and increases of 0.24 and 0.23 of a standard deviation for Year 3 numeracy and literacy scores (respectively) in the National Assessment Program – Literacy and Numeracy (NAPLAN) (Khanam and Nghiem 2016:616). The authors obtained their estimates not from standard regression procedures but from the generalized method of moments (GMM) estimator.

The purpose of this commentary is to provide evidence that these estimates are not plausible. Therefore, the conclusion that family income substantially matters for cognitive outcomes in Australia is misplaced. I analyze the same data as Khanam and Nghiem (2016): the kindergarten cohort of the Longitudinal Study of Australian Children (LSAC). The outcome variables are the same: early childhood scores in PPVT, MR, and Year 3 performance in NAPLAN. I also include Year 5 NAPLAN performance to include a measure of permanent family income calculated from data over a longer period.

The LSAC measures of family income were derived from the weekly incomes of both parents from all sources. Major efforts were made to construct the most accurate family income measures possible (Mullan and Redmond 2011). For these analyses, the family income measures are first adjusted to 2014 dollars through the annual consumer price index (CPI) and then are logged and centered. I include average income over a longer period given the claim that single-year measures of income include too much measurement error and thus downwardly bias the estimates (Blau 1999). This measure of “permanent income” is calculated by averaging the CPI-adjusted family income measures from the current and previous waves, then taking the log of the average and finally centering.

I estimate the effects of income on these outcomes in a series of models beginning with family income as the only predictor variable. Subsequent models add, in turn, two other indicators of socioeconomic status (SES): parental education and occupational status. These two SES indicators are commonly used in analyses of cognitive skills and student achievement. The final models in the analyses of PPVT and MR scores include the respective prior cognitive skill measure. The analyses of student achievement include a measure of cognitive ability from combining PPVT and MR scores. The analyses of Year 5 achievement also include same-domain prior achievement. I present both the metric and standardized estimates. I handle missing values with multiple imputation (Baraldi and Enders 2010).

Before discussing the results, I note that the estimates reported by Khanam and Nghiem (2016) seem larger than they are. Standardized effects of between 0.23 and 0.29 are comparatively large in studies of the effects of parental measures (apart from parental cognitive ability) on cognitive outcomes. However, the effects cannot be interpreted in the same way that standardized effects are for nonlogged independent variables. To simplify, assume that the estimates were all 0.30. That means that for a doubling of family income, the predicted achievement score would increase by 0.17 of a standard deviation. A tripling of family income would translate to an increase of 0.27 of a standard deviation.¹ Therefore, large changes in family income are associated with only moderate changes in test scores, even when assuming an estimate of 0.30.

The first model in each of Tables 1, 2, 3, and 4 shows the estimates for just family income. Without the effects of other predictors being considered, family income has very weak effects on these cognitive outcomes. The variance explained is only 2 % or 3 % for PPVT and MR scores, increasing slightly to 4 % for Year 3 NAPLAN numeracy and literacy. Permanent family income—that is, family income averaged over four waves—show slightly stronger effects, accounting for up to 5 % of the variance for Year 5 reading. Therefore, family income accounts for very little of the variance of children’s cognitive outcomes, even without the effects of other influences. The standardized estimates, which range from 0.13 to 0.21 (the latter for permanent income), are smaller than the standardized effects reported by Khanam and Nghiem (2016). It is not logical that Khanam and Nghiem’s effects—which are supposed to take into account unobserved heterogeneity and endogeneity—are larger than the much simpler bivariate effects reported here.

Table 1 Effects of family income (log) and other SES indicators on Peabody Picture Vocabulary Test (PPVT) scores

Table 2 Effects of family income (log) and other SES indicators on Matrix Reasoning (MR) test scores

Table 3 Effects of family income (log) and other SES indicators on Year 3 numeracy and reading test scores

Table 4 Effects of family income and other SES indicators on Year 5 numeracy and reading test scores

Subsequent models that include other relevant factors show even weaker effects for family income. The addition of parental education, which clearly has stronger effects than family income, generally halves the estimates for family income. The further addition of parental occupational status produces a smaller decline. Of the three SES indicators, family income consistently shows the weakest effects (Model 3). The effects of family income are miniscule and often are not statistically significant when prior cognitive scores are taken into account. This is also the case for the analysis of Year 5 achievement, which used a measure of permanent family income (Table 4). Based on the analysis of Year 3 NAPLAN scores, early childhood cognitive ability (PPVT + MR) clearly has much stronger effects (β ≈ 0.55) than the socioeconomic background variables (0.02 ≤ β ≤ 0.12), tripling the variance explained with its addition to the model. For Year 5 achievement, the effects of same-domain prior achievement are strong, with standardized effects around 0.40.

The conclusion that family income’s unadjusted effects on early childhood cognitive outcomes in Australia are small, with very small or negligible direct effects, is consistent with overseas studies. For the United States, Mayer (1997:90–91) estimated conventional standardized effects of 0.13 for family income on PPVT test scores, 0.06 for mathematics scores in the Peabody Individual Achievement Test (PIAT), and 0.14 for PIAT reading. Her “true” effects of family income were usually smaller and not statistically significant.² Analyzing mathematics achievement, Orr (2003:291, 293) reported no effect for family income (averaged over five years) on mathematics achievement, net of father’s occupational status, mother’s education, mother’s ability and other variables. Carlson and Corcoran’s (2001:789) analysis of reading scores in children aged 7–10 found that family income had an effect, but it was relatively weak. A doubling of family income increased child’s reading score by about 3.2 %. For Britain, Violato et al. (2011:1201) concluded “a weak or absent direct effect of family economic resources on child development.” Similarly, Aughinbaugh and Gittleman’s (2003:429) analysis of children’s test scores in the United States and Britain found that the effects of family income on test scores were quite small: the maximum effect was 0.08 of a standard deviation, net of other predictors including mother’s ability. Analyzing data from South Africa, Cherian and Malehase (1998:431) concluded that there was “no relationship between financial conditions at home and scholastic achievement of children from single-parent and two-parent families.” Analyzing student achievement in Denmark using data from the OECD Program for International Student Assessment study, Humlum (2011:994) noted that family income effects were small and statistically insignificant. Even a large change in permanent income of 100,000 Danish krone (equivalent to about US$15,000) was associated with a difference of only 2.6 PISA score points.³

Therefore, the often made claim that family income is important to children’s cognitive outcomes is not supported by the evidence from a range of studies.