Sickness Experience in England, 1870–1949

Abstract

Using data from the Hampshire Friendly Society, a sickness insurance institution in southern England, we examine morbidity trends in England between 1870 and 1949. Morbidity prevalence increased between 1870 and around 1890, mainly because of a rise in the average duration of sickness episodes, but after 1890 average durations fell markedly even though the incidence of sickness rose. During the first two decades of the twentieth century, sickness prevalence increased gradually, but this rise was entirely due to the greatly increased duration of claims made by men aged 65 years and over. After the early 1920s, both the incidence and the average duration of sickness claims declined. These trends seem to be measuring ‘objective morbidity’: they vary closely with year-on-year changes in the mortality of men of working age, but do not show any clear relationship with real wages or unemployment. Our conclusions are different from those of earlier research using English sickness insurance data. We believe that one reason for this was a methodological problem with the analysis performed by nineteenth-century actuaries.

4.1 Introduction

The question of how sickness, or morbidity, evolved during the period of mortality decline at the end of the nineteenth and the start of the twentieth centuries has been debated ever since Riley (1989, 1997) argued that sickness rates rose as mortality rates fell. Riley analysed aggregate data from sickness insurance schemes operated by the Ancient Order of Foresters (AOF) in Britain and concluded that there was a rise in reported morbidity between 1870 and 1910. This rise was not primarily attributable to changes in the age structure of the insured population, but represented an increase in age-specific sickness rates. Riley argued, following the nineteenth-century actuaries who had originally analysed similar data, that the increasing morbidity was due to the increased duration of periods of sickness: people were not ill more often, but they were ill for longer when they did succumb. He said this was a consequence of improved care of the sick, which meant, first, that a greater proportion of them recovered from their afflictions, but that those who recovered took longer to recover than their predecessors had taken to die; and, second, that those who still died took longer to do so. Both of these effects increased the average duration of sickness episodes and hence raised the prevalence of sickness at any time.

More recently Edwards et al. (2003) used individual-level sickness insurance data for the Hampshire Friendly Society (HFS) in southern England to examine morbidity trends. Contrary to Riley, they failed to find evidence of a rise in morbidity, except perhaps after the period spanned by Riley’s data, and they discussed the possibility that this later rise might be associated with the advent of national insurance in England in the early twentieth century. Using a larger sample drawn from the same source, Harris et al. (2012) analysed the prevalence and incidence of sickness by age between 1870 and 1950. They also found little evidence of a rise in prevalence, except perhaps among those aged 50–65 years between the 1870s and the 1890s and again between the 1920s and the 1940s, though the prevalence in this age group fell back in the intervening period (Harris et al. 2012, pp. 733–4). Among those aged under 65 years, neither the incidence nor the average duration of episodes of sickness showed an overall trend. Among those aged over 65 years (whom they only analysed for the period after 1900), prevalence did not change greatly. There was, however, clear evidence of a rise in duration and a fall in incidence among those aged over 65 years during the first half of the twentieth century.

The data used by Riley (1997), Edwards et al. (2003) and Harris et al. (2012) come from sickness insurance schemes. Such data are indirect estimates of sickness in that they measure absence from work.¹ Morbidity trends reported from sickness insurance schemes may vary for many reasons. One reason is that morbidity ‘objectively defined’ changes. This ‘objective morbidity’ is unobservable in practice, but denotes some kind of measure of sickness which would be consistent over time and space, and which would be independent of the context in which the measurement was made. In practice, we observe derivatives of ‘objective morbidity’, such as the inability to work, or more accurately a declaration by an individual (subject to the certification procedure, lay or medical, employed by the insurance scheme) that he or she is unable to work, this declaration being confirmed by those administering the scheme. How closely this measure captures ‘objective morbidity’ is not really a helpful question as we cannot observe the latter. What is important if we are to use sickness insurance data to infer morbidity trends is that the relationship between the measure of morbidity we use and ‘objective morbidity’ is, at least at the population level, consistent over time.

But this may not be so. Johansson (1991) suggested that whether or not a person is classified as too ill to work may depend on cultural views about how ‘objectively’ sick a person has to be in order to adopt the sick role. Reported sickness rates may thus rise or fall, even when morbidity ‘objectively defined’ is not changing. Specifically, she argued that the threshold for adopting the ‘sick role’ fell over time with economic and institutional development, and with the increased salience of scientific medicine, so that a rise in reported morbidity does not necessarily mean that morbidity ‘objectively defined’ also rose. In brief, people declared themselves (or were declared by the medical profession to be) unfit to work with increasingly minor ailments. She termed this the ‘cultural inflation’ of morbidity (Johansson 1991, 1992).

Whiteside (1987) argued that the sickness reported by sickness insurance schemes may be disguised unemployment, so that reported sickness rates might vary inversely with the health of the economy. The AOF (1928), p. 57) commented that the General Strike of 1926 was associated with higher ‘benefit expenditure’. A few years later, the High Chief Ranger of the AOF commented that ‘[t[he year … 1931 is showing a decided increase [in sickness claims]. That increase is undoubtedly much more closely associated with economic stress and unemployment than with real incapacity to work, even after allowing fully for the ill-effects of unemployment on health’ (AOF 1931, pp. 40–1). Macnicol (1998) suggested that changes in claim rates may have depended on the availability of alternative forms of insurance (e.g. statutory pensions) for underemployed older workers.

The trends exhibited by sickness insurance data may also vary with the nature of the insurance funds. Murray (2003) analysed what he referred to as ‘sickness absence’ from work using data from a series of large funds in continental Europe. Funds where membership was compulsory revealed different trends in sickness absence from those where membership was voluntary. Murray attributed this discrepancy to the changing financial health of the two types of fund over time and to the fact that they attracted different risk pools. The compulsory funds exhibited an increase in sickness absence between 1885 and 1905 which Murray interpreted as being due to their greater ability to pay benefits. The voluntary funds were always under pressure because their members were disproportionately drawn from persons who considered themselves to be less healthy than average. As the pressure increased, they sought to reduce the benefits paid, leading to a decline in the prevalence of sickness absence among their members.

Finally, reported levels of morbidity might be affected by changes and variations in members’ attitudes to the use of insurance schemes and changes in institutions’ preparedness to pay benefits (Harris 1999; Downing 2015). Such attitudes might vary between schemes, since they can arise from different procedures laid down in the constitutions of individual schemes, or the differential ability of schemes to monitor claims (Downing 2015). But they may also occur within the same scheme over time, especially if administrators’ views of the financial health of the scheme are the driving force.

Gorsky et al. (2011) addressed each of the effects mentioned above in the context of the HFS data. This was possible because the HFS has left a comprehensive set of Annual Reports and other documents in which changes in the volume and nature of sickness claims were discussed and actions proposed to maintain consistency in the processing and monitoring of claims. Although the HFS introduced a number of changes in the arrangements used to monitor the veracity of sickness insurance claims, the authors concluded that ‘most of the relative rise in morbidity seems to have been real, and not the result of cultural changes in the definition of the sick role or in the generosity or policing’ of insurance benefits (Gorsky et al. 2011, p. 1,782). They suggested that sickness benefit might have been used from time to time as a substitute for other forms of benefit—mainly pensions—among a small number of older members (mostly aged over 65 years), thereby allowing some older workers to disguise their exit from the labour force by claiming long-term sickness benefit (Macnicol 1998). But they found little evidence of systematic variation in reported sickness rates with the state of the economy or of ‘diagnostic creep’ whereby claims were lodged for ever more trivial illnesses. The HFS’s actuary also pointed out repeatedly that members who were insured for sickness benefit at a higher rate tended to claim more from the fund (Gorsky and Harris 2005).

In this paper, we present a reanalysis of the data used by Edwards et al. (2003) and Harris et al. (2012) using an approach different from theirs. We measure the trend in morbidity over time using annual age-standardised sickness prevalence ratios and age-standardised incidence ratios for the period 1870–1949. We then use regression models to examine the association between the trends in reported ‘sickness absence’ and a range of factors which might be plausibly related to the tendency to claim sickness benefit. These factors include a more objective measure of ‘healthiness’ (based on mortality rates) as well as measures describing economic trends and changes in social policy.

Section 4.2 of the paper briefly describes the HFS data. Section 4.3 presents trends in age-standardised morbidity. The regression models are examined in Sect. 4.4. Section 4.5 discusses the findings, focusing on the differences between the trends revealed by the HFS data and those from the AOF. This section also presents evidence that an analysis by contemporaries which purported to show that the rise in morbidity in the late-nineteenth century was duration-driven was flawed. In Sect. 4.6, we summarise our conclusions.

4.2 Data

The HFS data have been described in detail elsewhere (Edwards et al. 2003; Gorsky et al. 2006), so only a brief description is given here. The HFS was an autonomous institution set up in rural southern England in 1825 to provide benefits to working people. Its membership grew slowly until about 1850 but the rate of recruitment then accelerated (Harris et al. 2012, p. 725). It was administered by the local gentry and consequently had a paternalistic character. Initially, its members were drawn from rural and small-town Hampshire although, as time went on, it expanded to a limited extent outside the county boundaries. It provided three principal types of benefit: sick pay for members unable to work temporarily because of illness or injury, life insurance, and a pension. Most HFS members were men; women were allowed to be members in the early years but in 1850 were prohibited from subscribing for sick pay. Therefore, our data relate only to males. Members could choose to subscribe for all three benefits, or just one or two. Our data relate only to those who subscribed for sick pay, and comprise a sample of approximately 10% of members, the sample consisting of 5552 men born between 1790 and 1926. The sickness histories of these men are based on details of the number of days’ sick pay each man received in each year from 1870 to 1894, and each quarter from 1895 onwards. Our analysis covers the period from 1870—the first year for which we have data on the number of sick-days each member experienced—to 1949. In all, there are 83,533 man-years of exposure in our data.

The data measure the length of time each man was (in any year or quarter) off work and claiming sickness benefit.² They provide direct estimates of sickness prevalence, but assumptions are required in order to estimate sickness incidence, since if a man received some sick pay in a given year (or quarter), we do not know whether the episode was a continuation of a previous sickness episode or how many separate sickness episodes this represented. Provided the same assumptions are made throughout, it is still possible to examine changes over time in the incidence of sickness. After experimenting with several algorithms, we settled on one which calculates the minimum number of distinct sickness episodes consistent with the observed data: the ‘minimum incidence’ assumption. In this case, any man who reported sickness in two successive quarters (or years before 1895) is assumed to have had only one period of sickness which started in the first quarter (or year) and ended in the second quarter (or year), unless this was incompatible with the pattern of sickness reported in adjacent quarters (or years).³ The assumptions required to estimate the incidence of sickness from our data are less demanding after 1895, once the data become available quarterly rather than annually.⁴

4.3 Trends in Sickness in the Hampshire Friendly Society

Figure 4.1 shows the number of sick weeks reported each year between 1870 and 1949 together with the total number of insured men exposed to the risk of sickness in our sample in each year. The graph also shows the number of insured men aged 55 years and over. The threshold of 55 years was chosen because there is evidence that age-specific morbidity rises much more rapidly after that age than it does at younger ages (Harris et al. 2012, p. 730). The exposed to risk rose gradually from 1870 until around 1920, during which period the proportion of the membership aged over 55 years also increased. Then a recruitment drive raised the number of new members rapidly from 1925 onwards. Since most new joiners were young, this reduced the proportion of members aged over 55 years. The fall in the exposed to risk after 1938 is because we only collected data for men who joined up to 1939. The total number of sick weeks in the sample rose fairly steadily to a peak in 1940.

Fig. 4.1

Total sick weeks and number of insured men exposed to the risk of sickness in each year from 1870 to 1949

Source: Hampshire Friendly Society data

Because we know the date of birth of every member, we can work out the age composition of the members in all years from 1870 to 1949. This allows us to control for variations in the age structure over time using standardised prevalence ratios(SPRs). The SPR for year i, SPR_i, is given by the formula

$$ SP{R}_i=\frac{P_i}{\sum_x{p}_x{E}_{x,i}} $$

(4.1)

where P_i is the total number of sick weeks (or sickness claims) reported in year i among those who were members of the society in that year and eligible for sick pay, E_x,i is the number of members in year i in age group x, and p_x is a standard age-specific morbidity prevalence for age group x. Table 4.1 gives details of the age groups and the standard age-specific morbidity prevalence, which was calculated as the average prevalence of morbidity in each age group over the whole period from 1870 to 1949. The rapid increase in p_x at older ages demonstrates the need for age-standardisation. We have also computed standardised incidence ratios (SIRs) for the period from 1895 onwards using the formula

$$ SI{R}_i=\frac{C_i}{\sum_x{c}_x{E}_{x,i}}, $$

(4.2)

where C_i is the total number of claims estimated to have been made in year under the ‘minimum incidence’ assumption and c_x is a standard age-specific incidence schedule (based on the average estimated incidence over the whole period). The outcome of the standardisation exercise is shown in Fig. 4.2. Because the annual SPR_is and SIR_is are noisy, we have also drawn moving averages to help highlight the trends.⁵

Table 4.1 Standard schedule of morbidity prevalence by age

Fig. 4.2

Standardised morbidity ratios based on sick weeks (SPR) and estimated incidence (SIR), 1870–1949

Source: Hampshire Friendly Society data

Note: SPR, standardised prevalence ratio; SIR, standardised incidence ratio. The SIR values for the periods 1870–1894 and 1895–1949 are not exactly comparable because of the different units of time used in the data. Accordingly, we do not compute moving averages of the SIR across the time point 1 January 1895. The SPRs are comparable across the whole period. We have not estimated the SIR for 1870 as we cannot be sure how many episodes of sickness reported in 1870 actually began in earlier years

The SPR_is increase by about 25% during the 1880s to peak in the early 1890s before falling back by the end of the century. They then rise gently and somewhat erratically to reach a second peak in the late 1920s before beginning a sustained fall, punctuated only by the morbid year of 1940. Comparing the SIR_is over the whole period is difficult because of the change to the data after 1 January 1895. Looking at the two periods separately, we can say that there seems to have been a gentle increase in the incidence between the mid-1870s and 1900, though the year-on-year variability is high in the early 1890s. Between 1895 and 1915, the SIR_is are roughly constant. There is a slight dip around 1920 followed by another period of roughly constant values. SIR₁₉₄₀ reveals the incidence of sickness in that year to have been exceptional. After 1940 there is a substantial decline.

Harris et al. (2012, p. 733) observed that the trend in the incidence and duration of sickness among those aged 65 years and over was different from that among younger men. Gorsky et al. (2011, p. 1,782) examined a belief by the HFS authorities that, during the early twentieth century, some men aged 65 years and over were using lengthy periods of sick pay as substitutes for pension payments (after an investigation, the Society concluded that this might have been happening in a handful of cases, but too few to be worth acting upon). We have repeated the analyses reported in Fig. 4.2 using only data for men aged under 65 years (Fig. 4.3). For the period to 1900, the trends for the under-65s are similar to those for all men (this is not surprising, as few members of the HFS were aged 65 years or over before 1900). After 1900, though, the SPR_is for the under 65s begin a slow, erratic decline. The peak in sickness prevalence in the 1920s vanishes, but the years of high morbidity just before World War I stand out more. Trends in the SIR_is for the under-65s are rather similar to those for all men. This is to be expected, as the over-65s tended to have lengthy sickness episodes, which would have a greater impact on prevalence than incidence.

Fig. 4.3

Standardised morbidity ratios based on sick weeks (SPR) and estimated incidence (SIR), 1870–1949 excluding men aged 65 years and over

Source: Hampshire Friendly Society data

Note: SPR, standardised prevalence ratio; SIR, standardised incidence ratio. The SIR values for the periods 1870–1894 and 1895–1949 are not exactly comparable because of the different units of time used in the data. Accordingly, we do not compute moving averages of the SIR across the time point 1 January 1895. The SPRs are comparable across the whole period. We have not estimated the SIR for 1870 as we cannot be sure how many episodes of sickness reported in 1870 actually began in earlier years

The ratio between the prevalence of sickness and its incidence is the average duration of sickness episodes. This is plotted for the raw (unstandardised) data in Fig. 4.4. Looking first at Fig. 4.4(a) and taking all men together, the average duration of episodes of sickness rose during the 1880s. It also rose between around 1900 and the early 1920s before falling until the mid-1930s. Obvious trends in average duration among the under-65s are more difficult to discern, apart from the rise during the 1880s.

Fig. 4.4

Mean durations of sickness, 1895–1949. (a) Using annual data for 1870–1894 and quarterly data for 1895–1949 (in weeks). (b) Using annual data throughout (1871 = 100)

Source: Hampshire Friendly Society data

Note: The mean durations have been calculated by dividing the total number of sick weeks in each year by the total number of claims made. In graph (a), the series for the period before 1895 and that for the period from 1895 onwards are not strictly comparable. In graph (b), all years may be compared but the level is likely to be inaccurate, so the series has been indexed to 1871 = 100

A limitation of Fig. 4.4(a) is that we cannot compare the periods before and after 1 January 1895. This is especially frustrating because a key conclusion of Riley’s (1997) analysis of the AOF data was that sickness durations were rising between 1870 and around 1910. Figure 4.4(a) reveals a clear rise in durations during the 1880s and a rise after about 1900, the latter being a characteristic largely of those aged 65 years and over, but trends in the 1890s are not clear.

Figure 4.4(b) attempts a consistent comparison of average durations across the whole period by artificially reducing the level of detail in the data for the period from 1895 onwards so that it matches that for the earlier period. Doing this means that the reported level of the average durations is certainly overstated for the post-1895 period but that we can compare across the 1 January 1895 and try to establish the trend over the whole period.⁶ The results do not differ greatly from those in Fig. 4.4(a).⁷ There was a rise in mean durations during the 1880s and a fall in the 1890s. After 1900 average durations rose sharply among those aged 65 years and over to a peak around 1925 before falling back quickly; among those aged under 65, there were fluctuations in the mean duration, but no obvious secular trend. The difference between those aged 65 and over and those aged under 65 is partly associated with the different conditions giving rise to claims for sick pay. The most widespread causes of sickness among elderly men were diseases of the circulatory system, diseases of the nervous system and diseases of the skin, whereas among younger men, diseases of the respiratory system and injury were most commonly cited (Gorsky et al. 2006).

Our results thus confirm the observations of Harris et al. (2012) that there was a rise in the duration of claims after 1900 among those aged over 65 years. The magnitude of this rise is worth emphasising, however. The average duration of a claim in the age group 65 years and over in 1900 was 18 weeks; in 1925 it was 60 weeks.

Figure 4.4 is based on unstandardised data. In other words, it does not adjust for changes in the age structure of the HFS membership. Mean durations of sickness were much greater among older men than among younger men. Obtaining a standardised measure of the duration involves adjusting the quantity ${P}_i/{C}_i$ to take account of the relative effect of age on the reported prevalence and incidence. One way of achieving this is to define a ‘standardised duration ratio’ ($SD{R}_i$) to be equal to SPR_i/SIR_i. It is straightforward to show that this implies that

$$ SD{R}_i=\frac{P_i}{\sum_x{E}_{x,i}{p}_x}.\frac{\sum_x{E}_{x,i}{c}_x}{C_i}=\frac{P_i}{C_i}.\frac{\sum_x{E}_{x,i}{c}_x}{\sum_x{E}_{x,i}{p}_x}. $$

(4.3)

In other words, it involves adjusting the mean durations estimated from the ‘raw’ prevalence and incidence (the quantities plotted in Fig. 4.4) by a factor $\frac{\sum_x{E}_{x,i}{c}_x}{\sum_x{E}_{x,i}{p}_x}$, which reflects the expected average duration of sickness in a population with the age structure of the HFS in year i and the average age-specific incidence and prevalence rates.

Figure 4.5 plots the SDR_is for all men, as well as the SIR_is adjusted to render them comparable for the periods before and after 1 January 1895. The incidence of sickness rises by about 25% between the 1870s and the 1890s. Thereafter, it does not change appreciably in the long run, though there are year-on-year fluctuations. There is evidence of a decrease in incidence after 1940. The ‘standardised duration ratios’ rise in the 1880s, but fall rapidly during the 1890s and, although they rise a bit during the first two decades of the twentieth century, they never again reach the values they attained in the late 1880s. After the early 1920s, they once more subside.

Fig. 4.5

Standardised incidence ratios (SIR_is) and ‘standardised duration ratios’ (SDR_is) adjusted to use annual data throughout

Source: Hampshire Friendly Society data

Note: Each SDR_i is computed as SPR_i/SIR_i, where SPR_i is the standardised prevalence ratio in year i

Riley’s analysis (1997) of data from the AOF found that ‘[b]etween the 1870s and the first decade of the twentieth century age-standardised sickness prevalence increased from about 9 to about 12.5 days per member per year (or by about 40%)’ (Riley 1999a, p. 121). Our results suggest that, during the 1880s, there was an increase in sickness prevalence, after adjusting for changes in the age structure, of about 25%, but this was largely reversed during the 1890s. This rise and fall was associated with a rise and fall in the average duration of periods of sickness among all age groups. Although the difference in the nature of the data makes a comparison across 1 January 1895 awkward, Fig. 4.5 suggests that there was a rise in the incidence of sickness between 1880 and 1895. After 1900, though there were short-term fluctuations in both the incidence and prevalence of sickness, the main trend was a large increase in the average duration of claims among men aged 65 years and over. The prolongation of claims among these old men was driving almost all of the overall increase in sickness prevalence in the early twentieth century. Once this trend changed after the mid-1920s, and the duration of claims among the over-65s started to be curtailed, both the incidence and duration of sickness fell away.

4.4 Factors Associated with Morbidity Trends

In this section, we consider the association between a range of covariates and the sickness trend revealed by the HFS data by regressing the SPR_is on a set of covariates designed to measure aspects of the social and economic environment which have been considered relevant in accounting for variations in reported sickness. We capture economic conditions using the annual unemployment rate and real wages (Mitchell 1988, pp. 60–2, 124, 168–9).⁸ We also include dummy variables for wartime years. Most of the members of the HFS were able-bodied males, and many of these would have been recruited by the armed forces during the World Wars. Those who were not serving in the forces are likely to have been in poorer health than the average, and may have been required to work harder and longer than their health could bear, so may have experienced increased morbidity rates. We included an interaction between wartime and unemployment, to examine whether the effect of unemployment was greater among those left behind during the period of conscription. We measure changes in the social policy environment with dummy variables distinguishing the pre-national insurance era from the later period, and the period after the introduction of the state contributory pension scheme.⁹

We include a measure of variations in the disease environment, or in the ‘general healthiness’ of each year. A rise in the mortality rate should indicate a more hostile disease environment. If reported sickness varies more closely with this proxy for the hostility of the disease environment than it does with unemployment or other economic indicators, it suggests that the morbidity trends we are capturing are ‘real’ in the sense that they reflect trends in ‘objective morbidity’. We experimented with a range of different measures of mortality: the national death rate from all causes for males aged 35–44, 45–54 and 55–64 years, and the death rate in Hampshire for males at all ages from influenza and bronchitis.¹⁰ All the mortality variables produced similar results, but we present only those using the national death rate from all causes for males aged 45–54 years and the death rate in Hampshire from influenza and bronchitis for males at all ages. We estimated models using SPR_i for all men and for those aged under 65 years only (Table 4.2).

Table 4.2 Results of models of standardised prevalence rate

The results are clear. In all four models, reported morbidity is associated consistently with our measures of the hostility of the disease environment, but does not seem to have been influenced as strongly by the economic outlook, wartime, or changes in social policy. War tended to reduce the prevalence of reported sickness, except among the unemployed. The fit of the models to the data is reasonably good in most years (Fig. 4.6). The effects of the introduction of national insurance and the state contributory pension scheme were not strong, but national insurance was associated with a reduction in claims among those aged under 65 years and an increase in those aged 65 years and over, whereas the state contributory pension scheme was associated with a reduction in claims among men aged 65 years and over, as we might expect.¹¹

Fig. 4.6

Actual standardised prevalence ratios and those predicted from the model, 1870–1949. (a) All men. (b) Excluding men aged 65 years and over

Source: Hampshire Friendly Society data

Note: The fitted values are from models using the national death rate for males aged 45–54 years: see Table 4.2

The conclusion of this modelling exercise is that reported sickness prevalence in the HFS data, which is based on a medically certified inability to work, seems to be reflecting ‘objective morbidity’ reasonably closely. It adjusts in response to temporal changes in the general ‘healthiness’, and does not seem to respond closely to any behavioural factors which might be associated with changes in the economic environment. Among men aged under 65 years, there is some evidence of systematically lower sickness rates in the era following the introduction of national insurance. Thus, national insurance may, as Edwards et al. (2003) speculate, have had some effect on reported sickness levels, but its effect was in the opposite direction from the one they expected. The increased reported morbidity of the over 65s in the first decades of the twentieth century was not associated with short-run changes in unemployment rates or real wages, but the decreased morbidity among this group after the 1920s may have been influenced by the appearance of state contributory pensions.¹²

4.5 Prevalence, Incidence and Duration of Sickness

Our results broadly confirm those of Edwards et al. (2003) and Harris et al. (2012). However, we have been able to provide a more systematic history of the prevalence, incidence and duration of reported sickness among HFS members. Our results are different from those obtained by Riley using data on AOF members. Looking at the period between 1870 and 1910, Riley found an increase in morbidity of close to 40% among the AOF members, and he attributed this mainly to a rise in the duration of sickness episodes. During the same period, we observe an increase of about 25% in age-standardised sickness prevalence, almost all of which occurred during the 1880s and was associated with an increase in both the incidence and the duration of episodes of sickness. After 1890 this increase in prevalence was reversed, the reversal being almost entirely due to a decline in the average duration of sickness episodes. The only period during which there was a rise in age-standardised sickness prevalence which is accounted for mainly by duration was the first two decades of the twentieth century and, at this time, the rise in duration was concentrated among men aged over 65 years.

The idea that, as mortality declines, morbidity rises due to the increasing duration of spells of sickness has a plausibility derived from the well-known model of the epidemiologic transition (Omran 1971). This model posits that, as mortality declined, infectious diseases retreated and were replaced by ‘degenerative and man-made’ diseases as causes of death, a process which happened in England and Wales between 1860 and 1960 (Omran 1971, pp. 738, 740).¹³ Infectious diseases tend to be of short duration and kill quickly or not at all, whereas degenerative disorders tend to be long-lasting, killing more slowly but more reliably. Assuming that the conditions which are the main causes of death are likely also to be significant causes of sickness, the movement of a population through the epidemiologic transition is, therefore, likely to be accompanied by a rise in the average duration of spells of sickness.

Some infectious diseases were declining rapidly as causes of death after 1870 in England and Wales. A good example is respiratory tuberculosis, or phthisis. No cure for phthisis existed at this time and, since recovery without treatment was rare once a person started to suffer serious ill health from the symptoms, we can suppose either that the incidence fell, or that improved medical care allowed patients to survive for longer before eventually succumbing, or both.¹⁴ But other, normally acute, infections were on the increase. Russian influenza, which arrived in the UK in 1889, was epidemic from 1890 to 1892 (Registrar General 1907, p. lxxv; Parsons 1891). Between 1887 and 1891, there was an increase from 1483 to 2095 in the total number of claims made to the HFS. Of this increase of 612, 435 (71%) was due to influenza (Edwards et al. 2003, p. 152). Figure 4.2 shows that, around this time, the incidence starts to fluctuate quite wildly from year to year, with peaks higher than had been experienced since 1870. The HFS authorities became concerned about the financial health of the Society following an actuarial valuation in 1889, and set in place a more rigorous system for policing claims (Gorsky et al. 2011, p. 1,781). Our analysis suggests that they were right to be concerned that claims in the late 1880s and early 1890s were running at unusually high levels. To what extent the subsequent return of the volume of claims to ‘normal’ levels was a response to the more stringent monitoring regime established in the 1890s and to what extent it derived simply from the natural waning of the Russian influenza pandemic we cannot say. However, influenza claims were still being made at a greater rate in 1910 than in they were during the 1870s (Edwards et al. 2003, p. 152). This reflects the continuing high mortality from influenza during the first decade of the twentieth century: the age-standardised national death rate for males from influenza was 22 per 100,000 in the 1880s, 385 per 100,000 in the 1890s, and still 221 per 100,000 in the decade 1901–1910 (Registrar General 1919, p. ccv).¹⁵

A duration-driven increase in sickness has not always been observed in British data. When Riley (1999a) looked at three local sickness insurance schemes for which he had individual-level—as opposed to aggregate-level—data, he found that morbidity trends were different in each. In Abthorpe, Northamptonshire, the average duration declined as well as the incidence; in Ashbourne in Derbyshire, incidence fell dramatically but duration was roughly constant; only in Morcott in Leicestershire was the pattern of increasing duration observed (Riley 1999a, p. 116). Even where morbidity was unambiguously rising, this rise seems to have been as much in the incidence of sickness as in its duration. The Guild of St George Friendly Society in Cheshire, for example, shows a ‘rate of falling sick’ which more than doubled between 1873 and 1913, whereas the average duration increased by about 40% between 1873 and 1903 and `by about 70% between 1871 and 1913 (Riley 1989, p. 187).

A key piece of evidence in support of increased durations comes from large surveys undertaken by nineteenth-century actuaries. Riley (1989, p. 164) wrote that ‘[t]he testimony from the actuaries is unambiguous. Sickness rates … increased … because the average sickness episode became more protracted’. In support of this statement, Riley cites two pieces of evidence: a survey by Samuel Hudson of the Ancient Order of Foresters in 1897 which ‘concluded that sickness time exceeded the expected amount by 16.5% because of heavy demands from members who were not dying’ (Riley 1997, p. 163) and the massive survey by Alfred Watson of the Independent Order of Oddfellows (IOOF) Manchester Unity between 1893 and 1897 (Watson 1903). Here we focus on the second of these, which was produced by a future Government Actuary, and the results of which are still used (with appropriate adjustments) today.¹⁶ Is Watson’s testimony really ‘unambiguous’?

Watson compared his results for 1893–1897 with those obtained in a survey of the IOOF by Henry Ratcliffe which covered the period 1866–1870. Riley (1997, p. 173) says that Ratcliffe and Watson’s tables ‘show very clearly that, among Oddfellows, the average duration of sickness episodes increased’.

However Watson’s and Ratcliffe’s investigations differed in how they treated spells of sickness already in progress at the start of the investigations. Watson asked for details of when these spells actually started, so that he could accurately assign them to the correct duration category. Ratcliffe assumed that all such spells started on the date which the investigation started. To see the difference this makes, consider a man who fell sick 24 months before the investigation started, and was sick for a period of 36 months, his spell ending 12 months after the investigation started. This spell contributes 12 person-months of sickness experience during the period of the investigation. Ratcliffe assigned these equally to the 0–6-month and 6–12-month duration categories, whereas Watson—armed with information as to when the spell really did begin—would correctly assign the 12-months to the 24–36-month duration category. The effect of this is that Ratcliffe underestimated the amount of sickness experience at longer durations compared with Watson.

Watson was aware of this difficulty. In his words:

[t]he returns prepared for the investigation of the experience of 1866–70 did not supply the dates when sickness attacks began and ended, and it is understood that all attacks which were current on 1st January 1866 were scheduled as having begun on that day, thus overstating the first-period sickness and correspondingly under-stating that falling within the after-periods (Watson 1903, pp. 38–9).

However, he then asserted that:

[w]hen … all due allowance has been made for this circumstance, there must still remain a great percentage of excess, and the conclusion seems to be irresistible that the serious increase of sickness previously noted is in great measure to be traced to the increase of permanent cases; and that these cases are not only more numerous at the older ages—where excess was perhaps anticipated—but that at every period of life protracted sickness now represents a much heavier liability than it did in the period 1866–70 (Watson 1903, p. 39).

Watson did not attempt to evaluate the potential impact of the different methods used by Ratcliffe and himself. Neither could he explain why sickness at longer durations had increased at all ages: ‘[n]o satisfactory explanation for this phenomenon can be suggested’ (Watson 1903, p. 39), although he did offer some tentative suggestions elsewhere (see Watson (1900) and Snow (1913)).

In the Appendix, we show that the different methods employed by the two investigations account for at least one third of the apparent increase in the durations reported by Watson, and may account for almost all of the increase. This explains why reported durations seem to have increased in the IOOF data at all ages, and not predominantly at older ages. The evidence from the two IOOF investigations of 1866–1870 and 1893–1897, therefore, does not necessarily imply a real increase in durations, but may be more closely associated with methodological differences.

In arguing that the increase in reported durations was much smaller than Riley or Watson supposed, we are not taking issue with the fact that the IOOF data reveal an increase in the prevalence of sickness. This being the case, then if spells of sickness only became protracted to a limited extent, there must have been more of them: in other words, the incidence of sickness must have risen.

4.6 Conclusion

In this paper, we have analysed the trend in morbidity in England between 1870 and 1949 using individual-level sickness data for several thousand members of a sickness insurance scheme in the southern county of Hampshire. Our conclusions may be summarised as follows.

First, age-standardised morbidity did rise between 1870 and 1890, and again towards the 1920s, but between 1870 and 1910, the magnitude of the increase was only just over half that observed by Riley (1997). Moreover, we find that the rise in morbidity was the result both of the increasing incidence of sickness and the increasing duration of sickness episodes. The view that the greater length of episodes of sickness led to the rise in reported morbidity derives, in part, from comparisons made between contemporary actuarial investigations that we have shown to be confounded by methodological differences. Riley explained his results as ‘a transition from frequent but brief episodes of sickness to less frequent but notably protracted episodes’ (Riley 1999b, p. 134). The HFS data show that only during the 1880s was a rise in morbidity being driven mainly by the increased duration of sickness episodes (Table 4.3). Before 1900 morbidity fluctuated. During the 1880s, it rose because the duration of sickness episodes increased but, during the 1890s, it fell (even though sickness incidence was rising) because the duration of episodes decreased markedly. Between 1900 and about 1920, there was a rise in morbidity among men aged 65 years and over because the duration of their sickness episodes increased. Among younger men, however, morbidity changed rather little. After the 1920s, a new phase dawned in which both morbidity and mortality declined, the decline in morbidity arising from a decrease in both the incidence and duration of periods of sickness.

Table 4.3 Summary of trends in age-standardised morbidity in the Hampshire Friendly Society data

Second, the trends we have reported based on the HFS data do seem to be measuring ‘objective morbidity’, in that our annual estimates of morbidity are associated more closely with independent measures of changes in the disease environment than they are with economic or social policy changes. Morbidity was not closely associated with the unemployment rate or real wages. The introduction of national insurance in 1911 seems to have had only a limited effect on the level of sickness benefit claimed. However, reported morbidity in the HFS data was associated with changes in the general health environment. To be sure, there are other elements of ‘cultural inflation’ (such as general attitudinal changes, increases in the number of doctors or diagnostic changes) which we have discussed elsewhere (e.g. Gorsky et al. 2011). Contemporaries also believed that the introduction of parallel or multiple insurance schemes would increase the propensity to claim (essentially, because it increased the benefit/wage ratio). It was also why they believed that the introduction of workmen’s compensation in 1897 led to an increase in sickness prevalence, and it was why some of them were hostile to the introduction of national insurance (Harris et al. 2011, pp. 648–9). However, secular historical changes in the relationship between individual and medical treatment or in cultural attitudes towards morbidity are not required to account for the morbidity trends we have observed.

Appendix: Analysis of the Apparent Increase in the Duration of Sickness Among the Independent Order of Oddfellows Between 1866–1870 and 1893–1897

Where does the idea that the increase in morbidity in the second half of the nineteenth century arose because of the increasing duration of episodes of sickness come from? In this Appendix, we focus on Watson’s report, since this is the weightiest piece of evidence.

Watson’s analysis (Watson 1903, pp. 38–9 and 143–59) was based on person-years of sickness. He classified the person-years according to duration since the episode of sickness began using the duration categories 0–6 months, 6–12 months, 12–24 months and over 24 months. He then compared the actual amount of sickness recorded in each of the duration categories with the amount which would have been expected on the basis of Henry Ratcliffe’s investigation of the Oddfellows’ sickness experience in 1866–1870 (Table 4.4). There are two key observations from this table.

• 1. The increase in morbidity between 1866–1870 and 1893–1897 is very largely a consequence of the increase in the amount of sickness experience recorded at durations over 24 months.

• 2. This increase occurs in all age categories.

Table 4.4 Watson’s results for standardised morbidity in 1893–1897 compared with 1866–1870

We need to explain both these observations.

4.1.1 Sickness Episodes in Progress at the Start of the Investigation

The increase in morbidity at longer durations between 1866–1870 and 1893–1897 was characteristic of all age groups: indeed, it was actually stronger among the younger members than among those aged over 65 years. This matters, because it means that whatever was causing it was affecting all age groups. Explanations such as a replacement of acute conditions by chronic degenerative conditions (Riley 1989, p. 172) are unlikely, as if they were the cause, we should expect the increase in morbidity at longer durations to be concentrated among older members. Riley (1989, p. 192) acknowledges that there was an increase in sickness at all ages and describes this as ‘unsettling’, presumably because it suggests that something other than the conventional epidemiological transition is at work. However, he does not suggest what this might be. Perhaps the same need to posit a cause which would affect all age groups stumped Watson?

It is possible to use Watson’s data to obtain some idea as to the proportion of the apparent increase in the duration of sickness between Ratcliffe’s investigation of 1866–1870 and Watson’s investigation of 1893–1897 which might have been due to the different methods employed by the two men.

Watson provides overall data concerning the amount of sickness observed in his investigation. This was, to the nearest person-year, 52,718 at durations 0–3 months, 12,436 at durations 3–6 months, 11,923 at durations 6–12 months, 12,660 at durations 12–24 months and 45,310 at durations over 24 months, making a total of 135,048 person-years (Watson 1903, p. 141). Consider spells of sickness of durations 0–3, 3–6, 6–12, 12–24 months and 24 months and over. Let the number of spells which last for 24 months or more be l₂₄, and the numbers lasting at least 12, 6 and 3 months be l₁₂, l₆ and l₃, respectively. Let the total number of spells be l₀. Let the person-years of experience in each of Watson’s duration categories be p_0-3, p_3-6, p_6-12, p_12-24 and p₂₄₊, respectively, and let the mean duration of the spells in each duration category be m_0-3, m_3-6, m_6-12, m_12-24 and m₂₄₊ years.

Using standard life table methods, we can show that the following relationships hold:

$$ {p}_{24+}=\left({m}_{24+}-2\right){l}_{24}, $$

$$ {p}_{12-24}={l}_{24}+\left({m}_{12-24}-1\right)\left({l}_{12}-{l}_{24}\right), $$

$$ {p}_{6-12}=0.5{l}_{12}+\left({m}_{6-12}-0.5\right)\left({l}_6-{l}_{12}\right), $$

$$ {p}_{3-6}=0.25{l}_6+\left({m}_{3-6}-0.25\right)\left({l}_3-{l}_6\right) $$

$$ {p}_{0-3}=0.25{l}_3+{m}_{0-3}\left({l}_0-{l}_3\right). $$

Thus, substituting the total number of person-years in each duration category calculated from Watson’s data, we have

$$ 45,310=\left({m}_{24+}-2\right){l}_{24}, $$

$$ 12,660={l}_{24}+\left({m}_{12-24}-1\right)\left({l}_{12}-{l}_{24}\right), $$

$$ 11,923=0.5{l}_{12}+\left({m}_{6-12}-0.5\right)\left({l}_6-{l}_{12}\right), $$

$$ 12,436=0.25{l}_6+\left({m}_{3-6}-0.25\right)\left({l}_3-{l}_6\right) $$

$$ 52,718=0.25{l}_3+{m}_{0-3}\left({l}_0-{l}_3\right). $$

This set of five equations with ten unknowns has many solutions, but there are restrictions on the values of some of the unknowns. We know that there are restrictions on the mean durations of spells in each category. Let us assume that m_0-3 = 0.125 m_3-6 = 0.375 m_6-12 = 0.75 and that m_12-24 =1.5 (i.e. that spells under 3 months long are, on average, 1.5 months long; those between 3 and 6 months long are, on average, 4.5 months long; that spells lasting between 6 and 12 months are, on average, 9 months long; and that spells lasting between 12 and 24 months are, on average, 18 months long). Then the five equations become

$$ 45,310=\left({m}_{24+}-2\right){l}_{24}, $$

(4.A1)

$$ 12,660=0.5\left({l}_{12}+{l}_{24}\right), $$

(4.A2)

$$ 11,923=0.25\left({l}_6+{l}_{12}\right), $$

(4.A3)

$$ 12,436=0.125\left({l}_3+{l}_6\right) $$

(4.A4)

$$ 52,718=0.125\left({l}_0+{l}_3\right). $$

(4.A5)

Since l₀ ≥ l₃ ≥ l₆ ≥ l₁₂ ≥ l₂₄ > 0, then eq. (4.A2) implies that l₂₄ ≤ 12, 660. Substituting this into eq. (4.A1) produces

${m}_{24+}\ge 2+\frac{45,310}{12,660}=5.58$, or that the average duration of spells longer than 24 months’ long is at least 5.58 years.

For simplicity, suppose it is 6 years. With m₂₄₊ = 6, we can solve eqs. (4.A1)–(4.A5) to give

$$ {\displaystyle \begin{array}{l}{l}_{24}=11,328,\\ {}{l}_{12}=13,992,\\ {}{l}_6=33,700,\\ {}{l}_3=65,788,\\ {}{l}_0=355,956.\end{array}} $$

With other values of m₂₄₊ we obtain different solutions (Table 4.5).

Table 4.5 Numbers of spells with durations greater than 0, 3, 6, 12 and 24 months according to mean length of spells over 24 months’ long

We now need to consider the impact of the difference between Watson’s and Ratcliffe’s treatment of the spells ongoing at the start of the period of investigation. A Lexis chart showing the sickness for a set of spells of more than 2 years’ duration illustrates the situation (Figure 4.7). Calendar time is on the horizontal axis, and duration of spell is on the vertical axis. Suppose these spells last m + 2 years and imagine that claims for these sickness spells are made at a rate which is constant over time. Then the person-years of sickness at durations over 2 years during the period of investigation are represented by the area of the rectangle ABDC. This is the person-years calculation used by Ratcliffe in his 1866–1870 investigation. However, the spells under way at the start of the investigation, which are represented by the vertical line AB, will have durations at the start ranging from just above 0 to just under m years distributed uniformly between 0 and m (because of the constant rate of claims). The person-years of sickness before the start of the investigation which they encompass is represented by the area of the triangle ABE. It is this additional sickness which Watson’s approach brings in.

Fig. 4.7

Lexis chart to illustrate difference between Watson and Ratcliffe’s treatment of spells under way at the start of the period of investigation

Note. This chart illustrates the case of m = 6 years

The ratio between Watson’s sickness prevalence and Ratcliffe’s sickness prevalence is equal to $\frac{0.5{m}^2+5m}{5m}=\frac{0.5m+5}{5}=\frac{m}{10}+1$. So, if the average length of spells of over 2 years duration is 6 years (the minimum that Watson’s own figures allow), then, relative to Ratcliffe, he has inflated the sickness in the 24 months and over duration segment by 1.4 times. If the average length is 10 years (by no means impossible given Watson’s data), the inflation factor is 1.8 times., and if the average length is 14 years, it will be 2.2 times. Note also that this inflation factor is the same for all age groups provided m is the same for all age groups.

There will also be some inflation in the shorter duration segments, but since m is much smaller in these, the extent of the inflation will be much less: indeed, it cannot be more than 5% in the 12–24-month category and 2.5% in the 0–6- and 6–12-month categories.

4.1.2 Watson’s 12-Month ‘Off’ Period

Watson also adopted a 12-month ‘off’ period when compiling his tables (Watson 1903, p. 15). This suggests that he treated a new sickness within 12 months of the previous one as a continuation of the previous one. According to Riley (1997, pp. 172–3), this was different from the treatment by Ratcliffe in earlier investigations. Riley points out, correctly, that this means that comparisons of the incidence of claims between Ratcliffe and Watson are therefore not possible (Watson will record a lower incidence than Ratcliffe). He fails to mention, however, that altering the definition of the ‘off’ period will also have an impact on the duration of claims, and confound the comparison of durations between the two surveys. Watson described this 12-month ‘off’ period as ‘moderately long’ (1903, p. 15). By comparison with shorter ‘off’ periods, it will tend to inflate the number of claims of long duration.

It therefore seems that the different treatment of spells in progress at the start of the investigation by Ratcliffe and Watson is likely to account for a substantial proportion of the apparent increase in morbidity at longer durations. Since Watson’s data show a rise of some 2.4–2.6 times (Table 4.4), then the changed methods account for a minimum of 30% (using the minimum possible duration of sickness episodes over 2 years long which Watson’s own data allow) and could account for close to 100% of the increase, especially if the rather extended ‘off’ period used by Watson is also factored into the calculations. Moreover, since the impact of this change in method is not necessarily age specific, the notable and ‘unsettling’ fact that the apparent increase was roughly the same for all age groups suggests that the changed methods might be the main reason.

4.1.3 Watson’s Treatment of Sickness Claims Spanning More Than One Calendar Year

According to Riley, Watson treated a claim spanning more than one calendar year as several separate episodes, the second and subsequent episodes starting on each 1 January. This will artificially inflate the number of episodes and, when comparing the incidence of claims between Ratcliffe’s and Watson’s surveys, will act in the opposite direction to Watson’s 12-month ‘off’ period.

It will, however, tend to change the distribution of claims by duration, as longer claims are more likely to cross the end of the calendar year and hence to be counted multiple times. Its effect is to increase the proportion of longer claims and, again, to make it look as if the mean duration of claims is rising faster than it actually is. Its effect, though, is likely to be fairly small. Assuming an exponential distribution of claim durations such that the mean claim duration is x years, then the impact on the longest duration claims involves multiplying the number of such claims by a factor which is less than or equal to (1 + x). So if x is, say, 0.2 years, it will involve inflating the number of long claims by no more than 20%.

4.1.4 Conclusion

Table 4.4 reports standardised morbidity ratios of between 238 and 264 in 1893–1897 for claims of over 24 months duration compared with 1866–1870. It seems possible, and may be more likely than not, that the majority of this increase is accounted for by the different methods used by Watson and Ratcliffe in their computations. There are three specific differences, and all will tend to mean that Watson inflates the proportion of claims of longer duration compared with Ratcliffe. It is possible, therefore, that the increase in the average duration of claims reported by these actuaries is entirely artefactual.¹⁷