Introduction

Economic growth can help to reduce material poverty and expand choices and capabilities in the developing world. It can also increase inequality and reduce social cohesion. If a major goal of economic and social policy is to maximise quality and length of life for the population served, and ensure that they are distributed equitably, it is surprising that subjective measures of quality of life (QoL) are not used more widely, or that psychometrically inferior ‘global’ questions predominate within social indicators research (see [8] for a review). Using subjective measures that are person-centred rather than health-focused would also enable the effect of health on people’s quality of life to be empirically established (for example, in this study it was the area mentioned most frequently by all respondents).

Greater availability of these instruments, especially if they included a higher degree of respondent participation, might provide empirical data that could validate causal theories of quality of life and its determinants. For example, the authors have recently proposed a general theory that both defines individual quality of life and explains its relationship to key causal determinants [32]. According to the general theory, indicators of MWB and indicators of health demonstrate a positive linear relationship with individual quality of life until a certain level is reached—known as the ‘basic capability threshold’. Beyond this, further marginal increases in levels of MWB and health give rise to rapidly diminishing marginal increases in quality of life. This is explained in the theory by the phenomenon of ‘cognitive homeostasis, which only begins to operate above the basic capability threshold. It acts to equilibrate a person’s expectation of what they would like to do and be at a stable point above what they are actually capable of doing and being, keeping quality of life stable. The theory can also be used to assess the construct validity of the GPGI as part of a process of qualitative validation which assesses the accuracy with which the GPGI has represented the respondent’s worldview.

Individualised approaches to QoL assessment are predicated on the belief that only the individual living a life is able to accurately judge its quality [27, 31]. These instruments allow individuals to select, rate and weight the relative importance of those aspects of their life they consider of greatest relevance to overall QoL. Two such instruments receiving considerable attention in the health field are the Patient Generated Index (PGI) of QoL [34], and the Schedule for the Evaluation of Individual Quality of Life (SEIQoL) [27]. Over the last 14 years the SEIQoL (and its shorter version, the SEIQoL-DW) and the PGI have been subjected to extensive testing across a range of clinical conditions in the UK, Australia and North America [28]. Psychometric properties of validity, reliability and responsiveness have been assessed for both instruments with variable results, although in general the findings have supported their use in clinical evaluations of health care interventions [1, 10, 27, 34]. Neither instrument is appropriate for QoL assessment in a developing country and measures are mainly used in research, or occasionally planning and evaluating development interventions. The PGI is health focused, the weighting method used in the SEIQoL is complex and time consuming for routine field use, and field workers felt that the visual aid used in the SEIQoL-DW would be difficult for older respondents to understand and manipulate. It is theoretically possible to blend the successful elements of each approach however—the weighting method employed in the third stage of the PGI (see Fig. 1), and the global nature of the QoL assessment in the first stage of the SEIQoL—to produce a new instrument. We have called this pilot instrument the Global Person Generated Index, referred to in this paper as the ‘GPGI’ to differentiate it from the original PGI.

Fig. 1
figure 1

The Global Person Generated Index

Full size image

The Wellbeing in Developing Countries ESRC Research Group (WeD) is exploring the social and cultural construction of wellbeing in four developing countries (see http://www.welldev.org.uk for further information), which provided an opportunity to develop and validate the GPGI in a sizeable sample of respondents across three WeD countries (Ethiopia, Bangladesh, and Thailand) at differing stages of economic development Footnote 1. The respondents were a representative selection of men and women of different ages, socio-economic statuses, and religions from urban, rural, and peri-urban areas. This paper describes a preliminary validation of the new instrument, using a mix of qualitative and quantitative research methods.

Methods

Development of the pilot instrument

WeD fieldworkers in each of the three countries were given a free hand to interpret and translate a culturally relevant version of the GPGI. Forward and back translation methods were used and the translations were also discussed with experts in the local language and fieldworkers prior to piloting. The thoroughness of this process (‘double translation’ with independent verification) was designed to ensure that the measure performed equivalently in the three countries [36]. The process of the pre-testing and piloting, which should ideally be as thorough as that required for the original measure, is described below.

A purposive sample of 36 (17 female, 19 male) respondents was identified by local fieldworkers in seven towns and villages. In Bangladesh, an alternative wording for step two was introduced using satisfaction (shontushtho) and dissatisfaction (oshontushtho) rather than ‘good’ (bhalo) and ‘bad’ (kharap) as these were too vague in Bangla. In Ethiopia it was difficult to find a word equating with ‘important’ so the word ‘wesagn’, literally meaning crucial or needed, was used. It was also decided to give respondents ten ten-cent pieces and to ask them to place the coins in the boxes on the questionnaire when ‘spending’ points in step three of the GPGI (see Fig. 1). In Thailand, three methods were tried for step 2: moons (pie-charts, originally depicted as full–empty, later light–dark), smiley faces (happy–sad) and numbers (labelled from good to bad). It was generally agreed that any method could be used, even just numbers, if it was sufficiently well explained. In Thailand three methods were also tried for step 3: spending Baht, placing stones and making tallies. The coins were the most successful as they could be explained in terms of everyday activities such as ‘making merit’ (donating to a Buddhist temple). Following the pre-pilot analysis, a final questionnaire wording was agreed for the pilot instrument in each country. All countries decided to reduce the original 10 point scale used in step two to a seven point (i.e. a 0–6) scale, and to use the method of spending 10 coins to assign importance weights in step three. A qualitative content analysis was undertaken to explore the elicited life areas, and the validity of rating and weighting methods. A GPGI index score was generated for all respondents. Although a range of life areas were generated, with similarities and differences observed between countries, ‘development’ focused areas were frequently mentioned in all field sites.

Sampling and data collection

The GPGI was administered as part of WeD’s exploratory research into Subjective Quality of Life, which involved semi-structured interviews (SSIs), focus group discussions, and the piloting of other measures, such as the Satisfaction with Life Scale in Ethiopia [14, 30]. It was carried out by local researchers, the majority of whom had spent at least 1 year attached to the site, which enabled them to build a good rapport with the inhabitants. The researchers received full training in the methods used, and the majority had also participated in the GPGI piloting. As the GPGI was interview-administered, the response rate was 100%, with the exception of Bangladesh where four female respondents (two of whom were illiterate) chose only to respond to the SSI. Their results have not been included in our analysis. About 36 GPGIs were administered in the pilot and a further 242 during the main fieldwork (120 in Ethiopia, 102 in Thailand, and 22 in Bangladesh).

In Ethiopia respondents were purposively selected from four rural and two urban sites in Oromiya, Amhara, and Addis Ababa regions to represent: young (< 24 years and 24–30 years), middle aged (30–50 years) and old (50+ years) men and women; and orthodox and protestant Christians and Muslims. Researchers in Ethiopia did not select on the basis of wealth. In Thailand respondents were selected from a mixture of five rural and peri-urban sites in the South and Northeast of the country to represent young (18–30 years), middle aged (31–59 years) and old (60+ years) men and women. In the South respondents were also selected for religion (Buddhist and Muslim) and by wealth (rich and poor), based on participatory wealth rankings from community profiles Footnote 2. In Bangladesh, four rural and two urban sites were selected in Manikgani and Dinaspur, although the GPGI was only administered in the urban site. Consequently, Bangladesh was the smallest of the country samples and had an unequal balance of men to women (1:4) since in Bangladesh women are rarely seen in public spaces in urban areas and so are less easy to access. Respondents were purposively sampled to obtain a representation of younger (< 40years) and older (40+years), Muslim and Hindu, and rich and poor (selected using participatory wealth rankings). However, more than half of the sample had further or higher education, which is obviously not typical, and only 9% were over 60, compared to 24% in the whole sample. Further discussion with the field staff revealed that many of these respondents were wealthy middle-aged businessmen, interviewed in a teashop in the business district of the city.

The SSIs used local versions of the following questions: How is your life at the moment? What is making you happy/sad? When were you happiest/unhappiest? What are your worries/hopes for the future? What are the characteristics of a man/woman who lives well/badly? Respondents were administered the SSI, followed by the GPGI (see Fig. 1 for an example of a back-translated GPGI from Ethiopia). In Ethiopia, the SWLS was administered after the SSI and GPGI. The order of administration may have generated a more reflective response to the GPGI and SWLS, which would be expected to relate closely to the values, norms, and aspirations reported in the SSI. The GPGI was administered without a list of potential areas, to avoid introducing a source of bias if areas were not equally salient across countries and locations. In order to derive numerical scores from the qualitative data contained within the SSIs, one of us (LC) undertook a qualitative content analysis of all SSIs from Bangladesh and Thailand. For each respondent, the researcher assigned an SSI derived score between 0 and 6 for each of three life areas—material well-being (MWB), health, and family or children—where 0 represents ‘the worst you can imagine for the respondent’ and 6 represents ‘exactly as the respondent would like to be’. Making accurate ratings was challenging, however, the data on the characteristics of a man/woman who lives well/badly and people’s hopes and fears were used to gain a sense of what the best and worst imaginable outcomes would be for the respondents. Both raters had read ‘community profiles’ of the sites and transcripts of the focus groups, which were conducted as part of this study (see [21] for the full methodology). Additionally, LC had organised and participated in the fieldwork in Thailand, and in the training of the researchers in Ethiopia and Bangladesh (on a previous occasion she had visited the Ethiopian sites and conducted informal interviews with respondents via a translator). DR undertook the same analysis for the Ethiopian sample. Both researchers conducted the SSI ratings before looking at the PGI results (with the obvious limitation that LC had participated in their collection). Inter-observer agreement between the two researchers, using the Kappa statistic, was first confirmed with a sample of 20 respondents, independently assessed by both researchers. The three life areas were chosen because they were the most frequently mentioned areas in the GPGI (see content validity below).

Quantitative validation

Content validity

Content validity, that is the extent to which a measure assesses content relevant to the underlying construct, was assessed in two ways. First, a frequency analysis was undertaken of the areas mentioned in respondents’ GPGI questionnaires in step 1. This analysis was undertaken separately for each of the three countries. Three commonly mentioned areas emerged: (1) Indicators of material well-being (MWB) such as income, crops, property, and debts; (2) Health, including health of the family; and (3) Family or Children, including children’s education. Therefore a second method of assessing content validity involved correlating, for each of these three areas, respondents’ ratings on a 0–6 scale as mentioned in step 2 of the GPGI, with scores derived from the SSIs. Non-parametric correlations were used where SSI derived scores were non-normally distributed.

Criterion validity

Assessing criterion validity (i.e. the extent to which a new measure correlates with established measures of the concept under study) is problematic in the absence of a gold standard measure of individual quality of life in developing countries. However in Ethiopia it was possible to correlate GPGI scores with scores on the five items of the Satisfaction with Life Scale (SWLS), an established measure of a closely related concept. Non-parametric correlations were used where SWLS sub-scale scores were non-normally distributed.

Construct validity

Construct validity has been defined as ‘the extent to which a new measure is related to specified variables in accordance with an established theory or hypothetical construct’ [39]. In this study we assessed construct validity by testing the extent to which GPGI scores were related to a number of specified variables in accordance with the general theory described in the introduction [33]. We tested the following: (1) GPGI scores should show a positive linear or curvi-linear relationship with MWB and health scores derived from SSIs; (2) Poor respondents in Bangladesh and Southern Thailand should have lower quality of life scores than rich respondents; and (3) Rural respondents would have a different quality of life from urban respondents. For completeness we also explored the relationship between GPGI scores and country, age, sex and educational attainment. Analysis of variance and t-tests were used to test for differences between groups. SPSS stepwise regression was used with specified independent variables (‘enter’ method) to model the relationship between the GPGI and all significantly correlated variables identified above. We tested each variable individually and those found not to be significant were omitted from subsequent analysis. All significant variables were entered into the regression model and an adjusted R2 was calculated. Of 40 respondents explicitly mentioning religion as an important area in their GPGI, 34 were from Ethiopia. Therefore we were able to test the hypothesis that Ethiopians who mentioned religion in their GPGI had higher quality of life scores than Ethiopians not mentioning religion in their GPGI.

Qualitative validation

Qualitative validation is a new technique, but one that is particularly appropriate for a measure like the GPGI which spans the border between qualitative and quantitative methodologies. Qualitative methods like semi-structured interviews, focus groups, and even ethnography [43] are commonly used to generate item content. More recently, cognitive debriefing has been used to understand more about how people respond to measures [2], both with established measures (e.g. [24]), and as part of pre-testing [6]. However, it is still unusual to see qualitative interviews used in a way that focuses on the respondent rather than the measure, by aiming to assess the accuracy with which a measure has represented their worldview Footnote 3. Two examples of this approach are Paterson and Britten’s use of open-ended SSIs to assess of the validity of three health status measures, two of which were individualised [29]; and Tully and Cantril’s assessment of the validity of the Patient Generated Index, which used a combined qualitative and quantitative approach [41]. The methodology used here differs from both of these studies in that the GPGI and the SSI were administered on the same occasion, which had the advantage of controlling for any subsequent changes, or differences in the way people respond to a postal survey and an interview. However, all methodologies involved interviews that were “qualitative in nature, non-directive and conversational” [41] to ensure that respondents felt able to discuss all aspects of importance to their lives.

The qualitative validation focused on content validity, aiming to establish whether (a) the area nominated in the GPGI appeared as an area of importance in the SSI; (b) whether the way the area was discussed in the SSI suggested that the appropriate number of points had been allocated to indicate its importance; (c) whether the respondent’s satisfaction with this area appeared to be adequately represented by their GPGI score; (d) whether there were areas in the SSI that appeared to be even more important to the respondent, but were not nominated in the GPGI; and (e) whether the overall picture given by the SSI of the extent to which the respondent’s reality met their expectations corresponded with the total score given in the GPGI. While Tully and Cantril were only able to assess (a) to (d), the inclusion of open-ended questions about people’s expectations (e.g. their hopes and fears, what they thought was a good or bad life for a person like them), enabled us to attempt an impressionistic assessment of the size of the gap. Any problems with scoring or spending points were also noted (e.g. where the number of points spent failed to add up to ten Footnote 4).

Results

Quantitative validation

Content validity

First stage In categorising the areas mentioned in respondents’ GPGI step 1, conceptually similar areas were grouped together under single category headings to facilitate the frequency analysis and interpretation (for example, Money, assets where a respondent specifically mentioned money, assets, wealth, or riches). Little grouping was required, which could suggest that some researchers summarised what the respondents said, rather than recording it in their own words. However, LC’s observation of the fieldwork in Thailand, and attendance of a ‘debriefing’ workshop for fieldworkers in Ethiopia confirms that people understood the task and responded in summary form (which would make their responses difficult to interpret without an accompanying interview). There are some idiosyncratic responses, for example, “fear of leaving”, which were not incorporated in another category, although the corresponding interview suggests that this relate to fear of abandonment by children.

Area Frequencies (see table 1 in the appendices)

The most mentioned area overall was Health (partly due to its popularity in Ethiopia, which reflects the importance of physical fitness to a population predominantly engaged in agriculture), and the second was Money, assets. Children was the third most mentioned area, and featured in every country’s ‘top five’, with Bangladesh also prioritising Children’s education/ future. Children, marriage/ spouse, family, and children’s education were the top four priorities in Bangladesh, reflecting the intrinsic and instrumental value of family in Bangladesh [9]. The fourth was Home, which was very important in Thailand (11.5% of respondents), and fairly important in Ethiopia (5.5%). Fifth and sixth most mentioned areas were Employmentand Family. Both of these areas were important in Thailand (to 9.8% and 9.3% of respondents respectively), withEmployment also important in Ethiopia (3rd priority, 6.9% of respondents) and Family in Bangladesh (2nd priority, 7.4% of respondents). Peace was only mentioned in Ethiopia (3.6% of respondents), which may relate to the high value given to Religion in Ethiopia (84% item frequency), which almost equalled that of Employment (6.8% of respondents vs. 6.9% for Employment). Debt and Land were only mentioned in Thailand; the former may reflect high levels of rural debt and growing consumerism, and the latter the insecurity of land tenure in two of the sites.

Table 1 Content Validity 1: Percentage of sample from each country that nominated the following areas (by area and by country)
Full size table

Second stage In the second method of content analysis, in which respondents’ ratings in step 2 of the GPGI were correlated with scores derived from the semi structured interviews, a slightly different categorisation was undertaken. Anything that related to money or work was re-categorised as material well-being (MWB), anything relating to health as ‘health’, and anything relating to children as ‘family/children’ Footnote 5. Table 2 shows the Spearman correlation coefficients. MWB as indicated in the SSI shows a moderate but statistically significant correlation (0.286) with MWB as indicated in step 2 of the GPGI. Interestingly the MWB score from the SSI shows a higher correlation with family/children (0.395) as indicated in the GPGI. The health score as indicated on the SSI shows a strong and statistically significant correlation (0.584) with health as measured by the GPGI. The family/children score on the SSI shows a moderate and significant correlation (0.361) with family/children as indicated in the GPGI. As with the SSI MWB score, the family/children score from the SSI shows a moderate but significant correlation (0.232) with the MWB score as indicated on the GPGI.

Table 2 Content validity 2: Spearman correlation of satisfaction with material well-being (MWB), health, and family/children scores (0–6) as indicated in semi-structured interviews (SSI), with scores (0–6) as indicated on step 2 of the GPGI
Full size table

Criterion validity

Table 3 shows the correlation between the GPGI QoL score and scores on the five items of the Satisfaction with Life Scale (SWLS) amongst respondents from Ethiopia. The comparison is valid as they tap a comparable underlying concept [3] and are both scored 0–6, although the GPGI satisfaction score is subsequently weighted by importance (scored 1–10) and converted into a percentage score to aid interpretation. The GPGI shows weak to moderate but statistically significant correlations with four items ranging from 0.202 to 0.351. No correlation is observed between the GPGI and the item ‘I would change nothing in my life’. The SWLS item that is conceptually closest to the GPGI, ‘My life is close to my ideal’, shows the strongest correlation.

Table 3 Criterion validity: Spearman correlation between the GPGI and the five items of the Satisfaction With Life Scale (SWLS) in Ethiopia
Full size table

Construct validity

The relationship between GPGI Index percentage scores and material wellbeing (MWB) as indicated in respondents’ semi structured interviews shows significant linearity (correlation coefficient 0.248, P < 0.001), and there is also a suggestion that GPGI Index scores may level off above MWB scores of 4 (on a 0–6 scale). A similar relationship is seen between GPGI Index percentage scores and health as indicated in respondents’ semi structured interviews (correlation coefficient 0.284, P < 0.01), although there is less evidence of a threshold effect as health increases.

Table 4 shows the results of further construct validity tests of the relationship between GPGI Index scores and other respondent characteristics. A 10% difference in GPGI scores was observed between rural and urban respondents (55.9 vs. 65.1%, P < 0.05), and a similar difference between respondents classified as poor and rich in Bangladesh and Southern Thailand (55.7 vs. 65.4%, P < 0.01). Although men reported a slightly higher mean GPGI score than women (60.2 vs. 56.7%) this was not statistically significant. Slight differences in mean GPGI scores were observed between different age groups but no significant or consistent trend emerged, even when GPGI scores were correlated with year of age as a continuous variable. No significant trend was observed in GPGI Index scores with increasing level of educational attainment. However, a significant difference in GPGI Index scores was seen between those respondents who had completed further or higher education and everyone else (67.8 vs, 58%, P < 0.05). Only MWB as indicated in respondents’ semi structured interview, and country remained in the model. Together these two variables were able to explain over 21% of the variation in respondents’ GPGI scores. Finally, the mean GPGI score for 34 Ethiopians who nominated religion as an important life area in their GPGI was compared with the mean score for the remaining 82 Ethiopian respondents. Those mentioning religion had a slightly higher but non-significant mean GPGI score (61.9 vs. 59.2%).

Table 4 Construct validity: GPGI QoL Scores by age, sex, educational attainment, country, rural/urban, rich/poor
Full size table

Qualitative validation

In the Ethiopian sample, a content analysis of 21 responses to the three steps of the GPGI, and comparison with the content of semi structured interviews (SSI), revealed a close correspondence between the two sources for eight respondents (four female and four male, aged from 19 to 76 years). In each case, the life areas nominated in the GPGI were mentioned several times in the corresponding SSI and few if any additional areas were emphasised within the SSI that did not appear in the GPGI. There appeared to be a good match between the extent to which respondents felt their reality met their expectations for each GPGI nominated area, as measured in step 2 of the GPGI, compared with the content of the SSI. The relative importance attached to each GPGI nominated life area, as indicated in the points spent in step 3 of the GPGI, was also consistent with the interview content. A further 16 responses were content analysed for Thailand, and another 16 for Bangladesh, sampled according to gender, type of site, region (Thailand), and age (Bangladesh). In Thailand only five responses showed close correspondence, although the picture was slightly better in Bangladesh where nine respondents showed close correspondence.

The majority of the Ethiopian sample (13 of 21 respondents) showed inconsistencies between the content of the GPGI and SSI due to (i) lack of comprehension on the part of respondents or interviewers; (ii) minor inconsistencies between one GPGI nominated area and the GPGI; and (iii) major inconsistencies where discrepancies were identified between two or more life areas, or where the SSI raised questions about the validity of a respondent’s overall quality of life index score as measured by the GPGI. The majority of respondents in the sample from Thailand also had inconsistencies (11 of 16 respondents), six of which were major. However, respondents with inconsistencies were a minority in the Bangladeshi sample (7 of 16 respondents) albeit that five of these inconsistencies were major. This is probably due to the size and composition of the Bangladeshi sample, as discussed previously.

As we can see from this brief review, similar findings emerged in the three samples, and for this reason we have chosen to focus on the sample from Ethiopia as this is the largest, and the most balanced (containing rural, peri-urban, and urban respondents), and should therefore provide the best examples.

Basic errors in completion and minor inconsistencies between GPGI and SSI

The content analysis of GPGI assessments revealed errors in completion apparently arising from a basic lack of comprehension for four respondents. Three types of error were observed. In two cases the total number of points spent to indicate relative importance in step 3 did not total 10 points, but did not appear to have resulted from simple arithmetic error (where such simple errors occurred— e.g. where points totalled 100 or 12—valid calculation of a GPGI index score was still possible).

The content comparison of four respondents’ interviews and GPGI assessments revealed minor inconsistencies, usually in relation to only one GPGI nominated life area. For example a 23 year old male labourer, reporting a high overall quality of life as measured by the GPGI (85%), nominated four important life areas in his GPGI. ‘Peace’ was nominated in his GPGI, and assigned the second highest relative weight (0.25). Closer examination of his SSI revealed that although peace was explicitly mentioned only once, one of the most significant life events mentioned was the loss of his best friend during the war. Yet he then nominated ‘social life’ in his GPGI, but made no mention of this in his SSI. Conversely he chose not to nominate his relationship with his girlfriend in the GPGI, yet mentioned her several times throughout his interview.

Major Inconsistencies between GPGI and SSI

Analysis of GPGI responses and corresponding SSI content revealed major inconsistencies for five Ethiopian respondents (all male). Discrepancies were defined as major when a lack of concordance was demonstrated in two or more life areas, or where a respondent’s overall GPGI index score bore little relation to the overall content of the corresponding SSI. One example of this is the response from a 73 year old male, a married farmer and orthodox Christian. The respondent nominated wealth, poverty, and assets in his GPGI, and mentioned these over 20 times throughout an extensive SSI that lasted 1 hr and 45 min. The importance of material wellbeing in his life and the impact of poverty on his overall quality of life were exemplified in statements such as:

“It is good to die rather than to live in poverty. I am very poor, and so I do not have money. I am very poor, and so I do not have money”

Although he scored his current state of wealth at 2 out of 6, which reflects his dissatisfaction, the area carried a relative weight of 0.3. While this was the highest weight assigned to an area, it seems low in the light of his SSI comments. The respondent also nominated ‘health’ as an important life area in his GPGI, rating it 6 out of 6, with a relative importance weight of 0.1. This assessment does not seem to correspond to the SSI, where he described himself as ‘getting physically weak and old’. In conclusion, this respondent’s self-reported quality of life using the GPGI (63%) does not appear to constitute a valid self assessment when compared to the picture that emerged from the SSI. An example from another respondent is given in the appendix.

Discussion

We have taken individualised quality of life measuring instruments originally developed and validated in the health field, and produced a global person generated index (GPGI) measure of quality of life in a development context (see [5, 33]). Our pilot instrument was successfully translated into four languages (Bangla, Thai, Amharic, and Oromiffa), and we demonstrated the feasibility of administering the instrument by interview to respondents of all ages from a variety of social and cultural backgrounds. Some residual problems of comprehension remained however, even after pre-piloting and training, which emphasise the challenge in communicating complex concepts to respondents, and asking them to engage in a sophisticated and often taxing cognitive process. Our findings suggest that the understanding and skill required of the interviewer needs to be developed over a period of time, and slightly more structured training is required in future.

When subjected to quantitative validation using traditional psychometric methods, the GPGI seemed to show the ability to capture the important areas and aspects of life one would expect to impact on quality of life in developing countries. Further statistical analysis of GPGI content in those respondents nominating aspects of material wellbeing and children showed only moderate correlation between the GPGI ratings and ratings derived from corresponding SSIs for the same areas. It is possible that the need to amalgamate several disparate aspects of material wellbeing nominated in the GPGI to form a single category and the reliance on the researchers’ judgement when assigning a score to the content of SSIs introduced a degree of ‘noise’ or observer variation into the analysis not actually attributable to the GPGI. A stronger correlation was observed between health scores derived from the GPGI and those derived from the SSI, which may be because the more homogenous health category was less prone to observer variation, or that making a subjective assessment of one’s health state is less cognitively challenging than assessing one’s material wellbeing across a number of possibly inconsistent dimensions.

When quality of life scores as measured by the GPGI Index were correlated with scores from the Satisfaction with Life Scale items, weak to moderate coefficients were obtained. The highest coefficient was found between the GPGI and the most conceptually similar item (‘my life is close to my ideal’). However, for practitioners and researchers in international development, the ‘unique selling point’ of the GPGI is not the final score, which is comparable to the SWLS, but the insight into what people value, and how they feel they are performing in these valued domains. The process additionally offers an opportunity for the respondent to reflect, and enables rapport to develop between administrator and respondent, in a similar manner to participatory learning approaches Footnote 6.

The results of the construct validation provided further evidence for the validity of the GPGI. Statistically significant differences were seen in GPGI QoL scores between rich and poor, and between urban and rural respondents. Health and material wellbeing scores, derived from the SSIs, also showed a linear relationship, with a suggestion of curvilinearity at the higher levels of health and MWB, as predicted by the model. However the relationship was not strong and considerable variation was observed. Future empirical studies of construct validity may need to track GPGI QoL scores in individuals over time in selected groups likely to experience significant change in health or MWB over the period of study (for example, participants in a credit and savings scheme). The results of the construct validation are summarised in the regression model, which shows that only two variables, MWB and country, were able to explain 21% of the observed variation in GPGI QoL scores. If as suspected, the influence of country is explained by the inclusion of wealthy Bangladeshi businessmen, then this finding would appear to demonstrate the overall impact of MWB on GPGI scores, confirming its effect on QoL.

Perhaps the most revealing insights into the validity of the GPGI come from the qualitative analysis, although interpretation is not easy, and arguably presents greater challenges than traditional psychometric analysis. Our qualitative findings raise some questions about the level of validity of the GPGI in the majority of respondents, as there were discrepancies between the content of the GPGI and the content of the SSIs. However, in the majority of cases the discrepancies were minor: discrepancies in GPGI scores, or relative weights in one area, or basic errors of comprehension. More serious discrepancies occurred when a respondent’s GPGI rating or relative weighting of several life areas did not match the picture revealed in the same respondent’s SSI. Within the limitations of the current study, it is difficult to establish if these observed discrepancies are an indication of limitations in the GPGI, or in the method of qualitative analysis employed. For example take the first case cited earlier of the man who described the impact of poverty on his life in his SSI. It was noted that in his GPGI he nominated ‘wealth’ as an important life area but gave it a relative weight of only 0.3. This was interpreted as a discrepancy between the GPGI and the SSI. However the same respondent also nominated ‘labour/work’ in his GPGI and gave this area a relative weight of 0.2. It could be hypothesised that these two GPGI areas combined capture the impact of poverty on QoL that is described in the SSI, and that therefore their relative weights should also be combined when comparing GPGI and SSI content. In this case the ‘apparent’ discrepancy disappears. In Bangladesh the discrepancies between the GPGI and the SSI mostly related to areas that were highly abstract, or personal, and thus difficult to capture in a few words, e.g. “own boredom and lack of fulfilment”. However, in Thailand they almost all relate to debt, which other studies report as a major preoccupation (e.g. [21]. This suggests that people will talk about different things in the more relaxed context of a semi-structured interview, not merely topics that are abstract, or idiosyncratic (i.e. important to them, but not ‘important’), but also ones that are potentially shameful.

To achieve greater improvements in validity it may be necessary to undertake some form of routine interview, similar to the SSI used in the study, prior to administration of the GPGI, in order to promote ‘deep’ cognition Footnote 7. This approach to questionnaire administration is supported by recent research in cognitive processing [40]. There is, however, an inevitable trade off between improved validity and reduced applicability as the length of time required to administer the GPGI is increased; the context and aims of QoL measurement, and the resources available will determine how that trade off is made.

Further work is also required to confirm the reliability of the GPGI, and to establish its responsiveness to changes in QoL over time. The successful blend of qualitative and quantitative methods employed here holds out the promise of newer and more innovative methods for the validation of individualised measures such as the GPGI. For example respondents might be asked to interpret their GPGI index score and comment on its validity. More sophisticated quantitative methods such as structural equation modelling [39], based on large population samples, followed up and repeatedly assessed over longer time periods, with more objective assessment of material wellbeing and other influencing factors, could be used to complement qualitative validation techniques.

In summary, we believe our study provides preliminary evidence, using a mix of quantitative and qualitative methods, to suggest that a GPGI approach to QoL assessment in developing countries is feasible. We would go further and conclude that the instrument is able to identify those areas of an individual’s life that make the greatest impact on QoL, and that the resulting index measure of QoL has the potential to evaluate and guide development, economic and social policy in the developing world.