Abstract
This chapter starts with an introduction to the incomparability problem and an overview of the diversity in survey items. This is followed by a description of the problem of incomparability of the time series on life satisfaction in the USA, Japan and The Netherlands to illustrate the problem. Next two conventional methods for scale transformation are described: the Linear Stretch Method and the Semantic Judgment of Fixed Word Value Method. We explain why they fall short to solve the comparability problem and conclude that these shortcomings require further investigations and innovative solutions to solve them.
Access this chapter
Tax calculation will be finalised at checkout
Purchases are for personal use only
Notes
- 1.
We use the term ‘item’ for a survey question and its corresponding response options.
- 2.
We use the term ‘anchor points’ for the response options at both ends of a discrete scale. In the case of a continuous distribution, we use the term ‘extremes’ to refer to the boundaries of the continuum that bounds this distribution.
- 3.
http://www.gallup.com/products/170987/gallup-analytics.aspx. Assessed 3 February 2016.
- 4.
The Weighted Average Approach is a generalization of the Rank Method, but does not require that the numbers assigned to the response options are equal to the ranks of these response options to calculate a sample mean.
- 5.
Ridit stands for Relative to an Identified Distribution Integral Transformation.
- 6.
A VAS is a measurement instrument used to ask respondents to specify their level of agreement with a statement on a subjective topic by indicating a position along a continuous line between two end-points.
- 7.
A network of international multidisciplinary researchers committed to the measurement of health-related quality of life, www.euroqol.org.
References
Andrews, F. M., & Withey, S. B. (1976). Social indicator of well-being, Americans’ perceptions of life quality. New York: Plenum Press.
Bălţătescu, S. (2002). Problems of transforming scales of life satisfaction. Euromodule workshop, Berlin. Retrieved January 29, 2014, http://worlddatabaseofhappiness.eur.nl-hap_bib-freetexts-baltatescu_sm_2002A.pdf
Batz, C., Parrigon, S. & Tay, L. (2015). The impact of scale transformations on National subjective well-being scores. Social Indicators Research, Published online: 04 Sep 2015. doi: 10.1007/s11205-015-1088-1
Bjørnskov, C. (2010). How comparable are the gallup world poll life satisfaction data? Journal of Happiness Studies, 11, 41–60. doi:10.1007/s10902-008-9121-6.
Bross, I. D. J. (1958). How to use ridit analysis. Biometrics, 14, 18–38.
Campbell, A., Converse, P. E., & Rodgers, W. L. (1976). The quality of American life. Perceptions, evaluations, and satisfactions. New York: Russel Sage Foundation. Available at: http://babel.hathitrust.org/cgi/pt?id=mdp.39015071884634;view=2up;seq=6.
Cummins, R. A. (1997). The comprehensive quality of life scale – intellectual-cognitive disability (ComQol-I5) (5th ed.). Melbourne: School of Psychology, Deakin University.
Cummins, R. A. & Gullone, E. (2000). Why we should not use 5-point Likert scales: The case for subjective quality of life measurement. Proceedings, second international conference on quality of life in cities (pp. 74–93). Singapore: National University of Singapore
Cummins, R. A. (2003). Normative life satisfaction: Measurement issues and homeostatic model. Social Indicators Research, 64, 225–240.
DeJonge, T. (2009). The state of play in measuring SWB in the Netherlands. Paper for the OECD meeting Measuring subjective well-being: An opportunity for National Statistical Offices?, Florence, Italy. Retrieved January 29, 2016, Available at: http://worlddatabaseofhappiness.eur.nl/hap_bib/freetexts/dejonge_t_2009a.pdf
DeJonge, T., Veenhoven, R., & Arends, L. R. (2015). ‘Very Happy’ is not always equally happy on the meaning of verbal response options in survey questions. Journal of Happiness Studies, 16(1), 77–101. doi:10.1007-s10902-013-9497-9.
Diener, E., & Diener, C. (1996). Most people are happy. Psychological Science, 7, 181–185.
Donaldson, G. W. (1998). Ridit scores for analysis and interpretation of ordinal pain data. European Journal of Pain, 2, 221–227.
Frijters, P., Johnston, D. W., & Shields, M. A. (2008). Happiness dynamics with quarterly life event data (IZA Discussion Paper No. 3604).
Gallup (2016). Understanding how gallup uses the Cantril scale. Retrieved January 29, 2016, http://www.gallup.com/poll/122453/understanding-gallup-uses-cantril-scale.aspx
Guven, C., Senik, C., & Stichnoth, H. (2011). You can’t be happier than your wife. Happiness gaps and divorce (Paris School of Economics, Working Paper No. 2011–01, halshs-00555427).
Hull, C. L. (1922). The conversion of test scores into series which shall have any assigned mean and degree of dispersion. Journal of Applied Psychology, 6(3), 298–300.
Jones, L. V., & Thurstone, L. L. (1955). The psychophysics of semantics. An experimental investigation. The Journal of Applied Psychology, 39(1), 31–36.
Kalmijn, W. M. (2010). Quantification of happiness inequality. Dissertation, Erasmus University Rotterdam, The Netherlands. Enschede: Ipskamp Drukkers. Available at http://repub.eur.nl/pub/21777
Kalmijn, W. M., Arends, L. R., & Veenhoven, R. (2011). Happiness scale interval study, methodological considerations. Social Indicators Research, 102(3), 497–515. doi:10.1007/s11205-010-9688-2.
Kapteyn, A., Smith, J. P., & van Soest, A. (2007). Vignettes and self-reports of work disability in the United States and the Netherlands. American Economic Review, 97(1), 461–473.
Lee, H., Kim, K. D., & Shin, D. C. (1982). Perceptions of quality of life in an industrializing country: The case of the Republic of Korea. Social Indicators Research, 10, 297–317. doi:10.1007/BF00301097.
Lim, H. E. (2008). The use of different happiness rating scales: Bias and comparison problem? Social Indicators Research, 87, 259–267. doi:10.1007/s11205-007-9171-x.
Lodge, M. (1981). Magnitude scaling: Quantitative measurement of opinions. Beverly Hills: Sage Publications, Series Quantitative applications in the social sciences, ISSN 0149-192X; 07-025. doi: 10.4135/9781412984874
Mazaheri, M., & Theuns, P. (2009). Effects of varying response formats on self-ratings of life-satisfaction. Social Indicators Research, 90, 381–395. doi:10.1007/s11205-008-9263-2.
McDowell, I. (2006). Measuring health. A guide to rating scales and questionnaires (3rd ed.). Oxford: Oxford University Press. http://a4ebm.org/sites/default/files/Measuring%20Health.pdf.
Pudney, S. (2010). An experimental analysis of the impact of survey design on measures and models of subjective wellbeing (ISER Working Paper Series, 2010–20). Economic & Social Research Council. Retrieved February 3, 2016, www.iser.essex.ac.uk/research/publications/working-papers/iser/2010-20.pdf
Sangster, R. L., Willits, F. K., Saltiel, J., Lorenz, F. O., & Rockwood, T. H. (2001). The effect of numerical labels on response scales. Article presented at the Annual Meeting of the American Statistical Association, Atlanta. http://www.bls.gov-osmr-pdf-st010120.pdf
Saris, W. E. & Gallhofer, I. N. (2007). Design, evaluation, and analysis of questionnaires for survey research(Wiley series in survey methodology). New York: Wiley-Interscience. ISBN 978-0-470-11495-7, e-ISBN 978-0-470-16519-5
Schwarz, N., Knauper, B., Hippler, H. J., Noelle-Neumann, E., & Clark, W. (1991). Rating scales: Numeric values may change the meaning of scale labels. The Public Opinion Quarterly, 55, 570–582. http://www.jstor.org-stable-2749407
Szende, A., Oppe, M., & Devlin, N. (Eds.). (2007). EQ-5D value sets. Inventory, comparative review and user guide. Dordrecht: Springer.
Szende, A., Janssen, B., & Cabases, J. (Eds.). (2014). Self-reported population health: An international perspective based on EQ-5D. Dordrecht: Springer. doi:10.1007/978-94-007-7596-1.
Uwawunkonye, E. G., & Anaene, O. I. C. (2013). A comparative study between ridit and modified ridit analysis. American Journal of Theoretical and Applied Statistics, 2(6), 248–254. doi:10.11648/j.ajtas.20130206.23.
Valentine, N., Verdes-Tennant, & Bonsel, G. (2015). Health systems’ responsiveness and reporting behaviour: Multilevel analysis of the influence of individual-level factors in 64 countries. Social Science & Medicine, 138, 152–160.
van Beuningen, J., van der Houwen, K., & Moonen, L. (2014). Measuring well-being, an analysis of different response scales (Discussion Paper, Statistics Netherlands). Retrieved July 20, 2015, http://www.cbs.nl-NR-rdonlyres-FF644A99-580C-4D7E-B214-BEE54A947D46-0-Measuringwellbeing.pdf
Veenhoven, R. (1984). Databook of happiness. Dordrecht: Reidel Publishing. ISBN 90-277-1793-1
Veenhoven, R. (1993). Happiness in nations, subjective appreciation of life in 56 nations, 1946–1992. Studies in Social-Cultural Transformation, 2. Risbo, Erasmus University Rotterdam, Netherlands. Available at: https://personal.eur.nl/veenhoven/Pub1990s/93b-con.html
Veenhoven, R. (2011). World database of happiness, example of a focused ‘Findings Archive’ (RatSWD, Working Paper Series, Working Paper, 169). Retrieved January 29, 2016, http://www.ratswd.de/download/RatSWD_WP_2011/RatSWD_WP_169.pdf
Veenhoven, R. (2016a). Measures of happiness. World Database of happiness, Erasmus University Rotterdam. Available at: http://worlddatabaseofhappiness.eur.nl/hap_quer/hqi_fp.htm
Veenhoven, R. (2016b). Happiness in Nations. World Database of Happiness, Erasmus University Rotterdam. Available at: http://worlddatabaseofhappiness.eur.nl/hap_nat/nat_fp.php?mode=1
Author information
Authors and Affiliations
Rights and permissions
Copyright information
© 2017 Springer International Publishing AG
About this chapter
Cite this chapter
de Jonge, T., Veenhoven, R., Kalmijn, W. (2017). Diversity in Survey Items and the Comparability Problem. In: Diversity in Survey Questions on the Same Topic. Social Indicators Research Series, vol 68. Springer, Cham. https://doi.org/10.1007/978-3-319-53261-5_1
Download citation
DOI: https://doi.org/10.1007/978-3-319-53261-5_1
Published:
Publisher Name: Springer, Cham
Print ISBN: 978-3-319-53260-8
Online ISBN: 978-3-319-53261-5
eBook Packages: Social SciencesSocial Sciences (R0)