Abstract
This chapter investigates a set of hypotheses linking dimensions of social status, power, diversity and culture to survey item nonresponse. Cross-national data drawn from 35 countries and 48,720 respondents who participated in the 2016 International Social Survey Programme (ISSP) are examined, along with a series of relevant country-level indicators. These data are analyzed using multilevel mixed-effects negative binomial regression models. Findings support the marginalized group perspective and confirm its generalizability across a broad cross-section of countries. Based on these findings, the authors recommend that questionnaire developers consider the different motivations for item nonresponse and work to design their instruments to better encourage responses from members of marginalized groups.
This is a preview of subscription content, log in via an institution to check access.
Access this chapter
Tax calculation will be finalised at checkout
Purchases are for personal use only
Similar content being viewed by others
Change history
29 October 2021
The original version of the chapter was inadvertently published with an error. The chapter has now been corrected.
Notes
- 1.
Exceptions for 2016 ISSP were Finland (15–74 years), Japan (16 years and over), Latvia (18–74 years), Norway (18–79 years), Sweden (18–79 years), South Africa (16 years and over), and Suriname (21–74 years).
- 2.
Age was not collected continuously but as response categories with age ranges in Denmark. We used the middle value of these age ranges as age values for Denmark.
- 3.
After consultation with an expert on cross-national classifications of education, we coded the categories of the ISSP degree variables as follows: no formal education, primary school, and lower secondary as low educational level; upper secondary and post secondary, non-tertiary as medium educational level; lower level tertiary and upper level tertiary as higher educational level.
- 4.
Most of the countries have either only data collection with interviewer involvement or self-completion. Two countries, Germany and Suriname, had both types of data collection. Therefore, we assigned these countries to the mode setting that was predominant in the respective country (Germany: self-completion; Suriname: interviewer).
- 5.
Further documentation on the UNR inequality index can be found here: http://hdr.undp.org/en/content/what-does-coefficient-human-inequality-measure.
- 6.
The GDP per capita data was taken from https://data.worldbank.org/indicator/NY.GDP.PCAP.CD.
- 7.
Values were retrieved from Hofstede’s website: https://geerthofstede.com/research-and-vsm/dimension-data-matrix/.
- 8.
We opted for the mode-curvature adaptive Gauss–Hermite quadrature integration method instead of the default integration method (Gauss–Hermite quadrature integration method) because the default setting created out-of-bound values for the country variance. We cross-checked our results with the Gauss–Hermite quadrature integration method with outputs in mlwin and the gllamm command in STATA. In all outputs, we found a similar pattern of significance for the regression coefficients. Contrary to the results with the default setting, not all regression coefficients are significant with the mode-curvature adaptive Gauss–Hermite quadrature integration method. Therefore, our results are more conservative than in the default setting.
References
Adua, L., & Sharp, J. S. (2010). Examining survey participation and response quality: The significance of topic salience and incentives. Survey Methodology, 26(1), 95–109. https://www150.statcan.gc.ca/n1/en/pub/12-001-x/2010001/article/11252-eng.pdf?st=kPlsEgKD.
Alesina, A., Devleeschauwer, A., Easterly, W., Kurlat, S., & Wacziarg, R. (2003). Fractionalization. Journal of Economic Growth, 8(2), 155–194. https://doi.org/10.1023/A:1024471506938.
Alexander, E. C. (2018). Don’t know or won’t say? Exploring how colorblind norms shape item nonresponse in social surveys. Sociology of Race and Ethnicity, 4(3), 417–433. https://doi.org/10.1177/2332649217705145.
Awaworyi Churchill, S., Ocloo, J. E., & Siawor-Robertson, D. (2017). Ethnic diversity and health outcomes. Social Indicators Research, 134, 1077–1112. https://doi.org/10.1007/s11205-016-1454-7.
Berinsky, A. (2002). Silent voices: Social welfare policy opinions and political equality in America. American Journal of Political Science, 46(2), 276–287. https://www.jstor.org/stable/3088376.
Callens, M., & Loosveldt, G. (2018). Don’t know responses to survey items on trust in police and criminal courts: A word of caution. Survey Methods: Insights from the Field. https://doi.org/10.13094/SMIF-2018-00002.
Candido, E., Kurdyak, P., & Alter, D. A. (2011). Item nonresponse to psychosocial questionnaires was associated with higher mortality after acute myocardial infarction. Journal of Clinical Epidemiology, 64(2), 213–222. https://doi.org/10.1016/j.jclinepi.2010.03.010.
Colsher, P. L., & Wallace, R. B. (1989). Data quality and age: Health and psychobehavioral correlates of item nonresponse and inconsistent responses. Journal of Gerontology, 44(2), 45–52. https://doi.org/10.1093/geronj/44.2.P45.
Dahl, R. A. (1957). The concept of power. Behavioral Science, 2, 201–215. https://doi.org/10.1002/bs.3830020303.
Davis, K., & Moore, W. E. (1945). Some principles of stratification. American Sociological Review, 10(2), 242–249. https://www-jstor-org.proxy.cc.uic.edu/stable/2085643.
Dillman, D. A., Eltinge, J. L., Groves, R. M., & Little, R. J. (2002). Survey nonresponse in design, data collection, and analysis. In R. M. Groves, D. A. Dillman, J. L. Eltinge, & R. J. Little (Eds.), Survey nonresponse (pp. 3–24). New York: Wiley. https://www.wiley.com/en-us/Survey+Nonresponse-p-9780471396277.
Elliott, M. N., Edwards, C., Angeles, J., Hambarsoomians, K., & Hays, R. D. (2005). Patterns of unit and item nonresponse in the CAHPS® hospital survey. Health Services Research, 40(6p2), 2096–2119. https://doi.org/10.1111/j.1475-6773.2005.00476.x.
Frick, J. R., & Grabka, M. M. (2007). Item non-response and imputation of annual labor income in panel surveys from a cross-national perspective. SOEPpapers on multidisciplinary panel data research, no. 49, Deutsches Institut für Wirtschaftsforschung (DIW), Berlin, Germany. https://ssrn.com/abstract=1020605.
Goerman, P. (2017). Development of a doorstep training initiative for bilingual interviewers at the U.S. Census Bureau. Mannheim: Presentation at the CSDI Workshop. Retrieved from https://csdiworkshop.org/wp-content/uploads/2019/01/Goerman_CSDI_2017_final_updated.pdf.
Goyder, J., Boyer, L., & Martinelli, G. (2006). Integrating exchange and heuristic theories of survey nonresponse. Bulletin de Méthodologie Sociologique, 92, 28–44. https://journals.openedition.org/bms/554.
Groves, R. M., & Couper, M. P. (1998). Nonresponse in Household Interview Surveys. New York: Wiley. https://www.wiley.com/en-us/Nonresponse+in+Household+Interview+Surveys-p-9780471182450.
Gudykunst, W. B., & Kim, Y. Y. (2003). Communicating with strangers (4th ed.). Boston: McGraw-Hill.
Heerwegh, D., & Loosveldt, G. (2008). Face-to-face versus web surveying in a high-internet-coverage population: Differences in response quality. Public Opinion Quarterly, 72(5), 836–846. https://doi.org/10.1093/poq/nfn045.
Hilbe, J. M. (2011). Negative binomial regression (2nd ed.). Cambridge: Cambridge University Press. https://doi.org/10.1017/CBO9780511973420.
Hofstede, G. (1980). Culture’s consequences. Beverly Hills, CA: Sage.
Hofstede, G. (2001). Culture’s consequences (2nd ed.). Thousand Oaks, CA: Sage. https://us.sagepub.com/en-us/nam/cultures-consequences/book9710.
Hofstede, G. (2010). Cultures and organizations: Software of the mind. Cultures Organ, 23, 362–365. https://doi.org/10.1057/jibs.1992.23.
Hox, J., Moerbeek, M., & van de Schoot, M. (2018). Multilevel analysis: Techniques and applications (2nd ed.). New York: Routledge. https://doi.org/10.4324/9780203852279.
ISSP Research Group. (2018). International social survey programme: Role of government V – ISSP 2016. Retrieved from https://dbk.gesis.org/dbksearch/sdesc2.asp?no=6900&db=e&doi=10.4232/1.13052.
Jenkins, C. R., & Dillman, D. A. (1997). Towards a theory of self-administered questionnaire design. In L. Lyberg, P. Biemer, M. Collings, E. de Leeuw, C. Dippo, N. Schwarz, & D. Trewin (Eds.), Survey measurement and process quality (pp. 165–196). New York: Wiley. https://onlinelibrary.wiley.com/doi/10.1002/9781118490013.ch7.
Kanbur, R., Rajaram, P. K., & Varshney, A. (2011). Ethnic diversity and ethnic strife. An interdisciplinary perspective. World Development, 39(2), 147–158. https://doi.org/10.1016/j.worlddev.2009.11.034.
Kane, E. W., & Macaulay, L. J. (1993). Interviewer gender and gender attitudes. Public Opinion Quarterly, 57(1), 1–28. https://doi.org/10.1086/269352.
Klein, D. J., Elliott, M. N., Haviland, A. M., Saliba, D., Burkhart, Q., Edwards, C., & Zaslavsky, A. M. (2011). Understanding nonresponse to the 2007 Medicare CAHPS survey. The Gerontologist, 51(6), 843–855. https://doi-org.proxy.cc.uic.edu/10.1093/geront/gnr046.
Koch, A., & Blohm, M. (2009). Item nonresponse in the European social survey. Ask: Research & Methods, 18(1), 45–65. http://hdl.handle.net/1811/69564.
Kupek, E. (1998). Determinants of item nonresponse in a large national sex survey. Archives of Sexual Behavior, 27(6), 581–594. https://doi.org/10.1023/A:1018721100903.
Lee, S., Liu, M., & Hu, M. (2017). Relationship between future time orientation and item nonresponse on subjective probability questions: A cross-cultural analysis. Journal of Cross-Cultural Psychology, 48(5), 698–717. https://doi.org/10.1177/0022022117698572.
Lipps, O. (2007). Interviewer and respondent survey quality effects in a CATI panel. Bulletin de Méthologie Sociologique, 95, 5–25. https://journals.openedition.org/bms/392.
Little, R. J. A., & Rubin, D. B. (2002). Single imputation methods. Statistical Analysis with Missing Data. https://doi.org/10.1002/9781119013563.ch4.
Little, R. J. A., & Rubin, D. B. (2014). Statistical analysis with missing data (2nd ed.). New York: Wiley. https://onlinelibrary.wiley.com/doi/book/10.1002/9781119013563.
McKnight, P. E., McKnight, K. M., SIdani, S., & Figueredo, A. J. (2007). Missing data: A gentle introduction. New York: Guilford Press. https://www.guilford.com/books/Missing-Data/McKnight-McKnight-Sidani-Figueredo/9781593853938.
Messer, B. L., Edwards, M. L., & Dillman, D. A. (2012). Determinants of item nonresponse to web and mail respondents in three address-based mixed-mode surveys of the general public. Survey Practice, 5(2). http://citeseerx.ist.psu.edu/viewdoc/download;jsessionid=A45DB36A0B8BD2FBCC02FD762FEECAC9?Doi=10.1.1.673.8650&rep=rep1&type=pdf.
Messmer, D. J., & Seymour, D. T. (1982). The effects of branching on item nonresponse. Public Opinion Quarterly, 46(2), 270–277. https://doi.org/10.1086/268718.
Mulder, F. (1977). The daily power game. Leiden: Martinus Nijhoff. https://www.springer.com/us/book/9781468469530.
Owens, L., Johnson, T. P., & O’Rourke, D. (2001). Culture and item nonresponse in health surveys. Seventh Conference on Health Survey Research Methods. DHHS publication no. (PHS) 01-1013. Hyattsville, MD: National Center for Health Statistics. https://www.cdc.gov/nchs/data/hsrmc/hsrmc_7th_proceedings_1999.pdf.
Patsiurko, N., Campbell, J. L., & Hall, J. A. (2012). Measuring cultural diversity: Ethnic, linguistic and religious fractionalization in the OECD. Ethnic and Racial Studies, 35(2), 195–217. https://doi.org/10.1080/01419870.2011.579136.
Pickery, J., & Loosveldt, G. (1998). The impact of respondent and interviewer characteristics on the number of “no opinion” answers. Quality & Quantity, 32(1), 31–45. https://doi.org/10.1023/A:1004268427793.
Putnam, R. D. (2007). E pluribus unum: Diversity and community in the twenty-first century the 2006 Johan Skytte prize lecture. Scandinavian Political Studies, 30(2), 137–174. https://doi.org/10.1111/j.1467-9477.2007.00176.x.
Rässler, S., & Riphahn, R. T. (2006). Survey item nonresponse and its treatment. Modern Econometric Analysis: Surveys on Recent Developments.https://doi.org/10.1007/3-540-32693-6_15.
Riphahn, R. T., & Serfling, O. (2005). Item non-response on income and wealth questions. Empirical Economics, 30, 521–538. https://doi.org/10.1007/s00181-005-0247-7.
Schafer, J. L., & Graham, J. W. (2002). Missing data: Our view of the state of the art. Psychological Methods, 7(2), 147–177. https://doi.org/10.1037/1082-989X.7.2.147.
Shoemaker, P. J., Eichholz, M., & Skewes, E. A. (2002). Item nonresponse: Distinguishing between don’t know and refuse. International Journal of Public Opinion Research, 14(2), 193–201. https://doi.org/10.1093/ijpor/14.2.193.
Stoop, I. A. L. (2016). Unit nonresponse. In C. Wolf, D. Joye, T. W. Smith, & Y.-c. Fu (Eds.), The SAGE handbook of survey methodology (pp. 409–424). Los Angeles: Sage.
Struminskaya, B., Weyandt, K., & Bosnjak, M. (2015). The effects of questionnaire completion using mobile devices on data quality. Evidence from a probability-based general population panel. Methods, Data, Analyses, 9(2), 261–292. https://www.gesis.org/fileadmin/upload/forschung/publikationen/zeitschriften/mda/Vol.9_Heft_2/MDA_Vol9_2015-2_Struminskaya.pdf.
Tourangeau, R., Rips, L. J., & Rasinski, K. (2000). The psychology of survey response. Cambridge: Cambridge University Press. https://www.cambridge.org/core/books/the-psychology-of-survey-response/46DE3D6F7C1399BCDC78D9441C630372.
Triandis, H. C. (1995). Individualism and collectivism. Boulder, CO: Westview Press. https://www.routledge.com/Individualism-AndCollectivism/Triandis/p/book/9780813318509.
Tu, S.-H., & Liao, P.-S. (2007). Social distance, respondent cooperation and item nonresponse in sex survey. Quality & Quantity, 41(2), 177–199. https://doi-org.proxy.cc.uic.edu/10.1007/s11135-007-9088-0.
Turner, C. F., Lessler, J. T., George, B. J., Hubbard, M. L., & Witt, M. B. (1992). Effects of mode of administration and wording on data quality. In C. F. Turner, J. T. Lessler, & J. C. Gfroerer (Eds.), Survey measurement of drug use: Methodological studies (pp. 221–243). Washington, DC.: National Institute of Drug Abuse, U.S. Department of Health and Human Services. http://qcpages.qc.cuny.edu/~cturner/TechPDFs/NIDA_BOOK.pdf.
Vandenplas, C., Loosveldt, G., Beullens, K., & Denies, K. (2018). Are interviewer effects on interview speed related to interviewer effects on straight-lining tendency in the European social survey? An interviewer-related analysis. Journal of Survey Statistics and Methodology, 6(4), 516–538. https://doi.org/10.1093/jssam/smx034.
Vercruyssen, A., Wuyts, C., & Loosveldt, G. (2017). The effect of sociodemographic (mis)match between interviewers and respondents on unit and item nonresponse in Belgium. Social Science Research, 67, 229–238. https://doi.org/10.1016/j.ssresearch.2017.02.007.
Yan, T., Curtin, R., & Jans, M. (2010). Trends in income nonresponse over two decades. Journal of Official Statistics, 26(1), 145–164. https://www.scb.se/contentassets/ca21efb41fee47d293bbee5bf7be7fb3/trends-in-income-nonresponse-over-two-decades.pdf.
Acknowledgements
We wish to gratefully thank the many colleagues who generously provided us with advice and insights regarding social cleavages and minority populations in their respective nations. Ann Evans & Karen Kellard (Australia), Markus Hadler (Austria), Jaak Billiet, Koen Verhoest and Marleen Brans (Belgium), Lilia Dimova (Bulgaria), Ricardo González and Marta Lagos (Chile), Yanjie Bian (China), Kresimir Znidar (Croatia), Dana Hamplova (Czech Republic), Dite Shamshiri-Petersen (Denmark), Kadri Täht (Estonia), Seppo Antiainen (Finland), Frédéric Gonthier (France), Lika Tsuladze (Georgia), Evi Scholz (Germany), Miranda Phillips (Miranda Phillips), Toth Istvan Gyorgy (Hungary), Hafsteinn Birgir Einarsson (Iceland), Yashwant Deshmukh and Sowmya Anand (India), Gal Ariely (Israel), Kuniaki Shishido (Japan), Mareks Niklass (Latvia), Egle Butkevicience (Lithuania), Alejandro Moreno (Mexico), Ineke Stoop (Netherlands), Barry Milne (New Zealand), Gry Karlsen (Norway), Linda Luz Guerrero (Philippines), Marcin Zieliński (Poland), Viriato Queiroga (Portugal), Bogdan Voicu (Romania), Ekaterina Lytkina (Russian Federation), Miloslav Bahna (Slovakia), Valentina Hlebec (Slovenia), Mari Harris (South Africa), Jibum Kim (South Korea), Jose-Luis Padilla (Spain), Jonas Edlund (Sweden), Marlène Sapin (Switzerland), Stithorn Thananithichot (Thailand), Yang-chih Fu (Taiwan), Wahab Benhafaiedh (Tunisia), Bulent Kilincarslan and Melike Sarac (Turkey), and Roberto Briceno-Leon and Roberto Briceno Rosas (Venezuela). We, however, assume complete responsibility for the operational definitions employed to represent majority vs. minority group membership within each nation.
Author information
Authors and Affiliations
Corresponding author
Editor information
Editors and Affiliations
Appendix
Appendix
Rights and permissions
Copyright information
© 2020 Springer Nature Switzerland AG
About this chapter
Cite this chapter
Meitinger, K.M., Johnson, T.P. (2020). Power, Culture and Item Nonresponse in Social Surveys. In: Brenner, P.S. (eds) Understanding Survey Methodology. Frontiers in Sociology and Social Research, vol 4. Springer, Cham. https://doi.org/10.1007/978-3-030-47256-6_8
Download citation
DOI: https://doi.org/10.1007/978-3-030-47256-6_8
Published:
Publisher Name: Springer, Cham
Print ISBN: 978-3-030-47255-9
Online ISBN: 978-3-030-47256-6
eBook Packages: Social SciencesSocial Sciences (R0)