1 Introduction

Major changes have taken place in organisations since the 1960s and 1970s, including technological advancements, globalisation of industry, organisational restructuring and job security (Sparks et al. 2001). As a result of such changes attempts have been made to understand factors involved in employee well-being given its close relationship to job ‘burn-out’, which can include both mental and, to a lesser degree, physical ill health (Faragher et al. 2005). Despite the importance of identifying and managing components of well-being in workplaces, there has been a paucity of valid and reliable measurement tools to judge levels of well-being in that context. The development of the Workplace Well-being Questionnaire or WWQ (Parker and Hyett 2011) sought to identify and explore the various factors underlying individual employee well-being in workplaces. Initial analyses of the WWQ suggested that four factors accounted for much of the variation in workplace well-being. Given the exploratory nature of the initial study, it remained to be determined whether such factors were replicable beyond the sample.

The WWQ enables those who are employed, or recently employed, to gauge their level of well-being on dimensions of (1) work satisfaction, (2) organisational respect for the employee, (3) employer care, and (4) intrusion of work into private life. Parker and Hyett (2011) described the development of the measure and examined its validity and reliability in three samples: a non-clinical population of 100 subjects, a sample of people with depression, measured on two separate occasions with depression severity also recorded, to examine for test–retest reliability and impact of state depression levels, and a web-based sample of 1,000 individuals to enable calibration of factor scores. The best-fitting factor analytic model for the 31-item measure suggested a four factor solution. Psychometric calibration of the WWQ was undertaken in terms of its relationship with work type (e.g., professional/managerial vs. non-professional/non-supervisory) and length of time in the individual’s current or most recent job, with both influencing the reporting of workplace well-being. No validity studies were undertaken.

Well-being is a complex construct that is often difficult to measure given its variation within individuals (i.e., across time due to ageing) and across contexts. It has been argued that there is no universality in what it means to possess well-being and, thus, this has led to the suggestion that well-being is relative and often dependent upon a range of external variables (e.g., cultural bias; Ryan and Deci 2001). As such, in developing a measure of well-being in workplaces, it is evident that much effort is needed to address generalizability. In our development paper, we first sought to include only items that would address well-being in the workplace—effectively constraining analysis to the work domain. Second, items were constructed such that they covered both hedonic and eudemonic well-being principles, given the practical and theoretical importance of these factors (Ryan and Deci 2001). Fulfilling the goal of developing a valid measurement tool for workplace well-being with greater generalizability does, however, require robust validation studies to be undertaken. Doing so allows for greater confidence in the eventual application of any given measure and its underlying principles—here, the principle of well-being as a function of a given individual’s workplace.

In a review of methods of factor analytic techniques in psychological research, Fabrigar et al. (1999) advocated the use of methods such as confirmatory factor analysis (CFA) to obtain estimates of factor structure where there is a priori information at hand that may assist in identifying factors. The CFA method also reduces the likelihood of investigator bias given the set hypothesis testing framework on which it is based. The basic premise of CFA is testing whether observed data fit a given a measurement model (e.g., hypothesised factors, which may be derived from previous analyses such as principal components analysis). Its goal is to identify latent structure in the data and, hence, CFA is related—though distinct in its final form—to structural equation modelling (Rosseel 2012). Combined with exploratory methods such as exploratory factor analysis (EFA), CFA is a useful method in validating data structure.

Given the small sample in the initial validation study, the current study sought to revisit the factor structure of the WWQ in a distinctly larger sample to test its validity. Using both EFA and CFA we pursued the question as to whether the original factor solution was able to be replicated. We hypothesised that, an EFA would find a similar factor structure to the original four-factor solution and, in a separate sample, a CFA would provide support for this solution, thus arguing for retaining the same items (from our first analyses) for each factor. In testing associations of the factors in a large sample, we further speculated that the resulting factors would have moderate predictive utility on several variables, namely:

  1. 1.

    While there would likely be exceptions (e.g., poor management), we hypothesise that, in our sample, with increasing age there would be an increase in work satisfaction, organisational respect for the employee and a decrease in intrusion of work into private life;

  2. 2.

    Salaried, hourly paid and volunteer workers combined would report less intrusion of work into private life compared to contract workers, but there would be no distinct contribution of work level on the remaining factors;

  3. 3.

    Males would report increased levels of intrusion of work into private life compared to females.

2 Method

Our previous report (Parker and Hyett 2011) provided details about the WWQ and, thus, only methodological issues relevant to the current study will be overviewed. In terms of data acquisition, individuals accessing the Black Dog Institute website (http://www.blackdoginstitute.org.au) were offered the option of completing the WWQ in order for them to gauge their levels of workplace well-being. Prior to completing the online questionnaire respondents were informed that data would be used for research purposes. Several general questions were asked prior to the WWQ being completed, covering age range (with the options: ‘under 18’; ‘18–25’; ‘26–35’; ‘36–45’; ‘46–55’; ‘56–65’; or ‘over 65’); gender; current or most recent job level (‘manager’; ‘professional’; ‘technical/trade worker’; ‘community/personal service worker’; ‘clerical/administrative worker’; ‘sales worker’; ‘machinery operator/driver’; or ‘labourer’); what best represented their pay structure (‘salaried employee’; ‘charge by the hour’; ‘contract worker’; or ‘unpaid/volunteer’); and how many years and/or months they had been in their current or most recent job. Following these questions participants were instructed to complete the WWQ after which they were provided with scale scores in terms of ‘Work Satisfaction’, ‘Organisational Respect for the Employee’, ‘Employer Care’ and ‘Intrusion of Work into Private Life’.

2.1 Sample and Analytic Approach

In order to judge its validity—an important step in the development of questionnaire-based measures—we first recruited a large sample of individuals to complete the WWQ. We received 7,719 responses to the survey between October 2009 and May 2013 before ceasing data collection. Of the total sample, all but two had full data on the variables described above resulting in a final sample of 7,717 for the current analyses. All analyses were performed in R and SPSS. We undertook exploratory factor analysis on an initial sample of 5,000 (of the original 7,717), allowing us to first treat items as continuous variables and then, in a separate analysis (on the same sample), as categorical (using a polychoric correlation matrix). Larger samples are typically needed for categorical variables [compared to continuous] (Bartlett et al. 2001) and, thus, an unequal splitting of the sample was deemed necessary, with the remaining 2,717 respondents comprising the second sample for the CFA. Confirmatory factor analysis was carried out on this second sample using the ‘lavaan’ package (Rosseel 2012) in R using a maximum likelihood estimation scheme. This requires a measurement model to be specified, which for the current analyses, was the output of the EFA (Sample 1). For the current CFA, each factor represents a latent variable and each item an indicator variable. Factor loadings of the first indicator were fixed for the CFA, which was fully specified thus allowing for inspection of item-level relationships to their respective factors, covariances amongst factors, and item and factor variances. Examination of overall model fit is reported in terms of the root mean square error of approximation (RMSEA), which incorporates a penalty function for poor model parsimony (Steiger and Lind 1980), the Tucker–Lewis Index (TLI) and the Comparative Fit Index (CFI).

3 Results

3.1 Sample Characteristics

Sample characteristics are reported for the whole (N = 7,717) sample. Of these, 0.82 % (n = 63) of individuals reported that they were under the age of 18; 8.68 % (n = 670) between the ages of 18 and 25; 24.67 % (n = 1,904) between 26 and 35; 27.83 % (n = 2,148) between 36 and 45; 27.67 % (n = 2,135) between 46 and 55; 9.67 % (n = 746) between 56 and 65; and 0.66 % (n = 51) over the age of 65. Female respondents made up 68.78 % (n = 5,308) of the sample, and males 31.22 % (n = 2,409). In terms of job level, the respondents classified themselves as follows: 43.18 % (n = 3,332) as professionals; 19.44 % (n = 1,500) as managers; 16.20 % (n = 1,250) as clerical or administrative workers; 8.59 % (n = 663) as community or personal service workers; 5.23 % (n = 404) as technical or trade workers; 4.08 % (n = 315) as sales workers; 1.96 % (n = 151) as labourers; and 1.32 % (n = 102) as machinery operators or drivers. The reported average length of service in their current or most recent job was 6.2 years (SD = 7.1 years) with a range of <1 month–50.9 years. The principal payment method (from employer to employee for time worked) across all participants was as a salaried employee, 81.18 % (n = 6,265) of the sample, with hourly billing, 12.70 % (n = 980), and contract workers, 4.96 % (n = 383) comprising the majority of the remainder. Only 1.16 % (n = 89) of respondents were unpaid/volunteers.

3.2 Exploratory and Confirmatory Factor Analysis of the WWQ Items

Table 1 illustrates a comparison of the factor structure for the original four factor solution, along with the current EFAs (with items treated as either continuous or categorical). Items in bold from the continuous and categorical EFAs indicate divergence from the originally extracted four factor solution of the WWQ. The differences can be summarised as follows. For the original Work Satisfaction factor, all items except one (q31) were found to load on a similar factor in both continuous and categorical EFAs. The original Factor 2 (“Organisational Respect for the Employee”) was partially replicated, except in the current EFAs it was extracted as Factor 3; q31 was found to load on the new Factor 3, as was q29 (originally identified under “Employer Care”). The Employer Care factor in the original WWQ was, in the current analyses, extracted as Factor 2 and retained all of the same items (except q29 as above). For “Intrusion of Work into Private Life”, all items were retained under a new Factor 4 in the current EFAs. The new factors accounted for the following amount of variance: Factor 1 = 17.5 %; Factor 2 = 17.1 %; Factor 3 = 15.9 %; Factor 4 = 11.1 %.

Table 1 Loadings of continuous and categorical exploratory factor analyses on original Workplace Well-being Questionnaire (WWQ) items
Full size table

A CFA was undertaken on the items and factors that were extracted from the EFAs using a validation sample of 2,717 respondents. Model fit was reasonable, as reflected in the following fit indices; RMSEA = 0.08 (90 % CI 0.08–0.09; Probability RMSEA ≤ 0.05 = 0.00); TLI = 0.86; CFI = 0.87. In order to judge item-level fit for each of the factors, Table 2 provides an overview of item/factor standardized parameter estimates, standard error values and Z-values along with a corresponding two-sided p value (denoting the probability of a given parameter being equal to zero). For the workplace satisfaction factor, indicator variable coefficients ranged between 0.49 and 0.85; for organisational respect for the employee the range for indicator variables was between 0.70 and 0.88; for employer care, 0.71 and 0.90; and for intrusion of work into private life, loadings ranged between 0.56 and 0.79. All p values for the items were significant, but the loadings for some of the items were low-to-moderate in strength (e.g., q19 and q27 on the workplace satisfaction factor), which may have resulted in the relatively poor fit indices.

Table 2 Standardized parameter estimates, standard errors and Z-values of the WWQ indicator items and factors
Full size table

3.3 Examining Relationships Between Age, Work Level and Pay Type on Factor Scores

Due to the number of comparisons and large sample size a strict significance level of α = 0.001 was imposed for each of the comparisons between age, work level and pay type. The variable ‘age range’ was treated as a continuous variable in testing for associations between age and factor scores. For work satisfaction there was a significant increase in scores with increasing age (β < 0.01, p < 0.01). The effect size (eta2) for this finding, as measured by a separate MANOVA, was <0.01 and, thus, deemed trivial. Increasing age was negatively related to organisational respect for the employee (β = −0.02, p < 0.01) and employer care (β = −0.01, p < 0.01), while there was no significant effect of age on intrusion of work into private life. Effect sizes across both Factors 2 and 3 were the same (eta2 = 0.01), indicating that the significance may have been driven by the large sample size. In testing our second hypothesis of whether employment type (contract work vs. salaried, volunteer, and hourly paid workers combined) was associated with any differential intrusion of work into ones private life, it was found that contract workers reported marginally less intrusion (M = 14.98, SD = 6.46) than salaried, volunteer and hourly paid workers (M = 14.21, SD = 6.68), with the effect size again significant but not meaningfully distinct (eta2 < 0.01, p = 0.02). For work satisfaction, contract workers reported higher scores (M = 16.47, SD = 7.59) than salaried, volunteer, and hourly paid workers (M 15.40, SD = 7.46), but this was also associated with a low effect size (eta2 < 0.01, p < 0.01). No significant differences were observed for pay type on organisational respect for the employee, but for the employer care dimension, contract workers reported higher levels (M = 16.28, SD = 8.46) than those in the combined pay type sample (M = 14.63, SD = 8.33), with small but significant effect (eta2 < 0.01, p < 0.01). There was also a small but significant effect (eta2 < 0.01, p < 0.01) of gender on work satisfaction, with females reporting higher levels (M = 15.91, SD = 7.48) than males (M = 14.44, SD = 7.35). In terms of organisational respect for the employee, females again rated more highly (M = 10.47, SD = 6.45) than males (M 9.78, SD = 6.18) (eta2 < 0.01, p < 0.01). A similar pattern was also found for employer care with females (M = 15.09, SD = 8.41) rating higher than males (M = 13.89, SD = 8.15; eta2 < 0.01, p < 0.01). Contrary to our hypothesis, no difference was observed in terms of intrusion of work into private life across genders.

4 Discussion

With added emphasis being placed on psychological well-being across many facets of life, including the workplace, and its significance in terms of impact on the individual and beyond, it is becoming increasingly important to accurately measure its components. A previous study (Parker and Hyett 2011) reported the development of a new measure of workplace well-being, the WWQ, with analyses indicating that four factors accounted for much of the variation across its 31 items. The current study set out to test the validity of the four factors of the WWQ by conducting both an EFA—to test for similarity in factor structure—and, on a validation sample, a CFA to either confirm or refute such structure. Both continuous and categorical EFAs provided moderate support for the original structure of the WWQ. Much of the original factor two was extracted as factor three in the EFAs, and the original factor three was extracted as factor two. Additionally, several items that previously loaded on a given factor were extracted onto differing factors in the EFAs. Hence, this exploratory analysis suggests that some modifications to the item-factor structure were necessary. This was confirmed by a CFA conducted on an independent sample, which showed a modest fit for the revised factor structure of the WWQ.

The current study also suggests that the different scales of the WWQ are only slightly affected by age and employment type and are not affected by gender. This may be interpreted in one of two ways. First, response styles to the measure may be invariant across individuals, effectively resulting in little variability, for example between different age ranges. Second, and more concerning, might be the issue of failing to find differences when such differences are likely to exist. This may be a result of the questionnaire assessing work-related characteristics that are consistent across comparator conditions leading to underestimation of role- and person-specific well-being. In light of the sample size and the results of the CFA, we favour the first explanation, and conjecture that the measure allows for consistent identification of four domains of workplace well-being, albeit with some change to its structure. However, other explanations (not pursued in the current study) should not be discounted. Recent research (Kahneman and Krueger 2006) suggests that individual subjective well-being does, in fact, vary as a function of life circumstances including their health and neurological functioning (e.g., memory of events being responded to). For example, abrupt changes to the work environment, through events such as accidents or pregnancy, may have deleterious effects on employee well-being (Kuoppala et al. 2011; Pattison and Gross 1996). Measured correctly, subjective well-being—including well-being in the workplace—has been suggested to have important consequences for informing public policy (Dolan and White 2007), such that policy goals can be met by taking into account subjective happiness in addition to objective measures of “well-being” (e.g., productivity). Such research highlights the importance of constructing accurate measurement tools for broad dissemination.

The current study had a number of limitations that could be addressed in future research into workplace well-being. First, despite the large sample size it was difficult to interrogate the data in a more refined manner to examine the influence of, for example, discrete occupational groups, or stratified income levels. This limits the application of findings to specific workplaces and would benefit from a focus in future research. Second, the use of the Internet to recruit the sample and administer the survey may have imposed a selection bias to those who have access to a computer and Internet connection, limiting the generalizability of the measure. Third, and as alluded to above, we did not explicitly measure factors known to impact on workplace well-being (e.g., job security), which have the potential to significantly impact study findings. Despite these limitations, there is substantial scope for broad application of the WWQ in allowing the valid and reliable assessment of workplace well-being.

5 Conclusion

Based on our review of the literature, the accurate measurement of workplace well-being has a number of implications for both employees and employers, particularly in that its measurement allows for assessment of positive qualities while also offering the capacity to take remediation steps where optimal well-being is not observed. The original aim in developing the WWQ was to provide a measure of workplace well-being applicable to a wide range of working individuals, and to close a gap in the literature given no such measure, specific to workplaces, existed. Fundamental to such an aim was to ensure that the measure had robust psychometric qualities, allowing it to be used across populations with confidence. The current study addressed this qualifier, and the WWQ, in its revised form, has the capacity to inform individuals and workplaces on the nature of subjective well-being across a number of domains. We hence judge the WWQ to be of significant benefit for individuals and organisations wishing to validly identify workplace well-being levels regardless of gender, age and job type.