Introduction

The primary intended beneficiaries of in-patient psychiatric care are the people admitted for treatment in such beds and facilities. In many countries of the world, most of the dedicated mental health budget is spent on the provision of psychiatric hospital beds and staff [1]. Yet there is a remarkable disregard in the scientific literature for the evaluation of both out- and in-patient mental health services as rated directly by patients. Similarly, there is a lack of reports from patients of how satisfied they were with their experiences of the processes of hospital treatment. Assessing and understanding the patient perspective is vital for service quality to improve. The focus on the treatment gap as the insufficient quantity of services is important in the global mental health agenda, but the quality of care through the eyes of people using it is crucial as well [2].

Previous research on patients’ satisfaction with mental health services used different instruments and methodological approaches to measure satisfaction [3] and therefore it is difficult to present a systematic overview and reflection on patients satisfaction with in-patient care. In narrative terms, one can say that higher ratings of patient satisfaction after episodes of in-patient care appear to be associated with: a clear discharge plan [4, 5]; less coercive treatment during the hospital stay [6]; more personalised; higher quality information and teaching to patients by staff about the mental disorder [7]; better therapeutic relationships with staff, especially nurses [8–10]—and specific treatment components which are well received by patients, such as physical exercise sessions [11]. Patients also tend to positively value staff members who spend more time with them, and interestingly this is reciprocated—staff ratings of work satisfaction are greater when they spend a greater proportion of their time in direct contact with patients [12]. Satisfaction with in-patient care has also been reported as lower among older patients [13, 14]; for non-white patients groups, in studies with a majority white population [15, 16]; and for patients treated with a greater number of different medications [21]. Overall, there is some evidence that psychiatric in-patients are less satisfied than patients discharged from hospital after treatment of acute, physical disorders [17].

While over a dozen scales have been developed to assess psychiatric in-patient satisfaction [14, 18–21], doubts remain about their methodological quality [3] and sensitivity to change [22]. Indeed it is often reported, across range of different in-patient settings, that most patients are ‘moderately satisfied’ with their care [23, 24]. It is relatively unusual in the literature to find sizable differences between patients group in treatment satisfaction ratings [15].

There are a notable number of shortcomings in this literature, namely, few follow up studies beyond the immediate post-discharge period; no scales which are patient generated; and the fact that these studies have been conducted in high-, rather than in low- or middle-income countries.

In this context, the aim of this study is to explore satisfaction of people treated in mental health in-patient facilities. In addition, we wanted to gain experience on the utilization and feasibility of questionnaire evaluating the service through the eyes of the patients in clinical settings. The project was a part of the Young Psychiatrists’ Program by the Association for the Improvement of Mental Health Programmes (AIMHP). The goals of the AIMHP programme were: (1) to develop a network of collaboration involving early career psychiatrists from different parts of the world and (2) to demonstrate early career psychiatrists that the partnership with patients helps in many ways, including the gaining of information about the perception of services. According to these goals, the IDEA research group consisting of senior researchers and early career psychiatrists from different parts of the world was established to pursue the current project, titled “IDEA: Inpatient Discharge: Experiences and Analysis”.

There were previous attempts to collect patients’ views on hospital admission internationally [25], but to the best of our knowledge, this is the first international multi-centre study specifically focused on patient satisfaction with in-patient care as a measure of services quality and functioning. Moreover, some of the countries involved in the study have never participated in similar research and therefore the study provides unique information about mental health services there.

Methods

Design, setting and study sample

In this cross-sectional international multi-centre survey, patients were interviewed on the day of discharge from psychiatric wards in 25 institutions across 11 countries (seven European, three African and one South American country). Participating countries and institutions were chosen to represent a wide range of clinical and economic settings. The selection happened on the ground of feasibility of establishing collaboration based on previous experience gained in the AIMHP Young Psychiatrists’ Program.

Patients were recruited from a single site in 8 of the 11 countries (Argentina, Bosnia and Herzegovina, Czech Republic, Italy, Romania, Tunisia, Uganda, Ukraine), and from multiple sites in Nigeria (n = 5), Russia (n = 5) and Croatia (n = 7). The target sample was 30 patients per institution, established on the grounds of feasibility. The research team approached a consecutive sample of discharged patients to participate until the recruitment target was reached. If the number of discharges exceeded the research team’s capacity to interview consecutively discharged patients, then systematic sampling was used instead (e.g. approaching every 2nd or 3rd discharged patient). Where possible, the patient was interviewed by a clinician not directly involved in the patient’s care. All interviewers received study protocol and study-specific training—included DVD training for familiarisation with the aims and use of study questionnaires. At any given site patients were recruited over a 3-month period. In this study, we excluded patients if data were missing for one or more items of the patient satisfaction questionnaire.

Study measures

The primary dependent variable was ‘low satisfaction’ with in-patient care. Satisfaction was measured using the following five-item study-specific questionnaire, which was developed with the leads for each study site:

Q1:

Do you feel your stay in hospital was beneficial?

Q2:

How satisfied were you with the staff?

Q3:

Do you feel you came to any harm during your stay in hospital?

Q4:

Were your individual preferences and rights taken into account?

Q5:

Was your right to confidentiality observed?

All questions were translated to the local languages by site leaders of participating countries, assessed by the expert panel of local bilingual researchers and discussed with international coordinator of the study in case of misunderstanding. For each question the patient rated their satisfaction on a Visual Analogue Scale (by drawing a cross on a line labelled ‘Not at all’ to ‘Very much’). This response was converted to a numerical value ranging from 0 to 10. Exploratory factor analysis was used to assess the number of latent factors measured by this five-item questionnaire. Factors were extracted using the principal factor method (as item responses were non-normally distributed), and a screeplot and Kasier’s criterion were used to choose the number of factors retained. Using this method, we found that the questionnaire measured a single underlying latent variable (conceptualised in this study as ‘overall satisfaction’).

To measure overall satisfaction for each participant, we summed their scores across the five questions—after reversing the score for Q3 such that for each question lower scores indicated poorer satisfaction—to calculate a total satisfaction score (range 0–50). We then derived a binary ‘low satisfaction’ variable for each participant, defined as having a total satisfaction score within the lowest quartile (for all participants across the study sites). We then calculated the proportion of patients within each site with ‘low satisfaction’ (i.e. the proportion of patients whose total satisfaction scores fall within the lowest quartile). We examined the individual and site-level correlates of ‘low satisfaction’ as detailed below.

Covariates

We examined individual and institution level correlates of ‘low satisfaction’, with variables chosen on the basis of past literature or face validity. All covariates were measured by researchers using semi-structured questionnaires. The individual-level correlates, as self-reported in the patient interview, were: (a) socio-demographic characteristics: age, sex, marital status, educational attainment, employment status; and (b) clinical characteristics: primary clinical diagnosis according to International Statistical Classification of Diseases and Related Health Problems 10th Revision (ICD-10) [26], legal status during admission, length of in-patient stay and total number of psychiatric hospitalisations. Institution-level correlates were: (a) type and size of institution: purpose built psychiatric hospital vs unit in general hospital, total number of beds, average length of stay; (b) patient population and staffing: male-to-female patient ratio, staff to patient ratios—overall and by professional group for nurses, psychiatrists and psychologists; (c) physical environment and accessibility: overcrowding, outdoor access, disability access, accessibility by public transport; and (d) complaint processes: hospital has complaints process, patients are aware of complaints process, patients are provided with means to complain.

Statistical analysis

All analyses were done using STATA version 13.0 for Windows. Descriptive statistics were used to summarise the institutions’ and patients’ characteristics. Box plots were used to summarise the medians (and interquartile ranges) of total satisfaction scores by institution. To account for the hierarchical nature of the data, with patients clustered within sites, multi-level multivariate logistic regression was used to examine the correlates of low satisfaction (using the xtmelogit Stata command). We built a series of models: (1) we estimated the variance of patient dissatisfaction across sites using an empty multilevel model (MLM); (2) we estimated crude associations for each individual-level variable, taking clustering into account (using an MLM) and then built a multivariate MLM retaining co-variates with a p ≤ 0.10; (3) we estimated crude associations for each institution-level variable and then built a multivariate model retaining co-variates with a p ≤ 0.10; retaining age and sex as a priori confounders (4) we built a model that included individual and institution level covariates with a p value of < = 0.10 from models 2 and 3 above; retaining age and sex as a priori confounders. We examined the extent to which patient dissatisfaction across sites was explained by the included co-variates by comparing models 1 and 4.

Ethical issues

Ethic approval for the study was obtained from the Research Ethics Office, King’s College London (PNM/10/11-91 on 09 May 2011; extended with modifications on 18 September 2012 and 08 February 2013). Approvals were also obtained from national or local ethical committees in each participating country and heads of each hospital signed an agreement of the facility to participate in the study. Every individual participant of the study achieved a copy of the patient information sheet and asked to sign an informed consent prior to data collection. Confidentiality was granted by exclusion of the personal data and codifying of the research protocols in a non-identifiable way.

Results

Sample flow and characteristics

A total of 701 patients were recruited across 25 sites in 11 countries (median = 30 patients/site; mean 28, range 14–34). This study included 673 patients—after excluding 28 (4.0%) respondents with missing satisfaction questionnaire data. Institution and patient characteristics are shown in Table 1. In terms of institutional characteristics, all but two of the sites were urban (with one rural site in each of Russia and Croatia). Fifteen sites (60%) were psychiatric units within general hospitals, and ten (40%) were purpose-built psychiatric hospitals. The sites varied widely in size, with a median of 100 beds (interquartile range IQR 50–600; range 20–1680 beds). The median length of stay in institutions was 28 days (IQR 21–51 days) and the median staff to patient ratio was 0.97 (IQR 0.53–1.2). In terms of personal characteristics, the median age of participants was 42 (IQR 30–52), 56% were male, 38% were married, 37% had college education and 29% were employed. Clinically, 35% patients had a diagnosis of schizophrenia and 29% a diagnosis of depression. Fourteen percent were legally detained during their admission, and the patients’ median admission duration was 28 days (IQR 17–45).

Table 1 Characteristics of study participants and institutions
Full size table

Patients' satisfaction

Total satisfaction scores were highly skewed towards the upper end of the scale, with a median total score (out of a maximum of 50) of 44 (IQR 38–48). Satisfaction scores by site are summarised in Fig. 1, with medians ranging from 31 (IQR 28–35) in one of the Nigerian sites to 48 (IQR 43–49) in Argentina. The proportion of patients with ‘low satisfaction’ (i.e. the proportion of patients whose total satisfaction scores fall within the lowest quartile) by site is shown in Fig. 2—and ranged from 0% (in one of the Croatian sites) to 93% (in one of the Nigerian sites), with a median of 23% (IQR 11–29%). Inspection of Fig. 2 showed that there were two outliers with a high proportion of patients reporting low satisfaction (in Nigeria and Tunisia) and two sites with very low proportion or absence of dissatisfied patients (in Croatian sites 3 and 4). We compared the characteristics of these four sites with the remaining sites, and found that the most notable difference was the high proportion of legally detained patients in those with low satisfaction (83% in the Nigerian site and 76% in Tunisia; compared to a median of 6.7% (IQR 0–7.7%) detained patient across the other sites). Sites with a very low proportion of dissatisfied patients did not have characteristics that clearly distinguished them from other sites’.

Fig. 1
figure 1

Boxplot for median/interquartile range of total satisfaction score, by site

Full size image
Fig. 2
figure 2

Proportion of patients with ‘low satisfaction’, by site

Full size image

Correlates of ‘low satisfaction’

We investigated the individual and institutional correlates of ‘low satisfaction’, taking into account clustering of patient responses within sites. The empty multilevel model was consistent with significant differences in the proportion of dissatisfied patients between sites (p < 0.001). This is shown graphically in Figure S1—a plot of the parameter estimates from the empty random effects model (the Best Linear Unbiased Predictor and its standard error) for each of the 25 sites. In this graph, where the 95% confidence interval of the parameter estimate for a given site does not overlap the horizontal line at zero, this indicates that the proportion of patients with low satisfaction at that site is estimated to be significantly above average (above the zero line) or below average (below the zero line). The graph shows that there were several sites where the proportion of patients reporting low satisfaction was estimated to be significantly above or below the mean.

The binary association of ‘low satisfaction’ with individual and institute-level covariates is shown in Table 2. After taking clustering within sites into account, low satisfaction was associated with following individual-level covariates: being unmarried (unadjusted p < 0.001; taking clustering into account p = 0.09), being unemployed (unadjusted p < 0.001; taking clustering into account p = 0.04), and having a diagnosis of schizophrenia or related disorders (unadjusted p < 0.001; taking clustering into account p = 0.07). It is of note that low satisfaction was most commonly reported by legally detained patients (52%), and legal detention was crudely associated with low satisfaction, but this correlation was no longer statistically significant after taking clustering into account (unadjusted p < 0.001; taking clustering into account p = 0.59). After taking clustering within sites into account, the only institutional characteristic that was associated with low satisfaction was having a lower psychiatrist-to-patient ratio (unadjusted p < 0.001; taking clustering into account p = 0.04).

Table 2 Associations between individual- and institutional-level variables and patient ‘low satisfaction’
Full size table

The final model included both individual and institution-level correlates. As shown in Table 3 the only independent correlates of low satisfaction in the final model were: having a diagnosis of schizophrenia or related disorders (OR 1.6, 95% CI 1.1–2.5, p = 0.03) and having a lower psychiatrist to patient ratio (OR for lowest vs highest ratio tertile = 2.7, 95% CI 0.93–7.8, p = 0.09). The median (IQR) psychiatrist:patient ratio in the highest to lowest tertile were: 0.29 (0.23–0.35); 0.11 (0.10–0.17) and 0.044 (0.039–0.064), respectively.

Table 3 Individual and institution-level correlates of patient ‘low satisfaction’, estimated with a multilevel model (patients clustered within sites)
Full size table

Discussion

The study aimed to explore the satisfaction of people treated in mental health in-patient facilities in terms of evaluating such services explicitly from a patient perspective and to assess its feasibility in multi-country clinical settings. Applying this approach towards service evaluation we could successfully establish a collaborative network of early career psychiatrists (IDEA-group) from different parts of the world and collect data with regard to the aforementioned aims.

The results of the study demonstrated high levels of respondents’ satisfaction with the in-patient services provided. The satisfaction scores were positively skewed across all study sites. These data are in line with results achieved in other studies reporting high patients’ satisfaction with mental health services [27–29] and showing that up to 90% of respondents report being satisfied with the in-patient psychiatric treatment [30]. Similar results of generally satisfied patients were achieved in studies conducted in different parts of the world, e.g. India [31], Kuwait [32], Kenya [33], Nigeria [34], Poland [35], Thailand [36], Finland [37], and Israel [23].

However, the overall picture changes a great deal when the focus of past research was switched from general satisfaction towards specific challenges and experience that people living with mental disorders and their caregivers face. In particularly, patients and caregivers reported, for example, experiences of abuse while staying in hospital [38], staff misbehaviour [39], unnecessary curtailment of freedom [40], bad accommodation conditions and poor access to information [41]. Moreover, analysis of patient and staff perception of psychiatric in-patient wards showed that it may significantly worsen over time even in high-income countries, such as UK [42]. The evidence of hardships is especially manifest when studies use a qualitative research design [43]. This disparity between the overall satisfaction and multiple reports of difficulties during hospital stay should be taken into account when interpreting the quantitative results of this study.

The literature on Quality Of Life (QOL) has also shown the disparity between subjective (self-rated) and objective (externally rated) components [44]. QOL was defined by Calman as “the gap between person’s expectations and achievements” [45] which may be kept small in two ways: by lowering expectation or increasing achievements [46]. When there is no possibility to achieve one’s aim, a person may adapt to their environmental conditions by lowering their expectations. This was described in relation to people with mental illness’s subjective QOL as a “standard drift fallacy” [46]: having low, or very low standard of living and life expectations can lead to positive self-ratings of QOL by people who live in objectively very hard and poor conditions. People experiencing many hardships in fulfilling their needs may simply “cut their coat to their cloth” and self-rate QOL as excellent, at odds with the evaluations given by an independent rater [47].

In this context, when considering patient satisfaction it is important to keep in mind the role of expectations that were met or neglected during the hospital stay. If expectations are met, people will be satisfied even in the face of poor quality of care [48]. As it has been shown in primary health care, patient satisfaction correlates with the extent to which physicians fulfilled patients’ expectations [49–52]. Moreover, the level of satisfaction was shown to be associated with greater mortality risk, increased health care expenditure and higher risk of in-patient care [53]. The authors of the study discuss the “Cost of satisfaction” and suggest that enhanced health services should follow a patient-centered care model [54], where evidence-based interventions are provided in accordance with the priorities and preferences of well-informed patients. Poorly informed patients may have treatment expectations that are distant from evidence-based clinical practice.

Expectations of patients towards care within mental hospital may be shaped by the information and experiences people have. Existing studies of patients’ expectations address patients’ opinions on what constitutes good psychiatric practice [8, 55] rather than what people anticipate to meet within existing services. However, taken into account poor access to appropriate information, stigma and discrimination, disempowerment of people with mental illnesses, and poor public images associated with mental hospitals, it may be assumed that many hardships of hospital treatment are expected and considered as an inevitable element of care if the main outcomes of treatment (e.g. symptoms reduction) are met. In other words, low expectations of disempowered people may be in line with discriminatory services despite high rates of service satisfaction. Further research on patients’ expectations in the sense of what they anticipate is needed to better understand patients’ satisfaction with mental health services.

Analysis of factors associated with satisfaction in this study revealed only two independent correlates of low satisfaction: schizophrenia diagnosis, and low psychiatrist:patient ratio. With regard to the diagnosis of schizophrenia, previous research on satisfaction with in-patient treatment have also reported lower satisfaction scores among people with schizophrenia [29, 56], but not when compared to depressed patients [27]. One possible explanation of lower satisfaction score among people with schizophrenia is lower level of insight in schizophrenia [57] which in its turn was shown to be associated with treatment satisfaction [30]. Alternatively, low satisfaction may result from higher stigma towards patients with schizophrenia and more sceptical attitude of clinicians in including these patients in the decision-making process. It is possible that mental health professionals devote less time to people with schizophrenia because of more pessimistic views on this disorder [58], providing patients with less hope for recovery and preferring to spend time with people having other diagnosis.

The ratio of psychiatrist per patient may be considered as a marker of the quality of the institution as a whole, and an indicator of time doctors have per patient. It is also possible that the number of psychiatrists per patient is a proxy for the total wealth of health care system. Thus, it is not surprisingly linked to patients’ satisfaction in the current study. Association of satisfaction with the amount of time doctor have per patient [12], quality of relationship [9], amount of information provided [7], and consequently with more or less personalised approach to people in hospital were all shown to play a role in patients’ satisfaction with the mental health service.

There are number of other factors that can potentially influence patients’ satisfaction as an indicator of quality of services, but were not taken into account in the current study, for example, application of restraints, physical isolation, access to psychological support and rehabilitation, etc. We hope that further research will examine them on both international and local levels.

Limitations

There are several limitations in the current study that need to be appreciated. First, the issue of the power imbalance between doctors and patients during the data collection must be carefully considered. It is possible that respondents tended to report more satisfaction as the data were collected within hospital settings and by medical staff, even though the study protocol indicated that the observer should not be the clinically treating doctor for each patient assessed. It was shown in other studies that satisfaction is lower if data are collected at home rather than in medical facilities [59]. On the other hand, collecting data after the discharge may lead to a higher non-response rate and bias as respondents may differ from non-respondents. Therefore, it was recommended that instruments measuring satisfaction are administered before patients discharge [18]. Additionally, data collection by medical personnel may lead to unwillingness of patients to express dissatisfaction with hospital conditions for fear of antagonism with service providers, or future ‘punitive’ treatment [60], i.e. “response acquiescence” (i.e. tendency to agree rather than disagree) [61]. To reduce risk of these biases in the current study we attempted to collect data by raters who were independent of treatment and included one reversed question into the questionnaire.

Second, there is a critique on the overall concept of satisfaction and its measurement [48]. Divergent validity of the satisfaction concept was questioned as it overlaps with other patients-reported outcome measures, such as subjective quality of life, needs and quality of therapeutic relationship [3]. Additionally, the reliability of studies on satisfaction among people with mental disorders was questioned as influenced by psychiatric symptoms and cognitive deficit [62]. However, as it was argued in a review by Reininghaus et al., symptoms and deficit do not compromise measurement of patients reported outcomes [3].

Third, since only a limited number of non-randomly selected sites per each country were included in the study, there might be an issue with representativeness and generalisability. The selection of the 25 facilities was not a random sample among all facilities operating in the 11 participating countries, but rather a convenience sample of institutions, based on the personal knowledge that the study initiators had with the country investigators. It limits external validity and generalisability of results to all in-patient facilities, as in some countries there might be a substantial difference and uneven provision of care across institutions (e.g. in case of Italy [63]). Moreover, since the participating countries were chosen to represent a wide range of clinical and economic settings on the ground of feasibility, we cannot assume that the study sites are generalisable to other sites elsewhere.

There are two notable strengths to this study. To begin, it is the first study to our knowledge that examined satisfaction with psychiatric in-patient care as rated by patients at discharge from multiple institutions in low-, middle- and high-income countries in a single study. Next, the sample size for this analysis was relatively large and diverse.

Implications of the findings

Service evaluation is usually carried out through health information systems that rarely contain indicators reflecting patient satisfaction [2]. However, patient experience of service utilisation has been increasingly discussed as an important indicator of health care quality [64]. The ‘number of patients and caregivers expressing satisfaction with received services’ was mentioned among 15 most stable and highly ranked indicators for monitoring mental health care [2]. However, our study raises a number of questions on how to measure and utilize patient satisfaction to guide quality improvement initiatives. Further studies on satisfaction should pay attention to patients’ expectations and their previous experience of treatment and service-related knowledge and minimise power misbalances by assuring anonymity from clinical personnel and involving peers and independent parties into the data collection. It would also be useful to expand the focus of similar studies to explore satisfaction of patients with care provided in out-patient facilities and those primarily concerned with rehabilitation; and to conduct similar in other countries and world regions (e.g. in Asia and Latin America).