Impacts on practice

  • Pharmacists now have a validated instrument to assess patient satisfaction with the pharmacy services provided to ambulatory care patients

  • The rating of the satisfaction of the clients will assist pharmacists to determine the extent to which their service has met or failed to meet the needs of patients.

  • The new questionnaire can assists pharmacists to improve the quality of the service delivered.

Introduction

The shift of the pharmacy profession from a purchaser of medications to patient-centred care has created a need to evaluate humanistic outcomes such as patient satisfaction [1]. Patient satisfaction is an important determinant of the viability and sustainability of healthcare services, as it is the patients’ own evaluation of the services provided [2]. Evidence shows that satisfied patients are more likely to continue to utilise healthcare services, to value and maintain relationship with healthcare providers, to follow the advice of healthcare professionals, to adhere to treatment and to have better health outcomes [3, 4]. Evaluating satisfaction can also assist healthcare professionals in improving healthcare services more effectively. Healthcare providers will be able to determine the extent to which their service has met the needs of patients, and to identify areas that failed to meet patients’ expectations. Subsequently, measures can be taken to improve the inadequacies identified [5], and to account for the quality of the service delivered [6].

Several questionnaires have been developed specifically to assess patient satisfaction towards an ambulatory care pharmacy service in the United States [7,8,9,10,11,12], Spain [13], United Arab Emirates [14], Australia [15] and Slovenia [16]. Among which, only a few have been validated [7, 9, 10, 12,13,14,15,16]. In Malaysia, only two questionnaires have been developed and validated to evaluate patients’ satisfaction [17, 18]. However, these two questionnaires were developed specifically to assess patient satisfaction towards a pharmacist-led osteoporosis screening program [17] and an osteoporosis intervention program [18]. To date, no instrument has been developed and validated specifically to assess patients’ satisfaction for an ambulatory care pharmacy service in Malaysia.

Aim of the study

The aim of our study was to develop and validate the Ambulatory Care Patient Satisfaction Questionnaire (ACPSQ) in Malaysia.

Ethics Approval

Ethics approval was obtained from the University Malaya Medical Centre Medical Ethics Committee (approval number: 938.16) prior to the study.

Method

Development of the Ambulatory Care Patient Satisfaction Questionnaire (ACPSQ)

The ACPSQ was developed based on literature search in Ovid, PubMed and Scopus; using key words such as “satisfaction’’, “tool”, “questionnaire”, “instrument”, “survey”, “pharmacy”, “pharmaceutical care”, “ambulatory care”, “pharmacist” and “validation”.” Language was limited to English. From our search, a list of 47 potential questions were compiled based on a framework for the development of a satisfaction questionnaire for health services [2]. This framework consisted of seven domains that may influence patients’ satisfaction when utilizing health services: information, accessibility, technical quality, interpersonal relationship, finance, continuity, and physical condition [2].

Face and content validity of the ACPSQ was verified by an expert panel which consisted of one family medicine specialist, two pharmacists in academia, and one hospital pharmacist. Each item was reviewed, and the relevance and appropriateness of each item was discussed, until the expert panel deemed that the ACPSQ covered all the important domains on patient satisfaction in ambulatory care pharmacy. Four items were used as they were, 23 items were deleted, 13 items were rephrased, the points in six items were summarized to one item, and two new items were added. Our final version of the ACPSQ consists of 20 items. We hypothesized that the ACPSQ would consist of five domains: “accessibility”, “information”, “relationship”, “continuity of care” and “counselling”. All responses were measured on a 5-point Likert scale, ranging from strongly agree to strongly disagree (“Appendix 1 of Supplementary material”). One indicates the lowest satisfaction for the item and five indicates the highest satisfaction. Scores ranged from 20 to 100, and were converted to percentage.

We decided to develop the ACPSQ in English, as English is an important second language in Malaysia [19], and is taught to all school-going children as a second language. Additionally, the advantage of developing the instrument in English is that it can be administered to expatriates working in Malaysia who would more likely understand English than Malay.

Flesch reading ease

Flesch reading ease was calculated to assess the reading comprehension level of the ACPSQ. This was calculated based on the average number of syllables per word and words per sentence. The higher the score, the easier it is to understand the document. An average document should have a score between 60 and 70 [20]. A pilot test was conducted on 10 participants who collected their medications from outpatient pharmacy. They were asked to evaluate verbally if any of the items were difficult for them to understand.

Validation of the Ambulatory Care Patient Satisfaction Questionnaire (ACPSQ)

Study design and setting

This validation study was conducted from November 2015 to June 2016 at an urban tertiary hospital in Malaysia.

Participants

Included were patients/carers who were 21 years old and above, waiting to collect their long-term medication(s) (defined as medications prescribed for more than 6 months) from the hospital’s pharmacy, and able to communicate in English. Excluded were patients with cognitive impairment (dementia, psychosis or were mentally challenged).

Participants were divided into two groups to access discriminative validity: those collecting medications from the hospital’s outpatient pharmacy and from PharmCARE.

Outpatient pharmacy group

This group consists of patients or carers who were obtaining their long-term medications from the outpatient pharmacy. In our setting, the outpatient pharmacy receives about 2000 prescriptions a day, and dispenses about 8000 items a day. The waiting time for a prescription to be prepared can range from 30 min to 2 h [21].

PharmCARE group

This group consists of patients or carers who obtain their long-term medications from PharmCARE. PharmCARE is a subsidiary of outpatient pharmacy. It was specifically set up to decrease the waiting time for those collecting repeat medications. Patients who have been prescribed medications for more than two months can register for this service, once they have obtained their initial supply from outpatient pharmacy, provided they own a mobile phone. Medications are prepared on a mutually agreed date between PharmCARE and the patient via short messaging service (sms). Hence, the waiting time is shorter, in comparison with medication collection from the traditional outpatient pharmacy (range 5–15 min vs. 30–120 min) [22]. We hypothesized that the satisfaction score of patients in the PharmCare group will be higher than the outpatient pharmacy group, due to the shorter waiting time at PharmCare.

Sample size

Sample size was calculated based on a 1:10 participants per item ratio to perform factor analysis [23]. There are 20 items in the ACPSQ. Hence, the minimum number of participants required is 200.

Procedure

A researcher explained the study’s objectives to eligible participants using a patient information sheet. For those that agreed to participate, written informed consent and baseline information data were obtained. Participants answered the questionnaire themselves, which took about 10–15 min. The researcher then checked the questionnaire to ensure that all questions were answered. The ACPSQ was re-administered over the telephone 2 weeks later to assess for reliability.

Data analyses

All analyses were performed using the Statistical Package for Social Sciences version 22.0 (Chicago, Illinois, USA). Normality was assessed using the Kolmogrov-Smirnov test. Since data was not normally distributed, non-parametric tests were used. A p value < 0.05 was considered statistically significant.

Validity

Exploratory factor analysis was used as a data reduction technique to look into the dimensionality of the ACPSQ. The Bartlett test of sphericity, the Keiser-Meir-Olkin (KMO) test (> 0.7), anti-image correlation matrix coefficients (> 0.5), factor loadings (> 0.4) and the extent to which a variable correlates with all other variables (i.e. communality) were checked. Items which displayed low communalities (< 0.3) were excluded [24]. The principal axis factoring method was used since the items in the questionnaire were reflective indicators [25]. As items in the questionnaire could be inter-related, the promax (oblique) rotation method was used. Multiple criteria for determining the optimum number of factors to be extracted were used: Kaiser’s criteria (i.e. eigenvalue > 1), percent cumulative variance extracted and scree-plot test. Regression factor scores analysis was performed to assess the presence of outliers. All regression factor scores > ±3 were considered to be outliers. Ceiling and floor effects of each item (ideally should be < 15%) were also assessed. Discriminative validity was assessed by comparing the satisfaction score between participants collecting their long-term medications from outpatient pharmacy and PharmCARE, using the Mann–Whitney U test.

Reliability

Cronbach’s α was used to assess the internal consistency of all the items measured on the 5-point Likert scale, as well as for each domain. Cronbach α values more than 0.9 suggest redundancy of some items, values 0.70–0.90 imply adequate internal consistency, values 0.50–0.69 indicate poor internal consistency, and values below 0.50 indicate unacceptable internal consistency [26]. Corrected interclass item-total correlations were then used to identify items which did not agree well with other items in the questionnaire. Corrected item-total correlation values should exceed 0.2 to be considered as acceptable [26].

Test–retest reliability was assessed using Cohen’s kappa coefficient. Kappa values can range from − 1 to + 1. Negative values are observed when the agreement is less than that expected by chance, and + 1 shows complete agreement. Kappa values can be interpreted as follows: < 0 less than chance agreement, 0.01–0.20 slight agreement, 0.21–0.40 fair agreement, 0.41–0.60 moderate agreement, 0.61–0.80 substantial agreement and 0.81–1.00 almost perfect agreement [27].

Results

No problems were reported during the pilot study. Hence, no further changes were made to the ACPSQ. Flesch reading ease of the ACPSQ was 51.9.

A total of 220 participants were approached, of whom 200 participants agreed to participate: 114 from outpatient pharmacy, 86 from PharmCARE (response rate = 90.9%). Participants that collected their medications from PharmCare were significantly older and unemployed compared to those who collected medications from outpatient pharmacy (Table 1).

Table 1 Demographic characteristics of participants
Full size table

Validity

EFA initially showed that the ACPSQ was a 5-factor model, with acceptable sampling adequacy (KMO = 0.823, Barlett’s test of sphericity: X2 = 1859; df = 190; p value < 0.001). Anti-image correlation matrix coefficients and communality values were all > 0.5 and > 0.3, respectively. We then selected the number of factors to be rotated as 5 (Fig. 1), fixed the cut-off point for factor loadings as 0.4, and selected promax rotation. We did this reiteratively and removed three problematic items (items no. 9, 10 and 12) as their factor loadings were < 0.4. This model explained 60.0% of the total variance. When regression analysis was performed, three cases were removed and this model explained 60.6% of the total variance. The ceiling and floor effects of each item was < 15% except for items no. 17–20. Our final instrument was a 5-factor model with 17 items; with five domains: “information”, “accessibility”, “relationship”, “outcomes” and “continuity of care” (Table 2).

Fig. 1
figure 1

The number of constructs in the Ambulatory Care Patient Satisfaction Questionnaire. Extraction method: principal axis factoring; Rotation method: promax with kaiser normalization. Three problematic items (items no. 9, 10 and 12) were removed as the loading factor was < 0.4

Full size image
Table 2 The final construct of the Ambulatory Care Patient Satisfaction Questionnaire (with outliers removed, n = 197)
Full size table

Participants who collected their medications from PharmCARE (median = 78.0, IQR = 72.8–81.0) were significantly more satisfied with the services provided compared to those who collected their medications from the outpatient pharmacy (median = 72.0, IQR = 68.0–76.0, p < 0.001).

Reliability

Reliability analysis was performed on the remaining 18 items. The Cronbach’s α value for the ACPSQ was 0.839, whilst the Cronbach’s α value for each domain ranged from 0.580 to 0.890 (Table 3). All items had corrected item-total correlation values of > 0.2. Of the 200 participants, only 171(85.5%) were available at retest, as the remaining 29 participants were uncontactable. In the test–retest reliability analysis, kappa values ranged from 0.681 to 0.914 (p < 0.001).

Table 3 The psychometric properties of the Ambulatory Care Patient Satisfaction Questionnaire (n = 197)
Full size table

Discussion

The ACPSQ was designed to assess the satisfaction of patients towards an ambulatory care pharmacy service. It was developed using a systematic and rigorous process according to standard guidelines for developing questionnaires [28]. The final version of the ACPSQ consists of 17 Likert-type items with five domains measuring information (4 items), accessibility (4 items), relationship (4 items), outcomes (2 items) and continuity of care (3 items).

The Flesch reading ease score of the ACPSQ was 51.9, indicating that the ACPSQ was fairly difficulty to read, and was suitable for those who have completed the equivalent of 10th to 12th grade studies in the United States [28]. In our study, the majority (95%) of participants completed secondary school education (> 13 years of education). Additionally, when we performed the pilot study to assess for face and content validity, pilot participants did not encounter any problems in answering our instrument. Therefore, despite the lower than desired Flesch reading ease score, we feel that the ACPSQ is suitable to be administered in adults who understand English in Malaysia.

Initially, the ACPSQ had 20 items. However, three items had to be omitted as it did not satisfy the criteria of a good model-of-fit. The final ACPSQ consists of 17 Likert-type items that measures attitude towards the salient characteristics of pharmacists, such as the provision of information (“information)”, their accessibility to patients (“accessibility”), the trust fostered between pharmacists and patients (“relationship”), outcomes of the intervention provided (“outcomes”) and the continuity of care provided (“continuity of care”). The domains in our instrument were similar to previous tools that assessed satisfaction [2, 12, 14].

However, the ceiling and floor effects of items in the “information” domain exceeded 15%. The high ceiling effect of items 17–20 were due to the participants recruited from PharmCARE who were on long-term medications. They reported that their understanding of their current health condition (item 17), purpose of taking their medications (items 18), of how best to take medications (item 19) and the possible side effects of medications (item 20) as high, as they have been on the same medications for some time. The presence of ceiling and floor effects can influence the sensitivity and responsiveness of an instrument [29], and this has been included as a limitation of our instrument.

As hypothesized, the satisfaction score of participants who obtained their medication from PharmCARE was higher than those obtaining their medications form outpatient pharmacy. Our finding was not surprising as literature found that waiting time was significantly associated with patient satisfaction [30]. This indicates that the ACPSQ was able to discriminate between these two groups.

It was interesting to note that participants from PharmCARE were significantly older and unemployed compared to those recruited from the traditional outpatient pharmacy. In order to utilise PharmCARE services, one is required to visit another location that is approximately 800 m away from the outpatient pharmacy to register for this service, and patients who were working may have found this an inconvenience. Hence, the majority of patients that utilized PharmCARE services were patients that have retired. A search of published literature found that the relationship between satisfaction and age remains complex. Satisfaction scores peaked at 65 years, but declined thereafter, as the decline in satisfaction in older patients could be due to worsening health [29].

The overall Cronbach’s α value for the ACPSQ was 0.839, indicating adequate internal consistency. Each domain’s cronbach’s α was > 0.7 except for the continuity of care domain (0.580). A possible reason why this domain did not perform as well could be due to item 1 (“the pharmacy was located at a location convenient to you”. The deletion of this item would increase the Cronbach’s α slightly from 0.580 to 0.674. Hence, we decided to retain this item. At retest, kappa values ranged from 0.681 to 0.914, indicating that the ACPSQ has achieved stable reliability. The psychometric properties of the ACPSQ were similar to previous validated instruments that assessed satisfaction [12, 31].

One of the limitations of this study was that the ACPSQ was developed in English. Malaysia is a multiracial country where Malay and Mandarin are also widely spoken. Further studies to translate and validate the Malay and Mandarin versions of the ACPSQ are required before this instrument can be administered to those not fluent in English. The presence of ceiling effects was high for 4/14 items, which may influence the sensitivity and responsiveness of our instrument. Convergent validity was also not performed, as there were no validated instruments to assess satisfaction towards an ambulatory care pharmacy service when this study was conducted.

Conclusion

Although more extensive testing of the questionnaire in different languages is required, the present study has demonstrated the reliability and validity of the English version of the ACPSQ. Therefore, it can be used to measure patient satisfaction towards an ambulatory care pharmacy service, to improve the quality of pharmacy services delivered in Malaysia.