Introduction

Patients are confronted with complex information regarding their illness, treatment decisions or alternative options, and their side effects (Maddock et al. 2011). The patients as well as their caregivers need trustworthy online information in coping with their cancer. Yet, information is often difficult to understand and patients do not always have the possibility to directly address the physician in charge. Accordingly, they need other available and trustworthy sources of information.

Moreover, patients are asked to participate in joint decision-making and to take on responsibilities. For this, the need for sufficient information and a thorough understanding are fundamental (Liebl et al. 2015). Already today, with longer duration of the disease and cancer becoming more and more a chronic disease, many patients become experts in managing their illness by accessing online information and arranging their own care with the help of clinicians (Maddock et al. 2011).

Mobile Health (mHealth) is an innovative and growing field, which is increasingly being used for patient care. With the help of mobile devices like smart phones and tablets and with apps specifically developed for mobile devices, cancer patients can get the information fast and at any time. This information includes: prevention, diagnostics, treatments, and therapy options.

While the authors of the study “Chancen und Risiken von Gesundheits-Apps” (Chances and Risks of Health Apps) conclude that health apps have the potential to effectively support the self-management of chronically ill people and to enlarge the treatment adherence, much information is false and/or not actual (Collado-Borrell et al. 2016). In fact, there is a need to enforce the transparency and implement an independent review of mHealth applications in oncology (Brouard et al. 2016). Yet, the CHARISMHA study reported that there is still no comprehensive and valid guidance for assessing the trustworthiness of health apps (Albrecht 2016).

When looking for trustworthy health information on the Internet, quality seals can help to assess the quality of information. Afgis (2013) and HONcode (2017) are institutions committed to the certification of health websites. They also promote the development and dissemination of quality standards for health information in other areas, such as health apps.

Other tools for the evaluation of patient information exist as the DISCERN (2009) instrument which may be used by the patient visiting a website. Moreover, organisations as the German Network for Evidence Based Medicine have published recommendations how to convey high-quality information for lay people in medical care (Steckelberg et al. 2005).

For patients with cancer it is not always easy to identify valuable apps. The search is tedious because very few cancer apps are hidden in long hit lists. Most apps are in foreign languages, without reference to cancer, or exclusively for use by medical professionals (HealthOn 2017). Transparent information about purpose, content, and scientific validation of the app is required. Accordingly, the aim of the present study was to gain data on the quality of apps for cancer patients using a standardised instrument.

Materials and methods

Description of the standardized questionnaire

In a first step we searched the literature for a validated instrument to assess medical apps. We only found one instrument: “Mobile app rating scale: a new tool for assessing the quality of health mobile apps” (MARS) (Stoyanov et al. 2015). This questionnaire provides a multidimensional and reliable app quality rating scale.

The parameters of the questionnaire are as follows:

  • Interactivity of the app, individualization and degree of involvement of the user and his social environment (engagement, 5 questions).

  • Functionality of the app, handling and ease of use (functionality, 4 questions).

  • Attractive presentation of the app, visual imagery that is likable and offers an intuitive handling (aesthetics, 3 questions).

  • Quality and presentation of information, credibility of the sources (information, 7 questions).

  • Assessment of the subjective quality (recommendations, etc. four questions).

  • Subjectively felt effect of the app concerning knowledge, attitude, intention of the user, etc. (6 questions).

MARS was developed for the evaluation of health apps in general (Stoyanov et al. 2015). As it did not perfectly meet our requirements, due to the lack of items evaluating quality of content, we decided to add items from other instruments. Criteria from MARS (Stoyanov et al. 2015) that were irrelevant for cancer patients were removed.

In order to assess the quality of websites on cancer, the working group Prevention and Integrative Oncology of the German Cancer Society (GCS) developed an instrument that comprises formal and content-related criteria (Liebl et al. 2015). The scientific basis for this instrument was criteria for evidence-based patient information from other instruments or manuals:

  • Afgis criteria (2013).

  • HONcode (2017).

  • DISCERN (2009).

  • Criteria for evidence-based patient information (Steckelberg et al. 2005).

  • Manual patient information: recommendations for the creation of evidence-based patient information (Pubmed/Medline (NLM) 12. Ärztliches Zentrum für Qualität in der Medizin (ÄZQ) 2006).

In order to combine MARS with the GCS instrument, criteria from MARS that focused on gamification and entertainment were removed (gamification, strategies to increase engagement, performance, graphics, and visual appeal). Furthermore, for the rating by scientists we removed the rating on subjective quality. Finally, we removed some items concerning likeliness to change attitudes, motivation or behaviour as evidence-based patient information aims at the provision of information and not at directing the patient in a certain direction.

Formal and content-related criteria from the GCS instrument applicable to apps were added. Especially the items in section “information” were specified and expanded.

We added following subscales:

  • Visual appeal.

  • Purpose of the information.

  • Separation of content and commercials.

  • Complementarity.

  • Requirements with regards to information.

  • No statement to sections without safe information.

  • Detailed information on therapy methods.

  • Statements refer to patient issues.

  • Additional material (material for the doctor–patient communication).

  • Use of a language that enhances participation.

  • Information on sources.

  • Data protection.

  • Findings about the presentation of numbers and results are being considered.

The questionnaire is divided into three sections (engagement, aesthetics, and information) and includes 22 aspects to be assessed:

  • The first section includes the subscales interactivity and target group: does it allow user input, provide feedback, contain prompts, and is the app content appropriate for the target audience?

  • Layout and visual appeal are the subscales of the second category.

  • Most of the additions relate to the third section, for example purpose of the information, requirements with regard to information, detailed information on therapy methods and data protection.

The detailed complete contents of the questionnaire are shown in Table 1.

Table 1 Categories and subscales of the questionnaire
Full size table

In a pilot study, we evaluated ten mobile apps in the field of oncology concerning their quality to assess the feasibility of assessing apps with the new instrument (Böhme et al. 2017).

Selection of the mobile applications and data collection

A comprehensive search was started in the Apple App Store between April 6, 2017 and April 17, 2017. The search was performed in the category “Medicine” available in the store.

We decided to include apps that were dedicated to the most common types of cancer for women (breast cancer), men (prostate cancer), and both genders (colorectal cancer). Furthermore, we only selected apps which were free of charge.

Assessment of the apps

The selected apps were assessed by three people (two doctors, one computer scientist) using a three-step Liker scale:

1 = the criteria were fully met

2 = the criteria were partly met

3 = the criteria were not met or only met insufficiently

These scientists did not get to know the answers of the colleagues before they finished their rating.

The results of the three authors were summarised and added up as arithmetic mean. The resulting means were coded as quality according to the following list:

  • arithmetic mean 1.00–1.50 means “very good”,

  • arithmetic mean 1.51–2.00 means “good”,

  • arithmetic mean 2.01–2.50 means “deficient”,

  • arithmetic mean 2.51–3.00 means “insufficient”.

In addition to evaluating the apps through the developed questionnaire, we also assessed the following features:

  • target group (general population, patients or healthcare professionals),

  • content (teaching and information, screening, patient support, congress, or glossary),

  • including advertising or no advertising.

Results

Description of cancer-related applications

In the Apple App Store a total of 203 mobile applications dedicated to breast cancer (n = 100), prostate cancer (n = 83), and colorectal cancer (n = 20) were identified. Out of these 203 apps a total of 115 apps were free of charge, of which 61 apps were in the category “medicine”. Five apps were doublets and two apps were exclusively in French, leaving 54 apps for assessment.

N = 13 mobile applications could not be rated for the following reasons:

  • lack of availability (n = 2),

  • technical error (n = 6),

  • limited access (n = 5).

Therefore, 41 apps were left for evaluation by the questionnaire. This selection process is shown in Fig. 1.

Fig. 1
figure 1

Selection process of breast cancer, prostate cancer, and colorectal cancer applications

Full size image

Considering the final 41 apps, the target groups were equally divided with a third addressing patients, health people or healthcare professionals, respectively (Table 1). With respect to the content most apps were focused on teaching and information (76%, n = 31), and nearly 90% (n = 36) included no advertising (Fig. 2).

Fig. 2
figure 2

The characteristics of the apps on breast, prostate and colorectal cancer

Full size image

Evaluation of the apps using the developed questionnaire

The assessment of any chosen apps by the three independent scientists led to homogenous results. The results are presented in Table 2.

Table 2 Classification of apps according to the arithmetic mean, target group, content, and advertising
Full size table

Six apps (14.6%) scored very high, 15 apps (36.6%) had good results, 17 apps (41.5%) were evaluated as deficient, and 3 apps (7.3%) were rated as insufficient. The resulting arithmetic mean of all apps is 1.956 (Table 3).

Table 3 Evaluation of the 18 subscales of category “Information” for the three target groups
Full size table

All apps in the group “very good” to “good” were from reliable sources, had a concrete intent/purpose in their app description, and a strict distinction of scientific content and advertisement.

Apps with the predicates “deficient” or “insufficient” had particularly poor ratings in the following subscales of the questionnaire:

  • Requirements with respect to information.

  • Statement to sections without safe information.

  • Detailed information on therapy methods.

  • Information on sources.

  • Data protection.

Quality of the apps and target group

Considering the quality of the apps for the different target groups (Fig. 3), the ranking for apps that were dedicated to the target group “patients” were better than the overall average distribution with 4 apps (26.7%) being very good and 8 apps (53.3%) good. No app was insufficient. In contrast, apps of the general population had a much lower scoring with 7 apps (50%) being deficient, and 3 apps (21.4%) being insufficient.

Fig. 3
figure 3

Quality of apps for the target groups

Full size image

The results for apps for the target group “healthcare professionals” were heterogeneous with only 1 (8.3%) very good app and 7 (58.3%) deficient apps.

In a subsequent step we assigned the 18 subscales of category “Information” to the three target groups.

For apps for all three target groups the results in subscales “Accuracy of app description”, “Purpose of the information”, and “Separation of content and commercials” are almost homogenous.

The apps for the target group healthcare professionals received good scorings in the subscales “Requirements with regards to information”, “ No statement to sections without safe information”, “ Information on sources: references”, and “Conclusions of results are included”. Those for the target groups general population and patients only received deficient or insufficient scores for the same subscales.

Apps for all three target groups received only deficient or insufficient scores in the subscales “Complementarity” and “Data protection”.

Discussion

In this study, we rated a total of 41 cancer apps dedicated to cancer types “breast cancer”, “prostate cancer”, and “colorectal cancer” using a questionnaire consisting of contend-related and formal criteria. The evaluations by three independent scientists were homogeneous. The questionnaire specifically developed for the rating of cancer apps proved to be very suitable (Böhme et al. 2017). Yet, the reliability of the questionnaire was not tested, which is one limitation of our study.

The results of the present study show that half of the apps are good to very good (14.6% very good, 36.59% good). Only 7% were insufficient.

“Cancer genetics”, an app which provides clinicians with streamlined risk assessment and referral guidance for hereditary cancer, has been awarded MHRA certification as a Class 1 medical device (Medicines and Healthcare products Regulatory Agency 2017). Licences for medicines are granted only when a product meets high standards of safety and quality and works for the purpose intended. In our rating the arithmetic mean was 1.525, which was significantly better than the resulting arithmetic mean of all apps of 1.956. For this app criteria were fully met for 11 out of 22 subscales, e.g., requirements with regard to information, statements refer to patient issues, credibility, evidence base.

The app “Breast cancer: Beyond the Shock” had the best result in the study and was developed by the National Breast Cancer Foundation, a self-help group. The German Cancer Society also came to similar conclusions when rating websites using the same formal and content criteria. In their study the best scores were achieved among others by three self-help groups (Liebl et al. 2015). Many cancer-related self-help groups were already assessed on the basis of formal criteria and received a quality seal (Afgis 2017).

In contrast in our pilot study using apps from the Google Store, 40% were rated as insufficient (Böhme et al. 2017). Due to the low number of apps in the pilot study, these data must be considered with caution. Yet, health care professionals should know that apps offered in the two stores are different and that there might be gaps in quality. Moreover, there was a difference between both Stores with respect to advertisement. The apps with poor to insufficient rating had no distinction between scientific content and advertisement.

Particularly noteworthy is that the apps offer little to no support in shared decision making for both patients and the general population. In fact the subscale “Complementarity” is almost always rated low. Yet, this feature is not easy to realise in the concise form of an app as much information must be presented and the user should be encouraged to weigh different arguments. One solution is used in apps for healthcare professionals, which offers scientific sources in most cases and includes conclusions of results.

Another deficit is the low ranking for “Data protection”. With increasing complexity of apps, more and more data on the user will be entered to increase tailoring and interactivity and data protection will become a highly relevant topic.

For the usability for patient’s quality criteria the offering of cancer apps is rather unclear due to the size and dynamic development. Inadequate app descriptions without explicit details on content and functionality aggravate searching for suitable and high-quality cancer apps. Currently there are no tools for cancer patients available to facilitate targeted searching of high-quality cancer apps.

Future efforts should focus on the development of guidelines that make it easier for patients to search for suitable cancer apps. Regulation is required in order to prevent these tools from becoming a safety problem instead of an aid for cancer patients.