Introduction

According to a recent study by Shaw et al. we can expect the number of diabetes patients to rise over 350 million by the year 2030 [1]. Therefore, it is of high importance to act and identify persons at increased risk of developing Type 2 Diabetes (T2D) as early as possible. Considering the high cost and other negative implications, there are many initiatives on global and local level to prevent the rise of diabetes prevalence by promotion of healthier lifestyle. An important approach to effect changes in lifestyle of wider population includes screening of population to detect persons at risk. Multiple approaches to screening have been proposed, with self-assessment questionnaires aiming to warn users of potential risk for diabetes being just one of them. Using this approach, we can target the broader population and select the individuals with higher risk of diabetes for further examinations. A typical diabetes risk self-assessment questionnaires consists of a set of multiple questions related to demographics, life style and health status of a person. To develop a reliable diabetes risk estimation tool one needs to collect enough data that will represent multiple groups of persons differing in multiple characteristics that can be assessed using the above-mentioned questionnaires. In case of diabetes it is also important to record the biomarker value that is represented by a blood sugar level that can be measured in multiple ways.

With the advent of mobile devices such as smart phones or tablets, a large percentage of population now uses mobile health applications on a daily basis. By 2015, 500 million users will use mobile device applications for sport, diet and management of chronical diseases [2]. Even today, one can already find a large number of mobile applications aiming to help users with chronic diseases. For example, Donsa et al. [3] present how computerized decision support systems and the role of machine learning can help improve the personalization of the patient’s diabetes treatment on different levels. Hanauer et al. [4] developed Computerized Automated Reminder Diabetes System for blood glucose monitoring. They argue that using cell phone text messaging offers a highly portable, well-accepted, and inexpensive modality for engaging management of diabetes. Waki et al. [5] introduced one of the first smartphone based self-management applications for T2D patients. They developed an interactive system named DialBetics consisting of three modules that focused on researching the safety, usability, and impact of remote health-data monitoring on T2D patients.

A study by Garcia-Gomez et al. [6] demonstrates that using mobile health applications one can influence T2D prevention thanks to healthier lifestyle habits and paying attention to detailed preventable diabetes complications associated with this disease. Collins et al. [7] compare questionnaire based risk scores for T2D risk estimation. They also mention that early identification of patients with undiagnosed T2D or those at an increased risk of developing T2D is crucial and conclude that risk prediction models are one way of identifying this group of people.

Gray et al. went a step further and focused exclusively on smart mobile phones based T2D risk calculators [8]. They emphasize that smart mobile phone applications allow people to assess their risk of developing T2D on their own and, more importantly, learn about how they can reduce their risk. In contrast to our study, Grey et al. compared only 7 mobile applications for assessing the risk of T2D using running on two mobile operating systems (Android and iOS).

This paper focuses on a review of currently available mobile applications for T2D risk estimation. At the same time, we try to answer the following research question:

  • How much information on the scientific background of the risk calculator used in a mobile application is available to a mobile application user?

Kollman et al. [9] used mobile phone to collect data from patients with type 1 diabetes mellitus. They show that using the mobile phone as patient terminal provides a ubiquitous, easy-to-use, and cost efficient solution for patient-centered data acquisition in the management of type 1 diabetes mellitus. Gaggioli et al. [10] were using their own mobile smart phone application to collect psychological, physiological, and activity information about mental health with mobile phone and additional sensors (wireless electrocardiogram and accelerometer). Their success of collecting and analyzing data was surprisingly high. For example, participants filled 214 reports (98 %), of which 197 were included in the analysis (90 %). A total of 220 ECG sampling were recorded (100 %), and 205 were included in the analysis (93 %). Pfaeffli et al. [11] were measuring physical activity in a cardiac rehabilitation population using a smartphone-based questionnaire (Mobile physical activity level questionnaire and International physical activity questionnaire). Their success in collecting data was 83 % with 30 out of 36 potential participants completing the study. Min et al. [12] present a study where data collecting was made using their own mobile smart phone application that resulted in 45 % (1215 out of 2000 notifications) response rate in daily self-reporting sleep-disturbance in breast cancer patients receiving chemotherapy. Faurholt-Jepsen et al. [13] uses MONARCA (MONitoring, treAtment and pRediCtion of bipolAr disorder episodes) application on Android smart phones to collect data and monitor activities of bipolar disorder. The adherence rate for self-assessments in the MONARCA application was 88 % and the collection of clinical ratings were complete (100 %). Smartphone and mobile applications therefore represent a practical opportunity to explore new modalities of monitoring, treatment, and research of psychiatric and mental health conditions.

Although they are usually based on a set of questions similar to examples described above, the risk calculator mobile applications differ from the typical questionnaires where they mainly aim to collect data for later analysis by researchers. Risk calculators use answers to questions in the mobile application to estimate the risk for an outcome of interest in real-time. Therefore it is highly important to know which risk estimation test is used by a specific mobile application. In T2D risk estimation there are multiple tests that were proposed by different studies and have been in use by the most respected diabetes associations in the world. Here, we describe some of the well-known tests. Most of them were developed to be used as paper and pen questionnaires, but have been widely adopted and converted to web and mobile applications recently.

American diabetes association T2D risk test (ADA)

This, widely used, online calculator is available from the official American Diabetes Association website and is based on the slightly adapted methodology that was published in a study by Bang et al. [14]. It is based on one of the simplest questionnaires with 7 questions where a user can score up to 11 points. The threshold for people at risk is at 5 points, instructing all persons who scored above this threshold that they are at increased risk for having T2D. Users at risk are further advised to see their doctor and check if additional testing is needed. The model was developed using data from NHANES [15], Atheroscleriosis Risk in Communities (ARIC) [16] and Cardiovascular Health Study (CHS) [17]. The final model by Bang et al. yielded an Area Under the ROC curve (AUC) of 0.83 on NHANES and 0.74 on ARIC/CHS for diabetes risk estimation and 0.72 for pre-diabetes [14].

Canadian diabetes risk questionnaire (CANRISK)

The online version of the CANRISK calculator is used by the Canadian Diabetes Association. It includes 13 questions and it takes more effort for the user to answer all questions in comparison to ADA questionnaire. In contrast to ADA, CANRISK uses two threshold values and stratifies persons into three categories of having pre-diabetes or T2D: low risk (cumulative score of less than 21), moderate risk (21–32) and high risk (33 and over) with maximal score of 86. In case of moderate risk, users are advised to consult with health care practitioner about their risk of developing diabetes. For the high-risk group, the questionnaire suggests to consult with a health care practitioner to discuss getting their blood sugar tested. CANRISK is based on the Finish Diabetes Risk model (FINDRISC) with adaptations to reflect Canada’s multi-ethnic population [18]. The CANRISK validation study by Robinson et al. [19] provides a regression model coefficients that can be used for “programmed risk calculators (e.g. iPad App, online web calculator)” and an additional “paper-based” format. Robinson et al. validated CANRISK and eCANRISK on 6223 adults of various ethnicities and obtained the same AUC scores for both versions (0.75, 95 % CI: 0.73–0.78).

Australian type 2 diabetes risk assessment tool (AUSDRISK)

In comparison to the first two online risk assessment tools that focus on identification of persons with high risk of diabetes or pre-diabetes, AUSDRISK focuses on assessment of risk of developing T2D over the next 5 years. The online version of the questionnaire is available at the Diabetes Australia website. Data from AusDiab - Australian Diabetes, Obesity and Lifestyle study (1999–2000) [20] with a 5-year follow up (2004–2005) was used to develop this risk assessment tool [21]. In the 5-year period 362 people out of 6060 from AusDiab study developed diabetes. Data from 1993 participants of the Blue Mountains Eye Study (BMES) [22] and 1465 participants from the North West Adelaide Health Study (NWAHS) [23] was used to validate the risk assessment tool. The AUC of AUSDRISK was 0.78 (95 % CI: 0.76-0.81) using a score threshold of 12 out of the maximal 35 points.

Finish diabetes risk model (FINDRISC)

Using data from a 10-year prospective study on the incidence of T2D in a population-based cohort, the Finnish diabetes risk score (FINDRISC) was developed to identify subjects at high risk for the future occurrence of T2D [24]. Multivariate logistic regression model coefficients were used to assign partial scores used to compute the overall FINDRISC score. FINRISC includes 8 questions that were found significant in a population of 4435 subjects with 182 incident cases of diabetes. The original FINDRISC study reported sensitivity of 0.78 and 0.81, specificity of 0.77 and 0.76, and positive predictive value of 0.13 and 0.05 in the 1987 and 1992 cohorts, respectively.

QDiabetes risk model

Hippisley-Cox et al. [25] used a large cohort of patients aged 25 to 79 years from 355 general practices in England and Wales to build a diabetes risk model for estimating 10-year risk of acquiring T2D. The QDiabetes model was validated on 176 separate general practices with high AUC scores of 0.85 (95 % CI: 0.85-0.86) for women and 0.83 (95 % CI: 0.83-0.84) for men. Inclusion of large cohorts of patients (2.5 million for derivation and 1.2 million for validation of the model) is demonstrated in narrow confidence intervals and high discrimination scores. Therefore QDScore represents one of the most reliable T2D risk calculators available today.

Methods

Our review of T2D mobile applications was conducted following a Preferred Reporting Items for Systematic reviews and Meta-Analyses (PRISMA) proposed by Moher et al. [26]. PRISMA is an evidence-based minimum set of items aimed at helping authors to report a wide array of systematic reviews and meta-analyses that assess the benefits and harms of a health care intervention. PRISMA focuses on ways in which authors can ensure a transparent and complete reporting of this type of research. Initially it has been used for systematic reviews in healthcare, however nowadays it has been applied also on other types of research and other domains. PRISMA has been recently used in the review of the healthcare mobile applications domain as well [27].

Operating systems

The selection of operating systems used in this study was based on their share in the mobile application market. The four most common smartphone operating systems, by market share in the third quarter of 2013 were Android by Google, with a distinct dominance at 81 %, iOS, on Apple’s iPhone, at 12.9 % Windows Phone by Microsoft at 3.6 % and BlackBerry at 1.7 % [28]. Although, this study does not explicitly focus on smart phones, but also includes other mobile devices like tablets, we believe the above mentioned data supports our decision to use the three major mobile device operating systems at the moment (Android, iOS and Windows Phone).

Search criteria

Three experts in the field of healthcare related mobile applications were involved in the review of the mobile applications. Initially, a set of search keywords that were used to search for T2D risk estimation applications was defined. The selected search terms were “diabetes”, “risk” and “health” that were used in combination with keywords “test”, “calculator”, “checker”, “tool”, and “score”. The initial step that resulted in more than 1500 mobile applications was followed by manual inspection of the application title and description where needed. After this step, each of the three reviewers presented a list of resulting applications at the review meeting. The inclusion of the applications that were identified by only one or two reviewers was discussed and followed by a final decision by all three experts. The inter-rater agreement based on Cohen’s Kappa statistic [29] between the reviewers ranged between 0.77 (reviewer 1 vs. reviewer 3) and 0.90 (reviewer 1 vs. reviewer 2).

An additional criteria for exclusion was language of the application. Only applications in English language were included in the final review. The applications for T2D risk test in German (one for Android and one for iOS), Spanish (two for Android) and Chinese language (one for Android and one for iOS) were therefore excluded from the review.

We also excluded applications based on a technical exclusion criteria that was composed of the following components:

  • We could not run the application (two Android and one iOS application),

  • We were not able to log into the application (one Android and one Windows Phone application),

  • After the installation we found out that the applications’ functionality and the description do not match (one Android and one Windows Phone application),

  • An application was location specific (one Android application).

Results

The results of this study are based on the search in three major mobile application stores that was performed in January 2015. We found 31 (16 Android OS; 8 iOS and 7 Windows Phone) (Table 1) eligible applications for T2D risk estimation that met all inclusion criteria. Together we compared 25 freely available and 6 applications where payment was required to download the application (one from Google Play Store, two from iTunes Store and three from Windows Phone Store; range from €0.80 to €2.00; total €7.66).

Table 1 T2D risk application information
Full size table

Figure 1 presents a comparison of Android, iOS and Windows Phone risk estimation applications using Preferred Reporting Items for Systematic reviews and Meta-Analyses (PRISMA) diagram.

Fig. 1
figure 1

PRISMA flow diagram

Full size image

Table 1 presents basic information for all applications included in this study. An additional list of reviewed applications including the web links is available as supplemental information. To obtain the final set of applications, a comparison between Android, iOS and Windows Phone applications was performed to find duplicate applications implemented for multiple mobile operating systems. Six applications were found that appeared in two or three mobile operating systems and were therefore available from two or three mobile application stores (Fig. 1).

All application searches were executed on a personal computer using the following web browsers:

  • Chrome version 39.0.2171.99 (Android and Windows Phone)

  • Safari 8.0.3 version 10600.3.18 (iOS)

For testing applications we used the following mobile devices and operating systems:

  • LG Nexus 5 (Android Lollipop 5.0.2)

  • iPhone 5S (for iOS 8.1.1)

  • Nokia Lumia 1320 (Windows Phone 8.1)

  • Asus Nexus 7 2013 (Android Lollipop 5.0.2)

  • iPad mini 1st gen (iOS 8.1.1)

  • HP ElitePad 1000 G2 (Windows Phone 8.1)

All iOS applications are compatible with iPhone and iPad, where application Findrisc (I4) only runs on iPad.

Additionally, Table 1 presents information on risk score method along with prognosis period and cost of the application. Prognosis period depends on the data that was used to build the model. Most models are used to predict the risk of T2D at current point in time, 5 years in advance or even 10 years in advance. Although it is sometimes known what prognosis period corresponds to a specific risk score method (e.g. ADA test predicts current risk of developing T2D), we did not report it in the table if it was not explicitly stated in the application.

Holzinger et al. [30] stress the importance of interoperability and usability of the application on multiple different mobile platforms. We found out that only applications Diabetes risk checker (A5, I3 and W4) appear in all three mobile stores, applications Diabetes Risk Score (A6 and I6), Diabetes risk (A3 and I7) as well as Diabetes test (A7 and I5) appear in Google Play Store and iTunes. Applications Meditas (A12 and W2) as well as Ausdrisk (A14 and W6) represent the same applications and can be found in Google Play Store and Windows Phone Store.

Most probably, the reason for smaller number of applications that can run on different mobile platforms lies in the fact that developers of such applications need to know a lot of different software development kits, programming languages, libraries and different ways to design user interfaces [30]. For example, Diabetes risk checker developers used three programming languages (Java for Android OS; Objective-C for iOS; C# for Windows Phone) to develop the application for different platforms.

Out of 24 applications (excluding duplicates), we found that 15 (63 %) applications did not mention which T2D risk score method they were using, four applications were using FINDRISC T2D risk test followed by AUSDRISK, ClinRisk qdiabetes, CANRISK, ADA and QDiabetes that were used only once.

When closely inspecting the applications, one can also observe that questions asked to obtain a T2D risk estimation differ to a great extent. As an example of large variance in implementation of different risk estimation apps, we examine the input method for one of the most basic questions asked. Table 2 summarizes information on one of the simplest and most frequent questions asked – i.e. age of the user. It can be observed that 11 applications relied on numerical (mostly integer) value, although even here the method of input differs. Other applications used different pre-defined intervals that differ significantly, especially in the lower age groups. In general most applications recognized the upper age threshold at the age of 65 years. Two applications did not include a question on age.

Table 2 Different age intervals used in 29 applications
Full size table

Table 3 presents questions from all applications that represent 28 risk predictors used in different applications. Our results can be compared with a study by Collins et al. [7] where they analyzed 43 risk prediction models. Both studies point to the very similar set of the most frequent risk predictors in age, weight, hypertension, family history of diabetes and waist circumference. There is some difference in physical activity that was identified in 24 (77 %) mobile applications vs. 8 (19 %) predictive models analyzed by Collins et al. Similarly, mobile applications used height (77 % vs. 12 %) and sex (97 % vs. 40 %) more frequently compared to predictive models described in the literature. Six risk predictors appeared only once and five predictors appeared only twice in T2D risk applications.

Table 3 List of most frequent predictors based on questionnaires used in T2D risk estimation mobile applications
Full size table

Discussion

The results show that more than half of mobile T2D risk estimation applications do not include information on the empirical scientific background of the risk estimation test used in the application. To some extent it is difficult to believe that some of the applications are widely used despite the fact that users do not get any information on the underlying risk estimation model. Additionally, in many cases where we were able to identify which risk estimation test an application was using, there was no information or link to the scientific paper that would offer more information for the user. As a consequence, even in cases where the name of the prediction model is disclosed, users need to find the scientific paper describing the method on their own.

It also has to be noted that a population using mobile T2D risk estimation calculators usually includes people under 50. The average level of computer literacy of older people (65–70 and above) is considerably lower than in younger generations and this also influences the late adoption of new technology in this age group [31, 32]. On the other hand a study by Pfaeffli et al. demonstrated that their sample of middle-aged to older adults were able to use the smartphone applications without any serious limitations [11]. From this point of view, developers of such applications should put a great emphasis on careful design of such applications (visualizations, font size, button size, etc.).

As reported in a study from 2011 by Liu et al. [33] the majority of mobile applications in healthcare sector were offered on Apple App Store. However, Peischl et al. [34] observe that Android applications might overtake the current market leader iOS by 2015 in terms of market penetration in the healthcare sector. In our study we already show significant difference (16 android vs. 8 iOS) in healthcare applications for T2D risk estimation in favor of Android operating system.

Different studies presenting novel T2D risk calculators were using various methods to collect data. The most widely used methods were based on paper and pen questionnaires [35–37], web-based tools [38, 39], telephones surveys [40–42] and computerized tools [40, 43] for data collection. Despite our efforts to find information in the literature or from the mobile applications included in this study, we were not able to identify cases where data collected from mobile phones would be used to improve the performance of risk estimation tests or offer additional descriptive analysis of collected data to users.

Conclusion

We see opportunity and great potential for mobile applications in different fields of healthcare, because if their user friendly characteristics, especially in younger generations that grow up with mobile devices. As already stated in discussion, we believe that in the future we should focus on collecting and re-using the collected data to improve mobile applications. Unfortunately this is still not the case in mobile applications nowadays.

There are still many open issues in development of T2D risk estimation applications like methods and formats of input to create a user friendly and accurate application. For example, for most users it is much easier to provide their height and weight instead of their BMI directly. However, the same measurements can be problematic if metric units are used when a user expects imperial units or vice versa.

This study points at higher availability of T2D risk estimation applications for Android users in comparison to iOS and Windows Phone users. However, this does not mean that number of available applications necessarily means better applications per se. Based on criteria used in this study, we could recommend an application like AUSDRISK (A14 and W6) as it uses a validated score (The Australian T2D Diabetes Risk Assessment Tool), offers information about population for which this score was validated and represent results for T2D risk in three forms (text, numerical and graphical). On the other side this application lacks some novel graphical approaches to make it more user friendly when compared to similar applications.

Evaluation of user interface and user-friendliness of T2D risk estimation applications could be one of the directions for our future work. Different studies stress the importance of user interface design in healthcare mobile applications where different populations of users can be met [44]. Similarly, studies by Kalz et al. [45], Gray et al. [8] and Peischl et al. [34] all recommend thorough testing and inclusion of domain experts in the development of mobile health applications. As recently stated by Ehrenfeld [46], by the rising complexity of medical systems, the challenges of ensuring their security, efficiency, and reliability also rise. This fact does not hold only for clinical medical systems, but should be taken into account by medical domain experts and mobile application developers during the development of the next generation of healthcare mobile applications as well.