Introduction

Many freely available and subscription mobile ophthalmology applications (MOAs) have been introduced to the marketplace in recent years. Among the approximately 130 MOAs available, 32% were for vision testing, 13% for eye relaxation exercises, 12% for professional education, and the remainder offered a range of functions, including vision aids and patient education [1]. MOA access was split between Apple iOS and Google Android devices, with only 3% available on both devices [1].

While MOAs may be utilized by users in various settings, three areas are of particular interest. First, eye care providers (ECPs) may be asked to examine patients in settings where such assessments may be logistically or financially limited. For example, inpatient wards, intensive care units, and rural healthcare clinics lacking formal ophthalmology testing equipment are situations wherein ECPs may use MOAs as efficient and economical means for assessing visual acuity to guide clinical decision-making, including evaluating the severity of vision impairment, screening large populations, and determining a need for referral for TVAM and other required interventions. Second, other healthcare providers (HCPs), especially those lacking access to formal vision testing equipment, may effectively utilize MOAs to triage the level of vision impairment and the need for ophthalmology referral. ECPs and HCPs alike may utilize MOAs for chronic care management and interdisciplinary care coordination of chronic ocular pathologies. Finally, patients with chronic ophthalmic disease under the care of an ECP may use MOAs to self-assess visual acuity and determine the need or urgency for follow-up appointments. The latter two categories may particularly benefit both developed and developing regions, especially when combined with the growing utilization of telemedicine.

Among currently available MOAs, Eye Handbook (EHB) (Cloud Nine Development, Overland Park, KS, USA) is a free MOA available on iOS and Android; it is one of the oldest, most popular, and most frequently downloaded applications in the marketplace [2,3,4]. EHB is a comprehensive diagnostic and treatment reference with various features, including an eye atlas, patient education information, forum discussion boards, journal portals, coding tools, and visual function testing tools [5]. Among these, several visual acuity and vision assessment tools for distance visual acuity (DVA), near visual acuity (NVA), colour vision testing (CVT), contrast sensitivity (CS), and pupillary distance (PD) can provide users with objective measurements of visual function.

However, before broad implementation or recommendations, the accuracy of MOAs should be studied. Clinical decisions based on inaccurate or untested MOAs could lead to erroneous diagnoses, avoidable office visits, increased patient anxiety, and/or inappropriate and unnecessary interventions. Of 42 similar recently reviewed MOAs, only one included validity data [6]. Despite its longevity and popularity, few studies have comprehensively assessed the accuracy of EHB’s visual assessment functions. In a recent study involving 200 eyes from 100 participants, near visual acuity (NVA) had an average of 0.11 logMAR decrease compared to traditional NVA testing, indicating an overestimation with EHB [7]. A separate study of 100 male patients showed no statistical difference in CVT between EHB and TVAMs [8].

To our knowledge, no study has been undertaken to measure and assess multiple visual function parameters using EHB. We sought to study five parameters (DVA, NVA, CVT, CS, and PD) using EHB and compare them with TVAMs in the ophthalmology clinic setting to better evaluate this MOA’s testing accuracy.

Methods

Ethics

This prospective crossover clinical trial study adhered to the tenets of the Declaration of Helsinki. Ethics approval was obtained from the study institution’s Institutional Review Board (IRB: 14348), and HIPAA regulations were followed.

Patient eligibility

Patients seen at the study institution for routine ophthalmological care were approached as potential participants. Patients were excluded if they were non-English speakers, less than 18 years of age, illiterate in English, received ocular surgery within seven days of enrolment, presented with acute ocular injury, or could not comply with testing instructions. Patients had various ophthalmic conditions, including glaucoma, myopia, cataracts, pseudophakia, repaired retinal detachment, diabetic retinopathy, retinitis pigmentosa, and macular degeneration. One eye was tested from each patient. The last digit of the patient identification number randomly determined the tested eye. The right eye was tested for even numbers, and the left eye for odd numbers.

Visual function assessments

Five tests for visual function were assessed using the EHB app provided on a tablet (sixth generation iPad; Apple Inc., Cupertino, CA, USA) and TVAMs for a total of ten measurements. The tablet was kept at maximum brightness throughout the testing. TVAM was consistently performed between 550 and 650 lux (lx) as measured by a handheld light metre (Urceri MT-912; Urceri Corporation, United Kingdom). All participants wore prescription glasses, contact lenses, or reading glasses if refractive errors were present. Patients were instructed to occlude the non-tested eye with a single-end occluder.

DVA was first tested using 10 feet as the distance measured by a tape measure. Patients were asked to read the lowest line they could read on the TV monitor-based Snellen vision test at 10 feet for this test. After recording the lowest line read without a mistake, patients were then asked to read the lowest line on the MOA using the 10-foot testing feature. The lowest successfully read line was recorded each time, providing two separate DVAs.

Similar techniques were used for NVA testing. Patients with the non-tested eye occluded read the lowest line possible on a Rosenbaum near vision card held 14 inches away. Next, patients read the lowest line on the MOA using the near vision setting at 14 inches. The examiner measured the distance with a tape measure and monitored it throughout the testing.

CVT was performed next. Patients were asked to read the numbers on Ishihara HRR plates using the tested eye. The number of correctly identified numbers was recorded. Next, patients were asked similarly to identify numbers using the colour vision function of the MOA. Both techniques were done at a distance of 14 inches from the examinee, and both methods had a maximum score of 12 if all numbers were identified.

Fourth, CS was measured using the Pelli-Robson contrast sensitivity chart at 20 feet and the contrast setting on the application at 14 inches. The difference in distance was to comply with the recommended distance for the two testing methods. In both methods, each patient was asked to read as many letters as possible, and the number of letters read successfully was recorded.

Finally, PD was measured first using a standard millimetre ruler with a Finnoff transilluminator and then with the MOA’s PD function, which uses the tablet’s camera.

Statistical analysis

For power testing, MOA and TVAM methods were considered equivalent in logMAR measures if they were within one line (0.1 logMAR) on the eye chart. We also assumed the same standard deviation (0.2) of logMAR measures between conventional and mobile application methods with a correlation coefficient ranging from 0.2 to 0.8. Under these assumptions, we determined that 86 patients would be needed to achieve at least 90% power for an equivalence test for the difference between two-paired means with a 0.025 alpha level, assuming a true difference of zero in logMAR. Note that the alpha level is Bonferroni adjusted for two comparisons, one for near vision and the other for distance vision.

These methods provided two values (MOA and TVAM) for the five studied parameters. Two-paired sample procedures were used to compare the MOA to TVAMs. The paired t-test assumes approximate normality while the Wilcox signed rank test does not; therefore, the latter yields a more conservative value with granular or skewed data. Bland-Altmann testing was applied to summarize the differences between the two methods through a 95% agreement interval. Linear regression was also done for each of the metrics. Finally, the relationship between NVA and CS was reviewed via a Fisher’s exact value test and linear regression.

Results

After applying exclusion criteria, 129 patients meeting the inclusion criteria were included in the five visual function assessment metrics analysis. Table 1 displays the demographic and diagnosis data.

Table 1 Demographic and ocular history data of study participants.
Full size table

Table 2 displays DVA and NVA expressed in logMAR. The negative delta values indicate that patients scored significantly better on the MOA than on the corresponding TVAM (accounting for paired comparisons within patients and potential violations of normality). Additionally, the overestimation in the MOA was more substantial for near vs. distance VA (p = 0.0054 via Wilcoxon signed-rank test). Overestimation of NVA using EHB was found in a similar study [7].

Table 2 Distance and near visual acuity scores from the mobile ophthalmology application and traditional vision assessment methods.
Full size table

Table 3 displays the results of CVT, CS, and PD. For CVT and CS, a higher score indicates better visual performance. A positive delta score indicates better performance on the MOA relative to TVAMs. This table demonstrates that patients scored significantly better on the MOA than on the TVAM for CVT and CS (accounting for paired comparisons within patients and potential violations of normality). Conversely, PD showed no significant difference between the two methods of testing. Given the overestimation (i.e., falsely better) of DVA, NVA, and CS on the EHB compared to TVAMs, these metrics were analysed separately. As shown in Table 4, patients with better CS in the EHB were marginally more likely (p = 0.0519) to have better near visual acuity (46.28%) compared to those with equal or lower CS as measured by the EHB (28.57%). Bland-Altman plots displaying the differences between the two testing methods against the average of the two testing methods are presented in Supplementary Fig. 1–5.

Table 3 Colour vision, contrast sensitivity, and pupillary distance measurements on the mobile ophthalmology application and traditional vision assessment methods.
Full size table
Table 4 Comparison of near visual acuity testing and contrast sensitivity scores between the mobile ophthalmology application and traditional vision assessment methods.
Full size table

Linear regression and Spearman correlation tests were performed to assess the association of contrast sensitivity on near VA (EHB—Traditional). To yield interpretable slope estimates, changes in delta scores were computed per every 10-point increase in contrast sensitivity. This showed no clear relationship between the difference in NVA and the difference in CS in Fig. 1, suggesting that the error in contrast measurements is not the root of the error for NVA.

Fig. 1
figure 1

Scatter plot showing near visual acuity delta as a function of contrast sensitivity delta with regression with no clear relationship.

Full size image

Discussion

Given the popularity and availability of MOAs, we sought to assess five visual function tools for DVA, NVA, CVT, CD, and PD using the EHB and compare them to results obtained with TVAMs. Compared to TVAMs, individuals using EHB scored better in DVA, NVA, CVT, and CS with statistical significance. PD assessment showed no difference between the two methods.

Objective assessment of DVA and NVA remains one of the key “vital signs” in ophthalmology. We observed a statistically significant difference between DVA and NVA assessment using the MOA compared to TVAMs. However, when determining clinical significance, we observed delta scores of −0.02 and −0.05 for DVA and NVA, respectively, suggesting that the MOA functions are within one line of formal visual acuity assessment techniques. The NVA delta value was also similar to the NVA delta value (−0.11) reported by a previous study [7]. These findings suggest that DVA and NVA may be artificially better when using EHB, but it may serve as a valuable screening tool in situations with limited equipment or resources. Therefore, abnormal results would likely be pathologic rather than a false positive. An abnormal finding on the MOA, especially if detected by an HCP or patient, could then prompt further investigation by an eye care specialist with formal testing methods.

Improved NVA scores when using MOAs have been suspected to be due to the greater contrast ratio in LCD screens (>1000:1) compared to printed NVA cards (33:1) [7]. Previous studies have associated a higher contrast ratio with better visual acuities [9]. Additionally, because the LCD monitor used for DVA testing has a higher contrast ratio than the near vision card, one would expect the ΔNVA to be greater than the ΔDVA if contrast differences were the main reason for visual acuity differences (Table 3). However, establishing this causation proved difficult even though this study showed better NVA and CS scores in the MOA than TVAMs.

The letters on the tablet app are farther apart than those on the Rosenbaum card. This may also contribute to the falsely-better NVA. The NVA function on the app uses the entire LCD screen regardless of the device being used. Therefore, letters on a large tablet using the app will be spaced farther apart than when using a small smartphone. Although we believe both are spaced far enough apart to avoid seeing a difference in VA between the two, they are both significantly farther apart compared to the Rosenbaum card. This may be responsible for the differences seen between TVAM and MOA NVA.

We observed similar findings with CVT, as patients averaged statistically significant better performance using the MOA compared to TVAM. Our findings are similar to those of previous studies assessing the EHB’s CVT function [8]. Clinically, given the findings from Table 3, the EHB’s ability to evaluate colour vision objectively remains useful. This is particularly helpful for settings when colour vision changes may be important assessments for conditions (e.g., optic nerve or macular pathology) that warrant referral to an ECP.

Patient performance on CS testing was significantly higher on the EHB compared to the TVAM (Pelli-Robson chart) (Table 3). Clinically and statistically, we observed that the EHB is currently not an accurate assessment for testing CS. Several factors may account for this discrepancy, as contrast sensitivity varies with the luminance, target, grating motion, and grating shape [10]. First, while the testing distance for DVA, NVA, and CVT was similar, there was a considerable testing distance difference between the MOA (14 inches) and Pelli-Robson chart (20 feet). Second, while attempts were made to keep lighting similar, the angle of the held Pelli-Robson chart may have affected the amount of light that reached the patient’s eye. With the MOA, the tablet contains its own backlight source, which was used at the maximum brightness setting, which may have caused a more favourable score.

A categorical analysis was performed to determine the proportion of patients with better CS scores in EHB who also had better DVA/NVA scores compared with TVAMs. This group was compared to the proportions of patients with equal or lower CS in EHB. We observed that patients with higher CS scores in EHB were more likely (p = 0.0519) to have a better NVA in EHB (46.28%) compared to those with equal or lower CS (28.57%) (Table 4). This is consistent with contrast sensitivity affecting visual acuity assessment. However, this analysis may have power and stability issues due to the low number of patients in the “EHB ≤ Traditional” category for contrast sensitivity. Plotting the difference in NVA versus the difference in CS in Fig. 1 would have likely yielded a negative slope if contrast sensitivity was responsible for the better near VA in the app. Instead, no clear relationship was seen, necessitating a deeper investigation into other contributing factors.

Given that ocular pathology associated with decreased contrast sensitivity can occur even with concurrent excellent visual acuity (e.g., retinal or optic nerve disease, glaucoma, and/or visual pathway disease), we propose two clinical utilities from these findings. First, abnormal CS on the MOA can prompt referral for formal testing with an ECP. Second, normal CS scores on the MOA should not be used as a proxy for formal testing, especially if clinical suspicion for other diseases is high.

Finally, PD had the most agreement between the two methods (Table 3). While formal PD testing is typically performed to assess ocular motility, the EHB’s relatively accurate PD measurement can potentially be used for situations without implications for significant ocular pathology, such as determining measurements for the construction of eyeglasses.

For all five tests on the MOA, the screen brightness was held constant throughout the testing. Previously, tablet screen brightness at 100% was found to be 0.013 logMAR better than when the screen brightness was 50% [11]. Because screen brightness is independent of room brightness, increasing the room brightness would theoretically improve traditional near VA while keeping EHB near VA the same. This would decrease the delta. Alternatively, lowering the screen brightness would have also given a smaller delta.

We propose that EHB is an MOA that can be used by ECPs, HCPs, and patients to efficiently and economically provide objective visual function metrics for efficient and economic screening, diagnostic, referral, and monitoring purposes. Combined with telemedicine and other communication applications, MOA can better connect these three groups to optimize the delivery of ophthalmic care. As the EHB is a freely downloadable MOA, it can be easily used by all three groups anywhere in the world.

However, some considerations need to be made when implementing this technology. For example, facilities would need to purchase tablets/mobile phones and incur costs for a stable internet connection to operate EHB. Because this study only evaluated EHB on a tablet, studies should also be done to assess the accuracy of EHB tools using smartphones. If providers use their personal smartphones, the additional cost of using EHB will be eliminated. While none of these studied vision assessments are separately billable procedures, considerations for insurance companies’ attitudes toward payment on clinical exams using MOAs to assess visual acuities remain a discussion for future studies. Finally, implementing new technology, like EHB, would require additional staff training. This obstacle can be minimized by improving the application’s user interface through software updates. Additionally, because this technology has its most considerable potential with HCPs who don’t use and/or have access to TVAMs, EHB training in these settings may be easier than learning TVAMs (especially in those already familiar with using a tablet).

The strengths of this study include it being a prospective, single-eye assessment of five visual function tests of a popular MOA among patients with a range of ocular pathology. However, this study is not without limitations. First, given the need to screen, recruit, and test many patients, multiple examiners were involved in data gathering. Although they were instructed similarly, different examiners may challenge the participants to various degrees: some recording the lowest line the participant voluntarily will read and some encouraging the individual beyond their comfort. Second, NVA testing in our study was performed in a single eye only; previous studies assessed both eyes of each patient [7]. Finally, based on logistical considerations (e.g., exam room availability in a busy clinical institution), TVAM testing was performed before or after the MOA testing. It is not known if the testing order affects the observed outcomes. Future studies may wish to improve upon these limitations.

In conclusion, the EHB is a free MOA that, among its other features, provides ECPs, HCPs, and patients the ability to objectively assess five visual function metrics: DVA, NVA, CVT, CS, and PD. Abnormal or rapidly changing results can better direct providers and patients regarding the need for timely referral for formal ophthalmological examination. Additional prospective, multicentre studies, including side-by-side comparisons of other MOAs, are warranted to build upon these observations.

Summary

What was known before

  • Mobile ophthalmology applications (MOAs) offer clinicians and patients an economical and efficient way to assess vision.

  • However, an objective, comprehensive assessment of the accuracy of these vision testing functions has not been performed.

  • Results from MOAs have not been compared with vision assessment results obtained with traditional testing methods performed in eye care clinic settings.

What this study adds

  • This study assessed the accuracy of the studied MOA for distance vision, near vision, colour vision, contrast sensitivity, and pupillary distance and compared these results with traditional testing methods.

  • The MOA can be used to screen for and monitor ocular pathologies that cause vision loss.

  • Abnormal MOA vision testing results can be used to guide clinicians, especially non-eye care providers, to refer patients for formal examination in the eye clinic setting.