Introduction

An accurate visual acuity (VA) measurement is the foundation of ophthalmic diagnosis [1,2,3,4]. Traditional wall-mounted VA charts, including Snellen and Early Treatment Diabetic Retinopathy Study (ETDRS) charts, are the most commonly used VA measurement tools today [5,6,7]. ETDRS is recognized as the gold standard in VA chart measurement [8,9,10].

To get an accurate VA measurement, people in need of monitoring vision changes go to hospitals and have their VA measured routinely. However, frequent hospital visits are inconvenient, particularly for the elderly and mobility-impaired patients [11, 12]. Such visits were further deterred during the Corona Virus Disease 2019 (COVID-19) pandemic, in which people avoided going to overcrowded places like hospitals [13]. The pandemic has altered many patients’ medical treatment patterns as they preferred virtual consultations over in-person appointments. Our previous study in 2015 had shown that most patients with eye disease were willing to participate in mobile health programs [14]. Therefore, to help patients adapt to this new way of receiving diagnosis, a reliable and convenient online VA testing system is necessary.

Patients face many problems on self-administered test with VA charts at home. Firstly, a standard VA chart is difficult to obtain. Secondly, it is difficult for people to have the standard lighting required for an accurate VA examination. Thirdly, it is difficult to determine when to end the tests, how to score VA, and measure low VA on the chart [15]. Studies have shown that the results of measuring VA with traditional VA charts vary greatly and have poor repeatability [16, 17]. The participant’s inherent memory of the answer will also significantly impact test results.

Since the prevalence of telemedicine and virtual care has rapidly increased during the pandemic [4, 18,19,20,21,22,23], automated digital VA tests using electronic devices have gradually emerged [1, 4, 7, 24,25,26,27,28,29,30,31,32,33,34,35]. The random display of letters in different sizes greatly reduces the chance of memorizing [10]. However, many automated digital VA tests have not been validated in clinical trials and thus do not have guaranteed accuracy, and they are also not free. In order to investigate the effect of a self-administered online VA test, the Stanford Acuity Test (StAT) was selected for this study. This online VA test is free and allows people to complete it at home, which lightens their financial burden. Similar to other online VA tests (which have a high degree of reliability and agreement with traditional VA chart tests) [1, 4, 10, 24,25,26, 29, 34, 36,37,38], Piech et al. found that the StAT had reduced errors significantly compared with traditional VA chart [39]. However, most online VA tests, including StAT, do not provide guidance on how to select the appropriate device and screen brightness. Moreover, studies that discuss the effect of different devices and screen brightness on the results of online VA tests are limited. This study aimed to compare the online VA tests with ETDRS and study the difference in test results under different devices and screen brightness conditions.

Methods

Recruitment

The healthy subjects who underwent eye examinations in Zhongshan Ophthalmic Center of Sun Yat-sen University from May to September 2021 were recruited. The study received approval from the Zhongshan Ophthalmic Center’s Ethical Review Committee and was conducted in accordance with the tenets of the Declaration of Helsinki. Written informed consent was obtained from all participants in this study.

All participants underwent complete ophthalmic evaluation, including manifest refraction, slit-lamp biomicroscopy, dilated indirect ophthalmoscopy, fundus photography, and Humphrey SITA (Swedish Interactive Threshold Algorithm) standard 24–2 or 30–2 visual field testing.

Considering that the incidence of ocular and systemic diseases increases with age, which would make the test results biased, we set the inclusion criteria as follows: (1) age between 18 and 40 years old; (2) best corrected visual acuity (BCVA) 20/20 or better; (3) no history of ocular disease, or current evidence of ocular disease or systemic disease; (4) refractive error not exceeding ± 6 diopters (D) spherical equivalent (hyperopia or myopia), cylinder ≤ 2.0 D (astigmatism); (5) proficient in mobile phone operation.

Stanford Acuity Test

The StAT is an online test (https://myeyes.ai) developed by Piech and his colleagues. It is a new type of vision test based on an improved acuity model and an intelligent inference process. It can make decisions based on intelligent probabilistic models and display letters of any size adaptively, which compensates for the shortcomings of traditional printed VA charts.

The StAT mainly relied on two devices: a computer and a smart phone. The letter “E” was displayed on the computer in four different orientations; 20 letters were displayed by default for each eye test; and only one letter was displayed at a time. The smart phone was controlled by the participant, and the answer was recorded by swiping across the screen in the same direction as the arm of the letter “E” [39].

There were four steps in the StAT. First, the participant needed to calibrate the screen size by following the prompts. Second, they needed to enter the measure distance between their position and the computer screen (the distance can be customized) and then followed the prompts to connect the smart phone to the computer. Next, they would submit answers on their phones when the computer screen randomly displayed the letter “E” of different sizes in four orientations. Then, the algorithm would adjust the size of the letter displayed next according to previous answers. As the test progressed, the algorithm was able to generate a more reliable estimate about the result [39, 40]. The results would be shown on the computer screen after the test.

Visual acuity measurements

All subjects wearing their habitual refractive correction underwent VA testing on the same day using the ETDRS and StAT in a controlled testing environment. All VA measurements were taken with a room luminance between 589 to 651 lx. If the subject’s two eyes met the inclusion criteria, the VA of both eyes would be measured; otherwise, only the eligible eye would be measured. While measuring one eye, full closure of the other eye was ensured by using an eye patch. Subjects were given at least a 1-min break between each test. A single ophthalmologist (L.C.) performed all the VA measurements. Two ophthalmologists (L.C. and S.P.) recorded the VA results and the testing time of each vision test.

The StAT (https://myeyes.ai) was performed on two different tablet computers, and the screen brightness of both tablets would switch between 50% brightness and 100% brightness. The two tablets were Microsoft Surface pro4 (SURFACE, windows10, 12.3-inch, PixelSense™ touchscreen display) and iPad Air2 (IPAD, iPadOS15, 9.7-inch, Liquid Retina display). The pixel resolution of two tablets had been uniformly set to 2048 × 1536. The static contrast ratio of the SURFACE’s screen was 1020:1, while the IPAD’s screen was 1126:1, both of which were very high. We measured the screen luminance values at the four corners and the center of the screen, a total of five points, to measure the average screen luminance values. At 100% screen brightness, the average SURFACE’s screen luminance was 360 cd/m2, and the IPAD’s was 358.4 cd/m2. At 50% screen brightness, the SURFACE had an average screen luminance of 180 cd/m2, and the IPAD had 179.2 cd/m2. The screen luminance of each device in this study met the standard requirements (range: 80–320 cd/m2) [15].

The VA tests were performed in the following sequence: (1) tested VA with ETDRS chart. The light box brightness of the 4-m ETDRS chart was 300 cd/m2. The subjects were instructed to point in the direction that the letter “E” was facing, and the ophthalmologist recorded the results based on the smallest line that four or more optotypes were correctly identified; (2) performed StAT on IPAD, measured VA at 50% screen brightness. Subjects stood at the StAT’s default standard test distance of 6.1 m and submitted the answers by swiping across the smart phone screen; (3) adjusted screen brightness to 100%, run StAT again on IPAD to repeat vision measurement; (4) performed StAT on SURFACE, measured VA at 50% screen brightness; (5) adjusted screen brightness to 100%, run StAT again on SURFACE to repeat vision measurement.

The different VA measurement methods were split into two categories: traditional ETDRS chart test (the ETDRS group) and online StAT (the StAT group). The StAT group was further divided into the following groups based on different tablet devices (two tablet computers: SURFACE and IPAD) and screen brightness (two screen brightness: 50% and 100%):

  1. (1)

    SURFACE50%: running StAT with SURFACE at 50% screen brightness;

  2. (2)

    SURFACE100%: running StAT with SURFACE at 100% screen brightness;

  3. (3)

    IPAD50%: running StAT with IPAD at 50% screen brightness;

  4. (4)

    IPAD100%: running StAT with IPAD at 100% screen brightness.

Data analysis

VA was converted to log minutes of arc (logMAR) for analysis. Comparisons of VA were performed in different group settings. For paired comparisons, the mean difference of VA was calculated, as well as the 95% confidence interval (CI) and the 95% limit of agreement (LOA). Bland–Altman plots were used to illustrate the agreement of VA test results across groups at different devices and screen brightness conditions. The mean testing time between ETDRS and StAT was compared using paired t tests. The above data were analyzed using SPSS v26.0 software (IBM, IL, USA). A p-value of < 0.05 was considered statistically significant.

Results

A total of 56 healthy subjects (101 eyes) participated in this study, including 17 males and 39 females. Eleven subjects had only one eye that satisfied the inclusion criteria; hence, only the data from their included eye were obtained, while their fellow eyes were excluded. The mean age was 26.70 ± 3.57 years old. No adverse events or complications occurred during the testing period. The mean VA of the ETDRS for all eyes tested (20/20 or better BCVA) was − 0.05 ± 0.07 logMAR.

Comparison of StAT and ETDRS on VA measurement

Compared with the traditional ETDRS chart, the VA results of the online VA test StAT were better (Table 1). The mean VA when using the ETDRS was − 0.05 ± 0.07 logMAR, and the mean VA using the StAT ranged from − 0.07 ± 0.09 logMAR to − 0.09 ± 0.10 logMAR for different devices and brightness conditions. Although the VA results were significantly different between the StAT and ETDRS (all P < 0.05), the maximum mean difference was only 0.032 logMAR, which was about 2 letters. Given that the clinically significant change was defined at 0.2 logMAR or greater (equivalent to ≥ 2 lines of acuity) [41], the difference of VA results between ETDRS and StAT had no clinical significance.

Table 1 Comparisons of ETDRS and StAT in VA results (N = 56,101 eyes)
Full size table

Comparison of StAT VAs under different brightness conditions

The mean VA results of StAT were better at 100% screen brightness than at 50% screen brightness (Table 2). In these two different brightness conditions, the VA results of StAT only had significant differences when IPAD was used (P = 0.009). However, the mean difference between IPAD50% and IPAD100% was 0.013 logMAR, which was equivalent to less than 1 letter and was clinically insignificant.

Table 2 Pairwise comparisons of StAT under different screen brightness in VA measurements (N = 56, 101 eyes)
Full size table

There was a high agreement and a small difference of averages between 50 and 100% screen brightness on IPAD (0.013 logMAR; 95% LOA, − 0.081 to 0.107, Fig. 1A), as well as on SURFACE (0.007 logMAR; 95% LOA, − 0.088 to 0.102, Fig. 1B).

Fig. 1
figure 1

Bland–Altman plots for StAT VA measurements by the IPAD and SURFACE in two different screen brightness (50% and 100%). A Agreement between IPAD50% brightness and IPAD100% brightness. B Agreement between SURFACE50% brightness and SURFACE100% brightness. C Agreement between IPAD50% brightness and SURFACE50% brightness. D Agreement between IPAD100% brightness and SURFACE100% brightness. E Agreement between IPAD50% brightness and SURFACE100% brightness. F Agreement between IPAD100% brightness and SURFACE50% brightness. The x-axis displays the average logMAR acuity, and the y-axis displays the difference in logMAR acuity of the two pairwise comparison groups. The black solid line represents the mean difference of VA between two pairwise comparison groups, black densely dashed line indicates 95% confidence interval (CI) of mean difference. The gray dashed lines represent the limits of agreement (LOA) (± 2 SD of the mean VA difference)

Full size image

Comparison of StAT VAs under different devices

The results revealed that the mean VA results on IPAD were slightly better than those on SURFACE (whether performing StAT under 50% or 100% screen brightness), but the difference was not significant (all P > 0.05) (Table 3).

Table 3 Pairwise comparisons of StAT under different devices in VA measurements (N = 56,101 eyes)
Full size table

Figure 1C displays the high agreement of VA results of StAT between IPAD and SURFACE under 50% screen brightness. The mean difference was − 0.003 logMAR (equivalent to less than 0.2 letters) (95% LOA, − 0.019 to 0.103). Figure 1D illustrates that the agreement of VA results between IPAD and SURFACE under 100% screen brightness was also promising, with a mean difference of − 0.009 logMAR (equivalent to less than 1 letter) (95% LOA, − 0.100 to 0.084).

Comparison of StAT VAs on different devices and brightnesses

Table 4 shows the differences in VA results of StAT when the device and screen brightness were both different. The VA results of SURFACE100% were better than IPAD50% with an insignificant mean difference of 0.004 logMAR (equivalent to about 0.2 letters). However, IPAD100% had better VA results and a significant difference from SURFACE50% (mean difference, − 0.016 logMAR) (P = 0.005), but the mean difference was clinically insignificant because it was equivalent to less than 1 letter.

Table 4 Pairwise comparisons of StAT under different devices and screen brightness in VA measurements (N = 56,101 eyes)
Full size table

IPAD50% showed high agreement with SURFACE100% in VA results (Fig. 1E), the 95% LOA ranged from − 0.007 to 0.051. Agreement of VA results between the IPAD100% and SURFACE50% (Fig. 1F) was also good, with 95% LOA of − 0.123 to 0.091. Overall, the effects of different devices and screen brightness conditions on StAT VAs were not significant from a clinical perspective. Screen brightness might play a major role among these subtle effects, because higher screen brightness resulted in better test results no matter which device was used.

Comparison of testing time between StAT and ETDRS

The comparison of the mean testing time of each group is shown in Table 5. All StAT groups used significantly less time on average (48.95 s at most) than ETDRS (56.54 s) (all P < 0.001), with the IPAD100% group having the shortest mean testing time (47.04 s). Among the StAT groups, the difference of testing time was only significant between SURFACE50% and IPAD100% (48.95 s vs. 47.04 s, P < 0.05).

Table 5 Comparison of testing time between ETDRS and StAT
Full size table

Discussion

Web-based VA tests provide various advantages. They provide a practical and high-quality vision test for patients who reside in distant places and require long-term eye monitoring, notably during the COVID-19 pandemic. However, there is little evidence to support the accuracy of a self-administered online VA test. In this study, StAT was utilized to investigate the differences between the online VA test and the ETDRS vision measurement, as well as the differences in test results under differed device and screen brightness conditions.

In the current study, we compared the online VA test with the traditional ETDRS chart and found that the VA results of the online VA test were better, although this difference did not have a significant impact from a clinical perspective. When we compared the effects of different devices and screen brightness on the VAs of the online VA test, the VA results measured at 100% screen brightness were better than those at 50% brightness, and the VA results measured by IPAD were better than SURFACE. Nevertheless, VA results of the online VA test did not differ significantly between these different conditions (all less than 1 letter).

Table 6 shows previous studies comparing traditional VA charts with online VA tests. The mean difference of VA results between the traditional VA charts and various online VA tests was equivalent to 1 letter to 1 line, which was clinically insignificant. Similarly, StAT also has a high agreement with the gold-standard ETDRS chart (the difference was less than two letters), so the selection of StAT as the research object in this study is representative to a certain extent.

Table 6 Comparisons of traditional visual acuity chart and online visual acuity test in previous studies
Full size table

Although SURFACE offers higher pixel density and image quality than IPAD, IPAD has a better overall screen layout due to the Retina Display’s fully laminated display technology and anti-reflective surface coating. The test results of several screens under the same brightness settings were strongly correlated in this study, and there was little difference in test results under the same screen resolution. This was in line with previous studies on the visual effects of various devices on people with low vision, which discovered no discernible difference between the effects of head-mounted displays and vertical displays on patients’ reading acuity [42].

However, the impacts of different devices on the results of online VA tests still exist. In the current study, performing an online VA test using IPAD under 100% brightness had the best VA results. Combined with the fact that IPAD50% could achieve a very similar effect as SURFACE100%, while SURFACE50% could not achieve a similar effect as IPAD100%, we infer that the unique material of the IPAD screen had a positive impact on the results. Consequently, the material of the electronic screen may be one of the factors affecting the testing effect.

Many studies have suggested that different screen brightness might affect the visual results. It has been reported that the agreement between the observers’ diagnostic results of carotid plaque duplex ultrasound images was poor at 50% and 100% brightness [43]. Another study found that raising screen brightness could boost readers’ speed and character recognition accuracy [44]. Furthermore, screen brightness aggravated blurred vision symptoms in computer vision syndrome [45, 46]. However, the VA results of the same device under different brightness had a good agreement in our study. Therefore, the difference in screen brightness may not have a noticeable impact on the results of the online VA test.

It has been proved that the “crowding effect” is constantly present in ETDRS [47], which may underestimate acuity measured in people with normal vision. In the Peek Acuity test, a box was set around the tumbling “E” to simulate the “crowding effect” [26, 48]. However, only a single optotype “E” appeared on the screen when testing StAT, which reduced the “crowding effect” and made the optotype look clearer [47, 49]. This may be one of the reasons why StAT had better VA results than ETDRS.

In previous studies, it took an average of 77 s to complete a vision test using Peek Acuity, which was 5 s faster than the Snellen chart (P = 0.13) [26, 48]. Also, in this study, the testing time of StAT was significantly shorter than ETDRS. StAT presents just one randomly orientated “E”, which can prevent the consequences of crowding effects and the potential for sequence learning if the same chart is used repeatedly. Compared to the ETDRS chart, it offers a simpler testing scenario for the tester. Furthermore, the StAT only requires the tester to observe a total of 20 letters for each eye, which is significantly less than the ETDRS chart test. These reasons may explain why the online VA test is more efficient.

There are limitations in this study. To ensure that the test results are uniform and unbiased, we excluded patients with a history of eye disease or those with decreased contrast sensitivity. As a result, the conclusions of this study may not be applicable to a large number of ophthalmic patients, thereby limiting its generalizability. However, we will explore the effect of online VA tests on ophthalmic patients in future studies.

Overall, the online VA tests were comparable to ETDRS in terms of the agreement and accuracy while taking much less time. It is critical to determine the accuracy of online VA tests because inaccurate and unreliable measures can lead to delayed treatment and management of ocular disorders. By studying StAT, this study proved the test validity of online VA tests and demonstrated that they could also have a high level of accuracy.

Conclusion

Our study demonstrated that the online VA test had the same high efficiency as the ETDRS chart and was more time-saving. Both screen brightness and the device had a clinically insignificant effect on online VA test’s results. Furthermore, using the IPAD tablet and a brighter screen resulted in better VA results of online VA test.