Introduction

HIV/AIDS persists as one of the most significant global public health challenges of our time. Since the first case was diagnosed in 1981, HIV-related illness has claimed almost 33 million lives worldwide [1]. Currently, there are approximately 38 million people living with HIV (PLWH) globally, all of whom require initiation and lifelong retention on antiretroviral therapy (ART) [1]. Through ART, HIV has become a lifelong chronic illness requiring complex medical care and treatment, including management of sometimes complex regimens to control HIV and treat or prevent opportunistic infections, particularly tuberculosis. PLWH are also at increased risk of certain cancers [2, 3], and face higher rates of liver disease, cardiovascular disease, dyslipidemia, diabetes, kidney disease, dementia, and osteoporosis as they age [4].

To be effective, HIV clinicians must be proficient in an ever-expanding range of topics as they begin providing treatment, and must engage in continuing medical education (CME) to maintain, develop, or increase their knowledge and capacity throughout their careers. However, traditional models of in-person medical education are burdensome and expensive, and the use of mid-level providers to increase the capacity of the HIV clinical workforce [5] has only increased the number of HIV clinicians who require CME. In this context, distance learning delivered over electronic platforms could be an attractive alternative to traditional models of in person CME, provided that they can be shown effective at improving knowledge among HIV clinicians. To assess this knowledge gap, we conducted a narrative review of the use of eHealth for training HIV medical providers. Our goals were to define the scope of eHealth CME strategies described in the literature and, where possible, to assess the strength of evidence about their effectiveness at improving knowledge of HIV clinical care.

Methods

We performed an electronic search in PubMed and Embase databases on September 10, 2020, using search terms related to remote learning (including digital learning, e-Learning, eHealth, and mHealth), HIV/AIDS, and health providers. We defined “HIV/AIDS providers” broadly to encompass doctors, nurses, medical students, community health workers, lay workers, physician’s assistants, and/or midwives. Our search encompassed interventions that covered a broad continuum of medical education, including undergraduate medical training, residency training/graduate medical education, and CME/continuous education and improvement. Additional inclusion criteria were any peer-reviewed articles published on or after January 1, 2015. This cutoff was selected in light of the rapidly changing landscape of eHealth tools and our interest in capturing the most current innovations in this field. We excluded articles for which full text was not available or were systematic reviews and meta-analyses, commentaries, conference abstracts, or articles on patient interventions. There were two review stages, the preliminary title and abstract screening and the full-text article review. Two authors reviewed each article at both stages. Conflicts at all stages of the review were resolved through a discussion with all authors.

Interventions Included in the Review

Figure 1 summarizes the search and review process that resulted in the final set of articles included in the analysis. Twenty-four articles were included in this review (Table 1). Reisach and Weilemann developed recommendations for an e-Learning framework after conducting tests with community health workers in South Africa [6]. The other 23 articles represented 19 unique eHealth HIV medical education interventions: 12 interventions used solely eHealth mechanisms, whereas seven combined in-person and eHealth components. Table 2 summarizes the geographic location, target provider, intervention contents, and mode of delivery, and offers a description of the 19 unique interventions.

Fig. 1
figure 1

Search and review process for articles included in the analysis

Full size image
Table 1 Articles included in review
Full size table
Table 2 Interventions* by learning format
Full size table

Training content varied according to the nature of the setting and the participant population. Nine interventions focused solely on HIV clinical management [7,8,9,10,11,12,13,14,15,16]. Three related to HIV testing and counseling [17,18,19]. One intervention—the mobile CME (mCME) intervention in Vietnam—was more comprehensive, addressing HIV clinical management, testing, counseling, palliative care, and comorbidities [20,21,22]. The remaining interventions addressed stigma in care and treatment of HIV/AIDS patients [23, 24], prevention of mother-to-child transmission [25], HIV-tuberculosis (TB) coinfection [26], and LGBTQ-sensitive HIV care. [27] One intervention, focused on global health leadership, contained a brief session on “HIV/AIDS updates” [28].

Synchronous Versus Asynchronous eHealth Medical Education

The most evident way of stratifying the different eHealth modalities was whether the learning was delivered in real time (synchronous learning); accessed independently at any time (asynchronous learning); or involved a combination of both approaches (hybrid model).

Synchronous learning models deliver remote education to all students participating at the same time, thus creating a virtual classroom. Five interventions provided live instruction over tele- or video-conference during which participants could ask questions and engage in discussions with clinical experts and with each other [9, 11,12,13,14]. Among these, the most notable model was the Extension for Community Healthcare Outcomes (ECHO), described in several articles [11, 12] and adapted in others [13]. ECHO connects community providers to both a multidisciplinary specialist team and local peer providers to create a network for clinical support and education. Tele-conferencing approaches were also described in two case studies: one delivered CME to rural providers in Haiti [14], while the other established a telecommunications platform for Italian physicians to share clinical data and discuss cases with providers in eSwatini [9].

Asynchronous learning, by contrast, does not require real-time interaction. Instead, participants can access content at a time convenient to them. Asynchronous learning strategies build on a variety of technological approaches. These include short message service (SMS) or other interventions using a messaging or chat function; downloadable course material interventions that can be viewed on a mobile device; and modules that are emailed or accessible online. Of the eight asynchronous interventions included in this review, two used SMS or WhatsApp for training purposes. WhatsApp is a free software that uses the phone’s cellular or Wi-Fi capability to send and receive text messages, photos, audio, video, documents, and locations, and make voice calls over the Internet [29]. In one intervention, a WhatsApp group for discussion of clinical cases was created for doctors who had undergone training on HIV and TB co-infection [26].

The mCME 2.0 intervention in Vietnam used SMS over a 6-month period to deliver daily multiple-choice quizzes to HIV providers, provide hyperlinks to online CME courses and readings, and send regular updates on individual performance on recent quizzes relative to peers. Together, delivery of this information was designed to encourage self-study on relevant topics to improve HIV-related knowledge [20,21,22].

The remaining six asynchronous interventions used modules accessible via website, email, or tablet. In the T-HIT intervention, tablets were given to health workers to share weekly educational material [25], whereas in Mozambique, tablets were used to share videos, vignettes, and case studies [18]. Four of the 19 interventions used self-paced online modules [7, 8, 15, 16] or emailed participants a new case study, questions, and relevant links to engage in self-study [30].

Hybrid models incorporate elements of synchronous and asynchronous learning into an integrated training strategy. Six interventions used a hybrid approach. For example, DriSti’s in-person training sessions were complemented with a tablet-based asynchronous component [23, 24]. Four interventions used online modules as a complement to in-person training sessions [10, 19, 27, 28]. One intervention supplemented in-person classroom sessions with tablets that shared videos, case studies, and quizzes, and a WhatsApp discussion group [17].

Evidence of Effectiveness and Lessons Learned

The articles included in the review employed a variety of research designs to evaluate the interventions (see Table 1). Of the 19 articles presenting evaluation data, most (10 of 19) employed a cross-sectional design to gain feedback from participants on acceptability and utility [7, 11,12,13, 15,16,17, 20, 26, 30]. Two articles employed a quasi-experimental design to assess knowledge outcomes [18, 19], and four conducted a single-group pre-post assessment of gains in knowledge post-intervention [8, 10, 25, 27]. Only a few studies aimed to demonstrate efficacy using a randomized control trial (RCT): mCME [20,21,22] and DriSti [23, 24]. Five articles presented narrative findings from case studies describing the design [6, 9, 24] and/or delivery [9, 14, 28] of their interventions, identifying lessons learned for implementation of eHealth interventions, but offering limited generalizability. We present findings related to the interventions’ effectiveness for each type of learning format according to their level of rigor.

Synchronous Learning

Our review adds to the growing evidence that providers find synchronous eHealth approaches acceptable [31, 32], particularly for community health providers in rural areas. Acceptability and utility of the five interventions that provided instruction over tele- or video-conference were demonstrated in the USA [11,12,13], and to a lesser extent due to their case study design, in Haiti [14] and eSwatini [9].

There is limited evidence on effectiveness of these interventions in changing knowledge, attitudes, and practices. Cross-sectional studies demonstrated mixed results. One ECHO intervention reported a null effect on improving knowledge [12], whereas another [11] demonstrated significant increases in knowledge. Some studies demonstrated increases in providers’ self-reported confidence to provide essential elements of HIV care [13], improved perceived case management [11], and an increased sense of being part of a community of practice [11, 13].

Synchronous virtual learning is interactive and can be engaging for participants, while eliminating the need for travel (often costly) required by in-person medical education. However, it poses several barriers to implementation and sustainability. When instructors and participants reside in different time zones, sessions can be challenging to schedule, especially if planned on a long-term basis. Regardless of location, these models also require participants to adhere to a fixed schedule that may conflict with individuals’ clinical and/or personal responsibilities.

Furthermore, as noted in the case study in eSwatini, resource-constrained settings may present additional challenges, such as ensuring the required levels of Internet bandwidth needed for strong connectivity to facilitate engagement and the security required to transfer medical records [9]. Scaling up such models can also be problematic given the technological resources required to engage participants (e.g., multiple computers and/or screens), and may be better suited to small groups.

While synchronous learning solves the problem of accessing students who may be scattered widely by geography, it does not solve the challenge of securing enough qualified teachers to deliver the online, real-time content or scheduling that suits the needs of all participants. Its value is the convenience of not having to travel to facilitate or attend trainings, which reduces the costs for training significantly, but can only function if a number of conditions are met. A tradeoff is that it does not fully re-create the experience or quality of in-person training, and only partially solves the problem of access and convenience.

Asynchronous Learning

Five asynchronous interventions provided data on effectiveness in increasing providers’ knowledge of HIV care and treatment [8, 18, 20,21,22, 25, 26]. The RCT of the mCME intervention in Vietnam found significant increases in knowledge based on performance on post-training exams, as well as a significant impact on increasing confidence in clinical practice [20,21,22].

Although wielding less power due to their cross-sectional and pre-post-test study designs, several other asynchronous interventions also demonstrated significant increases in knowledge based on performance on post-training exams [8, 18, 25]. In a another uncontrolled observational study evaluating the utility of a WhatsApp chat group formed around HIV/TB care, participating clinicians reported feeling more confident in their clinical management decisions. However, this study did not include any objective measures to see whether knowledge was improved [26]. In Tanzania, rural healthcare workers demonstrated increased knowledge and acceptability over time, but the difference was not significant [25].

Asynchronous eHealth education approaches allow students access to educational modules at any time, mitigating scheduling issues and need for high bandwidth required to sustain the Internet connectivity for tele- or video-conferencing. The content delivery is typically automated from a server and uploading new modules or questions can be done as easily as editing the source files in programs such as Excel, making it easy to edit or add new content with minimal information technology support. However, the modality is very impersonal, since it lacks a convenient way to foster discussion with the instructor or between students.

As the only fully asynchronous approach in this review evaluated by an RCT, the Vietnam mCME intervention offers important lessons for the field. In mCME, participants received daily SMS messages that contained multiple-choice quizzes and hyperlinks to direct users to more in-depth content, such as CME courses hosted by Hanoi Medical University. To increase participation in mCME version 2.0, the investigators incorporated a component of competition among the participants. After receiving responses to the daily quiz sent to them via SMS, participants received a report showing their scores compared to their colleagues. While the daily quiz did not require a significant time investment by users, the goal was to spur users to read the embedded hyperlinks to daily readings and to participate in the online courses [22]. In a companion analysis, the researchers showed that this strategy was effective at motivating clinicians to utilize the CME courses, while increasing unscripted self-study across a wider range of modalities (including textbooks) [33]. As a “cue to action,” the daily SMS was effective primarily because it prompted users to be better self-learners. This is a subtle but important point: the benefit of eHealth education modalities may extend beyond the content delivered by the intervention itself, serving as a behavioral change stimulus (e.g., more self-study). However, timing of the quizzes was also important, even though the questions could be answered at any time. This may reflect a limitation of the simple SMS interface, where individuals tend to respond to SMS in real time, rather than leaving them for review later.

Other interventions also offer important insights. In sub-Saharan Africa, online courses were said to be appealing due to knowledge gains, format flexibility, and desire for recognition of course completion [7]. Rehabilitation professionals in the USA and Canada found courses delivered via email to be acceptable; however, they preferred receiving them in PDF versus in the body of an email [30]. Tablets were also seen as an effective and engaging learning platform, especially when paired with supportive supervision [18].

Hybrid Approaches

Of the six interventions using a hybrid of synchronous and asynchronous learning formats, only three reported findings on the impact on knowledge, attitudes, or behavior. One quasi-experimental study reported significant increases in knowledge based on performance on post-training exams [19]. These findings were confirmed in a cross-sectional study [10]; another study also demonstrated a significant impact on increasing confidence in HIV prevention [27].

The DriSti intervention to eliminate HIV-related stigma in India was the only hybrid intervention evaluated by an RCT. The evaluation showed significant reductions in stigmatizing attitudes among participants. The authors also noted that tablets were cheaper, easier to distribute and charge than computers, and did not require an Internet connection if material was uploaded in advance and accessible offline [23, 24]. A limitation is that the authors could only show changes in reported stigmatizing attitudes among those who used the system, but did not show behavior change. It is also possible that changed attitudes reflected social desirability bias, rather than true changes in beliefs.

Although less rigorously evaluated, providers in South Africa gave positive feedback on acceptability and use of text messaging as an adjunct to training courses and described SMS as a useful platform for information seeking, support, and case discussion [17]. The professional relevance of the topic area and participant feedback were reported to be important motivators for engaging in hybrid eHealth medical education interventions [27, 28]; however, intervention effectiveness was uncertain given the lack of evidence provided on changes in knowledge or practice.

A number of challenges also exist in implementing hybrid interventions. One study identified reliable access to electricity, particularly in rural areas, as a barrier to successful implementation due to difficulty keeping tablets, phones, and computers charged. The lack of charge resulted in delays in accessing training content or adhering to assignment deadlines [17]. In addition, participants experienced limited space on phones for typing case descriptions, which could be better accommodated using tablets [17]. These technological challenges underscore the importance of providing supplemental material that is available offline and/or in a printable format, when possible [6].

A novel hybrid approach is virtual discussion groups that complement in-person training. Woods et al. reported a significant increase in participant engagement in a WhatsApp discussion group that followed an in-person training. The participants used the group to get guidance from their peers and obtain new clinical insights [26]. However, a limitation of this approach is that “engagement” is not the same as acquiring knowledge; at best, it can signal acceptance or enthusiasm, but is not itself proof that students acquired skills or concepts as a consequence of the intervention.

Discussion

This review demonstrates the wide variety of eHealth HIV medical education interventions that have been designed and delivered thus far, though the field is still rapidly evolving. There is no one-size-fits-all approach for eHealth medical education approaches. Rather, recommendations for developing interventions arise from exploring two overarching questions that emerged from lessons learned in this review.

  1. 1.

    What Are the Needs of the Learners?

The evidence shows that eHealth approaches for HIV medical education across multiple modes of delivery are acceptable, but a critical question is whether the mode of delivery matches the needs of learners. In our review, providers often sought support for HIV cases with complex medical and behavioral health issues, noting that access to mental health and substance abuse services remains particularly problematic in their communities. This underscores that when designing an eHealth education intervention, it is critical to involve a multidisciplinary panel that includes experts on an array of topics including mental health, substance use disorders, adherence support, and viral hepatitis coinfection [13]. Providers placed high value on learning from experts and observing and applying skills; an opportunity to engage with experts was found to be primary motivators for engaging in eHealth CME courses and should be considered in engaging instructors to design and/or deliver content [12]. Content should also be realistic, culturally competent, and relevant to the learner’s area of practice [30]. Lastly, it is important to have a specific learner population in mind, with content relevant to the area of practice, and case studies, vignettes, or supporting materials that are realistic and informed by the lived experiences of PLWH [20,21,22,23, 30].

  1. 2.

    What is the Digital Landscape of the Context?

Designing eHealth education interventions requires a deep understanding of the digital landscape in order to choose the most appropriate modality of delivery (e.g., SMS, WhatsApp, tele- or video-conference, email, online), the educational format (synchronous, asynchronous, or hybrid), and the roles of partners. Assessing opportunities for technology ownership and capacity strengthening by key stakeholders to make interventions more effective and sustainable is also critical. The mCME intervention demonstrated the importance of collaboration with local Ministries of Health and stakeholders from intervention inception. The collaborators on this project gave development and ownership of the eHealth technology to Vietnam’s Ministry of Health, thus enhancing its stake in the project and facilitating local adaptation [21]. Cross-national, technology-sharing collaborations like this one were observed infrequently in this review.

The key to answering these questions and designing successful interventions is working with the target population and tailoring the curriculum and delivery to its unique needs and resources. Collecting qualitative data prior to, during, and after implementation can play a critical role in this process.

Additional Evidence Needed to Advance the Field

This review demonstrates remarkable capacity for ingenuity in conceiving ways to employ eHealth approaches for HIV medical education. However, testing new strategies rigorously, disseminating findings, and focusing resources on translating best practices into widespread training are major gaps. Studies that cannot provide efficacy data cannot be used to prove or refute the effectiveness of eHealth modalities. A key finding of this review is that only two interventions were evaluated using an RCT design—the gold standard for assessing a new intervention. Among the remaining articles, the rigor and reliability of the evaluation findings varied. Thus, we found an absence of evidence of effect rather than evidence of absence of effect. As we have seen, the specific way in which content is delivered can have a profound impact on the effectiveness of the strategy, as shown between the mCME 1.0 and mCME 2.0 iterations [20, 21]. Given eHealth’s potential to provide critical education to providers while saving time and money, it is imperative to move this field from one of experimenting without testing or disseminating evidence, to one of evidence-based practices. This is analogous to the development of the wide array of HIV prevention and treatment methods that have emerged as a result of public health and medicine’s innovations over the last four decades. The current paucity of evidence suggests that the potential to utilize eHealth modalities to support HIV medical education is still largely untapped.

Measuring long-term impact of medical education on clinical outcomes is a challenge due to time, budget, and lack of ability to control for other factors in real-world settings. However, using more rigorous mixed-methods evaluations would provide the opportunity to demonstrate short-term impact on gains in knowledge (quantitatively) and improve understanding (qualitatively) about why certain approaches work in different contexts and with different cadres of providers. There are also inherent limitations in how we define impact. It is difficult to demonstrate that having knowledge will improve clinical care—even if it is intuitive that such a link exists—because there are many steps separating knowledge gained through medical education from improved health at a population level. For this reason, assessments of the impact of medical education are difficult to take beyond demonstrating a change in clinician study habits (as a mediating variable) and the clinician’s knowledge measured through a standardized examination. However, it is still essential that future evaluations of novel eHealth medical education strategies be accompanied by objective measures of impact. Improvement in knowledge can be assessed objectively and systematically; we see this as a relevant proxy measure for efficacy. Assessing changes in knowledge is feasible even in very small studies. Economic evaluations of these interventions would also generate valuable data on the cost implications of major eHealth interventions. Emerging evidence from a cost-effectiveness study of the mCME intervention in Vietnam shows that most of the costs are in upfront technology-based investment, which then can be used to promote CME (and perhaps support initial training) over the long run. Data such as these are crucial to persuading local and national health authorities to fund such programs, especially if they are meant to be long-term or large-scale solutions.

Conclusion

This review adds important lessons learned regarding the array of eHealth interventions for HIV medical education over the last 5 years and the methodology used to evaluate them. Technology has changed the landscape of training interventions, reaching people and making connections that would not have been possible before, especially at a much lower cost than in-person training. Despite the challenges of measuring effectiveness of eHealth for medical education, there is evidence that some interventions work well, but only from a small number of studies. Our review finds that eHealth approaches for medical education have been shown to be feasible and acceptable in synchronous, asynchronous, and hybrid learning formats. There is a need for rigorous studies to evaluate the impact of eHealth to support scale-up of these approaches to ensure an adequate number of trained HIV providers. These evaluations are also necessary to inform future interventions in different populations and settings. Lastly, our review’s findings on the effectiveness of HIV medical education may extend to other topic areas in order to inform future eHealth interventions and move this burgeoning field forward.