Introduction

Demand for evidence-based practices to address common behavioral health problems is high. Many large integrated healthcare systems such as the Veterans Health Administration (VHA) have difficulty meeting this demand due to travel distance to facilities, scheduling difficulties, stigma, availability of trained providers, and cost (Mojtabai et al. 2011).

Behavioral intervention technologies (BITs) are programs that deliver personalized, self-guided interventions over Internet-connected devices. Alternative terms such as “eHealth,” “mHealth,” “digital therapeutics,” and “internet interventions” are also used (Fleming et al. 2018). BITs intervene in a wide range of behavioral, mental health, psychosocial, or chronic health conditions, termed “behavioral health” conditions here, by assisting the user to change behaviors, cognitions, and emotional states (England et al. 2015; Mohr et al. 2013). BITs present therapeutic materials in formats including audio, video, and text, which provide varying levels of consumer interaction. Participants use BITs at a pace and in a setting of their choosing and are provided varying levels of support from care providers (Schueller et al. 2017). Evidence-based BITs have shown efficacy for the management or treatment of many disorders including diabetes, depression, insomnia, and substance use (Sander et al. 2016; Quinn et al. 2008; Campbell et al. 2014).

BITs may help mitigate barriers to healthcare access and utilization by providing more convenient forms of evidence-based care. BITs may reduce travel time, scheduling difficulties, and stigma while increasing self-care and health system productivity (Warmerdam et al. 2010). BITs may be especially advantageous to large integrated healthcare systems such as VHA, which has over 1200 facilities providing care to over nine million consumers a year (http://www.va.gov/health/FindCare.asp). Despite recent legislation and intense programmatic focus on improving access to care, such access to evidence-based behavioral healthcare is still a major issue in VHA, as it is in other health systems where distance and lack of trained providers are barriers (West and Weeks 2006).

VHA’s current and historical investment in health information technology, which has been mirrored by other integrated healthcare systems, has positioned it for the implementation and dissemination of BITs. VHA was one of the first integrated healthcare systems to develop and implement a system-wide, interconnected electronic health record (EHR) as well as a web-based patient portal where consumers can schedule appointments, access records, and communicate with providers (Evans et al. 2006). Additionally, VHA is a leader in the development of Veteran-centered computer-based BITs, such as “Moving Forward” and “Anger Management” found on the http://www.veterantraining.va.gov health information site, as well as BIT mobile applications such as “PTSD Coach” (http://www.ptsd.va.gov/public/materials/apps/ptsdcoach.asp). These programs and others are the product of multiple initiatives across many years of focus on the dissemination of health information technologies. Such initiatives have been guided by several administrative units within VHA such as the Offices of Information Technology and Connected Care.

However, VHA’s leadership in health information technology dissemination and BIT development has not yet resulted in widespread BIT implementation at the point of care with VHA consumers or dissemination across the entire system. While policy initiatives and focused dissemination efforts in some integrated healthcare systems have paved the way for BIT use, there are few examples of successful implementation and sustainment in routine practice settings (Common Mental Health Problems: Identification and Pathways to Care 2011; Titov et al. 2017; Quanbeck et al. 2018). This gap has developed despite the acceptability of BITs, their strong evidence base, and the recent explosion of BIT development, even within VHA (Kaltenthaler et al. 2008). In a recent major pragmatic study implementing BITs for depression in primary care sites across the U.K., there was low adoption among patients (Gilbody et al. 2015).

One remedy to this relative lack of BIT deployment may be the development of comprehensive strategies for BIT implementation and dissemination. The development of such strategies should start by first identifying the determinants of BIT use for a given health system or context. Determinants are defined as the human and system factors, which determine to what extent and manner interventions are used, and have been referred to elsewhere as “barriers and facilitators” (Baker et al. 2015). While a host of literature has discussed general determinants for the use of technology in healthcare settings and described general frameworks for the implementation of technology, little is known about such factors as they relate specifically to the use of BITs, which rely on the use of both technology and self-care (Mair et al. 2012; Grol et al. 2013). Given that BIT implementation and dissemination research is in its early stages, such determinants may be best investigated through discussions with key informants who are early adaptors of BITs and presumed to be the most knowledgeable about the determinants for BIT use in their system of care. The objective of this study was to explore key informant perspectives on organizational-level determinants for the implementation and dissemination of BITs across the VHA. Information on such determinants can support the decisions of healthcare leaders, policy-makers, and researchers.

Methods

Study Design

Perspectives of VHA primary care providers, administrators, and policy makers on determinants of BIT dissemination and implementation were explored through qualitative analysis of semi-structured interviews. The institutional review board of the (VA Connecticut Healthcare System) pre-approved this study.

Sample

VHA providers and administrators who were either using or had used BITs in patient care or were otherwise knowledgeable about such use in VHA were sought. A range of participants were purposefully sampled, including primary care providers, primary care mental health providers, clinic/facility administrators, and VHA system administrators or policy-makers, to gain local, regional, and national perspectives. Participants were physicians, psychologists, nurses, or social workers by training, all with administrative or leadership roles as well as clinical care responsibilities. VHA does not systematically track or record consumer or provider use of BITs. Therefore, a sampling strategy based on in-depth investigator knowledge of the VHA system was used, which identified an initial group of VHA key informants. This group was expanded by asking interviewees for referrals to additional key informants. Recruitment was through e-mail contact that included a project description. After a consideration of thematic saturation and the breadth of interviewee background, a total of twenty interviews were conducted, with 100% of those who were approached agreeing to participate.

Data Gathering

The principal investigator (EH), a psychiatrist working in VHA integrated mental health in primary care contexts and health services research, conducted and audio recorded semi-structured interviews. The interview guide focused on first defining BITs and giving examples of VHA developed BITs or other programs if needed. Determinants were explored using questions derived from the core constructs of the Unified Theory of Acceptance and Use of Technology (performance expectancy, effort expectancy, social influence, and facilitating conditions). Follow-up questions explored specific topics that emerged (Venkatesh et al. 2003). Initial analysis of the first five interviews led to the identification of two draft themes, “internet availability” and “the lack of BIT implementation strategies.” Information about these themes was subsequently queried in the final 15 interviews if the topics did not come up naturally in the course of discussion. The topics were specifically queried in two interviews.

Data Analysis

Interviews were transcribed verbatim. Thematic analysis, a foundational approach to the analysis of in-depth interviews, was used to analyze interview content through a five-step group process: (1) familiarization with transcribed data; (2) generation of initial content codes; (3) collating codes into themes; (4) reviewing, discussing, and modifying themes in relation to interview extracts; and (5) defining and developing a typology of themes (Braun and Clarke 2006). Codes were first generated by individual investigators. Codes were then discussed as a group to resolve discrepancies and develop a final set. Similarly, themes were then developed from codes using a group discussion process. Codes within themes were reviewed using the entire data set to identify examples and counter-factuals. Authors EH, LB, and EP participated in steps one through three, while all authors participated in steps four and five. Themes were organized according to socio-ecological levels within the integrated healthcare system: patients, providers, and the organization. This manuscript describes those themes associated with the implementation and dissemination of BITs across the VHA at the organizational level. Determinants of practice at the provider and consumer levels are described elsewhere (Hermes et al. 2018). Data was managed and analyzed using Atlas/ti. version 4.2 software.

Results

Overview

Characteristics of interviewees are described in Table 1. Slightly more males than females participated, while the training and clinical area of focus was relatively balanced among interviewees. All interviewees participated in both clinical and administrative or leadership duties within VHA, but relatively more participants had primarily clinical rather than administrative or leadership responsibilities. Participants represented ten distinct VHA Central Office or regional network programs. Themes overlapped extensively among interviews: 18 or more interviewees (≥ 90%) discussed 5 of 8 themes, no theme was present in less than 11 interviews (55%), and an average of 6.6 themes were present in each interview. In addition, no direct counterfactual statements regarding any theme were made in any interview.

Table 1 Characteristics of participants
Full size table

Determinant Themes

Care Delivery Models

All interviewees discussed how current healthcare delivery uniformly emphasizes face-to-face clinical interactions between patients and providers as the exclusive approach and gold-standard for service delivery. The self-care model embodied in the use of BITs represents a substantially different approach. There are no systems in place to track, bill, or credit providers for workload associated with using BITs for self-care, and the incorporation of BITs into current clinical workflows has yet to take place. One physician with national programmatic leadership responsibilities stated, “We’re still in a culture that really believes in the ultimate value of the clinical encounter, versus anything patients do on their own,” while a psychologist with national program administrative responsibilities stated, “How is their workload tracked? How are the patients tracked?” and “…then there’s the whole financial issue of how do you get reimbursed for that time.” There is also a perceived tension between the clinical goals of providing access to care and improving the quality of care. Some providers feel that offering BITs improves access at the expense of quality care. For instance, comparing BITs to VHAs dissemination of face-to-face evidence-based psychotherapy, one physician with administrative responsibilities stated, “People are going to feel like…unless they get these sophisticated therapies that we now have, they’re not getting their needs met.”

Process for BIT Use At the Point of Care

Nineteen interviewees (95%) discussed critical processes needed to implement BITs at the point-of-care with consumers. Interviewees stated that providers must have knowledge of and familiarity with available BITs as well as access to programs at the time they interact with patients. One social worker stated, “[A patient will say] ‘Oh I want help with this,’ then I go and look at either apps (smart phone applications) that I have saved on my phone or I have this running list written down of apps that other people have mentioned to me.” Providers must also have enough understanding of BITs and the providers’ role in the delivery of BITs to help individual consumers understand the program’s use and content as well as determine the acceptability and feasibility of BITs as a component of their care. Another psychologist involved in research and clinical care stated, “If folks are…given a menu of things that they could use and one of them is self-help websites or applications, there might need to be some assessment of the patient to determine if the patient has enough motivation to use the app on their own. Do they have the self-efficacy needed to persevere and do the things that are in there?”

Internet Infrastructure

Nineteen interviewees (95%) discussed the need for reliable or convenient Internet infrastructure in VHA facilities as a barrier to BIT use. Interviewees identified inconsistent Wi-Fi access, as well as slow internet speeds for the few wired work stations where patients have access. For example, one social worker stated, “My internet is laughably slow,” while a psychologist with administrative and research responsibilities stated “It’s very hard to sit down with a patient and say, ‘let’s download this free app’ and they can’t download it because the Wi-Fi is problematic.” Other interviewees noted a need for space or equipment to work on BITs in VHA facilities, including workstations or tablets for patients who may not have internet access at home. Several interviewees noted Veterans treated in community-based outpatient clinics, especially in rural areas, may be particularly affected by the lack of internet access at VHA facilities.

Actions and Characteristics of Leadership

Nineteen interviewees (95%) discussed how the actions and characteristics of persons directing healthcare operations or policy affect BIT use. Most interviewees discussed how VHA administrators and leaders support self-care and BIT use in the abstract but do not fully appreciate the extensive changes to care needed to implement and disseminate BITs, such as the need for more capital investment and other forms of material support. A psychologist with national administrative and leadership responsibilities stated, “The lack of support just makes it that much harder for clinicians to feel comfortable doing it, to be able to just treat it like it’s any other kind of intervention;” and more to the point, another psychologist with administrative and leadership responsibilities stated “Funding is a major barrier. Our program is funded on a shoestring budget…and that’s an example of a lack of recognition among leadership, that it’s important or that’s something that should be happening.” Leadership, in comparison to frontline care providers, were described as more adherent to traditional treatment models, because of their longer tenure. One psychologist with primarily clinical responsibilities stated “We have a lot of young providers and those providers in particular are obviously gung-ho about this stuff. I think we have a window of opportunity to do something special. I just don’t know that we are capitalizing on that opportunity.”

System-Wide Dissemination Strategies

In 18 interviews (90%), processes or strategies to spread the use of BITs across the entire health system were discussed. Interviewees described the current dissemination practices such as word of mouth, interest groups, and limited press coverage as inadequate. One physician in a leadership role stated, “I think for a lot of these tools many of these are being diffused by word of mouth or within local communities and not necessarily rolled out with an implementation [strategy].” Interviewees provided a list of additional implementation practices that are needed such as provider training, clinical support materials, and direct to consumer advertising. For example, a psychologist with research and administrative responsibilities stated, “…materials could be developed to help support front line providers, particularly in primary care settings.” Most prominent among these recommendations was the need for a robust learning management system, i.e. an information technology resource that serves as a repository for BITs that both patients and providers can access. VHA has yet to fully develop or invest in such a resource. A psychologist with national administrative responsibilities stated, “We don’t have a learning management system for people to sign into where therapists can make a prescription for it and where it can be really part of the system.” Some interviewees discussed several current systems within VHA to manage BITs, describing the changes that needed to be made in order to better integrate them with other systems, such as secure messaging and the EHR.

Strategic Priorities of the Health System

Fourteen interviewees (70%) discussed the fact that health systems develop strategic priorities, which promote policies affecting the dissemination and implementation of BITs. Interviewees identified multiple current strategic priorities such as access to care, measurement-based care, EHR development, and maintenance of legacy health information technology systems, which overlap, but also compete, for funding with BIT development, implementation, and dissemination. One physician in a leadership position stated, “When you have mission critical business functions like your billing system, your appointment system, and your EHR, all of which need a lot of attention, getting a patient self-help tool to the top of a prioritization list for funding and development and roll-out can be a bit of a challenge.” Concerns voiced by media, Congress, and Veteran Support Organizations often direct funding decisions, rather than VHA leadership, who may have a clearer understanding of the complex needs of the system with respect to the system’s strategic priorities. Interviewees suggested that this climate incentivizes leaders to go for “simple wins” with respect to BITs, such as mobile application development that gain media attention, rather than planning for long-term system change. To this point, a physician with national administrative responsibilities stated, “[VHA] Central Office has been more about creating a ‘gee whiz’ moment, getting a little app out there, getting attention to yourself which may help your office,” while another physician in a network leadership role stated “It’s hard to have time and resources available for innovation when people feel like they’re constantly putting out fires or addressing mandates in other areas.”

Organizational Structure of Health System Administration

Twelve interviewees (60%) identified the hierarchical division of the health system’s administration into separate specialized offices as an important determinant. Two points supported this theme. First, the administration of care in VHA can be described as federated, whereby policy is generated by a centralized administrative group, but responsibility for care and system performance is maintained at the individual facility. This structure has the effect of siloing offices within the central administration and between the central administration and care facilities, preventing bidirectional communication about strategic objectives, performance, and facility needs. As observed by a nurse with local administrative and leadership responsibilities, “I think that one of the barriers is just the disconnect between what the people at Central Office say and what happens in the field,” while a physician with national leadership responsibilities stated, “We push out an awful lot of things at Central Office that we are not sure if they actually achieve a clinical objective or not.” Second, VHA Central Office relies heavily on the division of clinical care into service lines and the use of subject matter experts that focus on the treatment of specific clinical conditions, but may be relatively disconnected from primary care and care platforms, which cut across disease areas. One interviewee who is a physician in an administrative role stated, “Central Office is stuck farming out all of its clinical problems to the person who is the clinical subject matter expert and that serves Central Office politics but it has no relation to what happens in reality.” In relation to these points, BITs are a platform for the care of multiple behavioral and mental health conditions that cross these different divisions. Consequently, no one office coordinates the administration of BITs. This arrangement may support innovation as there are multiple centers developing BITs, but limits dissemination and implementation as there is no centralized authority on policy-making for BITs.

Health System Regulations and Policy

Eleven interviewees (55%) discussed a number of regulation and policy issues, primarily dealing with privacy and information security, which affect the use of health information technology, including BITs. Many BITs as well as most health information websites and applications recognize returning users and save consumer information. As an information security measure, VHA does not routinely allow the collection and storage of identifying information by websites or applications, nor does it allow the transmission of patient-reported information to the EHR or other health information dashboards. Consequently, the development and use of BITs that track users, as well as those that measure and report patient data are generally restricted within VHA. Thus, a consumer or provider’s ability to track BIT use or clinical progress is limited as the programs are not integrated with the EHR. To this point, one psychologist with administrative and research responsibilities stated, “If you’re a user and you go to the AIMs website [a VHA developed program for anger management], it cannot…keep up with where you are. It can’t keep any of the information you’ve entered…All of the data that you enter goes nowhere and is not saved and it’s a completely user-unfriendly experience.”

Interviewees also stated that communication with patients outside of the clinical encounter through platforms other than the telephone or the current secure messaging system is limited. For example, texting, push notifications, or “in app” messaging (a communication platform that is integral to a specific BIT), are not in use within VHA except for research protocols. One psychologist with clinical and research responsibilities stated, “You can e-mail yourself the data but you’re not allowed to e-mail it to your provider so there’s no way for the provider to get access to the data unless they say, ‘OK, can you go back to Thursday and tell me what you did on Thursday?’ So, then they’re forced to sit there and either look over the patient’s shoulder on their device or discuss it verbally.”

Interviewees also discussed that even when their technology development and use plans fell within current regulatory boundaries, project execution was difficult due to the number of steps and required approvals. Similarly, interviewees noted that individuals at the facility-level who are responsible for enforcing regulations are inconsistent in their interpretation of regulations. One physician with national leadership responsibilities stated, “That’s a challenge we face on a regular basis, working through the contracting and the like. If we’re going to use a commercial tool, it can be a sufficient enough barrier that sometimes we don’t pursue that.”

Discussion

The implementation and dissemination of BITs can potentially help integrated healthcare systems address the demand for accessible, low-cost, evidence-based treatment for behavioral health conditions. However, there is little use of such programs in integrated healthcare systems, including VHA, while major BIT implementation efforts have been difficult (Gilbody et al. 2015). In order to explore these issues and support the development of comprehensive strategies for BIT implementation and dissemination, we interviewed health system providers, administrators, and leaders familiar with current BIT practices within VHA, the largest integrated healthcare system in the U.S. Organizational-level determinants were numerous and overlapped extensively among interviewees who primarily described determinants as barriers to the dissemination and use of BITs. Some determinant themes were based on complex and longstanding structural and cultural issues, while other themes were less complex and may be amenable to policy and strategy changes. While all determinants were identified within VHA and may be more prominent in this healthcare system because of this sample, they likely apply to other integrated healthcare systems as well (Grol et al. 2013).

Interviewees identified the assimilation of BIT use into current service models as the most prevalent and complex theme. With the advent of hand-held devices and software designed for usability, we have become accustomed to the rapid dissemination of technology in many areas of daily life. However, BIT dissemination within healthcare systems requires the use of both technology and self-care by consumers, providers, support personnel and administrators. While health information technology use is becoming ever more common among these groups, technology use for self-care is still relatively novel (Tsai and Rosenheck 2012). One reason is that the current service models of many healthcare systems, even capitated systems such as VHA, still rely on traditional face-to-face care as the expected modus operandi and the basic measure of service delivery (Wasserman et al. 2001). Therefore, the depth of health system change required to deploy technology driven self-care resources cannot be underestimated, especially as the provider’s role in the provision of care via BITs varies and is, in some cases, still ambiguous (Schueller et al. 2017). Moreover, while VHA can be considered a leader among integrated healthcare systems for behavioral health outcome monitoring, quality improvement, and evidence-based program implementation, these processes have yet to fully address self-care delivered by BITs or methods to facilitate BIT implementation (Demakis et al. 2000; Jha et al. 2003). As health systems move to reimbursement models based more on quality outcomes rather than traditional face-to-face care, there will presumably be more pull to implement self-care and BITs as a way of increasing the ratio of positive outcomes to cost. VHA and other accountable care organizations are leading this transition—one driven by cultural shifts, legislation, and the implementation of other forms of health information technology-based care (Fortney et al. 2011; Tang and Smith 2016).

Other long-standing cultural and structural issues within VHA, likely found in other integrated healthcare systems as well, also need attention to improve BIT dissemination and implementation (Mair et al. 2012). While there are individual leaders and groups championing BIT use, the responsibility for the system-wide deployment of BITs is diffused over multiple offices within VHA. This structure has encouraged the organic development of BITs for a host of behavioral health issues but does not fully support the dissemination and implementation of those programs, which require integrated system-wide strategies and policies. Moreover, the leadership and organizational structures of many integrated healthcare systems such as VHA were developed to manage a model of care based on in-person services. Recent reforms focusing on access to care and wait times within VHA are prime examples of the focus on the provision of face-to-face care (Veterans’ Access, Choice and Accountability Act 2014). Thus, a recognition of and advocacy for the need to deploy self-care resources, which promote quality in the form of increased access utilization, irrespective of the format of service delivery, must be prioritized as highly as access to-face care to support BIT use.

Major structural and cultural changes are difficult for all large organizations, let alone the second largest department of the U.S. executive. However, VHA has paralleled other integrated healthcare systems in the initiation of several innovative patient-centered and health outcome-oriented programs for behavioral healthcare in recent years. First, VHA initiated a reorganization of primary care services toward a medical home model in 2010 to increase access to quality care and streamline the integration of services. The use of telecare and secure messaging is emphasized, as are stepped-care models, in the provision of collaborative care for behavioral health treatment in primary care. This transformation is associated with improved clinical outcomes, cost savings, and patient satisfaction, and has likely laid a foundation for the deployment of BITs (Nelson et al. 2014). Second, while the recent “Anywhere to Anywhere” initiative announced by the former Department of Veterans Affairs Secretary David Shulkin primarily supports virtual face-to-face care via VHAs robust telehealth program, this program will likely support the use of technology-driven self-care resources as well (Wicklund 2018). Finally, the recent decision to disseminate measurement-based care practices throughout VHA is another initiative that may eventually support the system-wide of use self-care and BITs (Fortney et al. 2016).

Other determinants identified were of a less complex nature and may be more amenable to changes in or clarification of policies, application of funding, or the development of strategies that could support BIT dissemination and implementation. Such changes, although identified in this group of VHA specific interviews, likely apply to other integrated healthcare systems and include the creation of a learning management system, supporting the development and testing of strategies that include provider and administrator education for the implementation of BITs at the point of care, and changing policies to securely accommodate the transmission of consumer data to providers and the health system. A description of these determinants along with general recommendations that may address them are listed in Table 2. We reiterate that such changes should not be viewed as simple, especially for a healthcare system where Congressional oversight and political considerations are paramount in an environment where overriding concerns are the security of personal health information. Moreover, we acknowledge that such changes may also compete directly or indirectly with other priorities of the health system, which we have documented.

Table 2 Recommended changes addressing system-wide determinants for the implementation and dissemination of BITs in VHA
Full size table

The goal of this study is to provide the healthcare leadership, operations, and research communities with actionable data to improve the dissemination and implementation of BITs. Our analysis clearly identified two groups of themes, one group directly affecting BIT use at the point of care with consumers and providers described in a previous publication (Hermes et al. 2018), and another described here that operates at the organizational level. We felt that these thematic groups diverged enough in content and applicability to various stakeholder groups as to warrant different manuscripts. However, the two thematic groups are complementary and to some extent the division is subjective. For instance, it is clear that policy changes at the organizational level can affect how providers and consumers interact at the point of care, in developing greater familiarity with BITs or improving strategies that provide face-to-face human contact that supports BIT use. Similarly, this issue highlights the limitations that theories describing technology use such as UTAUT, focused on factors related actual technology use by participants, have in describing how organizational factors such as policy, organizational structure, or culture affect the deployment of healthcare technology such as BITs. Future efforts should link the determinants identified here and those from previous work with theory-driven frameworks for implementation, as BIT use within healthcare organizations matures.

This study has several other limitations that must be noted. As with most qualitative research using semi-structured interviews, the sample involved a relatively small number of individuals and should not be considered representative of the official position of VHA or the opinions of all VHA providers, administrators, or leaders. As such, these findings may represent only an initial indication of the important determinants for BIT implementation and dissemination according to those who can be considered early adopters and likely the most knowledgeable about the platform. An understanding of these determinants may be strengthened by a larger sample size or the purposeful inclusion of individuals who have chosen not to use BITs in care. However, the consistency of themes among interviews was high, evidence that saturation was reached for a breadth of determinants. In addition, we could find no examples of clear counterfactuals, which may be due to several factors: (1) the natural consistency of themes as described above; (2) the inductive process of thematic analysis that may favor identifying those themes where counterfactuals are not present; and (3) only two themes, not the final eight, were identified in initial interviews and specifically queried in subsequent interviews. The relatively small sample size and consistency of themes across interviews also precluded an exploratory analysis of how themes may have differed by participant background characteristics. Future work could use the current findings as a basis pursue this objective using mixed or more quantitative methods. It also must be noted that determinants were developed from the perspectives of providers, administrators, and leaders. We did not undertake to verify these determinants with objective data. In addition, we interviewed individuals who were largely working separately, within a large organization. Interviewees may not have had a complete understanding of how the VHA health system is administered, changes currently taking place within the system, or plans for BIT implementation. For instance, the VHA’s Office of Connected Care and other groups have on-going efforts to develop and test technology platforms and processes allowing the flow consumer-reported health information to providers and the health record, of which some interviewees in this study may not have been aware (https://connectedcare.va.gov/). Finally, as our objectives were to develop and explore a broad catalog of organizational-level determinants for the implementation and dissemination of BITs across the VHA, we did not extensively analyze the effect the single interviewer’s perspectives and potential biases as a psychiatrist and BIT researcher may have had on the conduct of interviews or their analysis.

This study takes an important initial step in identifying a broad group of determinants affecting BIT deployment in VHA. These determinants point to the development of potentially effective dissemination and implementation strategies for the system-wide use of such programs, a worthy aim given that BITs will increasingly become an integral part of the progress toward a more patient-centered and democratized healthcare system (Tang and Smith 2016). However, the dissemination and implementation of such programs will not be a straight forward task, and the organizational determinants presented here may help health system administrators and leaders better plan for and facilitate this transformation. Future work in other integrated healthcare systems, as well as VHA, should add to these results and begin to develop and test the effectiveness of strategies to integrate the use of BITs in healthcare systems.