Introduction

Telemedicine, and telepsychiatry in particular, are burgeoning fields of both academic and clinical interest. However, analytic research into the impact and effectiveness of the teledelivery of health services has not kept pace with the proliferation of telehealth programs. Whereas most program descriptions express the goals of improved access to necessary services, reduced costs, convenience for patients, and increased knowledge among local practitioners, little is written about how to achieve these goals, and whether these goals are being met. Whereas studies have considered issues of user comfort and satisfaction with the technology,1 , 2 few have tested the educative and knowledge transfer potential of telepsychiatry programs.3 , 4 Some strong quantitative investigations and notable review articles on telepsychiatry have been published.5 Qualitative research in the area of telepsychiatry, however, with some exceptions is still notably in its infancy and has tended to be more descriptive than analytic.6 – 9

In July 2000, a pediatric telepsychiatry program was initiated by the Division of Child Psychiatry at the University of Toronto and was administered through The Hospital for Sick Children in Toronto, Ontario. Funded by the provincial Ministry of Children and Youth Services, the program was given a 3-year mandate to explore the feasibility and effectiveness of extending telepsychiatry services to 10 sites in remote and rural areas in Ontario. Children’s mental health agencies receive education and consultation services through the telepsychiatry program to assist the agencies in supporting local families with pediatric mental health concerns. The agencies have access, via a videoconferencing system, to pediatric psychiatrists with a range of expertise. The intended clients of the telepsychiatry program are the agencies, and not the families themselves, although it is reasonable to contend that they, too, benefit from the service. The program is thus designed as a professional support service and not as a method of delivering direct care to patients.

In Canada, which has a third-party payer health system, the effective and efficient delivery of health services to the almost 10 million Canadians who live in rural and remote areas is an immediate concern for government policy makers, health program administrators, and the rural residents themselves. The need for pediatric mental health services is urgent and significant. Families in rural areas of Canada may face more obstacles to obtaining services and support than those in urban areas, resulting from geographic, economic, and cultural factors.10 , 11 In sparsely populated areas, travel expenses increase the costs of both providing and obtaining care, and medical and social support services are frequently underresourced or are lacking altogether. It is difficult to recruit and retain medical specialists, who tend to concentrate in larger urban areas.12 Thus, telepsychiatry is increasingly proposed as a way to extend specialized expertise into areas where there are no resident psychiatrists. Whereas outcome studies of the impact of the extension of expertise into rural areas via video technologies are much needed, the study described below addresses a different gap in the literature, that of the lack of telepsychiatry user perspectives in the evaluations of these programs.

To assess the impact of telepsychiatry services on service provider agencies and on families in rural and remote areas, a qualitative pilot study was conducted that involved both open-ended telephone interviews and focus groups. The intent was to explore qualitatively the dimensions of the major impacts of pediatric telepsychiatric consultation services from the perspective of mental health service providers and family caregivers of children with mental health issues in rural areas.

Methods

Data were collected between March and April of 2002. Because of the evolving nature of the area of telepsychiatry and the goal of facilitating the voice of the service users themselves, an exploratory framework for the evaluation of the pediatric telepsychiatry service was chosen. This framework was developed by the authors using a participatory process that constituted a prior phase of this pediatric telepsychiatry evaluation project. This research and the participatory framework are described elsewhere.13 Two qualitative methods, focus groups and semistructured interviews, were utilized, each with a different subject population drawn from the same sites. Five sites were selected, in consultation with telepsychiatry program administrators, using maximum variation sampling14 to take into account the range of size of community, frequency of use of the service, and degree of rurality, among other important socioenvironmental and sociocultural factors. One selected site was ultimately unable to participate in the study. Because of lack of notice, time, and resources, it was not possible to replace this site with an alternate.

The use of focus group methodology and the participatory framework permitted the researchers to facilitate an organized discussion that maximized the amount of information collected.15 – 17 Focus groups are a well-researched and respected mode of qualitative inquiry, and a few words about the method are warranted. There are many definitions of a focus group in the literature, but features such as organized discussion,15 collective activity,16 , 18 social events,17 and interaction19 identify the contribution that focus groups make to social research. The main purpose of focus group research is to draw upon respondents’ attitudes, feelings, beliefs, experiences, and reactions that are most likely to emerge in a group context. Compared to longer-term observation, a focus group enables the researcher to gain a large amount of information in a shorter period of time, in part because the researcher follows an interview guide in a focus group. In this sense, focus groups are not natural but organized events. Focus groups are particularly useful when one wants to explore the degree of consensus on a given topic.20 The recommended number of people per group is usually 6–10.21 Depending on the subject of study, researchers using focus group methodology may conduct a single meeting with each of several focus groups,22 whereas others may meet with the same group several times. Holding focus groups in relatively neutral locations can be helpful for avoiding either negative or positive associations with a particular site or building.18

Focus groups were conducted by videoconference with mental health and welfare service providers associated with the rural and remote mental health agencies serving as pediatric telepsychiatry host sites. Although only four communities participated in the study, two focus groups were conducted in one community because of the significant geographical distance spanned by this community and the number of service providers interested in participating. Thus, a total of five focus groups were completed. The focus groups were attended by an average of seven service providers, with groups ranging in size from six to nine people. Participants represented various child health and welfare agencies and services. Among the participants were child protection workers, family counselors/child and family therapists, native child protection workers, mental health case workers and agency managers, family support workers, social workers, psychometrists, psychologists, Integrated Services for Northern Children workers, day treatment case managers, permanent care workers, and treatment unit workers.

The second component of the study involved semistructured telephone interviews conducted with caregivers of children having a first-time telepsychiatry consultation and living in one of the four participating communities. In-depth interviewing as a method allows researchers to engage in directed conversations with participants to elicit detailed and nuanced descriptions of respondents’ experiences, observations, and worldviews.23 , 24 Eligibility criteria for participation in the interviews for this study included the requirement that the caregiver reside with the child, the child had to be between the ages of 4 and 14, and had to have symptoms of an externalizing disorder as the primary reason for the referral to the telepsychiatry service. The selection of externalizing disorders as an eligibility criterion was based on a preliminary analysis of the first 200 referrals made to the telepsychiatry program, which revealed these disorders to be the most common reason for referral. Twelve interviews with caregivers were successfully completed, comprising the majority of the eligible families from the selected sites and meeting the eligibility criteria who received consultations during the data collection period. The relatively small sample group does not indicate a lack of need for service, but rather the combination of the limited time frame of data collection, small number of included sites, and eligibility criteria. Interviews were conducted in either English or French, according to the preference of the interviewee. Signed informed consent was obtained from all participants. In each interview, detailed notes were taken by the interviewer, and an audio recording was made of the interview, which was later used to check and validate the notes and confirm quotes. Ten of the interview participants were carers for a male child, and two cared for a female child. Eleven of the children resided with at least one biological parent, and one had lived with adoptive parents from birth. The average age of the children undergoing a telepsychiatric consultation was 9.3, with a range of 4–14 years. The average age of caregivers was 34.8 years and ranged between 20 and 54. One third of caregivers spoke French as a first language. Caregivers estimated the length of time that their children had been experiencing mental health problems to be between 1 and 10 years.

Detailed observational and verbatim field notes were taken during focus groups and interviews and constituted the textual data. Analysis of the textual data involved thematic content analysis, a dynamic form of analysis of verbal data oriented toward summarizing the informational content and describing the patterns and regularities in the data. Each member of the three-person research team reviewed the data transcripts first to gain an overall sense of the content, and then again to identify important themes and emphases. These themes were then discussed and agreed upon by the team as a whole and formalized in a codebook used to standardize the coding of the transcripts. This approach to the analysis of qualitative data follows a generally understood analytic process of reading transcripts, noting and comparing themes for coding, transforming codes into categories, and finally, contrasting and reviewing themes among the members of the research team.14 , 24 , 25 In the case of this research, the process of thematic coding was facilitated by the categories established in the participatory framework used in conducting both the interviews and the focus groups.13

Within the context of qualitative research, issues of reliability of the data and validity of research findings are included under the umbrella of trustworthiness, which comprises attributes of credibility, transferability, dependability, and confirmability.26 , 27 Team-based peer debriefing was conducted with relative agreement on emerging trends and observations, prolonged engagement with the topic in terms of immersion in the literature occurred, and an audit trail of field notes, meeting minutes, and analytic decisions were available to research team members.28

Results

Many important themes emerged from the analysis of the data as a whole. Here, key findings are extracted and highlighted. The pervasive message from both service providers and caregivers was that pediatric telepsychiatry is a much needed and very welcome service. There were also, however, concerns expressed about the limitations of the available services, both in terms of the extent of the services available via the technology and in terms of support and management services available locally. A major concern of caregivers and service providers alike focused on the need for further and more elaborate local support and medical services to oversee the implementation of treatment recommendations resulting from telepsychiatry consultations. More detailed analysis of the data highlighted the particular importance of two key functions served by the availability of telepsychiatry services, namely, enhanced capacity for service providers and reduced burden on caregivers, as well as a key frustration with the limitations of available telepsychiatry and support services, which was described by both service provider and caregivers.

Enhanced Capacity of Service Providers

Feelings of enhanced capacity among frontline workers with mental health and social service agencies in remote and rural areas were identified by the service providers themselves. For service providers, the local availability of telepsychiatry services has resulted in access to a specific mental health expertise. This access has, in turn, increased their knowledge, their confidence, and their sense of competence in assisting their clients and has decreased the feeling of isolation. This has had a trickle-down benefit for families in terms of the local availability of more expert assistance. In the words of the service providers:

[Previously] it was embarrassing to be a mental health agency and not have access to psychiatric services.

I have changed some of the ways I practice, the questions I ask clients.

The doctors support us, they give the clients reassurance that we are doing a good job, that we know what we are doing.

These findings highlight the importance of the educative component of the telepsychiatry program. The results point to the need for additional mental health training for rural service providers in order for them to successfully support their clients with mental health needs and their families.

Reduced Burden on Caregivers

The families of children with mental health issues interviewed for this study indicated that in their opinions, the availability of telepsychiatry services in or near their communities has led to reduced expenses associated with travel and a reduction in missed time at work for caregivers and missed days at school for the ill child and siblings. They also felt that the availability of local services has led to an increased probability of being able to remain active members of their communities while still obtaining help for their children. For many caregivers, an essential benefit of local services is that they are able to minimize the disruption to their paid employment caused by their child’s illness. Overall, this benefits the stability and well-being of the family unit. If this is indeed the case, increasing the number and range of health services available locally has the potential to contribute to the stability of rural and remote communities. Finally, the availability of the telepsychiatry service has resulted in shortened stressful wait times until expert help can be obtained, and the problems affecting the life of the family can begin to be resolved. These findings are perhaps best reinforced in the words of caregivers themselves:

I appreciated the fact that we could do this without travelling, or being on list for months. It attends to the problems right away, gives suggestions that we can put in place.

I honestly felt it was very productive. I felt that with the things [the psychiatrist] did suggest, and with the medication changes, things could get better for our family... He gave me hope that life that can be better than what it is now. Not unrealistically—he said it’s not going to be perfect. But it can be better.

Frustrations with Limitations on Available Telepsychiatry and Support Services

Both family caretakers and service providers expressed profound frustration with the distinct limitations of the existing telepsychiatry services. Their primary concerns focused on whether a one-time consultation with no follow-up and little local expert support could really address the children’s needs. Service providers and caregivers describe a two-pronged need, for both increased access to a wider range of telepsychiatry services and a larger cadre of well-trained mental health workers working within the rural communities. The caregivers most powerfully summed up these concerns:

I have one question: How can you help a kid with this kind of program—one session that is half an hour to one hour in length?

When the kid needs help, what the [consulting] psychiatrists are getting is a before taste, but [they] don’t get the rest of it. An evaluation—that’s the starting point—but if the kids need psychiatric treatment, there is none locally. They [the psychiatrists] make recommendations, but there is no follow-up.

Implications for Behavioral Health

Clearly, there is a need for more in-depth research into the diverse challenges of providing and using telepsychiatry services, especially if health policy planners hope to extend the reach and range of telepsychiatric care in the future. Further qualitative, quantitative, and longitudinal investigations would add immensely to the small body of existing literature on pediatric telepsychiatry. Longitudinal studies could better evaluate the impact of locally available services on family functioning, which may be improved by the ability to remain in the family home and continue with daily routines while receiving care. Costing studies would highlight the broader impacts on family well-being of reduced expenses and missed employment created by access to health care consultations conducted locally. To maximize the utility of existing telepsychiatry services, greater flexibility in terms of time and scheduling of appointments to accommodate the work and school day could further reduce missed employment and education, likely further contributing to family well-being. It is essential to note that in the planning of research, the importance of contextual sensitivity when dispensing advice and planning treatment regimens from large urban centers for small, rural, and remote areas cannot be overstated. It will be important for policy makers in particular to understand, too, the extent of the shortage of qualified mental health providers in rural and remote areas, and how this impacts the uptake of care recommendations made during telepsychiatry consultations. Clarification and standardization of the priorities and goals of telepsychiatry programs would assist in the planning and delivery of services, particularly in terms of how best to divide available time and funds between individual case consultations and broader educational programs to enhance capacity among service providers. Although pediatric telepsychiatry services can contribute to improvements in children’s mental health in rural and remote communities, the need to make concerted efforts to increase the concentration of expertise in rural and remote communities, rather than providing expertise by means of remote delivery, remains paramount.