Introduction

Primary care practice is complex and unpredictable. Patients present with a sometimes bewildering set of complaints that can stem from a wide variety of bio-psycho-social domains. In order to address these complaints, the “selection, combination, and order of application of resources and activities vary according to the [specific] requirements of the problem at hand.” (Stabell and Fjelstad 1998) Healthcare is usually provided by interprofessional teams, but training is mostly done in profession-specific silos. These siloed training experiences lead to different profession-specific cultural perspectives. Culture has been defined as:

“Shared patterns of behaviors and interactions, cognitive constructs and understanding that are learned by socialization. (Zimmerman 2017)

Different professional cultures contribute to the challenges of interprofessional teamwork (Hall 2005). They can lead to incompatible discourses and ongoing power differentials and conflict. (Haddera and Lingard 2013), (Paradis and Whitehead 2015)

In 2011, the Department of Veterans Affairs (VA) sought to address these issues through a large scale transformation of interprofessional training (Centers of Excellence for Primary Care Education, Request for Proposals, 2021). They created five Centers of Excellence in Primary Care Education. Each site was required to design, implement, and assess new interprofessional curriculum focusing on four broad domains: shared decision-making, interprofessional collaboration, sustained relationships, and performance improvement. Sites were further encouraged to develop and share specific curricular elements based on an experiential ‘apprentice’ model of learning (learn by doing, proximate to expert performance, graduated responsibility). Our study site was one of these initial Centers of Excellence.

One of our core assumptions as we began this journey was that the social structure of our clinics and teaching system would significantly affect function (Merton, 1968). We hypothesized that these structures could range from a rigid hierarchical structure to an egalitarian ‘small world’ network (defined below). Hierarchical structures (top down, command-and-control) are common in health care. (Haddera and Lingard 2013; Currie and White, 2012). They are efficient and are good at executing stable evidence-based algorithms, such as executing a ‘Code Blue’ in a hospital setting, but they have difficulty with flexibility. Top down configurations have trouble in complex environments, where adaptive precision (novelty, creativity, context awareness) is key, because there is often a lag time between the dynamic requirements of tasks and the understanding of the leaders in the hierarchy.

On the other end of this spectrum, the “team of teams” structure, in graph theoretical mathematics, is known as a small-world network (i.e., six degrees of separation). Examples of small-world networks include electric power grids, airline route maps, and social influencer maps. These are.

Highly clustered like regular latices, yet have small characteristic path lengths, like random graphs. They have specialized regions and yet exhibit shared or distributed processing and tend to facilitate communication, decision-making, and resilience. Small-world networks are stable. They resist change due to the filtering apparatus of using highly connected nodes. They are also efficient, being effective in relaying information by keeping the number of links required to connect a network to a minimum. (Watts and Strogatz, 1998). 

As an example of this difference between hierarchical and small-world teams, when General Stanley McChrystal took command of the Joint Special Operations Task Force in Iraq the traditional hierarchical command structure was not working. He and his staff created a “team of teams” structure that combined transparent communication with decentralized decision-making authority (small-worlds). Outcomes improved dramatically. (McChrystal 2015).  

Empirically, small-world networks have been shown to be superior to hierarchical networks in human systems for intelligence gathering in NATO (Stanton et al., 2012) and for creating more patents in innovative multinational corporations (Song et al., 2014). In health care, small-world networks have been found to have lower hospital costs and fewer readmissions than hierarchical teams (Uddin et al., 2013). Jippes, et al. (Jippes et al., 2013) studied dissemination of new innovations. They found that the efficacy of dissemination was better predicted by centrality in a small-world network than by participation in a train-the-trainer course. Holtrop, et al. (Holtrop et al., 2017) found that care managers embedded in the small-world network developed more communication, trust, and engagement than those working more remotely. Because of this evidence, one of our key program outcomes was to increase small-world social network structure in our training clinic.

Social network analysis (SNA) is a tool that examines actors (as nodes) and relationships (as connecting lines). It can reveal whether there is interaction between different nodes, including the strength, direction and mechanisms of the interaction. To understand SNA better, imagine a connected graph of internet users that shows the strength and direction of interactions. Social influencers would have several thick lines (high strength) directed outwards to multiple users.

SNA was one of the primary outcome measures in both the Jippes (Jippes et al., 2013) and Holtrop (Holtrop et al., 2017) studies mentioned above. Smit et al., (2021), created a six-week, competency-based interprofessional collaboration in practice (IPCP) curriculum consisting of face-to-face discussion, on-line training, and observation of colleagues. Pre/post SNA evaluation showed a larger, more collaborative, diverse interprofessional network after the curriculum. Yao et al., (2018) used electronic medical record data and SNA to study the care of 100 surgical colorectal cancer patients. They identified 6800 unique users representing over 150 roles or occupations. They were able to identify the development of clusters of professionals with frequent interactions (small-worlds?). Phillips, et al., (2016) used SNA to asses a 6 h interprofessional care curriculum consisting of knowledge, psychological strategies, and collaborative practice. Using SNA, they demonstrated a sustained increase in interprofessional referral networks. Although rare, the use of recurrent graph networks has been used to improve prediction of growth cascades with novel information (Huang et al., 2019). Large reviews by both Chambers et al. (2012) and Sabot et al. (2017) conclude that SNA shows promise for various analyses in health care settings and is underutilized and tends to use weak designs, such as self-report surveys. We did not set out to study SNA at the onset of this educational program. However, we were intrigued by these data and wished to test SNA despite the limitations of our data source (no strength or direction data).

Therefore, the objective of this pilot study was to assess the ability of repeated social network analyses (SNA) to signal a potential structural change from hierarchical to small-world networks as part of a multi-method program evaluation of an educational intervention. Our rational was to assess the potential of SNA to provide important data to evaluate educational programs, particularly if strength and direction of interaction data could be obtained.

Methods

This evaluation proposal was formally reviewed by the Administrative Officer/Research and Development at our medical center and was designated as a program evaluation and exempt from further human subjects review.

The study site is a health care system in the United States that has > 100,000 eligible patients within a 160 mile radius and serves military Veterans. It consists of a 46-bed hospital, an 11-bed inpatient substance abuse facility, a 28-bed nursing home, a large co-located ambulatory care facility, and five community-based outpatient satellite clinics. It is primarily affiliated with one medical school, one school of pharmacy, and four nursing schools. It trains professional students from social work, nursing, medicine, and physician assistant programs and post- and pre-graduate trainees (i.e., residents and fellows) from psychology, pharmacy, nurse practitioner, internal medicine, psychiatry, and family medicine.

We conducted this pilot study as a repeated SNA, performing individual SNA analyses sequentially on each academic year’s full data set, to assess whether structural change might be detected in our educational intervention using this method. This study was conducted on data from the last four years of phase one of the program (2013–2016). Data from the first year was excluded because curricula were being designed and organizational details were being worked out such that trainee exposure was rudimentary, making it inadequate as a baseline.

We chose the data for our analysis as co-presentation of lectures, particularly between professions. We chose this for two reasons:

- If the relationship between these training programs was indeed changing, then having more than one perspective of any topic during a lecture would become a cultural norm. This would be reflected in more egalitarian interactions between professions.

- Co-presentation represented at least two of the areas of focus for the greater program, interprofessional collaboration and sustained relationships.

In early 2017, data collection was conducted. We obtained the speaker schedules for all academic conferences in each core training program (psychology, nurse practitioner, pharmacy, medicine) for each entire year, for each year of the study (2013–2016). Data for SNA input is summarized in Table 1. We were not able to assess the strength of interaction or direction for this data.

Table 1 This table shows the total number of lectures in each department (not just in the educational program) over the four years of the study. One can see that the total remains relatively stable, while the percentage of co-lectures is increasing
Full size table

For each SNA, we created NxN matrices, where N was the total number of speakers, which were coded by profession. Each matrix element was filled by the number of co-presentations between that teacher pair, that is, when both presenters co-delivered a lecture. These matrices were imported into Ucinet (Borgatti, et al. 2002). In the graphical analysis, each speaker was a node and each co-presentation (interaction) was an edge. For each year, we calculated the clustering coefficient, the small-world index, and Eigenvalue centrality. We then graphed all speaker pairings (See Fig. 1).

Fig. 1
figure 1

This figure explains the components of our Social Network Analysis (SNA) using two examples made of 15 nodes each: a command-and-control structure and a small-world structure. A The Clustering Coefficient measures the extent of ‘grouping’ between nodes. Are there relationships within multiple node clusters and between clusters? Small-world networks demonstrate high clustering coefficients due to small independent groups working together on a specific project. To share resources and information they have relationships with other node clusters. The command-and-control structure nodes are hierarchical without relationships between clusters or within groups. B Small World Index measures how easy it is for two nodes to share resources and information with each other. It is inversely proportional to average path length and is also a function of the number of nodes. Do they have to go through several channels or layers to communicate (high path length)? Or, can they communicate almost directly (low path length)? Average communication across the small-world example is shorter, while the command-and-control example requires communication to pass through multiple layers and is a longer process. C Eigenvalue Centrality measures how influential, or connected, a node is to other nodes. The larger the node, the more influential or connected it is to the network, representing higher influence (eigenvalue centrality). Do relatively larger nodes connect with other large nodes throughout the system? Small-worlds demonstrate high eigenvalue centrality that is dispersed throughout the system, while the command-and-control example has overall low eigenvalue centrality due to the less interconnected nature of the system

Full size image

The clustering coefficient is a measure of the degree to which the nodes of the graph tend to cluster together. For instance, if your friends all know one another, you have a high clustering coefficient. The small-world index is inversely proportional to average path length between any two nodes and is also a function of the number of nodes in the network. So, if the larger group is made up of several tight cliques like your friend group, the clustering coefficient would also be high. These should both increase as the structure changes from a hierarchical network to a small-world network. Eigenvalue centrality is a measure of the influence of a node in the network. In the graphic displays, the larger the node, the more important the node is to the function of the group. Eigenvalue centrality would be high for ‘influencers’ in social media. This would be expected to change from a single dominant group in hierarchical models (typically physicians in health care) to more balance between the node sizes of each profession.

Results

Outcomes were depicted by SNA graphs that covered the four years of the study. The teachers associated with the educational intervention are located inside the circles. Other teachers not associated with the educational intervention are located outside the circles. We paid particular attention to the number of co-presentations (thickness of lines) between two ‘teacher’ nodes, the amount of influence each participant had (size of node), and the complexity of groupings formed. Changes outside the circles (in non-participating teachers) were considered to potentially be to diffusion of norms.

Year one

Year one of the study showed a rather hierarchical structure with a physician as the apex node. There was also very little co-teaching outside the program participants in the rest of the academic institution in that year. The clustering coefficient was 0.5 and the ‘small world’ index was 3.149. These are both rather low and suggest very little ‘small world’ behavior (See Fig. 2).

Fig. 2
figure 2

= physician; = nurse; = nurse practitioner; = pharmacist; = psychologist; Node size = Eigenvalue centrality (importance in the network); Tie thickness = number of co-teaching events

Social Network Analysis for year one of the study. The center oval represents the program boundary, e.g., teaching dyads occurring within the program faculty. Single instructor teaching not shown. Small dyads outside the oval represent non-program co-teaching in the broader institution.

Full size image

Year two

Year two shows that the structure inside the program is flatter and more extensive, with several professions having significant Eigenvalue centrality (importance in the network, represented by node size). There is more non-program co-teaching outside, suggesting some influence of program examples on the broader training program teaching norms. The clustering coefficient was 0.555 and the ‘small world’ index was 30.809. These values suggest a moderate amount of ‘small world’ behavior within the CoEPCE (See Fig. 3).

Fig. 3
figure 3

Social Network Analysis for year two of the study

Full size image

Year three

Year three shows greater network importance (Eigenvalue ratio) for several nonphysician members and a more interconnected and flatter network. The clustering coefficient was 1.167 and the ‘small world’ index was 27.459, suggesting continued evolution toward ‘small world’ behavior. Their continues to be a significant amount of co-teaching outside the program, and it is taking on more complex and interconnected structures (See Fig. 4).

Fig. 4
figure 4

Social Network Analysis for year three of the study

Full size image

Year four

Year four demonstrates continued flattening and clustering of the program participants. High Eigenvalue centrality is now seen for all core disciplines, and the highest was a psychologist. Co-teaching outside the program is expanding. The clustering coefficient is now 1.27 and the ‘small world’ index is 31.39, suggesting a more mature ‘small world’ functional structure.

Over the four years of the study, clustering coefficients increased from 0.5 to 1.027, the ‘small world’ index increased from 3.149 to 31.39, and Eigenvalue centrality grew to include persons with large values (more importance) from each profession. All of these suggest a change from a physician-centric hierarchical to an egalitarian ‘small worlds’ social structure. Coteaching events outside the program also increased and developed more relational structure (connectedness), suggesting an influence on institutional norms of co-teaching (See Fig. 5).

Fig. 5
figure 5

Social Network Analysis for year four of the study

Full size image

Discussion

Social network analysis (SNA) is a sensitive method for detecting social structure. This was a pilot study to assess whether repeated SNA might detect a structural change from hierarchical to small-world structure that we presumed was occurring. We assumed this (based on our curricular goals and the experience of others) delivering a curriculum like ours. Similar to Yao et. al., (2018) we were able to demonstrate increased clustering of professionals with increased interactions. Smit et al., (2021) evaluated a curriculum intervention with pre/post SNA that demonstrated larger, more collaborative, and diverse professional networks. While their curriculum was 6 weeks long, ours was four years long and showed larger effects.

Our analysis supports the use of repeated SNA to document important structural changes, moving away from a hierarchical physician-in-charge structure toward an egalitarian small-world structure. It also suggests a normalizing influence toward co-teaching in the broader academic institution. Further.

There are several limitations of this study. We were not able to obtain data on the direction of influence or strength of the interactions. This weakens any causative inferences. It was conducted at a single institution. We used co-teaching events as our measure, which may represent confirmational bias (selecting data that fit our expectations). Other measures, such as recording text messaging between individuals, might have shown different results. We studied only the last four years of data from stage one and may not have observed broader patterns in the project. The largest limitation is that, for this study, we did not examine the rich interactions between actors and how they changed over time. Therefore, we cannot make causative inferences about the mechanisms of any structural change. Although it shows promise, further mixed method studies that include repeated SNA should be done prior to wide adoption of this as a program evaluation technique (perhaps using electronic communication between teachers, which could be graded for strength and contain directionality). Although we have presented a simplified version of SNA and small-world networks, these are very complex and somewhat controversial.

Conclusions

Co-teaching, as demonstrated graphically, increased over the four years of the study. Clustering coefficients, the small-world index, and Eigenvalue centrality all showed meaningful increases. Centrality grew a to include persons with more influence from each core profession. All of these suggest the change from a hierarchical physician-centric social structure to an egalitarian interprofessional small-world social structure. Co-teaching events outside the program also increased and developed more complexity, suggesting an influence on institutional teaching norms.

One benefit of Social Network Analysis was the opportunity to visualize, as a marker or indicator, the social change that was occurring. If further validated, results such as this could be used to represent the outcome or impact of an educational program. They could be used to help reinforce leadership buy-in for continued support for the program and highlight the role of highly connected individuals (connectors) in role-modeling behaviors. We believe that repeated Social Network Analysis appears to work well for documenting structural change as part of a program evaluation and should be studied further.