Significance Statement

What is already known on this subject? Effective leadership development programs can improve capacity and job performance among public health professionals and are an important part of continuing education of leaders in professional areas.

What does this study add? This study illustrates the breadth of how MCH leaders implement their skills and impact both their organizations and the populations they serve. Further, in addition to MCH Knowledge Base, Self-Reflection, Ethics, and Critical Thinking, three additional areas of key interest to these leaders emerged from the data: networking, confidence, and career advancement.

Introduction

Skilled leaders in Maternal and Child Health (MCH) are needed to help address the significant and complex challenges facing communities today. The Maternal and Child Health Bureau (MCHB) has a long-standing commitment to training leaders, as is evidenced by the funding of multiple training programs (Belcher et al. 2015; Dodds et al. 2010; Margolis et al. 2013; Rosenberg et al. 2018) and the development of the MCH Leadership Competencies (MCH LCs) (Health Resources and Services Administration 2018; Kavanagh et al. 2015; Mouradian & Huebner 2007). While several contemporary researchers have highlighted the ongoing need for leadership development in the field (Baer et al. 2015; Belcher et al. 2015; Fernandez et al. 2015; Grason et al. 2012; Reynolds et al. 2015; Streeter 2015), the call to train public health leaders has spanned decades (Fraser et al. 2017; Halverson et al. 1997; Institute of Medicine 1988; Lachance & Oxendine 2015; Olson 2013; Roper et al. 1992).

The Maternal and Child Health Public Health Leadership Institute (MCH PHLI) has been described elsewhere (Fernandez et al. 2014, 2017), however, in brief, MCH PHLI was a 13-month experience combining three intensive in-person residential training components (13 days of training with 82.5 h of face-to-face learning expereinces) with a robust distance-based (synchronous and asychronous) education and coaching program in which participants completed an individual leadership project. The program was conceived to address the 12 MCH LCs and was thus driven by three educational objectives: 1) to enhance individual leadership skills, 2) to develop knowledge and skills for organizational leadership and change, and 3) to broaden the MCH systems perspective in current MCH leaders.

Up to 30 participants enrolled annually, with recruitment targeted toward leaders working in Title V and similar positions across the US and territories, family leaders, and associations serving the MCH workforce. Participants were recruited via national outreach, primarily using social media, email, conferences, brochures, MCH regional phone calls and Partner Boards. Applications were competitively reviewed by a selection panel with approximately 20% of applicants were accepted for each Cohort. The project was fully funded by the MCHB from 2009–2014 for all participant travel and training.

The MCH PHLI incorporated leadership development strategies that have been tested and validated with a variety of public health, academic and health care audiences (Dodds et al. 2010; Fernandez & Steffen 2013; Fernandez et al. 2012, 2016a, 2016b, 2017; Margolis et al. 2013; Orton et al. 2007; Saleh et al. 2004; Umble et al. 2006, 2011a, 2011b). The program coupled adult-learning theory based workshop-style components and simulations with 8 evidence-based psychological and leadership assessment tools commonly used in high level leadership training. Workshops and simulations were provided by nationally prominent faculty. Previous studies of participants in the MCH PHLI have explored participant-directed learning choices (Fernandez et al. 2017) and have shown statistically significant increases in self-reported skill competency across the twenty different skill areas targeted in the program (Fernandez et al. 2014). Yet those data illustrate neither how participants implemented their skills nor if they found any additional skills to be of key importance. This study examines how the 111 participating MCH leaders characterized their implementation of personal leadership skills. The goal was to gain greater understanding of how participants’ personal leadership develoment not only led to changes in their individual understanding, knowledge and skills, but also how developing these “Self” level of skills might lead to impacts and outcomes in the organizations and systems within which they work. The program was supported in full by Project T04 MC12783 from the Maternal and Child Health Bureau (Title V, Social Security Act), Health Resources and Services Administration, Department of Health and Human Services.

Methods

Design

For this analysis, a qualitative case study design was utilized to gather examples of how MCH PHLI participants implemented their learning from the program at the MCH LC “Self” level to effect subsequent impacts in their organizations or their community, with the classic Kirkpatrick Four-Level Training Evaluation Model (Kirkpatrick and Kirkpatrick 2006) (Fig. 1) serving as a guide. For the purposes of this study, an impact is defined as a change in a target audience, or an activity the Fellow presented as a measurable quantity (for example, development of a new program in their agency, improved skill level of a team they lead, changes in local policies, etc.) as a result of skills obtained through the MCH PHLI. The Consolidated Criteria for Reporting Qualitative Studies (Tong et al. 2007) is utilized to guide reporting of findings in this manuscript.

Fig. 1
figure 1

The Kirkpatrick Program Evaluation Model Applied to MCH PHLI

Full size image

Participants

Of the 113 participants enrolled, 112 (99%) successfully completed the MCH PHLI program, representing 46 states and territories. One participant declined to provide consent for use of their data, bringing the total sample size to 111. MCH PHLI Fellows ranged in ages from 28 to 64 years and were overwhelmingly female (97%). With respect to racial diversity of the cohorts, 67% reported as Caucasian/White, 25% African American/Black; 1.8% Native American/Alaska Native; 2.5% Asian/Pacific Islander, 3.5% with 2 or more races; and 7% identifying as Hispanic. All participants worked in organizations focused on maternal, child, and adolescent health; most served at the mid- to senior levels of management or leadership. Of these participants, 58% were Title V–related professionals, 20% were family leaders, 8.0% were representatives of professional organizations serving MCH, and 13% representatives of other groups (i.e., federal system participants, county agencies, etc.). Some participants were both family leaders and Title V employees. In addition to the special focus brought to the program by the Family Leaders (parents of children with special health care needs), participants represented a wide range of disciplines including medicine, social work, law, nursing, nutrition, public administration, business, marketing, maternal and child health, public health, and psychology.

Data Collection and Analysis

About one month prior to the final retreat participants were asked to reflect upon their training and prepare a 4–5 min oral presentation. They were instructed to answer the question, “What is your Ripple Effect? How did your experience in the MCH PHLI help you to impact or touch the lives of others?” Participants were instructed to consider the individuals, organizations, or communities they had reached as a result of using the skills gained during the training year, in essence to describe the difference they had made to others during the previous year. Broad parameters were purposefully given in the presentation instructions to allow participants to be responsible for determining which activities were most meaningful. Examples were provided upon request. The presentation was intentionally brief so that participants were required to make critical decisions about what information was most valuable to share and to be purposefully succinct and articulate. Presentations were made to their peers and MCH PHLI program staff at the program’s final retreat. This study was reviewed by the UNC Institutional Review Board and was determined to be exempt (UNC IRB Study # 11–0715.)

Two of the three investigators were in-person for the data collection. One investigator facilitated the presentation sessions and another was onsite and took notes during the presentations. One investigator is a faculty at the MCH PHLI hosting institution (University of North Carolina at Chapel Hill), one was the program evaluator, and one was an MPH candidate at the time of analysis. All investigators are female and have extensive training and experience in qualitative research methods, and in the MCH LCs (Health Resources and Services Administration, 2018) that guided the coding process. Two of the investigators had in-person interactions with the participants prior to data collection event through their roles as staff on the MCH PHLI program. Participants were aware of MCH PHLI program goals, that the information provided in their Ripple Effect presentations would be utilized for program evaluation, and that findings may be used for publication in the future.

The live presentations were recorded and transcribed for analysis. Three investigators independently coded the presentation transcripts for themes related to the research question. A content analysis approach was utilized, and coding was guided by the four MCH LCs under the “Self” level. Three new codes emerged during the process (Networking, Personal Confidence, and Career advancement) (See Table 1 for definitions.) Each reported activity was assigned at least one code. Some leadership activities were coded multiple times if applicable to more than one competency, however, codes were applied conservatively. For example, the code “Self-Reflection” was used in only those instances in which self-reflection skills were specifically mentioned; as opposed to when “self-reflection” was implied by the actions taken by the participants. Activities were also analyzed for scope of impact. The independently coded transcripts were then compared to validate the selected codes. Any discrepancies in selected codes were discussed by all three investigators. Code definitions were extensively reviewed during the comparison discussions, final codes were only assigned if a competency was specifically described, and final codes were only assigned to activities for which all three coders were in agreement. The research team then tallied the number of times each code was assigned. To protect participant anonymity, no distinctions were made between participants from different categories (for example family leaders vs Title V employees or Title V employees vs. county/federal-level workforce). Names of states or other highly identifiable information was redacted before reporting.

Table 1 Definition and Frequency by MCH Leadership Competencies 4.0 “Self” level and four newly emerging codes among Maternal and Child Health Public Health Leadership Institute participants
Full size table

Results

In total there were 562 examples of the leadership activities reported by MCH PHLI participants which were coded and classified under the four families of competencies and three newly emerging ones. Each competency family below is presented in order of most frequently cited outcomes/impacts and includes an exemplar quote from the data with identifying information redacted. Table 1 illustrates how these codes align with the four MCH LCs plus the three newly emerging codes. On average, in their short presentations program participants each cited 5.67 examples that fell into these seven codes, with most participants mentioning more than one code in their presentation. Exemplar quotes from the data are provided for each category (see Table 2).

Table 2 Example quotes for each coded competency in the MCH Leadership Competencies 4.0 “Self” level and four newly emerging codes among Maternal and Child Health Public Health Leadership Institute participants
Full size table

Self-Reflection (MCH Competency #2)

Across all cohorts, 87 participants (78%) shared 170 examples of self-reflection-related activities. Participants reported a greater sense of self-knowledge and self-awareness as a result of the MCH PHLI components such as assessments, personal coaching sessions, and interactions with other participants. A common theme emerged as participants indicated their involvement in the MCH PHLI helped them to better understand and appreciate the differences between themselves and others, and particularly how they interact with the people they work for and with.

Critical Thinking (MCH Competency #4)

Improved critical thinking skills were cited 116 times by 70 participants (63%) in their reports. Twenty participants (18% of total) reported the use of improved critical thinking skills in the development of important new programs in their home agencies, particularly in the areas of curriculum development, mentoring program development, and training program development for Title V staff and family/parent leaders. Dealing with the results of budget cuts, two participants reported their use of critical thinking skills specifically to come up with solutions for these challenges.

Newly Emerged Competency/Concept: Confidence

A total of 51 participants (46%) shared 78 instances of impacts in the area of confidence. Areas in which increased confidence were reported were the ability to take on difficult tasks, increased awareness of one’s identity as an MCH practitioner, the ability to speak to stakeholders such as legislators, and the ability to take on new and expanded responsibilities, among others.

Newly Emerged Competency/Concept: Networking

Increased networking was reported 72 times by 53 participants (48%) as an impact of involvement in the MCH PHLI. One theme that emerged is that participants viewed the relationships they had developed within their cohort as important resources for improving their work at their home agencies. Participants reported that the development of a network within their cohorts led to a better understanding of different aspects of MCH practice, shared knowledge, sharing of best practices, use of fellow cohort members as resources in future programming, and development of a nationwide resource sharing network. In addition, participants reported that the networking within the MCH PHLI increased their awareness of the value of family participation in MCH programming. By design, the MCH PHLI includes family leaders in each cohort, which is an important factor in increased awareness of the value of family leadership. Title V and other agency leaders reported how networking with family leaders improved their understanding of the role of families and their ability to speak with family leaders in their home states.

MCH Knowledge Base (MCH Competency #1)

An increase in MCH knowledge base was reported 47 times by 36 participants (32%); impacts which affected either themselves or the people they work with. Participants discussed expanded knowledge, awareness, and utilization of various key MCH and Public Health concepts such as Life Course Theory (Pies & Kotelchuck 2013) the MCH Pyramid, a tiered pyramid-approach to public health (Health Resources and Services Administration n.d.), the MCH Leadership Competencies (Health Resources and Services Administration, 2012), population-based programming, and focusing on prevention. MCH skills and perspectives gained through the MCH PHLI were used in a variety of programs that target specific MCH issues in the arenas in which participants work. Additionally, 16 participants (14%) reported that the MCH knowledge gained from the MCH PHLI gave them a broader understanding of MCH issues and 21 (19%) noted that it provided a framework for the programs they lead.

Newly Emerged Competency/Concept: Career Advancement

A total of 34 participants (31%) reported 42 instances of career advancement as a result of their involvement in the MCH PHLI. Examples include using interviewing skills learned at the MCH PHLI to secure a new job, realizing the job they were in was a poor fit and finding a new position, getting promoted within their agency, publishing an article in a professional journal, and expansion of influence in their current job or in their professional community, among others. One public health employed-Fellow reported that she was able to use MCH PHLI skills to obtain a faculty appointment so that she could be involved in the pipeline of workforce development, while maintaining her current Title-V role.

Ethics and Professionalism (MCH Competency #3)

Participants reported that incorporation of leadership skills developed in the MCH PHLI led to improved ethical practice, with 33 participants (30%) sharing 37 instances of ethics in their presentations. In particular, participants indicated that they had the confidence to stand by their values and the values of MCH practice, increased exposure and focus on issues around health equity, and to lead others to be courageous in addressing ethical concerns. Participants reported instances where their actions resulted in increased or new programming for historically marginalized and vulnerable MCH populations.

Conclusions for Practice

It is encouraging that broad evidence of meaningful outcomes and impacts was mined from such a simple communications exercise that helped participants realize shared outcomes. Other studies of leadership development in healthcare professionals (physicians) have highlighted the essential importance in collecting qualitative data to fully understand the impact of leadership training on both knowledge and skill development (Fassiotto et al. 2018; Frich et al. 2015; Throgmorton et al. 2016). This qualitative analysis helps provide insight into the actions and implementation of skills learned in the MCH PHLI program, which help broaden the understanding of potential outcomes and impacts occurring for organizations and communities when investments are made in leader development.

Applying the classic Kirkpatrick Four-Level Training Evaluation Model (Kirkpatrick and Kirkpatrick 2006) (Fig. 1) to the MCH PHLI program helps further understand the return on investment of such types of leader development activities. Previous work has investigated learner motivation (Fernandez et al. 2017), a component which is depicted in Fig. 1 as aligned with, but outside of, the traditional Kirkpatrick model. Additional studies have shown participants’ positive reaction to the program, which aligns with Level I (reaction) (Fernandez et al. 2017) and Level II (learning) (Fernandez et al. 2014). In this model a return on investment (ROI) is measured through both Level III: Behavior (changes in behavior) and Level IV: Results (tangible outcomes produced by more effective leadership behaviors). This paper explores the value of using intensive leadership training to catalyze changes in behavior (Kirkpatrick Level III) by MCH leaders and illustrate tangible outcomes (Kirkpatrick Level IV) resulting from these changes.

The depth and breadth of the examples of impacts given by participants provides important illustrations of how leaders implement and apply newly learned skills. One key finding from these data is that the investment in intensive training of MCH leaders not only impacts the leaders themselves, but also leads to demonstrable impacts on a much broader population of individuals, professionals, organizations, and communities. The data presented in this study indicate that MCH PHLI participants found relatively immediate and substantial value from their year-long training in the specific target areas of the MCH Leadership Competencies, which they could present and quantify as early as graduation from their training year. This finding suggests that the MCH PHLI curriculum was able to show successful results at the Kirkpatrick Levels III (changes in behavior) and IV (tangible outcomes produced by more effective leadership behaviors) (Kirkpatrick and Kirkpatrick 2006). In describing the impacts of their leadership skill development through their presentations, many participants referenced activities that were part of their Personal Leadership Projects, further substantiating the information shared.

While the examples above provide evidence to illustrate that investing in intensive leadership training in public health professionals from both MCH and family-based disciplines yields considerable impacts on the participants themselves, as well as the organizations and communities they serve, there are limitations to this study. Participants worked under a less than five-minute time frame to give their presentations—data presented might have been even more robust had more time been available. Further, while short term impacts are important and can lead to longer term ones, the MCH PHLI was funded for only four cohorts and no long-term follow-up data is available. Changes in communities and systems are often evolutionary rather than revolutionary, thus many of the most important changes that resulted from the training of these maternal and child health professionals would likely take place in the years and decades after the conclusion of their participation, and thus not be captured by this follow up. Longer term follow-up of MCH PHLI graduates could illuminate potential lasting effects of the intensive leadership training strategy utilized by the MCH PHLI, further clarifying its value.

As is the case with any method of study utilizing self-report data, a certain level of social desirability bias is to be expected (Furnham 1986). It is possible that participants were unwilling to share challenges they might have felt were insurmountable, however it is interesting to note that as they spoke many participants acknowledged a new realization of their achievements, with many commenting on their colleagues’ statements and remarking on their previously unrecognized shared experience. For many participants, this became a profound and empowering experience of realizing the broad impacts they had on the MCH community.

Findings from this analysis indicate that MCH PHLI participants gained significant skills which they applied in their organizations and communities, with the ultimate goal of improving the health and well-being of the MCH populations they serve. In this way, MCH PHLI participants are functioning as ambassadors of leadership training, as evidenced by their employing and sharing the specific skills they learn, effectively extending the reach of the initial investment in the leadership development program.