Introduction

In many parts of the world the practice of medicine and medical education increasingly focus on providing patient care within context of the larger healthcare system. This growing emphasis has emerged from a number of stimuli, including the following: (a) an increase in healthcare knowledge and technologies that require a more systems-based, rather than individual-practice, emphasis; (b) increased attention to severe and widely publicized errors in patient care and consequential demands to improve health system quality and equity both globally and in individual countries; and, (c) growing education and expectations of consumers to be able to access a complexity of services within healthcare systems (ACP 2008; IOM 2001; Lancet 2001; Owen and Roberts 2005; Palmer et al. 2002; Romanow 2002).

The ultimate goal of having physicians understand—and practice within—larger healthcare systems is to afford them the ability to assist patients in accessing a full range of services. In physician education, especially, the focus on practicing within the larger healthcare system is reflected in the published guidelines and accreditation regulations governing graduate medical education in various countries.

For instance in the U.S. context, in which authors of this article work, the Accreditation Council on Graduate Medical Education (ACGME) requires that all residents demonstrate competency in Systems-Based Practice (SBP).Footnote 1 The ACGME broadly defines SBP as the demonstration of “an awareness of and responsiveness to the larger context and system of health care and the ability to effectively call on system resources to provide care that is of optimal value” (ACGME 2007). To begin to operationalize this domain, the ACGME has said that residents must obtain and demonstrate competency by meeting the following Resident Expectations of:

  1. #1

    Understanding and working effectively in various health care delivery settings and systems relevant to their clinical specialty.

  2. #2

    Coordinating patient care within the health care system relevant to their clinical specialty.

  3. #3

    Incorporating considerations of cost awareness and risk-benefit analysis in patient and/or population-based care as appropriate.

  4. #4

    Advocating for quality patient care and optimal patient care systems.

  5. #5

    Knowing how to partner with health care managers and health care providers to assess, coordinate, and improve health care and know how these activities can affect system performance.

  6. #6

    Participating in identifying system errors and implementing potential systems solutions.

Internationally, within a similar time period of 1998–2008, other countries have adopted comparable expectations about roles that residents should learn to perform with competency. For instance, Canada’s CanMEDS Physician Competency Framework describes competency areas related to seven roles of specialists (Frank 2005). CanMEDS’ role of “Health Advocate” covers similar advocacy role topics as described by the ACGME for SBP Expectation #4. The CanMEDS role of “Collaborator” is primarily aligned with the SBP Expectation #5 that residents learn to partner with other members of the heath care team. CanMEDS’ role of “Manager” is similar to ACGME’s role of being a patient care coordinator who also considers cost awareness and risk benefit when prescribing treatment (SBP Expectation #3).

Furthermore, CanMEDS specifies “Key Competencies” to better operationalize each role. There is also a corresponding set of “Enabling Competencies.” For example, for the role of “Collaborator” two Key Competencies are described: (1) physicians are able to “participate effectively and appropriately in an inter-professional health care team;” and, (2) physicians are able to “effectively work with other health professionals to prevent, negotiate, and resolve interpersonal conflict” (Frank 2005). For the first Collaborator Key Competency, the associated Enabling Competencies include: “1.1. Clearly describe their roles and responsibilities to other professionals; 1.2. Describe the roles and responsibilities of other professionals within the health care team”, etc. In brief, CanMEDS-designated roles that define competent physicians are made further operational through associated Key Concepts and Enabling Competencies.

Other countries, including Denmark, the Netherlands, Australia, and New Zealand, have endorsed and implemented the CanMEDS framework in defining expectations for their clinicians-in-training (Ringsted et al. 2006). In Great Britain, the General Medical Council has established a similar series of standards for physicians. First published in 2001, these standards are updated periodically. The most recent version, made available in 2006, include 79 standards that lend themselves to a physician revalidating (licensure) process, and that include a number of standards similar to those discussed previously pertaining to practicing within the larger health care system (GMC 2006). For example, several GMC standards are specific in addressing ways in which physicians should work within healthcare teams, understand the available resources among professionals in these teams, and follow stringent patient hand-over procedures when delegating treatment and making referrals.

In spite of efforts to define and operationalize physician competencies, the collective international experiences with these requirements suggest a considerable amount of confusion among educators—apparently stemming from their perceptions about what competency guidelines actually require physicians to demonstrate vis-à-vis the larger healthcare system. In brief, the competencies guiding practice within larger healthcare systems seem to be stated “too broadly” in the United States (Englander et al. 2006; Zenni et al. 2006; Dyne et al. 2002), in Canada (Frank and Danoff 2007), in Denmark, the Netherlands, Australia, and New Zealand (Ringsted et al. 2006) as well as in the United Kingdom (Bleakley 2007).

One of the reasons for these broad definitions inevitably stems from the unenviable task that national medical education regulatory bodies have in issuing expectations that are broad enough to cross a large number of specialty areas. Therefore, definitions of large categories of competencies and roles are left fairly general, and specialty programs within medical education are asked to further tailor and specify these definitions to fit their respective specialty.

Perhaps an even more important cause of the ambiguity in defining ways that physicians should relate to the larger healthcare system has to do with the multiple perspectives of different stakeholder healthcare professional groups within this system. International experiences show that various types of health professionals contrast in the they way they conceptualize indicators of stated competencies, as well in the importance and confidence they assign to the roles and expectations that national governing bodies relate to physician competencies (Kvarnstrom and Cedersund 2006; Bleakley 2007; Dyne et al. 2002; Englander et al. 2006; Frank and Danoff 2007; Ringsted et al. 2006; Zenni et al. 2006). Moreover, there is some evidence that physician perceptions of what constitutes physician competency has taken precedence over other expert and stakeholder perceptions (Bleakley 2007; Moon 1999).

We believe that the potential for physician primacy in defining competency is an important point to consider, especially in the case of determining what accounts for a competent physician vis-à-vis the system in which multiple types of professionals operate. An overemphasis on physicians’ points of view in defining this area of competency, if true, is likely to result in only a partial or “lop-sided” definition of what it means to understand and operate competently within a healthcare system.

The present study uses a robust qualitative methodology to: (1) explore whether or not senior physicians who determine training and assessment approaches regarding resident competencies actually do differ from other members of the healthcare team who interact with residents; (2) if these differences in perceptions exist, describe what they are; and, (3) describe what implications these differences might have for future efforts to develop a more complete and operational definition of “understanding and practicing effectively within the healthcare system for the benefit of patients”, which in the U.S. accrediting agency is known as Systems-Based Practice (SBP).

Before engaging in this endeavor and developing our rationale for the present study, we first provide a brief description about the U.S. experience with attempting to define and operationalize SBP as a competency area. We will then describe the methods used in our study to explore multiple-professional perceptions of what constitutes competency in SBP, followed by a reporting of the study results. We conclude with a discussion of the implications of these findings for further and better operationalization of SBP in the U.S. When concluding, we also discuss possible international commonalities and implications for other countries attempting to operationalize healthcare system-related competencies, roles, and standards for medical practice and education.

Systems based practice

From the literature, Systems Based Practice (SBP) is regarded as one of the most difficult ACGME competencies to define (Pasquina et al. 2003; Reisdorff et al. 2002; Swing, 2002). Even though several studies in the graduate medical education literature mention or discuss the elusiveness of conceptualizing SBP (Englander et al. 2006; Zenni et al. 2006; Dyne et al. 2002), none report on the constituent attributes of SBP—that is, the set of behaviors and outcomes required to teach and assess this competency. Moreover, within the existing literature any SBP attributes identified have either been specialty-specific (e.g., emergency medicine or primary care) (Dyne et al. 2002; David and Reich 2005) or have addressed only a sub-set of SBP attributes like cost-effectiveness (Panek et al. 2006).

Surprisingly, none of the studies published to date focus on the potential ambiguities that exist within the ACGME definition, nor have they tried to highlight the gaps between the perceptions of all those that are involved in teaching and assessing SBP and the actual requirements of SBP. We believe that one of the reasons for this prevailing ambiguity is the use of broadly worded statements by the ACGME to define SBP (see Appendix A) (ACGME 2007). We understand why the ACGME needs to be wide-ranging in its wording of SBP since it is issuing these definitions for multiple types of programs and institutions. One drawback of the need to be more general, however, is the lack of enough specificity to move our understanding towards what exactly it is residents need to demonstrate in order to indicate proficiency in SBP.

We believe that it is time to focus on a detailed analysis of the language in the SBP definition. This will enable better assessment of residents’ demonstration of this competency than currently exists. According to principles of designing and using tools for educational assessment, there are a number of steps involved in order to do so, oftentimes including developing a taxonomy of observable behaviors from which training curriculums and assessment instruments can then be generated (Chatterji 2004). Prior to this happening for SBP, however, it is necessary to first understand the current perceptions of the stakeholders of resident practices and compare these perceptions to the actual requirements of SBP (Moon 1999).

Historically, efforts to define important attributes where residents must demonstrate their competence (e.g., professionalism) have focused only on physicians’ input as a primary source of expert and stakeholder knowledge (Moon 1999). Few efforts have been directed towards identifying the perceptions of other professional staff, such as the nurses, social workers, pharmacists, physical therapists, hospital administrators, residents, and attending physicians about which behaviors indicate competence in SBP. This is so even though attending physicians are shown to interact with residents in ways that are distinct from other care team members (Pasquina et al. 2003). It is these other stakeholders in the healthcare system who see different aspects of a resident’s day-to-day practice: from bedside patient care to presentation at rounds to managing interdisciplinary interactions (Moon 1999).

Rationale

Due to the wide-ranging interactions that each care team member has with a resident, there is the potential for a number of different perspectives regarding what each stakeholder believes makes a resident “competent” within a given system. Some of these perspectives are ones that the attending physician never sees yet are integral to a well-functioning system. Therefore, the intention of the present study is to collect qualitative indicators about potential differences in stakeholders’ perspectives regarding residents’ competence in SBP. Our purpose is to identify the differences (if any) between the key stakeholders’ perspectives regarding the System Based Practice competency.

Methods

Participants

A total of eighty-eight healthcare professionals (n = 88) from seven different care categories in two large, urban hospital systems participated in this process. Each care category chosen has multiple daily interactions with residents in order to meet patient care responsibilities directly (e.g., nursing) or indirectly (e.g. pharmacy). All participants were members of a healthcare team. To the best of our knowledge, however, none had extensive formal training on working (a) in multidisciplinary teams (Kvarnstrom and Cedersund 2006) or (b) within a contemporary conceptualization of a ‘team approach’ to health care (i.e., with a theoretical focus on authentic collaborative practice rooted in team members’ own needs and goals rather than the more traditional focus on mutual goals and collaboration) (for more complete discussion, see Lingard et al. 2004). Non-clinical individuals, such as patients and family members, were excluded from this study. Including them would have gone beyond the parameters of the investigation questions; nevertheless, the solicitation of patients’ and family members’ perceptions in the future may be of value (we discuss this further in Limitations section).

Data collection

After the researchers received approval from the University’s Institutional Review Board, fourteen Nominal Groups Process (NGP) sessions were conducted. The Nominal Groups Process was selected as our data collection procedure because, unlike other labor intensive qualitative data collection processes, it provides an opportunity to achieve several relevant perspectives in a short space of time (Carney et al. 1996; Saleh et al. 2001) The general procedure for conducting nominal groups, adapted from Moon (1999) and Carney et al. (1996), involves the steps outlined in Table 1. This explanation includes questions used to focus discussions and procedures for collecting and clarifying ideas that emerged from group discussion. As noted in Table 1, participants’ ideas were acquired on index cards and flip charts, which permitted both individual and group generation, clarification, and prioritization of ideas. Sessions, therefore, were not audio-recorded. These procedures are standard for these types of nominal groups (Moon 1999).

Table 1 Steps for the Nominal Group Process
Full size table

Recruitment for NGP participants varied depending on the healthcare profession, and was based on non-random, convenience sampling. Each of the nominal groups consisted of members from one profession only (e.g. pharmacists or nurses or residents) without regard to specialty. The number of members in each group is listed in Table 2. This process outlined in Table 1 was applied with consistency to each of the 14 groups.

Table 2 Participants of NGP (n = 88 participants in 14 groups)*
Full size table

Analysis

Our analysis of the Nominal Group Process (NGP) data involved the following: (1) collecting the prioritized statements; (2) combining statements with overlapping or redundant meaning; (3) content-analyzing the combined statements to identify themes embedded within the statements; (4) identifying the performance-oriented commonalities between the themes; and (5) classifying the themes into resident roles.

More specifically, as shown in Table 1 (within Step 4 and Step 5), the five statements having the highest priority rankings in each of the fourteen NGP groups were compiled. This resulted in a list of 70 statements. Some of the 70 prioritized statements were, however, nearly identical in meaning and concept. If two or more statements were deemed sufficiently similar by two of three author-investigators who were tasked with compiling the statements, then each of the similar statements were collapsed into one. For example, the three independent statements of “[the resident] is accessible” and “makes self available” and “is approachable” were agreed upon as similar enough to combine in order to reduce data and remove redundancies. In this instance, makes self available was retained to represent all three. As a result of this process, a smaller, aggregated list of non-redundant, prioritized statements emerged (for reference, see Table 3 in Results section).

Table 3 Resident roles that emerged from the themes and statements obtained in the NGP of prioritizing residents as being competent in systems-based practice
Full size table

With this new list of statements, we next applied a content analysis process to identify underlying themes embedded within these statements (Baker 2004; Legrow and Rossen 2005). This analysis involved three of the present authors and one research assistant. Each individual independently identified a theme nested within each of the statements. The coded themes for each of the four investigators were then set side-by-side and grouped together thematically by two of the four individuals-based upon structural similarity and through consensus agreement wherever possible.

For example, we identified four prioritized and similarly grouped statements from the 40. These were then coded, and classified under one theme, ‘Using Other Personnel’. This theme represented the following statements: (1) [The resident] calls for consults when beyond expertise; (2) uses other people to get things done well; (3) knows one’s limits and when one needs help; and (4) asks questions of other health care providers to improve patient care. Similarly, the coding team identified another five prioritized and similarly grouped statements from the 40. These went through the same process, resulting in one theme: ‘Demonstrating Communication Skills’. This theme represented the following statements: (1) [The resident] communicates necessary information about patient to other disciplines or departments and team members; (2) uses respectful communication; (3) uses active listening and follow-up questions; (4) writes orders properly; and (5) fosters relationships through positive communication.

Subsequent consensus-reaching sessions among the remaining investigators and additional institution-based physicians serving in the role of project consultants were conducted over a series of meetings. At this stage, because the task was solely for clarification of language and elimination of redundancy, other stakeholders (e.g., nurses, social workers, pharmacists) or members from the NGP meetings were not involved due to scheduling difficulties. As a result of theme identification and elimination of redundancy, this process further reduced the statements to an aggregated set of themes.

In a final consensus-building group meeting with 10 participants, the statements and themes were shown to representatives from each stakeholder group who took part in the earlier NGP data collection (nurses, residents, physical and occupational therapists, social workers, attending physicians, and pharmacists). This group represented 15% of the actual NGP participants, and at least one individual from each of the seven stakeholder groups (e.g., nurses) was present.

During this meeting, the group reviewed the statements and themes with the goal being to achieve a content validation of the themes only (Chatterji 2004). This was done to insure that the theme identification process that the investigators and consulting physicians completed was in agreement with each stakeholder group. It was generally not possible to have all participating stakeholders meet again. Changes or deletions were made by group consensus. In addition, one additional step of the analysis was to collapse and combine the conceptually similar themes into what we call the more general, overarching resident roles. Through this process a conceptual structure of the data emerged, which we will now describe.

Results

All results reported are included in Table 3. The framework that emerged through iterative analysis and subsequent consultation with the expert panel resulted in three categories of resident roles in SBP. We believe these roles represent how the healthcare team believes residents’ should function within the system and are similar in wording and structure to some of the CanMEDS competencies. The SBP resident roles identified by the stakeholder groups are the following: resident as Self-Manager, Team Collaborator, and Patient Advocate.

Based on the qualitative analysis from the nominal groups, and separate from the CanMEDS language, we arrived at a SBP-based definition for the Self Manager role as creating and using networks within the system to help themselves (as residents) get their own job done; the Team Collaborator role as using personal and professional skills to help the system get the job done; and, the Patient Advocate role as understanding and using the system to help patients through the system.

Table 3 represents the 20 themes—along with a sample of participants’ actual descriptors (which we call ‘sample statements’)—from the NGP and through which each of these three roles were identified. The themes associated with each role represent at least one (or more) of the nominal group’s top voted rankings. For example, items related to the theme “using resources” (Table 3) were prioritized as being most important solely by three physician groups. Similarly, five NGP descriptors (i.e., ‘sample statements’) that yielded the theme demonstrating communication skills were prioritized by eight different groups: four out of these eight groups were physical and occupational therapy (PT/OT) and another two were social work (SW) (Table 3).

SBP role 1: Resident as a self-manager

This role of the residents for SBP involves creating and using networks within the system to help themselves [as residents] get their own job done. This category emerged from a class of themes related to behaviors about residents’ interactions within the system to create relationships that are of benefit to them. As can be seen in Table 3, representatives from all participating groups generally endorsed these themes. Statements from the raw data dealt with aspects of ‘communication skills’, ‘information system skills’, and ‘interpersonal skills’. System-relevant communication skills involved residents’ use of written, verbal, and all-around communication to create partnerships and networks with the system. System-related information skills involved residents’ knowledge, utilization, and application of all kinds of information technology such as the telephone, pagers, interpreters, the computer help desk, files and paperwork, as well as proper forms and written orders. System-related interpersonal skills involved statements about how residents interact with other departments, staff, and hospital services. For the purposes of the present analysis, it is important to distinguish that these statements were congruent to the accrediting body’s definition of Systems Based Practice due to emphasis being placed on the effects that such actions specifically have on the system, as opposed to interactions that demonstrate other competencies outlined by the accrediting body, such as Communication Skills or Patient Care.

SBP role #2: Resident as team collaborator

This role of the resident for SBP involves using personal and professional skills to help other members of the system to get their job done. Table 3 shows the themes that emerged in this role. For example, if a resident is ‘not organized with files and paperwork’, then the nurse will be unable to carry out the patient orders appropriately. Or, if a resident is ‘not approachable’ or ‘not available’ when needed, a physical therapist may not be able to clarify problems that may arise during the therapy sessions. The major finding here is that groups endorsing these themes consist exclusively of healthcare personnel other than the attending physicians.

SBP role #3: Resident as a patient advocate

This role for SBP includes themes that relate to the role residents has in helping their patients to use the healthcare system (see Table 3). One emphasis that emerged is on the residents’ understanding and using the different aspects of the system to empower their patients. Another emphasis is on using the systems’ resources for educating patients and facilitating resources for patients effectively. For this role, it is notable that only the attending physicians have identified these concepts (along with one or two sporadic endorsements by other groups). For all the other healthcare groups, it appears that the presence or absence of these characteristics in residents is either not directly important for functioning of the system or it is something that they perhaps believe is ultimately their own responsibility to do. For the attending physicians, however, these aspects are ranked as being much more relevant.

Comparison of the ACGME Definition to the Acquired Resident Roles

Table 4 provides a comparison of the ACGME definition of SBP and the identified roles. Of note, for this group of 88 stakeholders of the hospital system, none emphasize awareness about cost effectiveness and the ability to identify errors important for practicing within the system.

Table 4 Comparison of the identified residency roles and ACGME definitions
Full size table

Discussion

The prime focus of this study was to collect and generate from the NGP a list of qualitative variables that key stakeholders within the healthcare team use to describe SBP, and then to see the extent to which there was unanimous agreement. We selected the NGP as the primary data collection procedure because it is reported to be relevant in exploratory stages of concept building where the researcher seeks input from potential end users (Aspinal et al. 2005). The methodology has previously been used to validate and revise existing medical classifications systems and theory-generation, to later develop reliable and valid research protocols, curricula, and evaluation strategies (Carney et al. 1996; Saleh et al. 2001).

According to the pattern of responses that emerged from each stakeholder group’s rankings, the representative groups of the “system” did exhibit a level of collective agreement on some roles, though some group-independent views were found as well. All stakeholder groups generally agreed about the characteristics that constitute the role of resident as self-manager. This role certainly represents a set of attributes necessary for the residents to use the system in a way that helps them to meet their own responsibilities. For example, knowing when to call for a consult when a situation is beyond their level of expertise. In such cases, interacting with other members of the ‘system’ for inviting a consult is a demonstrated competency by the resident to self-manage and get help from the system for completing their own tasks of caring for their patients. This is the fundamental and defining attribute that constitutes the role of resident as a systems-based self-manager.

We discovered, however, that some groups placed emphasis on an interesting yet underdeveloped domain of SBP: the resident as team collaborator. This role represents a set of attributes that are necessary for the resident to interact with the various systems (and individuals within each system) in ways that allow the other stakeholders in the system to get their tasks done. For example, the nurse cannot give medication to the patient unless the resident effectively uses the system to clearly and properly communicate the order to the pharmacist. Incomplete work or neglected attention to protocol will only delay the nurse from administering medication. Therefore, contrary to the self-manager role, the defining aspect of residents as a team collaborator is that a resident helps the ‘other’ members of the ‘system’ (i.e. the healthcare team) to complete their patient care responsibilities.

What also emerged from the role of team collaborator was the level of agreement between the health care professionals and residents. As stated previously, these groups and residents are more likely to rely upon each other for the day-to-day aspects of system functioning and are more dependent upon each other to get their own work done. Thus, their joint endorsement regarding the essential requirements for proficiency in SBP adds another level of importance to the uniqueness of the themes in this category. In addition, this is in sharp contrast to the absence of opinions from the attending physicians. It is possible the attending physicians are largely outside of these aspects of the system that were identified by others. Another possibility is that the attending physicians simply do not perceive the importance of these ‘behind the scenes’ attributes. Attending physicians value these characteristics, certainly; yet what emerged through the data from the NGP does indicate that attending physicians did not rank the characteristics of team collaborator nearly as high when compared to the nurses, social workers, pharmacists, and physical and occupational therapists.

Another finding that was unique to one group of stakeholders was that the attending physicians emphasized a third role, the resident as patient advocate. For instance, empowering patients to learn more about their insurance plans, or helping patients make a decision in choosing a surgical or conservative treatment option are specific resident-physician patient advocacy roles. Even though the attributes in this role may not directly affect the overall functioning of the health-care-system, they are nevertheless crucial to insuring that patients do not suffer in some physical, economic, or social way. While unexpected, it is notable that the attending physicians identified these characteristics almost uniformly but without endorsement from the other groups.

Finally, the differences between what the system stakeholders perceive as important and what the ACGME actually requires at residency exit suggest that there is a strong need to articulate the working ACGME definition in further detail. This should be done in order to provide residency directors a comprehensive list of what should be included in resident training prior to conducting and designing comprehensive assessments of SBP. The results indicate that involving stakeholders to collect perspectives can lead to generation of several interesting theoretical hypothesis regarding the existing resident roles, and roles that are presently de-emphasized but should be incorporated for comprehensive training and 360° assessment of the SBP construct. Otherwise, the construct of SBP runs the risk of being defined only in domains where the attending physicians are either are in unison with the other stakeholders (as with the role of self-manager) or have an independent set of viewpoints that are not emphasized by other stakeholders (as with role of patient advocate).

Implications of our findings for better operationalizing SBP through expanded stakeholder involvement for those in other countries who are attempting to better define physician competencies. As discussed in the Introduction, researchers and practitioners in other countries have pointed out that competencies relating to practicing within the healthcare system often are too broadly stated and reflect mainly physician rather than multiple stakeholder perspectives. A number of international papers have pointed out that non-physician members of multi-professional healthcare teams think differently from physicians (and sometimes think congruently with them) about what it means for a physician to practice competently vis-à-vis the team and larger healthcare system (Bleakley 2007; Kvarnstro and Cedersund 2006; Ringsted et al. 2006; Zenni et al. 2006). Even though these authors point out differences in multi-professional perspectives, they do not propose ways of integrating various viewpoints into a unified operationalization of physician competencies that can be used to both train and assess residents.

Our work provides an example of a process by which other countries can collect multiple perceptions and incorporate these into an integrated set of standards for physician practice in the healthcare system. Moreover, other countries might adapt the specific indicators that we include as ‘sample statements’ in Table 3, to further articulate expectations that relate to expected roles that have already been delineated. For instance, CanMEDS expected physician role as “Advocate” is closely aligned with the same as specified by ACGME. Our sub-themes connected to the role of patient advocate—e.g., demonstrating knowledge of resources available for patients, and taking a stand (for patients) even when it is unpopular or inconvenient—might provide additional understanding.

Limitations

The identified roles, themes, and statements that emerged herein reflect at least four of the six resident expectations of what constitutes the ACGME’s general definition of systems-based practice (ACGME 2007) (Tomolo et al. 2005). Nevertheless, participants in this study did not emphasize understanding of cost issues and resource allocation. This may be a limiting factor because our participants are from two urban academic medical centers and not other types of hospital environments where much more direct emphasis may be placed on costs (Ranz et al. 2006).

Another limitation of this study is our inability to interpret why attending physicians were alone in noting the importance of residents being patient advocates using the system to help patients through empowerment, education, and the facilitation of resources. From our NGP procedures, it is not possible to investigate why the other healthcare providers did not identify and prioritize the patient advocate positions that the physicians did—even though we believe that the other stakeholders would find these attributes important. One reason for their absence in ranking could be that they believe that it is something that they (as nurses, residents, social workers, physical therapists) do themselves, perhaps as a shared responsibility among the healthcare team.

A final limitation, and one that should be resolved in future studies in order to be even more comprehensive about SBP, is that we did not include patients and their families as stakeholders because of their ‘non-professional’ status. Nevertheless, based upon the priority patterns found for different stakeholders, it would be of great interest to see how patients or patients’ families would respond.

Given these limitations, the results presented should be regarded as the first step towards developing a comprehensive, relevant, and valid taxonomy of observable attributes and behaviors for SBP, which is a project that we currently have underway. The main finding of this study indicates that multiple key stakeholder (e.g. nurses, pharmacists, attending physicians, residents, and perhaps even patients or patients’ families) must be part of the SBP taxonomy development process. We believe that a taxonomy of relevant SBP attributes will provide a relevant and valid framework for developing more objective training programs and more comprehensive evaluation tools for assessing residents’ competence at different stages of development.

Conclusion

Through the NGP we have systematically constructed a representation of professional opinions for what constitutes residents competence in SBP. We have done so through multiple medical disciplines and multiple representatives of the “system”. Our results suggest that in order to avoid physician primacy in defining systems-based competency, input into a working definition of SBP needs to extend beyond physicians and include the other members of the healthcare team. When all healthcare provider groups contribute to thinking about how to operationally define SBP, the perspective widens to include an approach to practice that encompasses the role of team collaborator. This is a role that attending physicians may not witness regularly, due to the nature of their responsibilities. At present the working definition of SBP is being inappropriately limited when it is compartmentalized into what we call the resident “self-management role”, where all stakeholders are in agreement anyway. Meaningful and reliable assessment in medical education is becoming increasingly recognized as being multi-faceted and complicated yet necessary to do (Epstein 2007). In the case of SBP, without including the perspectives of all stakeholders the important role of the resident as a system-based team collaborator in SBP assessment is susceptible to be forgotten, diminished, or ignored in graduate medical education training and evaluation.