Introduction

As the instructional design field is characteristically dynamic (Sharif and Cho 2015), instructional designers (IDs) continuously attempt to keep up with the evolvement of the field such as on the topics related to emerging information and communication technologies (Ritzhaupt and Kumar 2015). When IDs do not find suitable resources at their work environment, they likely turn to external resources, such as a community of practice (CoP) facilitated through social media and professional organizations to seek knowledge and innovative best practices (Muljana and Luo 2020). For example, IDs may be limited by workplace budget and workload. When the budget and schedule do not permit, IDs are willing to seek alternative options to meet their professional learning needs.

As technology continues to develop rapidly, working professionals have taken advantage of the benefits technologies provide to improve their knowledge and skills without the limitations of geographical and temporal boundaries. Social media streams offer digital spaces for “educators to learn with and from each other” (Krutka et al. 2017). For instance, faculty and staff members in higher education have extended their learning network by taking advantages of the Internet and social media, enabling them to engage with each other virtually (Trust et al. 2017).

As revealed by Schwier et al. (2004), IDs are willing to participate in a virtual or online CoP since it can be fit into their spare time, and they can adjust their participation level in order to make the best practical and convenient use of learning materials. Therefore, facilitating an online CoP through 1-week long, bi-annual, asynchronous professional development (PD) event in Canvas learning management system (LMS) without charging a fee is an opportunity to assist IDs around the world with meeting their PD needs which are not limited by geographical and temporal boundaries. Essentially, IDs can learn about what other colleagues have been performing differently and innovatively, and access resources at any time when their schedule permits within the week the PD is offered. This paper presents the development of the CoP-based PD events for IDs and the formative evaluation results of the events for determining the efficacy and future improvements that can increase the participation of the members.

Conceptual Framework

Community of Practice

Wenger’s Community of Practice (CoP) explains how working professionals learn daily through social interaction and knowledge sharing. According to Wenger (2001), a CoP is “a group of people who share an interest in a domain of human endeavor and engage in a process of collective learning that creates bonds between them” (p. 1). Additionally, CoPs provide rich resources for professional learning (Duncan-Howell 2010; Woo 2015). This offers a possible explanation of why CoPs exist within education. For instance, facilitation of CoPs is beneficial for the PD of teachers (Tsai 2012; Woodgate-Jones 2012) and faculty members in higher education (Bond and Lockee 2018). Another study investigated the CoP phenomenon from the IDs’ perspective (Schwier et al. 2004); however, further studies exploring the facilitation of CoPs for IDs are still rare. Paradoxically, the field of instructional design is continually evolving; therefore, the professional development for IDs should also be dynamic, demanding “constant training and professional development” (Sharif and Cho 2015, p. 82).

Wenger (2001) also believes that the facilitation of informal learning can take place through an online CoP. Online CoPs are aimed to address the geographical and temporal limitations (Woo 2015). Hence, Wenger’s work expanded to further concept supporting CoP by connecting the members living in disperse locations (Wenger et al. 2002). A few years later, technology affordances were additionally discussed as a supporting factor of CoP (Wenger et al. 2009; Woo 2015). Alongside these technology affordances, scholars have turned to an exploration of online CoPs (Tseng and Kuo 2014), including the CoP facilitation through Twitter (Weseley 2013) and Massive Open Online Courses (MOOCs) (Jones et al. 2016).

Participation Levels in a CoP Group and Factors Influencing the Participation

CoP members participate at various levels (Wenger 1998b). The three levels of participation proposed by Wenger et al. (2002) are (1) core, (2) active, and (3) peripheral. The core members are highly active and likely to offer insights. In a lower participation degree, there are members whose participation fluctuates between active and limited (Lesser and Everest 2001); these members are in between the core and peripheral positions. The remaining members are on the peripheral position, preferring to follow the discussions rather than offering insights.

The core members comprise of the leader and facilitators “who play key roles in supporting the evolution of peripheral members” (Borzillo et al. 2011), followed by active members in the lower level of participation. Being in the center of the group, the core members create a CoP group (Borzillo et al. 2011) and then promote the activities, ensuring that the group remains interactive and effective (Wenger et al. 2002). Serving as leaders, the core members additionally seek and respond to the members’ request regarding the needed information and knowledge (Wenger and Snyder 2000). In other words, these core members plan and facilitate CoP events. Active members, although not resided in the core position, are willing to participate and share their knowledge and resources, as well as learn from other members within a CoP group.

The peripheral members are those who post occasionally (or even rarely) in an online CoP setting (Zhang and Storck 2001). Commonly labeled as lurkers, the peripheral members typically either ask questions without answering others’ questions, read the conversations without offering insights, and access or obtain information without sharing information (Kollock and Smith 1996). They are also the majority of a CoP group (Marett and Joshi 2009; Rafaeli et al. 2004). Several possible reasons may explain why the less-active members prefer not to increase their participation: (a) they do not have time, (b) reading the discussions already provides adequate information, (c) they are still getting to know other people in the group, (d) they do not have insights to post (Preece et al. 2004), (e) their opinion may be contradicted, and (f) they are afraid of being judged and criticized (Guan 2006) which is related to their self-efficacy (Lai and Chen 2014).

Based on the abovementioned reasons, it appears that the factors influencing the levels of less-active CoP members’ participation include time constraints (Preece et al. 2004) in addition to self-efficacy about their own knowledge (Tseng and Kuo 2014), trust and bonding with other members (Gorrell et al. 2013) which can help open up the communication or dialog (Cadiz et al. 2009a), and perceived value of learning from other members (Cadiz et al. 2009a; Wenger 1998a). Knowledge sharing occurs if CoP members have suitable self-efficacy regarding their own knowledge (Tseng and Kuo 2014). The level of self-efficacy, in turn, influences their knowledge-sharing behavior. Furthermore, the decision to share knowledge is determined by bonding with other members (Gorrell et al. 2013). When the bonding exists, it can open communication and impact the level of engagement among CoP members. Echoing Cadiz et al. (2009a, p. 1039), “without open communication, interaction would not occur between community members and, therefore, the community would dissolve.” Another imperative factor influencing the members’ participation is their perception regarding the value of learning from each other. If the members do not perceive the value of learning from other CoP members, the group will simply function as an online social gathering. Gradually, the CoP will fade away (Cadiz et al. 2009a; Wenger 1998a).

On a positive side, the participation in a CoP group can be dynamic and change over time, allowing the movement from the periphery to the core (Guldberg and MacKness 2009). As the peripheral members are more comfortable with the group, they may feel convinced to be participate more actively (Correll 1995; Lai and Chen 2014). CoP-related studies have offered implications on how to encourage CoP members to participate more actively; however, such implications for promoting IDs’ participation in a CoP group deserve further exploration so that IDs can also have “systematic venues for sharing their knowledge” with one another (Schwier et al. 2004, p. 97).

Designing CoP-Based-Free PD Events for IDs

Based on the premises of CoP concept, we designed three CoP-based PD events for IDs at no cost. The co-founders of CoP, serving as the core members, planned and coordinated the events. Each PD event offered ID-related topics presented by active members and provided professional learning opportunities to the remaining members that were mostly in the periphery. The three CoP-based PD events took place in December 2017, July 2018, and December 2018 respectively. Combining both synchronous and asynchronous delivery modes, each PD event was typically (a) a 1-week long, allowing IDs to participate asynchronously within the 1-week duration; (b) hosted in June or July and December each year; (c) and integrated with a live, but recorded, keynote webinar, allowing IDs to participate synchronously. To accommodate those who could not participate during the 1-week schedule, the co-founders transformed the LMS site into a self-pace mode by the end of the PD week.

Before commencing each PD event, the co-founders of the group, also serving as PD coordinators, developed a short online, anonymous survey to allow IDs to propose and vote for relevant topics. This was to ensure that all members, even the peripheral ones, had an opportunity to express interests and be involved through the determination of the PD theme. Based on the most-voted topics, a call for presenters was announced to invite fellow IDs experienced in the respective topics to propose a 30-minute module. This would encourage IDs to be active members by sharing successful practices and helpful resources that corresponded to the most-voted topics. In a sense, this approach overall aligned with the definition of CoP that the PD events gathered a group of people with the same interest and professions, which motivated the group to share knowledge and resources (Bond and Lockee 2018; Lave and Wenger 1991; Wenger 1998b). Furthermore, all members, regardless of their positions (whether core members, active members, or peripheral members), were involved at various degrees.

Once the incoming proposals were received, the three cofounders conducted a peer-review and offered acceptance to those that fit the requested topics. The co-founders, next, were in communication with each presenter to move forward to the development of a 30-min asynchronous module. Since knowledge-sharing and resource-sharing could occur through discussions, activities, artifacts, and documents, promoting meaning-making within a CoP group (Bond and Lockee 2018; Wenger 2010), the coordinators requested presenters to provide relevant documents and resources in their individual module, in addition to including essential components such as (1) an introduction of the presenter welcoming the participation from the attendees, (2) relevant discussion(s) and meaningful activities to promote IDs’ participation throughout the week, and (3) networking opportunity with the presenter beyond the event. This also resonated with a recommendation by Jones et al. (2016) that creating a welcoming learning environment should be considered as it can promote connections among learners and with the presenter.

Each presenter could use their creativity to share their knowledge, expertise, and resources through their own 30-min module. Most presenters preferred video presentations, followed by facilitation of discussions. Several others added self-assessments using the quiz feature. The remaining presenters took extra miles by sharing their personal resources such as templates and job aids.

Simultaneously, a call for registration was announced via social media, allowing IDs around the world to register for the PD opportunity without any cost. Such an approach represented a provision of learning opportunities by taking advantage of technology affordances, in which IDs were welcome to participate anytime, anywhere without costing them money. We further describe each event in the subsequent sections.

First PD Event: December 2017

According to the results of the survey asking IDs to vote for the most requested topic, the theme gamification was selected for PD in December 2017. There were nine proposals received by the learning group. After reviewing the proposals using the three criteria, one was accepted as an opening keynote webinar, and four of them were accepted for the asynchronous modules. There were four modules during this PD event.

Second PD Event: June 2018

The most requested topics for the second PD event in June 2018 were adaptive learning, virtual and augmented reality, and microlearning. These topics received almost equal votes. The call for proposals garnered eight proposals from international and regional prospective presenters. Seven of them were accepted, and each of the seven presenters was offered to develop an asynchronous module. A live keynote presentation was not offered this time due to a schedule conflict. However, to give a variety of topics, the PD included four tracks that IDs could pick and choose according to their learning interest. The tracks included (1) adaptive learning/personalized learning containing one module, (2) augmented reality/virtual reality containing two modules, (3) interactive learning tools containing three modules, and (4) learning design tools containing one module. IDs could select any module(s) within the track that they were interested in.

Third PD Event: December 2018

For the PD event hosted in December 2018, the requested topics included learning analytics, universal design and accessibility, and design thinking. The call for presenters garnered seven proposal submissions, but only five proposals were accepted. The PD did not offer a live keynote due to the schedule constraint. Three tracks were offered: (1) design thinking containing three modules, (2) learning analytics containing one module, and (3) universal design and accessibility containing one module. Figure 1 shows an example of the Canvas LMS navigation menu included in the third PD event, consisting of Home, Announcements, Modules, Discussions, and Quizzes links.

Fig. 1
figure 1

The homepage and navigation menu of the LMS course site for December 2018 PD event

Full size image

Formative Evaluation Questions

A formative evaluation was conducted to investigate the efficacy of the three PD events that followed an application of CoP and provided an open learning environment. In addition, this evaluation was also used to obtain insights about improvements that would encourage the members to participate more actively. Three questions were formulated in order to formatively evaluate the three PD events in terms of collecting knowledge, information, and resources, sharing knowledge and resources via discussions, and factors influencing the participation. The three questions guiding this formative evaluation were the following:

  • Q1—To what extent did IDs collect information from each PD event by viewing the content and resources in Canvas LMS?

  • Q2—To what extent did IDs participate in each PD through online discussion forums in Canvas LMS?

  • Q3—What were the factors that influenced IDs’ levels of participation in the PD events?

Evaluation Methods

Collecting LMS Usage Data

To address Q1 and Q2, we collected data from the LMS usage log. The number of page views and type of resources accessed were collected from Canvas to address Q1. Further, we obtained the number of participations in discussion forums within Canvas, such as the number of IDs participating in and viewing discussions, for addressing Q2. These types of data can be used as evidence of participation degree (Dietz-Uhler and Hurn 2013; Dietz et al. 2018; Schwier et al. 2004), notably that Canvas recorded the number of accesses to information, content, and resources, as well as the participation in the discussion forums of the PD events.

In March 2019, after all three PD events had concluded, we downloaded the LMS usage data from the week when each PD event took place, capturing the number of page views performed by IDs with student role all across the three PD events. Specifically, the downloaded data included the number of page views occurring in the course components, the number of participants accessing the course, the number of participants viewing the discussions, and the number of participants posting insights in the discussions. Next, we obtained additional LMS data reflecting page views beyond the 1-week duration of each PD event. The LMS data were mined using the tools and following the steps discussed in a free online course entitled Analytics in Course Design: Leveraging Canvas Data (Qi and Reid 2016). We installed a user script, developed by Qi and Reid (2016), in Chrome browser to collect the LMS course usage data.

Administering Survey

Also in March 2019, we administered an online, anonymous survey to address Q3. To explore the knowledge-sharing self-efficacy, we adapted three items from Tseng and Kuo’s (2010) Online Knowledge Sharing Self-Efficacy Measure. Four additional items were adopted from Gorrell et al.’ (2013) Community Practice Scale for Schools to assess IDs’ bonding level and general perceptions about the community. Six items from Cadiz et al. (2009b) Experienced Community of Practice Scale were added to examine the open communication among IDs and IDs’ perceived value of learning from others. The co-founders of the group and all past presenters and participants across the three PD events were invited to complete the survey.

Although the survey inventories have been validated multiple times in the past and we could trust the validity and reliability, we still modified the questions to fit the context. For example, several items originally used the term teachers and, therefore, we changed it to IDs. We also added a few more items inquiring causes that hindered IDs’ participation and open-ended responses to attain in-depth insights. The survey was administered to co-founders who coordinated the PD events (serving as core members), past presenters (serving as active members), and past participants of the PD events (alternately serving as active and peripheral members).

Data Analysis

After saving the LMS usage data in a comma-separated value file, the file was loaded to a Shiny app, provided by Qi and Reid’s (2016) free online course entitled Analytics in Course Design: Leveraging Canvas Data, to display data analysis. The comma-separated value file was also imported in Microsoft Excel for further data organization and analysis. Descriptive statistical analyses were conducted to analyze the survey results. We used an open-coding technique to explore the codes and themes emerged from the open-ended insights. Furthermore, we, among authors, discussed the themes to resolve any disagreements.

Results

Q1—To What Extent Did IDs Collect Information from Each PD Event by Viewing the Content and Resources in Canvas LMS?

Before analyzing the LMS usage data to address Q1, we analyzed the total numbers of registrants and participants. We discovered that there was a total of 328 participants (out of 559 registrants) who accessed the LMS site as follows: (a) 164 in December 2017 PD event, (b) 48 in June 2018 PD event, and (c) 116 in December 2018 PD event. These data displayed a gap between the number of registrants and the number of IDs accessing the LMS in each PD event (see Table 1).

Table 1 The number of registrants and actual participants accessing the LMS in all three PD events
Full size table

December 2017 PD Event

On the LMS course site hosting this PD event, the course navigation consisted of the following navigation links: Home, Announcements, Discussions, Quizzes, Course Modules, and Conferences (keynote). LMS data regarding the number of page views to these menu links were obtained. These data revealed that the highest content views occurred in Home, receiving 676 views from 169 participants, and Modules, receiving 515 views from 138 participants. Interestingly, these results showed that several participants might not have furthered accessed the Modules navigation link after clicking on the home page navigation link.

We further investigated the activities within the Modules. Although the navigational menu link Modules received 515 page views, the access activities within the module content received distinctly higher views, 870 views. IDs who visited the module content seemed to explore the content and materials in a deep manner.

The same data analysis was used for analyzing LMS usage data from June 2018 and December 2018 PD events as described subsequently. Next, data from each PD event are depicted side-by-side to display results from the three PD events (see Fig. 2 and Table 2 for the number of page views occurring within course navigation menus in all three PD events; see Fig. 3 and Table 3 for the number of page views occurring within each module unit).

Fig. 2
figure 2

The number of page views occurring within course navigation menus in all three PD events

Full size image
Table 2 The number of page views occurring within course navigation menus in all three PD events
Full size table
Fig. 3
figure 3figure 3

The number of page views occurring within each module

Full size image
Table 3 The number of page views occurring within each module unit
Full size table

June 2018 PD Event

During this event duration, the highest page views similarly occurred in the course Home page, receiving 141 views from 50 participants, and Course Modules pages, receiving 161 views from 39 participants. Unlike the previous PD, the navigation link Modules was accessed by more participants, compared to the accesses occurred in the Home navigation link. Several IDs somehow accessed the Modules navigation link without clicking the Home link.

We obtained further data capturing the number of page views within the modules content. Although the navigational menu link Modules received 161 views, a similar phenomenon occurred—the access activities within the module content received higher number of views, 591 views. Within the module content, IDs viewed the PD Introduction pages the most. Module 2 received the highest number of page views when compared with other modules; it was possibly because IDs found an interesting topic in this module or it might contain more materials.

December 2018 PD Event

We discovered a larger gap in the number of page views for the December 2018 event. The highest number of page views occurred in the Home navigational link, receiving 457 views from 127 participants. However, the Modules navigation link was viewed by lower participants. Only 69 participants accessed the Course Module link, which garnered a total of 357 page views. It appears that many ID participants who clicked on the course home page navigation link might not have decided to explore further.

As far as the access activities within the Modules, LMS data showed that although the navigational menu link Modules received 357 page views, the access activities within the module content received higher number of page views, 1685 views. Again, this was consistent with the two previous PD events. Within the module content pages, the most page views occurred in Module 1.

Page Views Beyond the 1-Week PD Duration

After the completion of the 1-week period, each LMS course site was converted to a self-paced format. We discovered additional page views occurring after the 1-week event duration was over. Table 4 displays the number of page views occurring in the corresponding course components after the 1-week term was over until Summer 2019.

Table 4 The number of additional hits occurring in the three PD LMS sites beyond the 1-week duration
Full size table

Q2—To What Extent Did IDs Participate in Each PD Through Online Discussion Forums in Canvas LMS?

Participation in the LMS online discussions was low: only 20 participants in December 2017, four participants in June 2018, and 32 participants in December 2018 PD. This translated to a total of only 8 to 27% of participants who accessed LMS actually participated in online discussions.

We perceived the importance to obtain additional data regarding the IDs’ participation in the online discussions. Despite the low number of IDs participating in the online discussions, we discovered a higher number of IDs visiting the discussions regardless of their contributions. These data show that IDs accessed the discussions although they did not post any insight in the discussion forums (see Fig. 4).

Fig. 4
figure 4

The number of registrants, participants accessing the LMS, offering insights in online discussions, and viewing the discussions. There were more IDs who viewed the discussions than those who posted insights in online discussions

Full size image

The above findings from the analysis of LMS data display a less-optimal participation level. The next question sought IDs’ insights allowing us to understand the factors underlying the levels of engagement. Therefore, we would be able to identify interventions to optimize IDs’ participation in future PD events.

Q3—What Were the Factors that Influenced IDs’ Levels of Participation in the PD Events?

We invited the co-founders of the group and all past presenters and participants across the three PD events to complete the survey. A total of 28 IDs (N = 28) completed the survey. Among them, two were co-founders of the group coordinating the PD events (N1 = 2) who served as core members, seven were past presenters (N2 = 7) who served as active members, and 19 were past participants (N3 = 19) comprised of active and peripheral members. The two co-founders of the group were included so that the data analysis could potentially detect different perspectives of members in various levels of participation.

Demographics

Demographic responses showed that most IDs were female (22 indicated female, five indicated male, and one preferred not to mention). Years of working experiences varied. Eight IDs had been working in the ID field for more than 8 years. Seven IDs worked between 6 and 8 years. Six IDs worked between 3 and 5 years. The remaining three IDs worked in the field for less than 2 years. All IDs participating in the formative evaluation were from various education sectors. Eight of them were from a public higher education setting; six were from a private higher education setting; seven worked in a for-profit organization; two worked in a non-profit organization; five worked as an independent consultant. The remaining IDs either worked in a military, private K-12, public K-12, or other settings. Demographic information additionally shows that IDs worked in multiple settings simultaneously. While they had a primary job, a few of them were also concurrently affiliated with other organization, company, or managed their own consulting business.

In the subsequent sections, further survey results are presented. Results of multiple factors influencing the levels of participation are also summarized in one table (see Table 5).

Table 5 Knowledge-sharing self-efficacy
Full size table

Knowledge-Sharing Self-Efficacy

Co-founders were confident about their knowledge and were excited about coordinating the next PD events. Participants also self-rated their knowledge to be high but were not sure about their capabilities to demonstrate and share opinions (M = 3.31, SD = 0.58). Based on the means, the co-founders overall displayed the highest rates, followed by presenters and participants.

Bonding among IDs

We discovered that co-founders self-rated the statements higher than the past presenters and past participants did. Presenters and participants approximately rated the sense of belonging (M = 3.86, SD = 0.69, and M = 3.74, SD = 1.11) and trust (M = 3.57, SD = 0.53, and M = 3.89, SD = 0.94) statements at similar levels. However, they overall enjoyed working together (M = 4.57, SD = 0.53, and M = 4.26, SD = 0.73). The co-founders strongly agreed that their ideas were valued by other ID colleagues. The presenters’ perception regarding this statement seemed stronger than the participants’ perception was.

Open Communication

Co-founders were confident that there was open communication. They rated all the statements higher than the past presenters and past participants did. Based on the means, both presenters and participants similarly rated the open environment (M = 3.71, SD = 0.75, and M = 3.79, SD = 0.78) and easy communication (M = 3.57, SD = 0.97, and M = 3.89, SD = 0.94) statements.

Perceived Value of Learning from Others

Overall, co-founders rated the statements about the opportunity to learn from each other almost on the highest end. Everybody mostly agreed that they interacted with others with the intention to learn (M = 4.50, SD = 0.71; M = 4.28, SD = 0.76; and M = 4.37, SD = 0.68).

Causes Hindering IDs’ Participation

When we asked IDs (regardless of their roles such as co-founders, presenters, and participants) to order the potential causes, they mostly voted the scheduling and timing as a primary cause that hindered their participation. After counting the frequency of each rank for each cause, the top-rated causes were discovered as follows: (1) lack of time, (2) intention to review materials only without participating in the discussion, (3) complex discussion prompts, and (4) uninteresting discussion prompts (see Table 6).

Table 6 Frequency of each rank for the causes hindering IDs participation
Full size table

Open-Ended Responses

A total of 23 IDs contributed open-ended statements. We analyzed these responses and discovered four highlighted themes as follows.

Lack of Time

Twenty-two percent of IDs who contributed statements confirmed that the lack of time was a primary issue. They did not have time to go through the materials. Other IDs said that they were too busy at work to participate. For example, while December was thought to be a slow month, one ID noted otherwise, “I think the tyranny of the urgent in December with year-end project competed for my attention and won.”

Insufficient Comfort Level to Participate

Other responses corroborated that there was hesitation due to low comfort levels to participate. Approximately 17.39% of IDs contributing statements expressed that they were unfamiliar with other learners. One representative comment noted, “… [one of the] drawbacks of online discussions for working professionals [is] that [they] are not as familiar with each other.” Other IDs preferred to wait for more IDs to participate in the online discussions rather than posting additional insights. This following statement provides an example: “… when I saw not many folks were participating, I opted not to continue.”

A Need for More Engaging and Motivating Materials

IDs (21.74% of those contributing statements) expressed that the materials and discussion activities could have been designed to be more relevant for their context. They felt like a student instead of being treated as a professional colleague. One outstanding comment conveyed that “…discussion forums feel too student-like…[it should] enable folks to respond in a less academic, more conversational way.” Other responses suggest that the materials could have been more engaging and motivating. For example, one ID recommended, “I think guiding the presenters to create more interactive materials, rather than direct instruction videos could really help.”

Other Recommendations

Other responses implying recommendations for the co-founders to better coordinate future events were offered by 21.74% of IDs who contributed statements. IDs asked for more activities targeting higher-order skills. One ID suggested to “make it more like a problem-solving exercise.” Another ID gave an example of activities such as “… mini projects, sharing ideas, collaborating with participants.” Another recommendation offered was conducting a strategic promotion of the event. One noted that the co-founders could have conducted a “better promotion beyond internal social channels. [Co-founders] need to invest in public forums.”

Discussion and Implications

This investigation was performed as a formative evaluation of CoP-based PD events delivered through an open learning environment for determining the efficacy of the events and future improvements that can increase the participation of the members. Discussion and implications can be extrapolated, addressing each question that guided the formative evaluation.

Q1—To What Extent Did IDs Collect Information from Each PD Event by Viewing the Content and Resources in Canvas LMS?

By analyzing the LMS data, we discovered a gap between the number of registrants and the number of actual IDs accessing the PD course sites. Additionally, among those who accessed the PD course sites, there was a lower number of IDs accessing the modules and even a lower number of those contributing to the discussions. While this may sound discouraging, we also noted that either existing participants continued to access, or additional participants accessed the PD Canvas sites for the next several weeks after the 1-week schedule was over. It is possibly because the topics offered in the PD events are found to be interesting by IDs. Additionally, the number of page views occurring within each module unit varies. Modules receiving higher number of page views may suggest that its topic is found to be more interesting and relevant than others. Two implications are perceived from this set of findings.

Offer Relevant Topics

Based on the above finding, there appeared to be a learning value perceived by the members. IDs wish for the resources and context that they can conveniently and practically use (Schwier et al. 2004). As an implication, coordinators and presenters should concentrate on the relevant topics and activities in order to promote participants’ interests and assist them with meaning-making relevantly to what they need to gain professionally. Additionally, the practice to keep the PD Canvas LMS sites open as a self-paced format should continue to allow flexibility and provide timely resources as IDs’ PD needs arise. This notion corresponds with Trust et al. (2017) that working professionals seek for flexible learning opportunities according to their immediate needs, interests, and career goals.

Provide Intuitive Course Structure

Findings also discover the page views occurring outside the 1-week schedule. As the professional learning needs arise, the participants either access the content again or new participants decide to access the PD Canvas sites through the self-paced mode in the absence of co-founders’ or PD coordinators’ and presenters’ live facilitation. One recommendation is that the course navigation within LMS should be intuitively kept simple and consistent with logical structure (Preece 2000). By applying this strategy, the self-paced participants can easily find content and resources without any support; otherwise, they may feel frustrated and lose motivation to explore further (Simunich et al. 2015).

Q2—To What Extent Did IDs Participate in Each PD Through Online Discussion Forums in Canvas LMS?

Results display that only a small percentage of IDs participants views the discussion forums (between 33 and 38% of the number of IDs accessing Canvas LMS). The number of IDs offering insights in the discussion forums is even lower (between 8 and 27% of the number of IDs accessing Canvas LMS). These results suggest that some of IDs visited the discussions although they did not post any insights in the discussion forums. Two implications are additionally offered.

Enable Connections Among PD Coordinators, Presenters, and Participants

There are several factors as to why IDs did not offer insights in discussion forums. IDs may be coming from different settings and may not know one another. It is therefore imperative for the coordinators and presenters to pay special attention to providing welcoming environments (Jones et al. 2016). For example, encouragement to post (Preece et al. 2004) from the PD coordinators can be helpful. Instead of relying on the presenters to facilitate an ice-breaker discussion, the coordinators can create a special general forum serving as an introductory forum. Additionally, coordinators can suggest an introduction from all presenters and participants, discussing why the PD topic was mostly voted by IDs and how it is relevant with their current professional needs. This special general forum may also serve as an icebreaker to promote IDs’ comfort level for carrying out a dialog with the colleagues they have not met.

Stay Engaged with Those Who Already Provide Insights

Coordinators and presenters may consider staying engaged with the IDs who already contribute insights in the discussion forum(s). One way is performable through provisions of responses and feedback in the discussions. Carrying out regular dialog with them can form meaningful connections that strengthen the CoP existence. As reminded by Weseley (2013), a CoP is “a community of learners who engage regularly in dialogue and create meaningful relationships” (p. 313).

Q3—What Were the Factors that Influenced IDs’ Levels of Participation in the PD Events?

Findings from Q3 overall validate the CoP concept. For an instance, the coordinators reported a more positive perception regarding their self-efficacy and the existing open communication than the presenters and the participants did. IDs holding the co-founder role seemed to be more comfortable in the group and enjoy working with others, possibly because they are on the core position within the CoP group (Wenger 1998b). Other IDs mentioned time constraints as a factor influencing their participation. This set of formative evaluation results can be further expounded into four implications to alleviate factors hindering the participation of CoP members.

Minimize Potential Technical Issues

Most IDs rated the lack of time as the number-one factor hindering their participation, explaining why they remained less active (Preece et al. 2004). To mitigate this issue, a recommendation to keep the course structure and navigation as simple as possible is reemphasized. Ensuring the ease of use, testing the course in Canvas before inaugurating a PD event is a requirement (Preece et al. 2004). Consequently, it will reduce the potential technical issues that can take away IDs’ time from attaining and sharing knowledge, mainly when time is an issue for them.

Facilitate Pre-Dialogs Before Inaugurating a PD Event

Results further suggest that some IDs may not be confident nor feel comfortable in sharing their insights with others due to low self-efficacy and issues related to bonding. This may explain a reason why most IDs intended to only review the materials of the PD without participating in online discussions. As revealed by a previous study, when people do not feel comfortable with one another (Preece et al. 2004) and are afraid of being judged by others (Guan 2006; Lai and Chen 2014), they may be hesitant to offer insights and participate in discussions. To assuage this issue, the coordinators may consider facilitating pre-conversations in social media before a PD event is commenced. To reiterate, this can serve as an early icebreaker to promote ID’s enthusiasm about the topics in the upcoming PD and to facilitate early connections among members. When they are comfortable with one another, they may participate more actively during the PD event. As the ease of communication increases in the community, it can enhance intellectual conversations among members (Lai and Chen 2014; Phang et al. 2009).

Support the Less-Active Members

Our findings indicate that IDs participation in the CoP often occurs in a less tangible manner. Some IDs who did not post any comments in the online discussions visited the discussion forums, and even after the 1-week PD period. There is a possibility that IDs gain information by reading others’ posts without responding to the discussions (Preece et al. 2004). Coordinators and presenters can remain supporting those who do not intend to participate in the discussions. For instance, placing the discussion forums and classifying the discussion topics logically in LMS are imperative in supporting the peripheral IDs to review the discussions at their own time (Preece et al. 2004). To reiterate, the current practice to keep the LMS course sites available in a self-pace mode after the completion of the 1-week PD period should also continue.

Select Appropriate Activities

Another set of results display that IDs ask for relevant, engaging activities and interactive materials for the PD. This request resonates with the CoP characteristic that CoP members must “engage in joint activities” (Wenger 2006, p. 2). It is not surprising that ID professionals asked for activities like “a problem-solving exercise” in addition to “… mini projects, sharing ideas, [and] collaborating with participants.” Moreover, such activities have been perceived as valuable to professional development (Shepherd and Cosgriff 1998).

The coordinators may consider a different approach for guiding the presenters in designing the activities. For example, if the selected topic centers on how to succeed in a job interview, the IDs can be facilitated to collaboratively share resources regarding preparations of the job interview, such as a collection of potential interview questions, tips on how to dress for success, and strategies on how to demonstrate an appropriate attitude during the interview. A wiki feature in the LMS can be utilized for accumulating these resources (Bonk et al. 2018).

Another example, if the most-voted topic is about building a portfolio, the co-founders may consider inviting an expert to facilitate the PD event. The expert can provide guidance on selecting the portfolio-building tools appropriately and how to present portfolio artifacts effectively and competitively. By the end of the PD event, participants will have a portfolio, or at least have several portfolio components and a plan on how to complete the portfolio. After all, participating in online discussions may not be the only way to connect with other CoP members since an effective CoP is also portrayed through the reification such as via artifacts and documents representing the knowledge gained from the community (Wenger 2010; Wenger et al. 2002).

Conclusion

Due to the nature of the field, IDs are required to stay current on trends and issues related to instructional design. Therefore, obtaining professional development without being limited by geographical location and temporal barriers is perceived as important by IDs. According to Trust et al. (2017), working professionals seek flexible professional learning opportunities based on their immediate, relevant needs and career goals. The CoP-based PD events delivered through an open environment are an initial attempt to help IDs meet their timely professional learning needs. However, there remains much to be learned about facilitating these CoP-based PD events for IDs especially that the number IDs participating in the formative evaluation is low. The results are still insightful as “the function of formative evaluation is to improve” (Nieveen and Folmer 2013, p. 158). This formative evaluation promotes an awareness regarding the areas to improve and offers implications for how open, CoP-based PD can be designed suitably for instructional design professionals. As technology affordances provide flexibility to the PD venue, more online CoPs may emerge from other professional groups. Findings from this formative evaluation will not only inform IDs’ CoPs exclusively, but also offer insights to other CoP groups and extend CoP-related literature on both research and practical aspects.