Providing Continuous Learning and Professional Development Through a Toolkit Design

Abstract

We devised a Continuous Professional Learning and Development (CPLD) initiative for all academic staff in our university, many of whom had limited or no experience of online learning and teaching, to enable them to shift their teaching online in response to the restrictions to campus attendance during the first wave of the Covid-19 pandemic in early 2020.

We adapted the ABC Curriculum Design Framework (UCL, 2018) into a ‘Toolkit’ for online delivery. The Toolkit was aimed at harnessing the features of the ABC Framework identified by its authors (Young & Perović, 2016) via work by JISC, (2012) as appealing to busy academics (rapid and intensive) and as effective for online learning design (robust theoretical underpinning, graphical representations of course designs). Uptake of the CPLD was high, and initial feedback was positive. A year later, we conducted a small-scale qualitative study exploring staff perspectives on the CPLD and their teaching practice. We found some positive benefits of the initiative, but also some less successful outcomes. Our findings show the limitation of metrics-based evaluations of academic development (Bamber, 2013; Bamber & Stefani, 2016), and we present some practical ideas for enhancing the Toolkit.

1 Introduction/Background

Glasgow Caledonian University is a post-1992 institution with its main campus in Glasgow City Centre. It contains three academic schools, each of which has a strong background in professional disciplines.

When the global coronavirus pandemic first began to impact UK universities, it was evident our campus would be unavailable for an unpredictable time. Like many other Higher Education Institutions we looked to our Virtual Learning Environment (VLE) to provide a continued learning experience. We devised a Continuing Professional Learning and Development (CPLD) initiative to augment staff pedagogical and digital capabilities at scale in preparation for teaching fully online. In doing so, we aimed to minimise disruption to teaching and learning while increasing staff capabilities in online teaching.

In March 2020, our institution employed around 780 academic staff who, pre-Covid, varied considerably in their levels of expertise in online pedagogy: from staff who already had experience of teaching and learning online, to those whose involvement had been minimal before that point. We had recently completed our annual survey of staff digital capabilities, and although the response rate was low (n = 138), the data provided a useful snapshot of staff digital capabilities immediately before the pandemic. Respondents were nearly evenly split between those who felt confident in the use of digital technologies and those who considered themselves in need of some or lots of further development.

Previous CPLD initiatives related to online pedagogy were fairly small-scale, and it would be reasonable to represent pre-pandemic teaching practice in the university as variable, with many staff using the VLE as a document repository and class noticeboard, a commonplace approach to using VLEs in higher education according to O’Rourke et al. (2015) to supplement on-campus learning. At the other end of the scale, other staff were using the VLE to provide an engaging and active online learning experience. Only a relatively small number of online (distance) programmes were on offer.

This provided a challenging starting place for designing a CPLD initiative for online teaching. Early conversations with faculty leads suggested the academic schools favoured impactful, practical solutions over slower burning theoretical learning, prompting us to consider in depth both the content and the format of the CPLD initiative. Constrained by time, we decided to adapt an existing, successful framework for learning design, aiming for an approach that was suitable for even the most novice academics to develop knowledge and understanding of good online pedagogy.

2 ABC Learning Design

The ABC Curriculum Design Framework (UCL, 2018) developed by Nataša Perović and Clive Young builds upon Laurillard’s (2012) Learning Types theory, underpinned by her Conversational Framework (Laurillard, 2002) and the University of Ulster’s (2012) Viewpoints project storyboarding approach, to enable rapid or ‘sprint’ learning design that appeals to busy, ‘time-pressured’ academics (Young & Perović, 2016, p. 390). The method is described by the authors as ‘lightweight and streamlined’ (Young & Perović, 2016, p. 391), combining several of the successful methods identified in the JISC Institutional Approaches to Curriculum Design Programme (JISC, 2012), i.e. short, intensive workshop-style delivery where staff can come together to discuss and collaboratively ‘design’ courses using some form of graphical representation, for example, a storyboard, road map and/or timeline (Young & Perović, 2016, p. 391; Beetham, 2014, p. 5). The Viewpoints method of storyboarding adapted for ABC was centred around a paper-based storyboard and cards to guide discussion, enabling sequencing of activities on a flexible course timeline.

Storyboarding requires, as a bare minimum, a set of course intended learning outcomes. From here, participants work to build their courses in layers: deciding upon the overarching timeline and unit of learning (e.g. blocked into weeks, topics and/or themes); then agreeing on a suitable pattern of learning types within each unit; next making decisions about the most suitable means and blend of delivery; and then looking for opportunities for formative and summative assessment (Young & Perović, 2016, pp. 392–393). An explicit design process mapped out sequentially in this way, whilst arguably implicit in the design process for in-person teaching amongst experienced practitioners (Moallem,1998), is arguably essential when designing online learning (Tennyson & Schott, 2010), particularly where participants lack experience.

The ABC approach was highly appealing. The method was already well received across the sector. Moreover, it was the ‘built in’ nature of robust theory in online pedagogy from Laurillard’s work (2002, 2012) and a simple and engaging design process implicitly underpinned by the steps required to enable constructive alignment (Biggs, 1996) that offered sufficient scaffolding for even the most novice lecturer to create the basis of a good online learning experience.

2.1 Adapting the Framework

Pre-pandemic, the ABC Curriculum Design Framework was supported by a global community of practitioners, many of whom were keen to incorporate it into their support for staff during the shift to online teaching. As 2020 progressed, the online resources for the ABC community began to evolve to incorporate adaptations of the framework by the original authors (Young, 2020), as well as other institutions in the UK and Europe. Because the paper-based materials for the workshops were central, reshaping them for online delivery was the primary challenge of any adaptation. At GCU, we adopted the basis of an early adaptation by the University of Laurea, (2020) that had reconceptualised the paper-based course ‘canvas’ as a Padlet in the ‘shelf’ format, embedding the learning types cards and a set of instructions into the Padlet for users to copy and paste as required.

There was a range of other digital tools aside from Padlet, such as Trello or even Excel, that would have fulfilled the storyboard function. Padlet was not a perfect solution by any means. At the time of writing, there are unresolved issues with the accessibility of Padlet, the main issue being there is no way to add an alternative description to images. However, Padlet is very intuitive and simple to use; is very flexible and versatile; and has simple functionality for collaboration and sharing. Our institution had a licence for the platform, and it was already being used well by some academics for student learning. If staff were motivated to engage with Padlet for curriculum design, we hoped they would go on and apply it for student learning: a kind of CPLD by the back door, so to speak.

2.2 A Toolkit Approach

Extending the ‘built-in’ design features of the ABC framework, we developed a Toolkit, incorporating resources for online design and delivery in one self-service package. Even when not dealing with a crisis, academic staff have reported deprioritising their own development needs in favour of other essential tasks. Of course, curriculum design is integral to teaching. However, the majority of our academic staff prior to this did not use systematic instructional design processes for developing their curricula, accounted for by Moallem, (1998) who suggests that traditionally, teachers, especially experienced ones, treat the design of teaching and learning as implicit. Therefore, we anticipated CPLD focused on curriculum design may have been judged by academic staff as ‘nice to have’ but superfluous to their immediate needs.

Since Moallem’s (1998) research, blended and online learning has become increasingly prevalent in Higher Education, facilitated via the emergence of Virtual Learning Environments (VLEs) and, more recently, Massive Open Online Courses (MOOCs). Institutions whose academics have been involved in creating content for MOOCs are likely to have had exposure to instructional design approaches (e.g. see Lackner et al., 2015; Warburton & Mor, 2015), thus making the curriculum design process far more explicit than ever before: arguably a concrete design process is highly advantageous to the effectiveness of online courses (Tennyson & Schott, 2010; Beetham & Sharpe, 2013). Our institution had not previously had any notable involvement with MOOC design; moreover, the provision of CPLD in curriculum design was limited and variable across disciplines. The Toolkit was the first attempt to promote systematically to all academic staff the benefits of instructional design by embedding it within a CPLD initiative aimed at building capabilities during a time of unprecedented change. By packaging and communicating the initiative as a practical, developmental toolkit solution rather than traditional ‘training’, we hoped to increase its perceived value and entice high participation rates in spite of significant workloads. The term ‘toolkit’ has connotations of practicality, problem-solving and efficiency, which suitably characterise instructional design in general (Moallem, 1998), and was intended to appeal to and reassure staff as they faced the challenge of ‘pivoting’ to online teaching.

The initiative also provided an opportunity to model best practices in online learning by effectively blending a range of different learning types and digital tools and demonstrating how to balance synchronous and asynchronous learning content. As far as possible, the Toolkit exploited a number of the same digital tools academics would have at their disposal in creating learning activities for their own learners, providing them with a useful end-user perspective. This approach (it was hoped) would facilitate reflective practice as part of experiential learning (Kolb, 1984) during the design process, especially when making decisions about which digital tools might work well for delivering different ‘learning types’ (Laurillard, 2012).

The key elements contained in the Toolkit were embedded into a single webpage on an internal SharePoint site and included the following:

• 1. An introductory webinar via Blackboard Collaborate Ultra. The only synchronous aspect of the Toolkit, the webinar was recorded for flexible access: important given the unpredictable impact of the rapid move to remote working for staff.

• 2. A printable PDF supplemented the webinar, foregrounding additional relevant models useful to the process of instructional design, including a reminder of the concept of constructive alignment (Biggs, 1996) and also an introduction and hyperlink to the Universal Design for Learning (UDL) framework (CAST, 2018) to promote the practice of accessible, inclusive design-thinking. The storyboarding activity was illustrated as occupying the initial stages of a bigger, overarching instructional design cycle similar to the cyclical ADDIE instructional design model (Culatta, 2018) with five discrete phases of activity: Plan, Design, Build, Implement and Evaluate. Locating the CPLD activity into a larger process helped staff divide the bigger task of moving teaching online into smaller, more manageable stages.

• 3. A Padlet storyboard (see Figure 1) provided the platform for online learning design. New users were advised to complete ‘An introduction to Padlet’ CPLD, so they understood the basic features of the tool.

• 4. An online discussion forum provided a way of channelling questions after the webinar as the participants began to put the CPLD into practice.

Fig. 1

The GCU Padlet storyboard template

The cards in the template contain the learning types cards (front and back), some basic instructions about how to use the storyboard, and sections where key module information can be added. The space on the right of the template below is the blank storyboarding space within which users can add activity cards and code them with the relevant learning type/s so they can see the pattern of learning at-a-glance.

It was made clear that the Toolkit was aligned with the early stages of a wider instructional design process. After using the Toolkit to design, the next stage would be to build. So, we also provided clear linkages (e.g. within the Padlet and in the PDF, which contained a table for recording development needs) to further CPLD in the use of digital tools for online delivery.

For example, when virtually ‘turning over’ learning types cards in the storyboarding process, a number of digital tools were suggested. The card corresponding to collaborative learning had on the back suggestions such as Wiki, Padlet, a discussion forum or a student-led activity in Blackboard Collaborate Ultra. Each suggestion corresponded with a further CPLD opportunity, usually a webinar or self-service resource.

2.3 Evaluation

It is somewhat difficult to pin down the exact figures for attendance because the live webinar was also recorded. However, our broad analytics show that the live version was attended by 463 individuals, and we delivered a team-based session 16 times: attendance much greater than any previous local CPLD initiative by far. Attendance of webinars that supported staff CPLD in the use of various digital tools was also high: with 1575 live attendees in total and many others who accessed the session recordings at a later time.

The relatively high level of attendance described above and some generally positive qualitative feedback from staff via post-webinar evaluations that the workshop was useful to them suggest at least some indirect positive impact (i.e. attendance and staff satisfaction) of the CPLD initiative on aspects of teaching practice:

“I benefited from being able to actively participate in the webinar as this mimics the live classroom experience.”

“The Responsive Curriculum Design tool kit was my saving grace when planning out my module and how I was going to deliver the content on-line in a meaningful, professional manner.”

“Using the responsive curriculum design template has allowed me to develop my module ensuring I am using all the different learning types within.”

“Myself and my team are already using the storyboard/responsive curriculum design feature. This has enabled a level of co-creation.”

Following others’ (Bamber & Stefani, 2016; Hughes et al., 2016; Winter et al., 2017) acknowledgment of the complexities involved in evaluating academic development initiatives, especially their impact on student learning, an issue described vividly as ‘thorny’ by Winter et al., (2017) and ‘vexed’ by Geertsema & van der Rijst (2021), for our preliminary evaluation of the Toolkit we aimed to ‘evidence value’ (Bamber, 2013) in terms of demonstrable outcomes (i.e. changes in behaviour and practices) rather than outputs (attendance, staff satisfaction) (Bamber & Stefani, 2016).

3 Methodology

We conducted a small-scale qualitative study nearly one year after the CPLD. Qualitative research is criticised for its lack of generalisability, but generalisability was not our goal. We were seeking a detailed, contextualised understanding of the value of the CPLD in different disciplines and so we set out to explore a small number of in-depth participant experiences. Moreover, timing was key. Tick-box evaluations of academic development, usually deployed immediately after the intervention has taken place, may tell us about broad ‘satisfaction’, but can’t reveal resultant changes in behaviour. Winter et al. (2017) point to strong corroboration in the literature that waiting for around six months to evaluate an initiative is key to being able to uncover any changes or impacts to practice. On reflection, a year was possibly slightly too long: at times, some of the participants were no longer able to recall exact details about their experience of the CPLD. However, participants were able to articulate aspects of the short and longer term impact (if any) the CPLD had on their teaching over an extended period of time including any post-course student feedback. They could also show us their completed designs on Padlet.

Our sample consisted of six academic participants in total: two from each of the three academic schools. Participants attended online semi-structured interviews with the academic developer who delivered the original CPLD webinars. Although this posed a potential bias that might cause participants to under-report any negative feedback, this was mitigated by the informal tone of the discussions, assurance of full anonymity and a request for candidness. The interviews lasted around thirty minutes and were recorded using Microsoft Teams, then transcribed and anonymised.

3.1 Analysis

We carried out an inductive thematic analysis of the interview data through a step-by-step approach that included a standard process of reviewing the data for the purpose of coding and then combining these codes into themes, then further refining and defining those themes (Braun & Clarke, 2006).

4 Results

Three broad themes emerged from the analysis, all of which relate to the impact of the CPLD as well as being articulated by participants as generally characterising their experiences of shifting to an online delivery approach.

4.1 Theme 1: Time

Participant accounts of responding to the shift to online delivery and the role of the CPLD in that process were often keenly focused on the concept of time, most usually as a limited resource.

“I kind of felt we were getting bombarded with all you must use: Collaborate; or you can use Teams; you can use whatever, this is what you must do, and you must record it and you must provide transcripts and all in a short timeframe. It was all a bit nuts. Summer is usually a time to put your feet up … I don’t think I have worked so hard or felt under so much pressure as I have ever done.”

Participant 3

“I never actually finished mine. But I started doing mine and I loved doing it. I wanted to share it with students and was going to use it to help them at the start. Anytime I showed it to the team I said, it’s not finished but this is what I’ve got so far and it looked great. So it really was just the time.”

Participant 6

Some claimed the timing of the CPLD in the academic session was problematic:

“I think people needed the workshop at that time, but there was so much happening. Inevitably no-one did their storyboard. The template is great, it’s just the timing and it was so much for people.”

Participant 2

“There was an issue with the timing of when we have to give information. So for example, we’d already submitted our module timetables. So that includes the structure, the mode of delivery. We’d done it all as part of contingency planning … you’re planning so far in advance it’s just too late to do this.”

Participant 6

Time also cropped up as a feature of the learning design, where participants reflected upon the timings of their face-to-face lectures and how they would need to be adjusted for an online delivery, supported by the ABC design process:

“I had a lot of experience but very much of the traditional, you go into the classroom and you do your 50 minute, traditional lecture or your tutorial or whatever. It was showing how to take that resource in terms of the PowerPoints and whatever and break up, you know change the length for online delivery without simply doing it as the same live lecture.”

Participant 3

The theme of time was also articulated more positively in relation to the time commitment required to engage with the design process:

“I feel it was quite self-contained, which I thought was really useful actually because you could actually do it in the time specified. A lot of these things take longer.”

Participant 1

Helping students to manage time was also described as being made possible via the storyboarding approach:

“I used the Toolkit for planning the whole module. And then for each week, I set up an activity timeline used with the students so they could see what they needed to do and that was very useful.”

Participant 3

4.2 Theme 2: Knowledge

The concept of identifying and locating the knowledge needed to pivot online was discussed in different ways by the participants in relation to the CPLD. It’s important to note the distinction between knowing and understanding here. The former came through strongly, but understanding (i.e. the application of knowledge) did not emerge as a theme. Participants noted gaps in their existing knowledge that they hoped would be filled by the CPLD:

“I don’t know any hints and tips about how to engage students; it’s difficult to know how to engage them.”

Participant 1

“I didn’t really grasp what are the ways of dealing with really large classes online, its ok for acquisition but what about discussion and collaboration?”

Participant 4

The participants also identified potential sources of important knowledge they felt hadn’t been available to them in the CPLD initiative:

“It’s hard online but it would be good to be in the same room as others to do this. Maybe discussion with others may encourage those who were less experienced.”

Participant 1

“Would have been good to pair up with other staff who know more about how to do this.”

Participant 4

Participants also highlighted where the CPLD managed to build their knowledge:

“The curriculum design template helped me from the point of view of knowing what technologies are out there, what they’re capable of and knowing how to put them together.”

Participant 2

“I was always thinking about the six different learning types and making sure they had a good mix through the module.”

Participant 4

4.3 Theme 3: Organisation

The final theme that was prominent in the data was related to the benefits of the CPLD approach for enhancing management and organisation of courses, applying to staff and students. For staff, the benefits were articulated as assisting them in preparing for online delivery:

“I found it useful in the planning process, very useful in the planning process. And, as I said, then deciding how to chop up the lectures into chunks and organize that. And then, you know, thinking about thinking about the different activities. I found it very useful for that, and just organizing everything on a week to week basis.”

Participant 3

Additionally, the advantages of the approach for giving a method of effectively structuring content came through:

“This year, it’s a lot better because even just the layout within it and the different learning activities that have made sure to tick the boxes that are linked better to the outcomes and assessment, that was one key thing I really did.”

Participant 4

“I got feedback last year that everyone loved the content but said they would like the structure of the module to be clearer. And I wasn’t really sure how to go about this. I think the curriculum design allowed me to do that really quite coherently.”

Participant 6

The direct result of providing the storyboard as a roadmap for the course to students was commented on by the participants as leading to a better student learning experience, and in some cases to positive student evaluations:

“What this allowed me to do, and it’s something the students have fed back positively on, it is allowed me to distinctly create a theme for every week, and there are certain aspects of the curriculum we would focus on.”

Participant 2

“In the evaluation questionnaires, one of the things they are focused on that they really like is the structure that has come out as a result of this.”

Participant 6

5 Discussion

The thematic analysis provides a more fine-grained, nuanced evaluation of the impact of the CPLD than would be possible using customary post-evaluation surveys, adding further weight to the argument that metrics alone are not a particularly useful or appropriate way to evidence the value of academic development (Bamber, 2013; Bamber & Stefani, 2016). The study data reveal a positive impact on some aspects of the participants’ practice, mostly in relation to better and more explicit organisation and management of their courses, which fits with the argument presented above about the benefits of getting academic staff to engage with an explicit design process, i.e. via instructional design frameworks. There is also tentative evidence to suggest, via the accounts of the participants, that this in some measure enhanced the student learning experience by giving learners a much clearer insight into the overall plan for their learning.

The participant accounts point to a tension between an acknowledgment of the strengths of the CPLD approach but claiming a lack of time or inappropriate timeframe in which to adopt it to enhance practice. Despite providing a framework designed specifically as a rapid solution to curriculum design, this was not seen by some staff as offering timeous assistance and was ultimately disregarded.

Finally, we have gained insight into the extent to which the initiative was felt by participants to support the development of their knowledge of aspects of designing and delivering online learning. It appears it was successful in more theoretical aspects of this, but less so in helping participants to know how to apply this knowledge in practice. Clearly articulated was the requirement for peer-based support and access to examples of real practice to help participants gain a more in-depth and practical understanding of online pedagogy.

6 Next Steps

Our plan is to use our findings to directly shape and inform the next iteration of this CPLD initiative. Similar to many universities, due to the ongoing restrictions on campus attendance, we are looking at provision of a portion of our teaching and learning being delivered in a ‘hybrid’ or ‘dual mode’ format, i.e. involving simultaneous participation from both in-person and online attendees: this will most certainly require staff to think carefully about their course designs that are currently delivered in a single mode. Additionally, we wish to continue to strengthen and enhance online learning and teaching activities provided via a single mode of delivery. Via our study findings, we acknowledge the need to deploy the CPLD in such a way that it doesn’t conflict with existing processes related to course delivery. Moreover, the Toolkit could be further enhanced by providing a range of pre-designed templates to scaffold good designs and further save staff time by facilitating higher levels of peer-support amongst participants, and now we have some actual examples of this method being successfully implemented by staff, to work towards effective ways of prompting peer sharing of good practice and authentic user experiences.