Introduction

Regardless of setting or industry, individuals are often in need of documenting their expertise and skillsets for professional advancement. Digital badges as a form of credentialing has become a topic of discussion in more recent years (Catalano and Doucet 2013; Grant 2016). By definition, digital badges are electronic symbols used to document performance and achievement. Open badges are intended to provide additional information via metadata in order for viewers of badges to verify issuer details, evaluation criteria, and evidence such as the actual work product used to earn the badge (Parker 2015). Badge activity completion can be verified manually by the instructor/issuer or automatically if the badge platform has the capability. Manual evaluation increases instructor workload, whereas systems which allow automatic assessment have less impact on workload (Jovanovic and Devedzic 2015).

Digital badging is a relatively new concept, and the topic is broached with varying opinions as to how badges can best be purposed (Devedžić and Jovanović 2015; Hickey et al. 2014; Joseph 2012; Jovanovic and Devedzic 2014). Badges are typically used as non-tangible representations of skill achievement and accomplishments. Their value and efficacy in educational settings are still being explored (Gamrat et al. 2014; Gibson et al. 2015; Glover and Latif 2013; Hickey and Soylu 2012; Rughinis 2013). Currently there are no uniform standards for badge programming which often negatively impacts the perceptions of credibility outside of the organization or system where the badge was issued (O’Byrne et al. 2015).

There is debate as to whether digital badges are effective pedagogical tools and for what purpose they are best suited (e.g., rewards, incentives, assessment, skill recognition) in higher education. One use for digital badges is to acknowledge content mastery and knowledge acquisition. Badges hold potential to generate peer discussion and feedback opportunities (Hickey et al. 2015). Research recommends avoiding the use of extrinsic motivators such as rewards and incentives (Deci et al. 2001; Schenke et al. 2013). However, if used appropriately, badges can be a form of alternative assessment that actually increases motivation in learners (Abramovich et al. 2013; Deci et al. 2001; Schenke et al. 2013). The use of digital badges can assist learners by truncating assessments through their learning trajectory in order to document their progression (Abramovich 2016). A recent study (Reid et al. 2015) has shown that the absence of assessments in badging programs has left students dissatisfied. This study explored the utility of implementing digital badges in higher education settings by interviewing individuals who had successfully implemented related programs at their institutions.

The future of higher education may benefit from open badges used as electronic, mobile credentialing systems that reach beyond the walls of the traditional university (Ash 2012a, b, Carey 2012). A large amount of learning occurs outside of traditional classrooms, and badges are a way to recognize this informal learning (Wilson et al. 2016). Hickey et al. (2015) incorporated badges into a Big Open Online Course (BOOC) titled Educational Assessment. Peer discussion and feedback was a fundamental component of the course. Earned badges could be shared over social networks and email. Participants could also link their final work product along with peer endorsements and comments to the badge.

Questions arise regarding if digital badging programs are more effective for lower division courses, and if there is a difference in learning outcomes when learners are required to participate in a badging program versus if participation is optional. Some believe digital badges are only effective when used as rewards for lower-level rather than higher-level skill acquisition (Abramovich et al. 2013). Others suggest using badging programs as a way to teach the required curriculum for an entire college course, rather than using badges as a supplement to the curriculum (Randall et al. 2013). They suggest that giving students a choice in which badges they want to complete will promote autonomy and motivate them to learn, such as in self-regulated learning theory (Ash 2012a; Dodd 2014; Gibson et al. 2015; Joseph 2012. Badges also have potential to strengthen traditional degree programs, to support competency-based programs, and to link badge earners to potential employers and professional organizations (Glover and Latif 2013; Tally 2012; Wilson et al. 2016).

Digital badges can be used as virtual credentialing systems to showcase acquired skills and recognize achievements. Implications related to badges used as credentialing systems include the importance of standardization in order to provide credibility for badges awarded for skill acquisition and achievement recognition (Glover and Latif 2013; Hickey et al. 2014; Olneck 2012). Other researchers suggest badges used as microcredentials could position universities as credentialing institutions (Shen 2014; Wilson et al. 2016). Possible investigations into the future of badges could address the impact of badging on professional development and employment (Dyjur and Lindstrom 2017; Gamrat et al. 2014). There are instances where standardization could provide a common language and framework in which human performance practitioners and training personnel could practice (Hickey et al. 2014; Laanpere et al. 2014; Lemoine and Richardson 2015; Tally 2012).

Instructional designers, badge issuers, and educators should consider both the motivation and the ability of learners when designing badges and including them in the curriculum in order to avoid adverse effects of motivational outcomes (Abramovich et al. 2013; Schenke et al. 2013). Directions for future research could address these and other concerns by providing empirical research from which we can improve practice and establish theory (Abramovich 2016; Jovanovic and Devedzic 2014; O’Byrne et al. 2015; Olneck 2012). Research is also needed to study the impact of badges in various educational settings and with multiple types of learners (Cucchiarra et al. 2014; Gamrat et al. 2014; Fields 2015; Gibson et al. 2015; Hickey et al. 2014).

There is only a handful of empirical research available on the topic of the use of digital badges (Abramovich et al. 2013; Gamrat et al. 2014; Hakulinen et al. 2013; O’Connor and McQuigge 2013; Reid et al. 2015). The existing literature primarily discusses potentials of badges, especially in terms of motivation and self-directed learning in non-traditional learning environments (Cucchiarra et al. 2014; Schenke et al. 2013). While there is discussion offering best practices for designing digital badge environments (Gibson et al. 2015; Hickey et al. 2014), there is a notable lack of discussion addressing instructional design considerations to promote digital badging in higher education.

In order to capitalize on the potential uses of badges, empirical evidence is needed to better understand what constitutes a well-designed badge program. To maximize engagement, motivation, and learning, heuristics are needed to inform badge design (Schenke et al. 2013). Instructional designers must be well-informed on what considerations need to be made in order to ensure badge rigor while providing accessibility to learners of varying abilities. In order for badges to be considered a reliable reflection of the intended learning and valid pieces of evidence which transfer to other environments, such as from the classroom to the job market, instructional designers need guidelines on how to effectively incorporate evaluation and verification components into badge design (Olneck 2012; Catalano and Doucet 2013).

Motivational Design of Digital Badges

A survey of the literature and research on badging reveals that several theoretical constructs can be associated with badges. Self-regulated learning theory asserts there are benefits to encouraging learners to plan, set goals, monitor, and evaluate their learning processes (Zimmerman 1990). Self-regulated learners have the propensity to know when they have mastered a skill. These types of learners are proactive and seek out needed information to support their learning. They are able to overcome learning obstacles and adapt as needed in order to be successful while taking responsibility for their own learning and achievement. Allowing learners a choice, such as in which badges to earn, increases a sense of autonomy and may increase motivation (Cucchiarra et al. 2014; Schenke et al. 2013). In fact, setting goals can be an integral part of a digital badge program that is designed using a hierarchical model where learners progress through increasingly more challenging levels of content while earning badges as evidence of mastery along the way, similar to the program design by Randall et al. (2013).

Achievement goal theory can be used as a context for interpreting the value of digital badging (Abramovich et al. 2013). Performance goals increase performance capacity. Badges, similar to achievement goal theory, have components of both mastery and performance (Elliot 1999). Badges represent electronic artifacts of accomplishments and can act as symbols of achievement and evidence of mastery (Dickey 2005).

Academic motivation is explored by Schunk (1991) in the context of self-efficacy, which is the learner’s perception of his/her capacity to succeed at a given task. He outlines self-efficacy theory and discusses self-efficacy in terms of academic motivation and calls for more research on how efficacy and motivation impact knowledge transfer. Schunk (1990) purports that learner satisfaction increases with goal attainment and this in turn increases self-efficacy perceptions. The cycle can be repeated as goals are continually set and achieved in order to repeat feelings of self-satisfaction. The inherent nature of digital badges lends itself well to principles of self-efficacy, goal attainment, achievement, and motivation.

Early research on digital badging by Abramovich et al. (2013) showed that the type and purpose of the badge will impact motivation and learning. They found that badges impact both intrinsic and extrinsic motivation and that the effects of badges vary across learner ability, prior knowledge level, and badge design. Monetary rewards used as extrinsic motivators actually decreased intrinsic motivation while verbal praise used as extrinsic motivation increased intrinsic motivation (Deci 1971). These findings should be considered when determining the purpose of a digital badge program. Incentives should be avoided as well, as they are forms of extrinsic rather than intrinsic motivators. There has been an increase in using extrinsic motivators in education and as a result, a decline in intrinsic motivation has occurred as a result of a system which relies heavily on rewards and incentives (Martinez 2010).

Research points to intrinsic motivation being more effective than extrinsic motivation with the former type being attributed to increased learning gains (Abramovich and Wardrip 2016; Randall et al. 2013). One activity known to increase intrinsic motivation is setting and completing goals, as indicated by achievement goal theory (Elliot 1999). Learners who actively participate in their own learning experience increased levels of motivation, according to Goligoski (2012). A particular feature of badges is active participation which manifests as learner choice in which badges they wish to earn (Schenke et al. 2013). Pintrich and De Groot (1990) conducted a correlational study to explore connections between motivation, self-regulation, and performance. Students self-reported on level of self-efficacy, intrinsic value, self-regulation, test anxiety, and learning strategy use in addition to the measurement of performance. Intrinsic value was found to positively correlate with engagement and performance. Learner choice activities have the potential to increase motivation and desire to learn and acquire new skills.

Digital Badges in Higher Education

Open badges can be used in higher education as a tool to teach soft skills which fall under Bloom’s affective domain. Well-designed badges can provide a form of alternative assessment which supports collaboration and engagement. Educational technologists at Purdue University developed an e-portfolio system called Passport, an assessment platform that aligns outcomes to challenges and incorporates scaffolding. The University of Central Oklahoma identifies soft skills they expect graduates to demonstrate and uses badges to document and track these skills in addition to grades (Parker 2015).

O’Connor and McQuigge (2013) conducted a pilot study to examine the impact of peer-reviewed badges in an online graduate course. Badges were issued for web skills in an emerging technologies course by peers based on criteria which were different than instructor’s evaluation criteria. Students had the option to post their badges via web-based technology such as in digital backpacks or on social media sites for prospective employers to view. The authors hypothesized that peer-awarded badges would promote intrinsic motivation for participation in the course and support lateral learning and evaluation. Their findings showed responsible engagement of learners throughout the process which they feel warrants the further exploration of badges used as peer-reviewed evaluation.

McDaniel and Fanfarelli (2015) examined digital badging used to provide feedback and reward in the online higher education environment. Due to the research that points to the negative impact of extrinsic motivators, badges were used in the study as a feedback mechanism which provided information related to performance. The study was designed to elicit both engagement-based and performance-based data through the implementation of a badging system in online sections of web design and graphic design courses.

Digital badges for Professional Development

Badges hold value as a system for potential employers to validate skills aside from content mastery and to allow learners to reflect upon and track their own soft skills (Parker 2015). Gamrat et al. (2014) tested a digital badge system’s capability to support custom professional development opportunities. They refer to digital badges as microcredentials which represent knowledge and skills acquired by the badge holder. They based their study on the theoretical framework of customized decision making to allow teachers choice in which professional development activities they pursue. The researchers partnered with NASA and National Science Teachers Association (NSTA) to design a badging system they call Teacher Learning Journeys (TLJ) and conducted a collective case study to answer their research questions focused on badging and customized decision making. Their findings, which indicate enhanced teacher productivity and fulfillment, are intended to guide design principles of badges used in workplace learning settings.

There are indicators that badges could be used for credentialing specific skills and competencies (Abramovich et al. 2013; West and Randall 2016). Possible forums for badges include the use of digital backpacks to collect and showcase skills to prospective employers. Mozilla Open Badges is a free and open online environment which provides an infrastructure for learning recognition and verified accomplishments to be collected in one place and showcased to multiple individuals and organizations. Data are integrated into each badge which correlates with the badge issuer and the criteria upon which it was evaluated. Mozilla Open Badges provides free software that allows users to earn, create, issue, and verify digital badges (Goligoski 2012).

Badges hold potential to provide a method of credentialing skills (West and Randall 2016). Open badges are an alternative to traditional college transcripts from expensive higher education institutions. Some purposes of open badges are to promote lifelong learning and provide a format to showcase skills and achievements acquired outside traditional educational institutions and beyond the immediate badge holder community (Goligoski 2012). Badges are cataloged in a digital backpack which serves as a dashboard that enables badge owners to determine privacy settings and where they will publish their badges, such as on personal websites and professional networking sites. Table 1 provides an overview of studies that have been conducted exploring the potential for implementing digital badges in higher education.

Table 1 Research considerations for digital badging
Full size table

Purpose of Study

The purpose of this study was to explore how digital badges are being implemented in higher education settings. The results were used to identify needs and formulate recommendations to inform instructional design practice in higher education learning environments. The goal of this study was to provide insight regarding guidelines needed for instructional designers and issuers of badges for badge program implementation (Gamrat et al. 2016). The following questions guided the study:

  1. 1.

    What standardization or verification processes exist for digital badges?

  2. 2.

    What factors contribute to increased motivation for badge earners?

  3. 3.

    What are user perceptions of digital badge programs?

  4. 4.

    What design considerations need to be made when developing badge systems?

Method

Research Design

Our study was designed to gain a better understanding for how digital badges are currently being used in higher education. We used semi-structured interviews (Creswell 2015) to provide participants with an opportunity to elaborate on their individual contributes to digital badging at their respective institutions. We created an open-ended, structured interview script containing 13 questions for to explore how badges are being used in various settings and to glean more information from the badge expert perspective.

Participants

A purposeful convenience sample was obtained for the study. The interview participants included prominent figures from the digital and open badging community who are considered leaders in the badging community. These professionals have published research studies and documented involvement in high-profile badge programs and initiatives, activities indicative of the inclusion criteria. Participant criteria were set to ensure that this phase of data collection came from credible sources in order to increase internal reliability of the data collected. We conducted interviews with a total of 10 individuals. Nine participants worked primarily in the higher education field as professors, a research associate, an instructional designer, and one was an open badge and digital credential consultant.

Procedure

This study was reviewed and approved by an Institutional Review Board prior to collecting any data. We recruited experts and leaders involved in the digital badging movement to participate in our interviews to provide insight into some of the challenges associated with the digital badge movement. These individuals had experience with the design and implementation of badge programs, and many had researched and published on the topic of badges. We contacted these individuals via email who had documented experience in badges as evidenced by their research, published articles, conference presentations, or professional networking profiles to request their participation. A structured interview script was utilized to ensure that all participants were asked the same questions to ensure consistency. Interviews were audio recorded to avoid information recall issues during transcription and transcripts were sent to respondents for verification of accuracy before transcription took place. This extra step of member checking was included as a measure to maximize trustworthiness (Hays and Singh 2012).

Data Analysis

All interviews were transcribed according to an interview protocol that was designed during this study. Our data analysis approach to coding followed phenomenological (Giorgi and Girogi 2003; Hsieh and Shannon 2005) and ethnographic (Spradley 1979) approaches to better understand how digital badges were being used in higher education. We avoided assigned codes a priori so that the data from the interviews could allow for new insights to emerge (Kondracki and Wellman 2002; Mayring 2000). Once we had coded the results of the interviews, we discussed the codes to determine whether codes needed to be adjusted. This exercise also allowed for us to group codes into meaningful themes and better understand how digital badges were being used by the participants (Coffey and Atkinson 1996; Lincoln and Guba 1985; Patton 2002).

Results

Standardization Processes for Digital Badge Programming

Participants described incorporating self-assessment and peer-assessment components into their badge programs and explained that open badges supply the artifact or evidence of the activity completion, as well as the criteria on which it was evaluated. One issue that requires consideration regarding objective evaluation is that of interrater reliability. One participant described conducting faculty workshops to provide training and opportunities to work in groups to promote evaluation standardization. Badge system dashboards also allow for transparency to view submissions approved by all faculty members in order to cross-check evaluation patterns. These methods can help ensure consistent standards and badge rigor.

Table 2 outlines the organizational framework that guided our discussions with the participants about badge context.

Table 2 Organizational framework for badge context
Full size table

Most of the digital badge leaders (n = 8) interviewed were in favor of some sort of common currency, framework, or standardization for badges. They believe that transferability necessitates a common language between institutions and across platforms. Proponents of standardizing badges suggest a common framework is necessary for credentialing, college credit, and endorsement by third parties. They advocate for a basic unit of currency in order to understand a badge’s meaning, especially within operational contexts such as K-12, higher education, business, and industry. One participant shared:

I think there’s a necessity from a technical perspective for standards. From a conceptual perspective, though, I think that there should be a variety of standards that are appropriate to the contexts within which they operate.

There are, however, some experts who emphatically oppose standardization. One participant claims that the validity of an open badge is supported by linked evidence in the badge and that standardization undermines the purpose of badges. Another participant strongly believes that badges should not be used for credentials, since other alternatives exist. It was also expressed that standardization entirely depends on the badge purpose. One who opposes standardization had the following to say:

I think, too, that as badges become a much more revolutionary way of being able to speak to the ways that people are educated in the modern world, by forcing particular standards, by forcing particular pedagogies, by forcing these particular types of frameworks, it not only undermines the purpose of the badge, but it would also create its own weird power structure, too. That’s the power in badges and micro-credentials in general, is that it gets past those traditional mechanisms that don’t necessarily match how students are educated today or how they enter the workforce.

Although most of the survey participants viewed standardization, or a common framework, as necessary to ensure rigor and establish a basic unit of currency, there were a number of participants who did not see the need, and in fact, thought that standardization undermines the purpose of badges. One participant pointed out that validity is supported by the evidence linked to the open badges. Although others insist that it depends on the purpose of the badge. Standardization across institutions is currently non-existent, and the need for a common language or currency in order for badges to be more valuable outside the awarding institution or organization is a need echoed by many participants in this study:

I think they have the potential to be useful, but because there’s no standard for what a badge is, what it represents, etc., they are of limited or no use at the present time.

Another shared:

I think there is great possibility in them, but it will take outside entities to validate their importance.

Factors Contributing to Motivation of Badge Earners

Motivation is described as a complex factor, but most participants mentioned that participation-based badges are less motivating than proficiency-based badges. One participant described a badge program with unexpected or hidden badges and believed that these badges are more motivating than badges that are visible. When discussing how digital badges could be used to motivate learners, one participant shared:

If the badge is just used as an external motivator, or a digital gold star, it’s unlikely the badge will be motivating over a long period of time.

When discussing the relationship between digital badges and learner engagement, one participant shared:

Badges on their own are not likely to drive engagement, though they can foster deeper engagement if the broader learning environment is designed well.

Another added:

The usefulness and potential to increase engagement depends on the design of the badge experience and value placed upon them by employers and peers.

Several participants emphasized that digital badges must be accompanied by goals in order for learners to attain credentials:

I think if well explained and with a clear purpose, badges can increase engagement for students. There has to be a rationale behind each badge and specifics to earn them.

In addition, another added that organizations should consider scaffolding how digital badges are attained by learners:

Badges can be useful to increase engagement provided that learners can choose which badges to pursue. I think badges designed as part of hierarchies, with the attainment of lower-level badges leading to earning a higher, macro-level badge, can lead to increased engagement and work as effective milestones of learning achievement.

User of Perceptions of Digital Badge Programming in Higher Education

Those involved with badges enthusiastically agree that badges can be meaningful if used intentionally and purposefully. Participants identified badges used as micro-credentials as one of the best purposes suited for badges. Conversely, some participants felt strongly that this is not a meaningful use of badges, unless a common currency or framework for badges is established:

I think they can be useful; however, they require extensive buy-in from everyone involved, extensive program nurturing to keep people engaged, and a perception of currency (doesn’t have to be academic credit, but it does have to have perceived value; without that perception of value, it is just irrelevant clutter).

Another shared:

I think if badges are to ever gain currency (i.e., formal academic credit) among institutions of higher education, they need to be validated by an accrediting agency (e.g., CAEP). Otherwise, I don’t see the utility in accruing badges at the college level.

Learner benefits

Badges afford recognition of granular learning and informal learning recognition. Open badges are uniquely positioned to provide information storage and knowledge retrieval and to demonstrate learner competencies and accomplishments over social networks. The flexibility of badges creates opportunities for individualized learner choice and autonomy over one’s learning trajectory. There has been recent interest in how badges fit with competency-based education systems Gibson et al. 2015; O’Byrne et al. 2015; Rughinis 2013). If designed purposefully, badges can enhance motivation and engagement; allow learners to build identities around specific content; and provide a built-in feedback mechanism. Learners particularly value badges which can be recognized by future employers or college admission boards. This transferability, or currency, adds value to the learner experience (Ash 2012a; Catalano and Doucet 2013; Glover and Latif 2013; Joseph 2012).

Organization or institution benefits

Participants describe such public relations benefits as raised profiles, published articles, and grant funding opportunities. The process of designing a badge program forces the alignment of learning objectives and assessments, and it focuses the institution on its goals. Badge system implementation can also standardize instruction and course outcomes through defined learning pathways (Devedžic and Jovanovic 2015; Olneck 2012; Schenke et al. 2013).

Challenges and barriers

There are challenges that may arise during the design, development, and implementation of badge programs. One important consideration in the higher education setting is to consider how to gain faculty-buy in and how to mitigate faculty workload. Technical challenges may arise due to platform issues, and there can be stakeholder adoption issues as well. Barriers and challenges associated with badges were discussed by the participants and are listed in Table 3.

Table 3 Barriers and challenges associated with badges
Full size table

Designing issues were emphasized in terms of grappling with which platform to choose and how to make badges transferable. Several interview participants (n = 3) commented on the difficulty of locating information about what other people are doing with badges and how the various platforms compare. Another theme was innovation adoption and the badge skeptic. When discussing issues surrounding the topic of the design and implementation of badges, one expert shared:

I would say that the primary surprise factor for badges and badge systems, is that people don’t realize how complicated it is. They start off thinking, ‘Oh, I’m just going to make some badges. I’m just going to slap some badges on top of this thing and we are going to be done.’ And you’re developing a system, right? And the system is constantly ongoing. And it’s necessary to understand that it’s an evolutionary process and one that’s iterative.

When discussing how to handle the badge skeptic, one participant offered the following strategy:

In terms of developing and rolling out those ideas, talking with people that say, ‘Oh, that’s a series of stickers.’ And often my response is, ‘They could be.’ They can easily be stickers and that’s a use of badges. However, you can do a lot more with them; it’s all dependent upon the design.

Another issue that arose during the interviews was of maintaining relevancy. Recognizing that technology seems to be evolving at an exponential pace, one participant shared their concerns:

How do you keep this digital artifact persistent in a rapidly growing and rapidly changing digital world? It’s foolish to think that in a decade we’re going to have the same systems and technology that we have today, but how do we keep the things, and especially the badges that people earn now, still keep them relevant, still keep them active in the future?

Faculty adoption issues were identified as an obstacle for badge implementation. Faculty members are already feeling the pressure to publish and present, so implementing a badge system can be met with resistance. One participant said, “If you really want to get technology integrated, you have to put it in tenure and review.” Another participant described a badge program used as the university’s assessment model so that all students would be evaluated using the same framework. The participant also described increasing faculty buy-in by including faculty members in the program evaluation process and tying the program to faculty salaries when the badges are used for college course credit. The same participant addressed faculty training solutions:

We did a lot of workshops and a lot of exercises with them where we showed them submissions in groups and then the faculty would talk through together whether or not [the badges] were accepted or denied.

The interview participants who research badges (n = 9) describe issues such as low response rate, lack of literature, and confounding variables. The responses discussed the wide variety of badges out there makes it very hard to use one particular analytical lens or theoretical framework consistently over several different studies. Others identified the need to take a more qualitative approach to focus more on anecdotal research.

Design Considerations for Badging Systems

The purpose of the badge system should drive the design and development of the badge program. It is important to determine if the badges will be skill-based or participation-based. Most participants from both phases placed more value on skills-based badges. For skill-based, there should be evidence of mastery associated with the badge, along with the evaluation criteria, and these components should be accessible through embedded metadata within the badge. One suggestion was to document learning objectives first, and then determine how those objectives might translate to badges and the criteria for earning them.

Badges afford the opportunity to structure hierarchal learning opportunities and create badge families. It is worth considering how badges within a badge family or hierarchy may fit together as a collection of related badges. Consideration must be given to learner characteristics, abilities, and existing levels of motivation. Instructional designers must consider how they will build in challenge, ensure rigor, and incorporate feedback opportunities and transferability so that the badges can be shared with others across platforms. Summative feedback was chosen as the type of feedback badge design should include.

It was recommended by the experts in the interviews to start small and focus on the instructional strategies and content when designing a badge program. Learner ability and current motivational level should be determined through a learner analysis phase of the design process. The technology platform is a foundational consideration and will drive the system’s capabilities and scalability affordances. Designers should assemble a design and implementation team which includes stakeholders at all levels in order to increase buy-in and system adoption. It is recommended that instructional designers be familiar with the research, theoretical frameworks, and cognitive psychology as they relate to badges. As one participant stressed, draw upon the learning sciences and design-based research. One badge expert shared concerns for how to assess learner knowledge acquisition:

I really see the value of badges in terms of professional development, but I think there has to be some assessment. There has to be some demonstration of skills, otherwise the badges just become sort of arbitrary, and I think that’s the real hang up for most people when they talk about badges.

Participants also shared that consideration into how digital badge activities will be used for a broader audience. One participant shared:

People tend to design solely for their own institution, with little regard to how a badge might live outside of that institution. But it’s really, really important to step back and take as long a view as possible and say, “How will what I’m designing mesh into the world that this person is going to move into?”

Another suggested focusing on commonalities for learners working in different contexts:

I suggest that we start focusing on those commonalities across different environments, because that’s actually where we get to the true framework… that’s an organic framework that really represents what it is we are aiming to do [with badges].

Discussion

The results of this study are intended to identify needs and formulate recommendations in order to inform instructional design practice and build theoretical constructs. The goal of the study was to provide insight to instructional designers and issuers of badges and to provide guidelines for badge program implementation. Interview participants described badge programs used as assessment models, professional development programs, teacher technology integration courses, and in a multitude of formal and informal learning settings. One program described is unique in that it is tied to college credit, faculty salaries, and tuition.

While reviewing responses to the interview questions, it became very clear that when studying the potential use of badges, the badge purpose and learning context drives the answer to most questions one may have about badges (Abramovich et al. 2013; Gamrat et al. 2016). When designing studies, one must carefully consider the context in which the badge system is used in order to know which questions to ask in order to avoid the response, “It depends”. Similarly, considering the purpose and context of badges is also very important for those who are exploring badge design and implementation in their settings (Ostashewski and Reid 2015).

Most agree that badges awarded for participation are less meaningful than skill-based badges as suggested by (Abramovich et al. 2013). This is not surprising when one considers the extensive amount of motivation literature in the field which favors intrinsic over extrinsic motivation (Deci 1971; Martinez 2010; Randall et al. 2013). Participants cited the gamification elements of badges as engaging along with the value of open badges linked to evidence of learning activities (A recently published study found that badges were only slightly motivating for high-level learners and even less motivating (in some cases, demotivating) for lower-level learners (Reid et al. 2015). Participant responses also echoed findings from the motivation research which points to badges being more motivating when used as intrinsic rather than extrinsic motivators such as rewards.

Digital badges and more specifically, open badges, are used in a multitude of learning environments and serve many purposes. One interesting use of badges is formal recognition for informal learning which takes place in after-school programs, libraries, museums, camps, and sports. The granularity which badges afford was made apparent through participant descriptions of how badges can recognize much more than a letter grade. Acknowledgement outside the awarding institution or organization was the top choice selected by participants for what makes a badge motivating to earn. Goligoski (2012) also described badges as a vehicle to represent skills outside the immediate community.

Considerations for Digital Badge Programming in Higher Education

Institutions who are interested in implementing digital badging must take into account scalability. Regardless of the level and extent that badging occurs (i.e. classroom level, program level, department level, college level, cross-university), it is important that badge programmers consider the degree of transferability. If an instructor is utilizing badges in a course, thought must be given towards how these badges will be received or counted in other courses at the program level. It is important that the process of badging is standardized across the system or sub-system within which it is being implemented.

Most badge programs today are heavily focused on observable skills. Faculty interested in using badges within a course should review their course competencies outlined in course syllabi and identify strategies or artifacts that would demonstrate that a student had mastered the task.

Assessments must be linked to the skills being assessed and communicated with students at the beginning of the badging opportunity (Abramovich 2016). Detailed guidelines and support must be provided to students to support their abilities to self-regulate their learning as they progress through the badging activities (Cucchiarra et al. 2014).

Limitations

One limitation of this study is that most of the data came from those who designed, implemented, and researched digital badge programs with limited data retrieved from badge earners. Although valuable data were collected from badge experts, more data were needed from the badge user perspective. Future studies exploring challenges with implementing badging programs in higher education settings are needed to better understand the infrasture needed to support and sustain badge programs. Another limitation is that only those instructional designers with badge experience were recruited for the study. Another important limitation is that the relatively small sample size restricted the opportunity to hear from a more diverse set of perspectives.

Conclusion

The existing body of badge literature includes empirical studies related to motivation (Abramovich et al. 2013; Reid et al. 2015) and badges used as peer review in college courses (O’Connor and McQuigge 2013) and teacher professional development programs (Gamrat et al. 2014). More empirical research is needed from which we can draw to improve practice and establish theory. Suggestions for future research include a need for comparative studies related to platforms and usability studies. Additionally, there appears to be a need for more case studies and current published program information, so those interested in badging systems can explore and connect with similar institutions and organizations that are using badges. Knowing where to look for example badge programs could provide a framework for those making design and implementation decisions for new badge programs.

More research is necessary to establish the need for a common framework or standardization of badges, especially those badges used for credentialing (Ostashewski and Reid 2015; West and Randall 2016). Possible research studies could address the impact of badging on professional development and employment. For example, if badges are used by potential employers it will be important to understand how the job application process may shift in particular fields, and how human resource professionals will evaluate applicants through the digital badge lens (Finkelstein et al. 2013; Shen 2014). These are instances where standardization could provide a common language and framework in which human performance practitioners and training personnel could practice. The long-term impact of badge systems has not been explored due to the relatively small number of years of badge existence, but as time goes on, it will be possible to study the long-term effects of badges in specific contexts. Exploratory research to help define common frameworks for badges needs to be conducted if we are to unlock the potential of badges used as micro-credentials. The future of badging holds promise, and it will be interesting to see how it evolves and takes hold in various settings.