Keywords

1 Introduction

Practitioners of applied human factors and cognitive systems engineering are required to balance the need to innovate while ensuring tangible outcomes for the scientific and technological activities they perform [1]. From consumer products to highly specialized technology devices, researchers, designers, and engineers seek to combine best practices in their respective areas [2]: human-centered analysis, research, and testing to ensure adherence of the product or service with how humans think or behave; iterative and incremental user experience design to guarantee usability, utility, delight, and trust; and agile prototype development to assure quality, scalability, and resilience of the end product. In turn, the project or program managers must balance antagonistic constraints in allocating resources towards innovating (increased front-end analysis, cycles of design, and iterative assessment) versus building (more back-end implementation, quality assurance testing, and end-user evaluations).

2 Motivation

While the balance between innovating and building products, systems, and solutions (whether for commercialization or pure research) can be achieved in large, multi-specialized, and well-funded organizations, small businesses and academic institutions must often partially sacrifice innovation for tangibility, or vice-versa.

For example, a typical method for human factors practitioners to understand the users, the context of use, or the job-to-be-done consists in performing a Cognitive Task Analysis (CTA) [3, 4]. However, the cost and logistics of conducting a full CTA are oftentimes beyond reach for small teams on small budgets, leading our community of practitioners to adopt variants on the CTA, such as the applied CTA [5] or the hybrid CTA [6].

More recently the applied human factors and cognitive systems engineering communities have started adopting and practicing design thinking activities [7, 8] like agile sprints [9] or spark sprints [10] that combine upfront, design-oriented sessions centering the human end-user and their pain points, with iterative prototyping and testing. Although these techniques do not necessarily generate the critical artifacts typical of applied human factors processes, they do shed light on the cognitive aspects of end-users’ work and their work environment. But the breadth and depth of the analysis, which, in turn, drive innovation, is by design (so to speak) limited to make way for building activities such as rapid prototyping.

In larger efforts that require additional development (e.g., “runway building,” back-end infrastructure, data wrangling and feeding) or that go beyond the scope of scientific research (i.e., products meant to be deployed ‘in the real world’), these design activities are subsumed to DevOps [11] or DevSecOps [12]. Intrinsically, these development approaches emphasize continuous development, that is, tangibility, over deep innovation.

The Scalable Agile Framework (SAFe) [13] provides a structured approach to combine most of the aforementioned approaches. But SAFe is prohibitively expensive to many practitioners: There is not sufficient time, money, and personnel to do it all. In critical domains such as healthcare and defense, this conundrum is further complexified with requirements like information security, privacy, or capability maturity [14].

3 SPADES

To resolve this gap, we propose the Spiraled Agile Design Sprinting (SPADES) methodology. SPADES is intended as an activity and schedule-based framework within which all stakeholders collaboratively optimize what they do and how, while ensuring that neither innovation nor tangibility is sacrificed for the other.

SPADES was established by our cognitive systems integration (CSI) team, which is composed of human factors and cognitive systems engineers, cognitive scientists, software and quality assurance (QA) engineers, data and machine learning (ML) scientists, product owners, and scrum masters. Typically, our CSI teams works on Small Business Innovation Research (SBIR) and Small Business Technology Transition (STTR) contracts to research innovative and tangible solutions for our government customers. SBIR and STTR efforts are examples of projects where small teams work with small budgets to seek forward-leaning, yet concrete and feasible solutions to urgent and critical problems.

SPADES entails successive spirals of agile design activities that follow the SAFe structure, preceded by a human factors analysis of the domain, data, and users. The concept at the core of SPADES is that of the “technical nugget,” defined as a key, prioritized capability that solves a specific user pain point and is understood by all team members. Table 1 lists examples of technical nuggets for various critical domains.

Table 1. Examples of SPADES technical nuggets for various domains.
Full size table

As described in Fig. 1, SPADES is structured in three phases:

Fig. 1.
figure 1

The Spiraled Agile Design Sprinting (SPADES) approach starts with an upfront analysis to identify technical focus areas, continues with a series of spirals within which development sprints successively focus on each technical nugget, and ends with a closing phase for reporting.

Full size image
  1. 1.

    An upfront analysis for discovery and understanding, which leads typically to the production of general design artifacts and the identification of the key technical nuggets of interest;

  2. 2.

    A series of spirals, which combine innovation design with tangible development in successive development sprints, each focusing on a single technical nugget; and

  3. 3.

    A concluding sequence, typically employed for writing a final report and debriefing customers.

3.1 Upfront Analysis

The pre-spirals analysis of SPADES is calibrated to last for about one-half spiral and to target the identification and specification of the technical nuggets to be developed in the spirals. In this part of the process, the team selects the combination of human factors and design thinking methods they are comfortable with performing within the time and resource constraints imposed by the schedule. We have found that a combination of iterative Creative Briefing or spark sprints [10] with the hybrid CTA methodology [6] typically supports a deep and wide analysis for innovation. In so doing, the team not only builds empathy (a key design thinking concept) for the end users, but also gets to understand the cognitive underpinnings of the problems to solve. This upfront analysis supports the creation of high-level use cases and prioritized system requirements. In turn, those help narrowing down which technical nuggets are critical to building a solution.

3.2 Spirals

Each spiral in SPADES starts with a design sprint [9] or its scaled-down version, the spark sprint [10], should resources constrain the team, for the purpose of focusing and scoping the spiral technically (as opposed to capability-wise, which is done in the upfront analysis). This innovation-focused sprint is constrained to a single week and generates additional or more refined design artifacts as well as a prioritized backlog of tasks and user stories, per the SAFe approach [13]. This sprint constitutes the SPADES equivalent to the SAFe Program Increment Planning sessions.

Each spiral then continues with a series of two-week agile development sprints, one for each technical nugget (as identified in the upfront analysis). Each sprint employs the traditional agile ceremonies (scrums, stand-ups, backlog grooming, retrospectives, and demonstrations). During these sprints, team members adopt key roles such as product owner (who generates and specifies user stories to represent the voice of the user), scrum master (who orchestrates the work and removes logistical impediments), and dev team member (e.g., encompassing front-end designers, software engineers, quality assurance engineers, data scientists and modelers). The driving force for each sprint is the constraint that one iteration of the technical nugget must be completed within two weeks. This unique focus shared by the team over a defined period of time ensures agility and productivity.

Finally, each spiral ends with a one-week retrospective analysis, which identifies and prioritizes gaps for subsequent iterations. During that week, the tangible artifacts, whether they are software instantiations, or design artifacts, or artificial intelligence (AI) and ML models, are released and demonstrated to the customer. Demonstration also constitute opportunities for collecting feedback from external stakeholders.

3.3 Concluding Sequence

After the series of spirals is completed, a project is effectively closed with a reporting sequence where all the artifacts designed and prototyped in the spirals (illustrating both innovation and tangibility) are documented per the contractual requirements set by the customer. The duration of this sequence varies from contract to contract, based on how many spirals are feasible within the budget and time constraints under which the team operates.

4 Application and Lessons Learned

Our team has employed SPADES on SBIR and STTR research projects for several services and agencies in the Department of Defense (Table 2). Recognizing that our customers require more than reports at the end of Phase I projects, we applied the SPADES approach to efforts ranging from 6 to 12 months, for the specific purpose of building a tangible proof-of-concept or prototype, without sacrificing the innovative requirements.

Table 2. Examples of innovative and tangible outcomes achieved using SPADES on SBIR and STTR Phase I efforts.
Full size table

Based on the application of SPADES to the efforts listed in Table 2, we share with our fellow practitioners the following lessons learned.

SAFe Ceremonies Are Key to Focus.

We have found that adopting all the SAFe ceremonies, which engage all team members, even for as little as 15 min, drastically improves team collaboration, awareness, and understanding. The benefits included delay reduction (e.g., through the immediate sharing of design artifacts to software engineers), less re-do or iterations (e.g., because of a better alignment of middleware analytics models with data feeds and front-end interactions), and better adherence to the scope, thereby reaching a more resilient and tangible outcome faster.

It’s Ok to Be Flexible.

None of our CSI team member works full time on a single project. Rather, each SPADES practitioner splits their time between several efforts, thereby reducing staff availability and overlap. But we have witnessed that, despite split schedules, our team members were able to operate within the time bounds of each SPADES phase, provided our scrum master built in flexibility in the timing and duration of ceremonies and in the scope of each development sprint within each spiral.

Quality Results from Managed Focus.

Between the rigor of SAFe ceremonies and the flexibility required to conform to a small business’ staffing model, we realized that managing the focus of the team was the key driver to the quality of innovation and tangibility. Because highly skilled and educated workers consistently strive to ‘do more’ and dig deeper into the science and novelty of a domain or problem, it is necessary for the product owner and the scrum master to diligently keep the team on task. The initial backlog and use cases of the upfront analysis set the first general boundaries. The scope and focus collected adopted at the start of each spiral refines those boundaries to design and development that is feasible during the spiral. Managing each team member’s focus to stay within those refined boundaries is essential to yield quality products.

5 Conclusion and Next Steps

We have presented here a novel method, SPADES, that structures research and development activities in a manner to balance the needs for innovation and tangibility, within the time and resource constraints of small teams working with a limited budget. We exemplified our application of SPADES to Phase I SBIR and STTR contracts and described how our CSI team was able to achieve balance between innovation and tangibility. We finally identified a series of lesson learned, tackling ceremonies, flexibility, and focus management.

We recognize both that (a) this paper is a limited account of our own experience, (b) we have only applied SPADES to one type of contract (although we are now embarking on Phase II efforts which will also leverage SPADES), and (c) that there may exist additional methods or approaches employed by other practitioners in the human factors and cognitive systems engineering communities to reconcile the combined needs for innovation and tangibility.

Therefore, we see our contribution as an invitation to our fellow researchers, scientists, technologists, and product developers, particularly in small businesses and in academia, to share with us their thoughts and feedback on SPADES.