Keywords

1 Introduction

Quality of products and processes concerns more and more the manufacturing firms because negative consequences do not show up until the product is actually produced or worse, until the customer returns it [1]. The acceleration of technological growth in the context of Industry 4.0 does not automatically result in improvements in manufacturing quality. The products’ complexity (i.e. number of components and their relations) and variety is continuously increasing, while the traditional quality management practices, such as Material Requirements Planning, Lean Manufacturing, Theory of Constraints and Six Sigma, have lost their effectiveness [2]. The future of quality management is to be proactive: adverse effects of quality flaws have to be prevented before they become evident in the actual use of a product [1, 3] or even before they cause inefficiencies in the production process. This trend is significantly facilitated by the latest advancements in machine learning [4] which dictate that production is gauged based upon the product quality, the process quality, and the quality of the services provided surrounding the product. To this end, predictive quality is an approach that moves beyond traditional quality evaluation methods towards extracting useful insights from various data sources by determining patterns, revealing correlations between products and defects and predicting future outcomes (e.g. product defects and fault localization) [1,2,3, 5].

Predictive quality approaches can be further enhanced by extending data analytics with augmented analytics. Augmented analytics aims at optimizing the use of data for decision making in order to augment human intelligence and contextual awareness [6]. It brings automation to the complete analytics cycle [7] by leveraging AI algorithms in order to transform how analytics content is developed, consumed and shared. Augmented analytics uses natural language processing and conversational interfaces, allowing all users to interact with AI through spoken and written language, without requiring advanced skills [6], something which has long been a goal of AI researchers [8]. On the other hand, prescriptive analytics moves beyond the provision of meaningful insights about the current or the anticipated states (e.g. prediction of future defects) and aims at prescribing the best decision options within time constraints in order to take advantage of the predicted future, e.g. to eliminate or mitigate the impact of a future undesired event [9, 10]. With augmented analytics, conversational interfaces can provide users fast access to non-intrusive recommendations and real-life feedback from the shop floor. Moreover, intelligent digital assistants allow users to interact with data and insights easily.

In this paper, we focus on Augmented Manufacturing Analytics (AMA). We present a voice-enabled Digital Intelligent Assistant (DIA), which interacts with a prescriptive quality analytics service. The assistant’s interface aims to allow workers to access and customize quality predictions and the prescribed mitigating measures. Our approach aims to reduce the number of quality issues through fast, mobile, and easy access to quality predictions for products and processes. The application case is the production of white goods. The next section briefly overviews the current state of the art in business analytics and digital intelligent assistants. Section 3 presents our conceptual framework for augmented quality analytics and highlights some of our innovation objectives. The fourth section outlines the case study in quality management in the home appliances industry, while the fifth section presents the concluding remarks.

2 State of the Art

2.1 Augmented Analytics

Companies have at their disposal staggering amounts of data, which, if analyzed and processed properly, can generate valuable insights and lead to better decisions. The use of advanced techniques that can analyze and process large and diverse data sets that include structured, semi-structured and unstructured data, from different sources, and in different sizes from terabytes to zettabytes, has led to the field of business analytics [11]. The data analytics lifecycle consists of three phases: description, prediction, and prescription. While descriptive analytics analyzes past events, predictive analytics predicts future events – both do not provide direct support for decision-making [12, 13]. Prescriptive analytics, on the other hand, is a newer data analytics type enabling data-driven optimization for decision support and planning [9]. Prescriptive analytics has the potential to provide the greatest benefit for business by providing insights about proactive mitigating actions for the predicted undesired events [14]. Current solutions, however, require that users have higher skills in data science and machine learning, which impairs wide adoption. To address this barrier, combining prescription with augmented analytics can enable users with lower data science and machine learning skills to find and surface the most important insights or changes in the business by interacting through spoken and written language taking advantage of natural language processing and conversational interfaces [15].

2.2 Digital Intelligent Assistants

Digital Intelligent Assistants (DIAs) emerged from fragile niche applications to everyday helpers. Consumers use them for home automation, to schedule appointments, and to search for facts. There are various voice-based assistants such as Amazon Alexa, and text-based assistants (chatbots) [16, 17]. Market researchers expect that AI-based digital assistants will become a key element in the future of work [18, 19]. A voice-enabled DIA processing pipeline has four core components [20, 21]: Speech-to-Text (STT) to transcribe voice inputs, Natural Language Understanding (NLU) to extract intents and entities, Dialog Management (DM) to track dialog states and decide the next actions, and Text-to-Speech (TTS) to generate a computer voice output. Morana et al. classified DIA’s along two dimensions: (1) the degree of interactivity enabled by the user assistance, and (2) the degree of intelligence of user assistance [22]. The former characterizes the DIA’s capability to support humans in an ongoing dialog and the latter describes its capability to assist the user considering, for instance personal and task characteristics.

Applications in manufacturing include, for instance, smart voice-enabled digital assistants for human-robot communication [23, 24]. Although further research is still needed for a more accurate voice recognition and conversational intelligence, the technology is mature enough for running experiments [25, 26] and building commercial applications in manufacturing, such as SpixFootnote 1. Realizing DIA in this domain is challenging though. Achieving robustness under industrial operating conditions, clarity about an assistant’s accountability, acceptance among employees, ethics issues, and data security risks are some examples that slow down wider adoption [27]. Tools and frameworks for building DIAs, however, prosper. Many global IT leaders, such as Google, Amazon, Microsoft, Baidu, or IBM offer tools to design, develop, deploy, and maintain assistants. A noteworthy effort in the Open Source domain is MycroftFootnote 2, which offers a privacy-centred digital assistant framework. Their solution offers exchangeable natural language processing modules, speech-to-text, intent parsing, and text-to-speech.

3 Augmented Manufacturing Analytics Framework for Human-AI Collaboration in Quality Control

Our framework (Fig. 1) implements quality analytics for decision augmentation through optimized human-AI interaction. The framework covers the whole data analytics lifecycle (descriptive, predictive, and prescriptive analytics) aiming at extracting increased value from quality data and prescribing appropriate mitigating actions through a voice-enabled DIA. As such, it can support tasks related to product and process quality control, such as detecting abnormal behaviors and root causes of defects, predicting their consequences in terms of product and process quality and prescribing appropriate actions in the form of voice-first advices. Specifically, the framework foresees stream processing to perform real-time data processing for: (i) detecting potential sources of defects and revealing correlations between products and defect rates; (ii) predicting future quality issues and their consequences; and (iii) prescribing mitigating actions to optimize relevant manufacturing performance indicators, such as Overall Equipment Effectiveness (OEE), uptime and scrap rate.

Fig. 1.
figure 1

Augmented manufacturing analytics framework

Full size image

We use the DIA to recognize worker intents, questions, and instructions in the industrial environment. The DIA has to manage typical interaction constraints in manufacturing, such as factory noise, multiple languages, jargon, and workers wearing earplugs, masks, safety googles, and safety gloves. Its STT component has to be reliable under these conditions, which is a technical challenge. Reliable solutions ground on thousands of hours of audio data and their transcription – each supported language needs its own dataset. Therefore, STT model training is costly and companies, such as Google, Amazon, Apple, and Baidu, invested millions of Euros in this field to develop proprietary solutions. Their closed solutions are a cause for ethical concerns because the training data are not transparent and biases hard to identify. Open solutions are not as reliable yet, but projects such as DeepSpeechFootnote 3 and CommonVoiceFootnote 4 develop transparent alternatives. The application of trustworthy STT mechanisms is challenging and it is not clear how successful open state of the art technology can be.

Furthermore, the DIA has to address: i) data security requirements of manufacturing environments through data access management, traceability, and encryption; ii) data privacy requirements derived from the legal frameworks through a privacy-by-design approach with clear data ownership, data processing scope, and user consent management. The assistant needs to operate on mobile devices to be usable wherever and whenever the user needs it. Since time pressure is a typical work condition, the DIA’s dialogs must be fast, unambiguous, and easy to use. Voice interactions are inherently fast, potentially hands-free, and we assume they are also easier to learn than interactions via graphical interfaces. For multi-dimensional analytics results (e.g. graphs or tables) voice can be inefficient because of the slower audio processing on the human side. For these cases, the assistant needs access to graphical interfaces. Relevant interaction options include voice-based drill-down, drill-up, and customization options for the analytics processes. Wake words should activate the assistant for hands-free interactions.

4 Case Study on Quality Control

The case study of our approach concerns the quality control procedures of Whirlpool, one of the leading companies in the home appliances industry, with around 92,000 employees and over 70 manufacturing and technology research centers worldwide. The use case will address the end-of-line quality control with human-AI collaboration with the aim to adopt a predictive quality strategy that will link the quality control of the finished product with the design stage and the shop floor.

4.1 As-is Situation

In the Whirlpool production model, the whole white goods production is tested from quality and safety point of view in order to ensure a high standard level of product quality to final customers. These tests are executed in all factories either through the usage of automatic dedicated machines at the end of production line or through automatic, semiautomatic or manual checks in some critical workstation along the production flow (visual quality checks, quality gates). To these testing actions, Whirlpool Production system adds also some statistical quality check actions that are applied both on internal production parts on quality critical processes (statistical process control stations) and on finished goods, after the packaging process. In particular, this last testing, called Zero Hour Testing (ZHT) is referring to the Statistical Quality Control applied in a dedicated laboratory out of production flow on some finished products retrieved from the quantities ready to be delivered to the markets; see Fig. 2.

Fig. 2.
figure 2

Zero Hour Testing (ZHT) laboratory at Whirlpool

Full size image

The main objectives of ZHT are to measure the quality level of the outgoing product from an aesthetic, functional, and normative point and to measure the effectiveness of process control. These tests are executed in dedicated laboratory environment, created in each production site, and following a specific STD operating procedure. This testing method is designed to replicate the customer approach to the product, simulating the normal product usage conditions at final customer first usage. Currently, the procedure is executed manually by a laboratory operator and it is fixed, statically defined during process design phase both for what concerns checklist and reference parameters, and for statistical product withdrawal rate.

4.2 To-be Scenario

With our proposed framework we aim at adopting a predictive quality strategy that will link the quality control of the finished product with the design stage and the shop floor. By integrating all available information sources (e.g. sensor data, historical operation-al data, and expert knowledge), the factory will be able to a) predict low-quality products and b) to plan effective control actions to mitigate the impact. In this way, Whirlpool will be able to effectively capture defects risk and proactively initiate resolution process before it may impact on the final customer level. The support to decision making process in mitigation actions identification and implementation will ensure a prompt reaction of the overall Whirlpool quality network system, ensuring fast and effective reconfiguration with the modification of control points to capture un/conformities. A second important contribution is in the facilitation of root cause analysis that, due to the high complexity of white goods production process (high number of components, several quality-critical processes within production flow, high production pace, high variety of product range), is very often difficult to be executed at shop floor level. An interactive system collaborating with workers and leading into a deep dive analysis, will facilitate the identification of early signal of quality derailing effects and the possibility to consolidate knowledge to be used for the future. An augmented human-AI interaction is of outmost importance to enhance the shop floor workers’ capabilities in identifying potential failures, investigating root-causes, and addressing the causes effectively, allowing them the opportunity to fully leverage on whole data availability to anticipate the events instead of reacting the events.

5 Conclusions and Further Work

Augmented analytics is an emerging topic which deals with the enhancement of analytics with conversational interfaces as well as the exploitation of the human knowledge representation through intelligent digital assistants allowing users to easily interact with data and insights. The communication with the user by voice poses new challenges to the development and execution of data analytics services. Apart from visualization dashboards, the data analytics outcomes should be structured in a way that can be translated to speech. On the other way around, they should be able to take as input parameters that are derived from the human speech. In this paper, we outlined a framework for implementing quality analytics for decision augmentation through optimized human-AI interaction. The coupling of analytics with augmentation is in line with the overall trend towards human-AI osmosis for Operator 4.0 as a grand challenge [28].