1 Introduction

Recently, many emerging information technologies (e.g., Internet of things (IoT), cyber-physical system (CPS), artificial intelligence (AI) and big data) have been introduced into the manufacturing environment (Lee et al. 2014). Against this backdrop, a strategic initiative called “smart manufacturing,” “connected industry” or “Industry 4.0” has been proposed which refers to the fourth stage of industrialization (Drath and Horch 2014; Davis et al. 2012). Roland Berger, a renowned German strategic consulting firm, believed that the essence of Industry 4.0 is based on information technology (Dujin et al. 2014). Based on the report, we categorize the main technical concepts in Industry 4.0 as “interconnection,” “replacement” and “creation,” all aiming at the improvement of production efficiency and quality. Specifically, IoT and CPS technologies “interconnect” manufacturing facilities and construct a data-rich environment to support “replacement” and “creation”; automation and immersive technologies are employed to “replace” human workers and even physical devices; AI and other advanced analytics extract meanings from industrial big data to facilitate value “creation” throughout the entire value chain of manufacturing industry.

Visualization has become an important tool in many areas for explaining and understanding large and complex data (Sun et al. 2013; Liu et al. 2014; Wu et al. 2016; Li et al. 2018). It can effectively combine machine intelligence with human intelligence to gain insight from the data to support informed decision-making under complicated scenarios. With the advance of Industry 4.0, various simulation and process data continue to grow exponentially. It is much more urgent than ever before to extract useful information from these data to promote innovative activities including “replacement” and “creation.” In recent years, visualization has been frequently introduced into a wide range of industrial data analysis scenarios. From strategic industries [e.g., nuclear fuel processing (Maljovec et al. 2016; Höllt et al. 2013) and aerospace manufacturing (Takeshima et al. 2013; Dutta et al. 2017)] to people’s livelihood industries [e.g., automotive manufacturing (Splechtna et al. 2015; Angelelli and Hauser 2011) and food processing (Boukhelifa et al. 2013; Chen et al. 2017)], a growing number of industry sectors have noticed the application value of visualization technology. From production planning (Wu 2001; Jo et al. 2014) and simulation (Dutta et al. 2018; Zhou 2011) to production monitoring (Xu et al. 2017; Wu et al. 2018) and testing (Reh et al. 2013; Pajer et al. 2017), visualization has been applied throughout almost the entire life cycle of the manufacturing process. However, to the best of our knowledge, no survey has been proposed to summarize the visualization technologies specifically tailored for smart manufacturing applications.

In this paper, we analyze the technical publications to establish a development basis of visualization for smart manufacturing. We first manually scanned through all the issues and proceedings of the major visualization venues from 2008 to 2018, including journals (e.g., IEEE Transactions on Visualization and Computer Graphics, Computer Graphics Forum and IEEE Computer Graphics and Applications) and conferences (e.g., IEEEVis, EuroVis and PacificVis). To ensure that this survey covers the state-of-the-art technologies, we further selectively searched through Google Scholar with keyword combinations. And this resulted in a total of 53 included papers that apply visualization to the manufacturing industry.

Based on a thorough literature review, a method of classification is proposed. We first classify the literature horizontally and vertically according to the application scenarios and industry sectors, respectively. We further classify the application scenarios based on the two concepts of “replacement” and “creation.” In terms of “replacement,” we focus on the use of simulation and visualization technologies to present non-physical internal and external environments of the production equipments. Regarding “creation,” we discuss the application values of visualization and visual analysis in the four phases (i.e., design, production, testing and service) of the entire life cycle of industry manufacturing. Considering that there are a wide range of industries, we categorize related work into 10 familiar industries and select three representative ones (i.e., automobile, energy and food processing) to discuss their application features. We provide an overview of this survey in Table 1. Finally, we conclude this survey and provide possible directions for future research.

Table 1 A taxonomy of visualization research for smart manufacturing applications
Full size table

2 Visualization for “replacement”

The concept of “replacement” in smart manufacturing possesses two meanings: liberating people from multifarious dull work by replacing human labors through intelligent devices (e.g., robots replacing human beings in repetitive operations and automation technology replacing monitoring personnel in real-time online fault diagnosis on production equipment) and presenting complex and dangerous work scenarios in a computer-generated virtual world where people can learn skills in a safe and informative way. The latter part that we focus on is related to the topic of this survey. In this part, immersive technology and scientific visualization are widely used to represent the digital structures of various devices and simulate the internal and external environment of production equipment, which both enriches human–machine communication modes and promotes the combination of expert knowledge and machine automation for better smart manufacturing.

Immersive technology refers to all forms of software and hardware that blur the boundaries between the physical world and the simulated or digital world (Govindarajan et al. 2018). Based on different implementation principles, immersive technology can be classified into virtual reality (VR), augmented reality (AR) and mixed reality (MR). VR creates non-physical production environment that allows users to “go inside” and thus familiarizes themselves with complex production operations and manufacturing processes. For example, Zhong et al. (2008) created a virtualized electronics assembly factory, as shown in Fig. 1a, to familiarize workers with the whole assembly process; Zhou et al. (2016) conducted an animated simulation shown as Fig. 1b to explain how hot gases escape from the door of reheating furnace in the production of flat-rolled steel products, and deepen experts’ understanding of the internal state of reheating furnace. In terms of AR, it provides users with additional contextual information in practical manufacturing scenarios by adding or deleting computer-generated virtual objects. As shown in Fig. 1c with the help of AR glasses, assemblers can quickly master the assembly order and method of different accessories by reading online instruction, and then efficiently assemble accessories in an appropriate sequence (Paelke 2014). Figure 1d shows a system planner who is modeling a missing object of a production line onsite using AR solutions (Dangelmaier et al. 2005). MR Billinghurst and Kato (1999) is a hybrid of VR and AR that combines digital reality with virtual digital images to present a semi-real semi-virtualized production environment and therefore bridges the real world with the virtual world to realize information sharing. For instance, Fig. 1e presents a MR environment where virtual equipment is registered to an image-based real workshop to assist users in planning the layout of the workshop (Lee et al. 2011), and Fig. 1f shows a hybrid interface that integrates the structural information of production equipment and its monitored data to help users with maintenance work (Espíndola et al. 2013).

Fig. 1
figure 1

Application cases of VR, AR and MR in industry manufacturing. a VR assembly factory; b VR furnace hot gases escaping; c an assembly worker wearing AR glasses; d AR-supported production line modeling; e MR workshop environment; f MR equipment interface

Full size image

Scientific visualization concerns graphical data representation and interactive data analysis of scientific and natural phenomena (Kehrer and Hauser 2013). A variety of computational models have been utilized in the simulation of the natural phenomena relating to production and manufacturing, such as furnace flame in steelmaking and airflow in aircraft manufacturing. Many scientific visualization researches have been well established to explore and analyze such data and to communicate results from data analysis, which mainly falls into two orientations depending on where the simulated phenomenon occurs, namely the internal and external environments of equipment.

During the process of production and manufacturing, the internal environment of some facilities and devices represents complex natural systems. Modeling and analyzing these complex natural systems are challenging problems. Scientific visualization can partially or totally replace physical devices for simulating and testing the internal environment to simplify the validation of simulation model and design prototype. For instance, in steel manufacturing, the flame inside a furnace is a complex gas combustion system. Zhou (2011) used a computational fluid dynamics model to simulate the velocity, temperature and species distributions of the system, and then united flow visualization, an important scientific visualization technique (Merzkirch 2003), to produce its graphical and dynamic representation. Engineers, shown as Fig. 2a, thus can go inside a non-physical furnace environment to deepen their understanding of such system for testing and optimizing furnace performance. Similarly, in aircraft manufacturing, advanced flow visualization is often used to illustrate the airflow inside a jet engine. Through observing and analyzing the visualization results, engineers can intuitively verify the performance of jet engine design prototype and improve its structure (Dutta et al. 2017, 2018), as shown in Fig. 2b.

Fig. 2
figure 2

Application cases of scientific visualization in industry manufacturing. a Steelmaking furnace internal environment visualization; b jet engine internal environment visualization; c oil exploration external environment visualization

Full size image

The external environment includes the production environment and natural environment outside the equipment. In most scenarios, the production environment mainly refers to the collaborative relationship among various production resources. In the scenarios required field operations, such as oil and gas exploration, the natural environment exerts a crucial influence on production safety. And through simulation and visualization, a virtual production background can be reconstructed to provide an intuitive revelation of such relationship and influence for achieving rational resource allocation and low-risk production scheme. For example, Wu et al. (2001) designed abstract graphical elements to represent the melting furnace and heating oven in metal ingots casting, thus allowing people to intuitively understand the synchronous relationship of scheduled capacity and the load between these two devices. Höllt et al. (2013) proposed a scientific visualization system, as shown in Fig. 2c, for oil exploration in the Mexico Gulf, which visualizes the ocean by combining sea surface rendering and topographic map, thus forecasting uncertainties quantified by a circulation ocean simulation model and ensuring low-risk oil mining schemes.

These two technologies are uniquely important in demonstrating the essence of “replacement.” With the development of industrial production, the production environment and equipment have become increasingly complex, and the degree of connection and integration of production resources have been manifestly enhanced. Therefore, one single technology is no longer enough to meet the representation requirements of complex and highly integrated work scenes. The advantageous combination of these two technologies and other technologies, such as computer-aided design technique (CAD), is expected to be a major development direction for the next stage of “replacement.” Many researchers have already taken the first step toward this direction. For example, scientific visualization and AR are combined to enhance users’ perception of hybrid wind tunnel (Takeshima et al. 2013) (Fig. 3a); CAD and scientific visualization are integrated at the same interface to illustrate the production structure of a jet engine and at the same time simulate its inside dynamic environment in real time (Xue et al. 2016) (Fig. 3b); DualCAD (Millette and McGuffin 2016) uses AR, VR and CAD technologies to provide designers with enhanced stereoscopic 3D Content to strengthen their all-round perception of product models (Fig. 3c).

Fig. 3
figure 3

Cases of the combination of CAD, immersive and scientific visualization technologies for industry manufacturing. a AR and scientific visualization; b CAD and scientific visualization; c AR, VR and CAD

Full size image

3 Visualization for “creation”

The concept of “creation” refers to creating new values for both makers and customers of products by conducting various innovative and creative activities. In the era of Industry 4.0, production machinery and equipment generate vast quantities of data on an ongoing basis. These data are generally characterized by multi-modality, strong correlation and high throughput. Visualization, regarded as one of the powerful tools for complex data analysis (Zhao et al. 2019; Chen et al. 2017; Shi et al. 2018; Liu et al. 2017), can intuitively present both the summary and details of various manufacturing data, and provide users with interactive manners to facilitate interesting pattern identification and valuable information exploration, hence supporting human-in-the-loop decision-making in a wide range of innovation and creation scenarios. We use the general life cycle of manufacturing as the main line to classify these scenarios into four phases from design and production to testing and service. In this section, we will analyze the scenario characteristics of the four phases and introduce representative researches in each phase.

3.1 Design phase

The design phase is a creative thinking process during which designers create and refine the appearance, function and performance of a product based on market demand and creative inspiration. With the increasing complexity of modern design, the considerations and knowledge scopes involved in this phase are becoming broad and extensive. To achieve a comprehensive design with high standards, designers begin to pay close attention to data-driven design (Yin et al. 2014), which means to extract useful information from design-related data to guide the creative thinking process. Visualization is an important approach for data-driven design, as it enables designers to deepen the unfathomable professional knowledge contained in the data, explore the constraint relationship between various considerations, and even verify the function and performance of design prototype (Coffey et al. 2013).

Product design generally involves multiple steps such as the design of appearance, function, performance and production environment. All these steps are closely linked and mutually restricted, and at the same time, there are also many design-related factors that need to be considered in each step. Striking a balance between design steps and numerous factors can be quite challenging. Take two scenes as examples. The first scene is about how to realize the coordination between appearance design and performance design. Appeal products trend to attract customers’ eyes, but excessive pursuit of attractive appearance will result in unsatisfactory performance. Kratz et al. (2014) proposed a visualization approach for mechanical component design, as shown in Fig. 4a. Using the design of a plastic brake lever as a case study, they used a tensor visualization technique to present and compare the mechanical stressing performances of different brake lever appearance designs. Designers thus can achieve the design balance between appearance and performance with intuitive visual experience. The other scene is related to the rational layout of a workshop or a production line. To design an efficient and comfortable production environment, planners need to consider the complex relationship between various devices as well as the work load of staff working there. Wörner and Ertl (2011, 2013) introduced a multi-view visualization system for production environment design with multifaceted factors taken into consideration, including devices, workers and production lines. Combined with automatic evaluation algorithms, this system enables planners to interactively and iteratively modify and verify the proposed environment layout, so as to ensure that the final layout is flexible, versatile and adaptable. CasCADe (Ivson et al. 2017) is an innovative 4D visualization system that maps time information as the fourth spatial dimension to visualize the construction event sequence of a virtual construction plan, thereby allowing planners to identify schedule uncertainties and work–space conflicts in the virtual plan to create a truly executable plan, as shown in Fig. 4d.

Fig. 4
figure 4

Visualizations for design phase: a structural design of product; c material characteristics analysis; d production environment design

Full size image

Modern industrial design requires designers to master multi-disciplinary professional knowledge, such as esthetics in appearance design, ergonomics in functional design and material science in material selection. However, professional knowledge is often difficult to embark and understand. Taking material science as an example, material selection is an important part of production design. To select appropriate materials, designers need to grasp the characteristics of all candidate materials. Beketayev et al. (2011) combined several visualization methods, including topological visualization, multi-dimensional scaling and graph layout, to display molecular structures of porous materials, as shown in Fig. 4b, which helps designers to understand the structural characteristics of materials from a microlevel and then select the applicable one in product design. Weissenbock et al. (2014) used blob visualization and fiber metadata visualization to depict the characteristics of fiber material from a macro level, such as fiber length and orientation, which facilitates the design of fiber-reinforced polymers, as shown in Fig. 4c.

Moreover, CAD techniques also play a very important role in the design phase. Different from visualization, CAD technique is a typical application of computer graphics that transforms human thinking into graphics (Sarcar et al. 2008), aiding in the creation, modification and optimization of manufacturing component design. CAD technique has been widely used in various manufacturing fields, such as machinery and electronics, to improve design efficiency and quality. Recently, 3D printing has changed CAD technique from effective design to rapid prototyping (Lipson and Kurman 2013). This new technology can convert the digital model generated by CAD software into solid samples and even be directly applied to product manufacturing, all of which significantly brings down costs and alleviates difficulties during the production.

3.2 Production phase

The production phase transforms products from ideology to physical forms, occupying the largest share of production costs in the manufacturing life cycle. The main content of this phase is to formulate and manage the production process. The pursued goal of this phase is to maximize production efficiency. However, the achievement of this goal is quite difficult. It is necessary to maintain innovative thinking all the time on management improvement and process optimization. Fortunately, highly digitalized and interconnected modern production lines provide rich process data. Visualization and visual analysis can provide production operators with intuitive presentations of real-time process data for production monitoring and onsite troubleshooting; it also provides production managers with deep insights into non-real-time historical process data for process innovation.

Real-time process data analysis generally refers to a method where production operators observe the process data captured from production lines, and then identify the current operating status of components and the whole production lines, so as to handle abnormal conditions in a timely manner. The real-time monitoring of production line might be the most basic requirement in manufacturing industry, and it is also regarded as one of the core applied scenarios for visualization and visual analysis technology in smart factories. Compared with traditional production monitoring, the visualization-supported method possesses two advantages. On the one hand, visual data presentation is conducive to achieving centralized control of spatially dispersed production devices. On the other hand, interactive analysis manner can integrate the specialized experience of operators to evaluate the running status of production lines for timely and pertinent troubleshooting. Xu et al. (2017) proposed an extended Marey’s graph to visualize the processing time and status of massive work stations in an automatic assembly line on a single screen, as shown in Fig. 5a. When some work stations appear to be delayed or have status failures, they will stand out in this graph momently. Wu et al. (2018) developed a suite of interactive visualization techniques assisting operators to pre-define trustworthy observation rules with domain experience. These rules are then integrated into an adaptive semi-supervised solution for real-time detection of production line anomalies. Zhou et al. (2017) noted that the routine monitoring requirements of some core production facilities are extremely complicated. They proposed a visual analysis system for routine monitoring of roller hearth kiln (RHK), a complex key lithium battery cathode material manufacturing equipment with a long continuous structure consisting of dozens of working zones. As shown in Fig. 5c, the system first uses a qualitative and quantitative situation assessment model to generate the comprehensive description of RHK, and then visualize the result of situation assessment and relevant short-term process data for insightful and efficient troubleshooting.

Fig. 5
figure 5

Visualizations for production phase. a Assembly line performance monitoring and troubleshooting; b CPS bottleneck exploration; c complex manufacturing equipment monitoring; d large-scale manufacturing schedule optimization

Full size image

Non-real-time process data analysis focuses on extracting patterns and discovering knowledge from historical process data to facilitate process optimization and management innovation. Visual analysis emphasizes the leading role of domain experts in innovation activities. It visually explains the results of advanced automatic data analysis and provides users with interactive functions to carry out human–computer collaborative intelligent analysis of process data. LiveGantt is a visual analysis approach used in analyzing large-scale manufacturing schedules consisting of numerous production tasks and resources (Jo et al. 2014). It utilizes task aggregation and resource reordering algorithms to deal with schedule data at first and then uses a new Gantt chart to visualize the results of these algorithms, as shown in Fig. 5d. Users can straightforwardly observe the inefficient part of the schedule and interactively reschedule it on the interface until a satisfactory result is obtained. Visplause (Arbesser et al. 2017) is a visual analysis system that combines statistical models, semantic inspection and a group of linked visualization views to achieve a man–machine joint exception checks on multi-dimensional temporal process data. Post et al. (2017) provided a series of visualizations and interactions shown as Fig. 5b to analyze the process data generated by a complex production system, thereby facilitating user-guided exploration of possible bottleneck process steps and excess devices for production system refinement.

3.3 Testing phase

Testing is a process that measures the functions and performances of products. Its core principle is to guarantee that the products are strictly manufactured following established standards, while also paying attention to lessons learned from the examined quality issues to guide design refinement and process optimization. Testing phase is the most data abundant phase in the whole manufacturing process due to the diversity of test requirements and examining methods. Many novel visualizations have been proposed in this phase to cope with various testing data and analysis requirements, including multi-dimensional visualization for multi-index evaluation test data, comparative visualization for ensemble testing data and 3D rendering for light detection data. These visualization techniques help experts understand test data, analyze product defects, improve inspection methods and provide guides for design and production optimization.

Multi-dimensional or even high-dimensional data are very common in testing phase since the quality evaluation system of a product generally contains multiple weighted indicators. But it is difficult for test engineers to understand data space over three dimensions (Xia et al. 2016, 2018), let alone a weighted multi-dimensional indicator system and its evaluation results. Pajer et al. (2017) developed a novel visual weight space exploration system, as shown in Fig. 6a, in which test engineers can adjust the weights of evaluation indexes interactively, and then get the visual presentations of both the weight spaces and evaluation results of the index system to explore which indexes are more sensitive to qualitative evaluation results. Radoš et al. (2016) proposed a novel linking and brushing interaction for multi-view coordinated visual analysis. With such interaction, test engineers can quantitatively analyze the internal relations between fuel consumption indicators and the automotive shaft design based on the testing data of automobile performance in two related visual views.

Fig. 6
figure 6

Visualizations for testing phase. a Multi-dimensional test data analysis; b ensemble testing data analysis; c image testing data analysis; d flow testing data analysis

Full size image

Product testing often requires multiple sets of tests with different input parameters to verify the range of product parameters and suitability. These tests will generate ensemble testing data, whose multivariate and multi-valued characteristics make the analysis of test results extremely tough. Matković et al. (2014) developed a visual analysis solution allowing test engineers to interactively steer ensembles generated in the performance test of automobile power system, as shown in Fig. 6b. In this solution, test engineers can select subsets from a multivariate parameter space, such as injection rates or injection pressures of car engines, to carry out a comparative analysis, which assists them in finding out the suitable input parameter combinations to increase the combustion efficiency of automobile power system.

Product testing sometimes produces unstructured data like texts and images. In terms of this issue, Amirkhanov et al. (2016) proposed a visualization-supported system for the analysis of CT image of glass fiber-reinforced polymers, as shown in Fig. 6c. They first proposed an image processing algorithm to recognize typical structural defects (e.g., fiber breakage and fiber pull-out) based on geometrical features of glass fiber. Then, they highlighted distributions of different defects with a color scale visualization and a 3D surface visualization, assisting quality control engineers to locate the physical position of defects on the production. Sedlmair et al. (2011) developed a visual analytics system for exploring millions of text messages captured by in-car communication network testing. The system first combines a state machine algorithm with an anomaly detection algorithm to find out the possible in-car communication network anomalies. And then, various visualization methods are used in the system to illustrate the in-car communication structure and highlight the communication nodes associated with those abnormalities, thus helping test engineers to achieve anomaly root cause reasoning.

In some product testing scenarios, special examination methods are required, such as light detection and fluid detection. These methods will produce scientific data related to natural phenomena happened on the examined products. Scientific visualization is a major way to help engineers understand these testing results. Huettenberger et al. (2015) used 3D triangulated surface rendering to visualize the optical measurement data generated by the structure testing of a car trunk lid, thus enabling test engineers to find its structure defects. Angelelli et al. (2011) proposed a new streamline flow visualization for the analysis of automotive exhaust system test data, as shown in Fig. 6d. With a novel idea that visually straightens the crooked tubular by following the main gas flow direction within the exhaust system, test engineers can intuitively comprehend its interior gas flow.

3.4 Service phase

In service phase, manufacturing enterprises will continue to track the product usage and users’ experience after the product leaving the factory. Data generated in this phase are characterized by long time span and wide-ranging involvement of both products and customers. Such service data often contain valuable information on defects of product quality or even product design. All these defects normally cannot be discovered in previous three phases. Therefore, effective data analyzing and information mining are important for enterprises to better develop after-sale service strategies and refine product design. Visualization and visual analysis have been repeatedly used to present intuitive data overviews and detailed visual clues to assist experts to mine valuable information based on field experience. Here, we take two typical data as examples, namely maintenance records and customer relationship data, to illustrate the positive role of visualization in promoting service upgrade and product iteration.

High-tech consumer products (e.g., automobile and cell phone) generally experience various large and small faults during usage. Empirically, there might be some internal relations between these faults, some regularities in their sequence or even some newly unexpected faults. Such information is very difficult to comprehensively forecast during design and testing phases but can be excavated from the long-term maintenance and repairing records accumulated in service phase. Alsallakh et al. (2017) skillfully used a treemap-based multi-sets visualization method shown as Fig. 7a to explore intersection relations between faults from massive car maintenance records, which helps maintenance engineers to summarize specific association rules with high confidence, such as the combination of what faults can cause “GPS inoperative.” Sequence synopsis (Chen et al. 2018) is a visual analysis system that facilitates the exploration of potential temporal development paths of vehicle faults from long-term maintenance data, which is conducive to realizing predictive maintenance of automobile failures, as shown in Fig. 7b. Guo et al. (2018), by taking both the datasets of automobile maintenance and repairing service into consideration, designed a novel visual analysis approach to explore possible event thread that reflects the impact laws of car usage habits on the service life of automobiles, thereby pushing guidelines on safe usage to customers, shown as Fig. 7c.

Fig. 7
figure 7

Visualizations for service phase. a Analysis of intersection relations between production faults; b analysis of potential development paths of vehicle faults; c possible thread analysis from automobile maintenance and repairing events

Full size image

The customer relationship data mainly refer to customers’ subjective evaluation of products in forms of revisit, etc. These data normally contain product quality issues and uncomfortable using experience commonly concerned by customers. UTOPIAN (Choo et al. 2013) is a visual analytics system for product review exploration. It uses a topic creation algorithm and a topic splitting algorithm to extract customer most concerned topics and then adopts a node–link graph to visualize topic clusters. Customer service personnel thus can quickly identify the consensus reached by customers on product quality and using experience. In addition, some manufacturers attach great significance to the identification of customer groups in service phase, which supports them to provide customized after-sale services and find out potential customers for future products. Perer et al. (2011) presented a people-centric visual analysis system to identify customer groups. This system first collects and aggregates customers’ public information on social media, and then builds a social graph to visualize the relationships and exchanged information between customers for customer group mining.

4 Industry sector analysis

Industrial manufactory covers a wide range of industry sectors. Each sector has its own unique production environment and production process, leading to diversified data and analysis requirements. And the focus of visualization in various industries is also quite different. For example, chemical fiber industry emphasizes on material characteristics analysis, whereas metallurgy industry pays more attention to process control. We therefore classify related work into familiar industry sectors, as shown in Table 1. In this section, we will further discuss three specific industries: automobile industry, energy industry and food production.

Automobile industry is a typical representative of modern industry. Since the first industrial revolution, it has always acted as a leading role in promoting the reform and innovation of manufacturing modes. For example, Ford Motor Company initiated a pipeline production model in the nineteenth century and Toyota Company firstly proposed the Lean Production mode in the twentieth century. In the current Industry 4.0, the automobile industry is obviously the pioneer of smart manufacturing. Table 1 depicts a successful application of visualization and visual analysis in the whole manufacturing life cycle. The key to such success lies in this industry’s leading informatization process and outstanding data-supporting environment. We believe that visualization and visual analysis technology will continue to play a major role in the transformation and upgrading of global automotive industry from traditional fuel consumption to new energy types.

The petroleum and nuclear energy industry is the core of national energy economy, which profoundly impacts the political and economic development of a country. Research and production costs are extremely high in the extraction, processing and utilization of energy. And safety factors also need to be considered seriously. Therefore, cost efficiency and production safety become the major concerns of this industry. Some researchers have used visualization and visual analysis to address the above two concerns. For instance, Höllt et al. (2011, 2013) visualized the weather and geography information data relevant to oil exploitation, so as to formulate an efficient and safe oil mining plan. Maljovec et al. (2016) visually simulated nuclear reactor processing status for the optimization of production safety settings. Sahaf et al. (2017) performed an interactive spatial–temporal analysis on historical production accident data for future prevention.

Food production is a typical light industry. As people regard food as their prime want, this industry exerts a great impact on people’s life. Food desert, safety and quality attract wide attention from the public. Recently, some visualization and visual analysis approaches have been proposed to help address these issues. For example, Chen et al. (2013) proposed a visual analysis system to analyze the spatial and temporal distribution of food desert issue, thus promoting more equal access to food and providing convenience for residents. Chen et al. (2017) developed another visual analysis system for pesticide residue data analysis. The system enables people to discover the time-dependent pattern of pesticides attached to crops and take more effective measures to reduce pesticide residues in crops. Sarkar et al. (2004) used a method of flow visualization to analyze the air flow fountain from air impingement device to improve its nozzle structure, thereby reducing raw material loss during the usage of this device in food processing operations such as drying and freezing.

In addition to meeting the special needs of each industry, researchers have also begun to explore general visualization applicable to multiple industry sectors. For example, Peng et al. (2012) focused on reducing maintenance costs of complex products. They introduced a visualization-supported solution to simulate product maintenance systems that are related to numerous similar products, including products in aerospace and general robots. This solution realizes product maintainability verification during the model design stage, so as to guide designers to iteratively adjust design schemes based on these results. Amirkhanov et al. (2011) proposed an image-based metal segmentation algorithm to analyze CT images. They also visualized the metal parts detected in CT image to obtain high efficiency metal inlay examination that has been widely used in the product testing of electronic and automotive industry.

5 Conclusion and opportunities

Visualization is of great significance in the era of the fourth industrial revolution. In this paper, we have presented a detailed literature review on industrial data visualization based on a taxonomy combining the corresponding application scenarios and industry sectors. With the review, we preliminarily conclude that visualization and visual analysis can effectively satisfy multiple demands of the new production and management models in Industry 4.0, such as digital and intelligent production, networked collaboration and service extension. Also, visualization and visual analysis can drive various innovation and creative activities throughout the entire manufacturing life cycle from design refinement, process optimization to business model innovation.

Although research on industrial data visualization has yielded remarkable results, there are still some major challenges that require further investigation.


Data integration: With the rapid development of CPS and IoT technologies, the future sources of industrial data will become more diverse, and at the same time, the heterogeneity and coupling of the data will continue to increase. To facilitate industrial data analysis, we should emphasize more on data quality and integrity. Visualization has performed well in other application domains for data cleaning and data fusion. In the future, visualization is supposed to be applied to heterogeneous data integration.


Data security: As the manufacturing process becomes increasingly intelligent and networked, the importance of data transmission, sharing and analyzing is growing. Meanwhile, industrial environment is facing more and more security threats, including issues in data transmission security and data sharing privacy. In the traditional data security field, many mature visualization applications have already existed (Shiravi et al. 2012; Zhao et al. 2014, 2018; Shi et al. 2016), but in the industrial field, visualization practices remain new. Therefore, we believe that the future of visualization technology in industrial data security is quite promising.


Online analysis of large-scale data: Real-time online analysis is a common requirement in the manufacturing industry, such as low-latency event detection, equipment health assessment and early fault diagnosis. This requires the ability of processing large-scale data within a short time period. Therefore, in future research, advanced large-scale industrial data processing methods, such as parallel computing, edge computing and cloud computing, are expected to be integrated into visualization and visual analysis solutions.


Domain knowledge integration: Industrial data analysis requires extensive professional and domain-specific knowledge. It is essential for interactive visualization systems to utilize such professional knowledge and domain experience for human-in-the-loop analysis. We need to further consider how to integrate machine learning, natural language processing and other methods to facilitate the collection, extraction and identification of knowledge, so as to enable human–computer collaborative intelligent data analysis.


Perception and cognition theory: Manufacturing process generally involves different types of users who have inconsistent perception and background knowledge on visual data presentation and interactive data analysis. User-centric research on the mental maps of different user groups is urgently required. Visual and interactive perception and task-driven design will remain a challenging research topic for academics and industrial engineers.


Versatility and extensibility: Currently, the application of visualization in industrial manufacturing is still in the development stage. Universally applicable technical concepts and methods have not been formed yet. Most existing works are case by case studies, which may easily lead to repetitive research and development. Therefore, versatility and extensibility will surely become one of the main directions for future research. This requires researchers and engineers to identify generic abstractions for industrial data and develop extensible architectures for visualization applications.