1 Introduction

With the rapid development of social economy, people’s material living standard has been significantly improved. People no longer only pay attention to the affluence of material life, but also begin to focus on the construction and pursuit of their own spiritual world. As the epitome of China's history and culture, Museum has become an important place for science popularization and plays an important role in meeting the spiritual needs of citizens. A museum is a place for collecting, collecting, displaying and studying the objects representing the natural and human cultural heritage, and classifying the objects with scientific, historical or artistic value. The United States, Japan, the United Kingdom and other developed countries attach great importance to the popularization of scientific knowledge to the public through museums, and constantly innovate their exhibits to meet the needs of visitors. Compared with developed countries, China’s museum science popularization started late, and the exhibition form of exhibits is relatively single (Linlin 2016; Afroz et al. 2020; Wenxia et al. 2014; Ke 2013). According to statistics, more than half of the visitors to most museums in China are young people. Young people become the main force of museum visitors. Therefore, we should pay attention to the museum's science popularization work and its forms. Comprehensively integrate and optimize its popular science resources, and popularize scientific knowledge to the public by means of education and entertainment (Junzhong 2018; Feng 2011; Lenjani et al. 2020; Huojiao 2016).

The development of medical image processing technology has revolutionized the diagnosis and treatment. At present, with the continuous innovation and improvement of medical imaging technology, the classification of medical images is becoming more and more complex. The common imaging techniques include positron emission computed tomography (PET), computed tomography (CT), ultrasound (US), digital subtraction angiography (DSA), X-ray, magnetic resonance imaging (MRI), single photon emission tomography (SPECT), etc. Scholars and doctors usually need to further process the above-mentioned medical images in order to make full use of the information provided by these images, and medical staff can provide accurate diagnosis information for patients, so as to improve the accuracy of diagnosis (Junzhong 2018; Feng 2011; Lenjani et al. 2020; Huojiao 2016). As a hot spot in the field of medical image processing, medical image registration has made great progress after decades of research work. With the continuous improvement of hardware conditions in recent years, medical image registration methods are also constantly improved. Compared with the traditional manual interactive method, modern medical registration has gradually changed into automatic registration method, which not only improves the registration accuracy, but also improves the registration speed.

The core of digital museum is to use digital technology to build a variety of information resources mainly collection information, which has the characteristics of openness and sharing. Digital museum integrates the collection information of each museum, supports the audience to browse independently through the network, and can provide convenient services and resources for social education. Digital museum contains the infinite vitality of network communication resources. It is an information system expressed by static and dynamic information (Xiaoli 2008; Zhang Fa Xin 2015; Al-Sumaiti et al. 2020; Xiangming 2006).

In recent years, with the development of intelligent terminal hardware and computer technology, augmented reality technology has gradually entered the public vision. This technology can integrate the virtual environment generated by computer with the real scene around the user, enhance and deepen the user's perception of the surrounding environment and the three-dimensional understanding of the object to be visited. Hybrid reality technology makes customers more deeply involved in it, and provides users with a more humanized human–computer interaction mode. At the same time, due to the increase of handheld device carriers, augmented reality technology has been more widely used. Mobile augmented reality represented by smart phones not only has the advantages of augmented reality technology itself, but also has the advantages of portability, connectivity and ease of use. Thus, the development prospect of hybrid reality is very broad (Huojiao 2016; Jingyi 2016; Sungheetha and Sharma 2021; Wang et al. 2020). Museums play an important role in improving the scientific quality of the public. The application of hybrid reality technology to museum exhibits can, on the one hand, mobilize the public's initiative and enthusiasm, and the public can construct relevant knowledge through interaction with objects. On the other hand, it can stimulate public curiosity, search for relevant background cultural knowledge through multiple channels, and help the public develop a scientific attitude of active inquiry. In addition, mixed reality does not separate the public from the real object, it can realize the superposition of virtual and real, and bring new experience and experience to museum visitors (Yi 2017; Zhaotong et al. 2017; Jin et al. 2016; Jianghan and Xiujun 2017).

To sum up, if the mobile augmented reality technology is widely used in museum guide, it can bring users a more humanized human–computer interaction mode, and can more vividly show users all kinds of information related to exhibits. The purpose of the augmented reality system is to provide users with a platform for information assistance and collaboration to enhance users' awareness of the real environment. Based on the needs of practical applications and the sensitivity of the human eye, the augmented reality system has high requirements for the accuracy of registration and the realism of virtual information. The user has the ability to communicate with the base station and the ability to forward data for other users at the same time. Through the multi-hop transmission of other users, the users in the low-rate area can obtain the high-bit-rate services provided by the cellular network, thereby achieving high-rate services and the rate business coverage expansion. The performance of ad hoc network in terms of business throughput, delay and power is better than that of cell structure of small area, but the communication range of ad hoc network is very limited, far from reaching the range of cellular network. Therefore, considering the advantages of the self-organized network and the existing cellular network in a comprehensive manner, a network structure of self-organized network transmission method introduced in the cellular network is proposed! In order to solve the current problem, the dead corner of the cellular network.

The most direct advantage brought by such technology is that it can greatly meet the tourists' personalized guidance needs, and can play a certain role in promoting the overall quality of tourism and the improvement of cognitive level. The application of augmented reality technology in museums, on the one hand, explores the new exhibition mode of museums. While changing the traditional explanation tour, it provides a subversive way of cultural relics exhibition. On the other hand, augmented reality technology perfectly combines historical culture with modern technology, which is of great theoretical significance for the protection and inheritance of ancient historical relics. The rest of the paper contains the following sections. (1) The literature review section discusses the state-of-the-art methods and approaches that support the proposed model. (2) The proposed methodology section clearly discusses the overall technologies of the proposed mode. (3) Experiment and verification section, the model is simulated. (4) Conclusion and prospect section summarizes the work and set up the future research directions.

2 Literature review

Foreign research on digital museum is earlier. Burgard et al. were the first to study the Museum. In 2002, they developed an interactive navigation robot for museum visits, which can explain cultural relics to visitors, thus reducing human labor. However, due to its high cost and low flexibility, the system has not been widely used. In 2004, Based on Burgard's research, Li-der et al. developed a multimedia museum navigation system using PDA as the platform. The system installs a corresponding infrared transmitter next to each artifact. When tourists hold a PDA near the cultural relics, the infrared transmitter will send the cultural relics ID to the PDA. Then, PDA gets the corresponding cultural relic multimedia information through this ID. The system plays a certain role in promoting the development of digital museum. Besides the traditional audio commentary and introduction, the system can also display pictures, videos and other multimedia information of cultural relics.

In 2005, Adfiano et al. developed a museum tour system based on computer vision, which initially presented the prototype of the AUGMENTED reality museum system. It USES computer vision technology, through the cultural image color, brightness, edge and other characteristic information, to realize the automatic recognition of museum painting cultural relics. The system adopts The C/S architecture, carries on the network communication through the wireless WiFi, has realized the client side and the server side data transmission.

In 2008, Herbert developed an interactive museum navigation system. This system is developed based on Table PC, which also USES computer vision for cultural relic identification and feature matching algorithm for cultural relic image matching. However, the system has installed a Bluetooth transmitter in the exhibition hall to locate the current area of visitors, so as to reduce the number of cultural relics to be identified, which improves the speed of cultural relics identification to some extent.

In 2009, Bruns et al. designed a museum tour system based on computer vision, which USES a lightweight two-layer neural network algorithm for cultural heritage identification. In 2013, Miyashita et al. first proposed a signalless, mixed tracking method. This system adopts inside-out non-standard tracking mode and USES sensor tracking mode for auxiliary identification, thus solving the previous problem that too little cultural relic feature information is not conducive to tracking (Wendrich and Vaneker 2017; Jibb et al. 2020; Ziyu 2016; Li 2017).

In 2015, Feng Weixia analyzed excellent cases of AUGMENTED reality at home and abroad, and summarized the artistic characteristics of AUGMENTED reality technology and its advantages in museum display. He designed an augmented reality APP based on mobile devices, and proved the feasibility and rationality of using augmented reality technology in museum display through practice.

However, the network-based digital museum system faces many problems in the implementation process, and the network-based virtual model data transmission is one of the main problems. In digital museums, virtual models are usually represented as triangular meshes. In order to describe their fine details, some models often require tens of thousands or hundreds of thousands of facets. A model with such a huge amount of data will inevitably cause real-time network browsing. The user needs to wait for the data download to complete before starting to browse the virtual scene. The cultural relics information resources of the digital cultural relics museum mainly refer to the recording or description of the state, change characteristics and the characteristics of cultural relics in different forms such as language, text, graphics, audio and video formed in the process of cultural relic discovery, unearthed, sorting, research, utilization and protection Original information contained in cultural relics such as the characteristics of the connection between different time and space and the objective environment, or processed and processed information stored in materials such as paper, tape, CD, film, etc.

In summary, the techniques can be separated into the following aspects. (1) The external display information system is a Flash-based webpage. The general geological phenomenon simulation, terrain formation process, and mineral evolution history displayed on the webpage are all animations made by Flash. In addition, virtual reality, three-dimensional space, images, sound, and ultra Text links and other methods make up for the shortcomings of boring geological exhibits and complex geological phenomena that are difficult to explain. (2) The information collection and management of museum collections must first create a unified standard, so that the collected information has a unified recording mode and regular data structure. (3) The Internet is a bridge for the exchange of information between the museum and the outside world. It can often collect relevant information from the Internet, retrieve information through search engines, and mine information to enrich the museum’s digital resources.

3 The proposed methodology

3.1 Hybrid reality technology

In a broad sense, virtual reality, augmented reality and mixed reality are generally referred to as virtual reality. Virtual reality technology uses computer to create and simulate virtual environment, which makes the experiencer immerse in a three-dimensional virtual space, and makes the experiencer feel as if he is in the real world by mobilizing his visual, auditory and other senses. The most important feature of virtual reality is to let the experience have a strong sense of immersion and presence. Augmented reality is the expansion and extension of virtual reality. It combines virtual objects with the real world with the help of computer vision and other technologies. The goal of augmented reality is to strengthen the experiencer's understanding of the real environment by combining the virtual object with the real environment. The main difference between virtual reality and augmented reality is that the experiencer will think that the virtual object is a part of the real environment. Hybrid reality is a new visual environment produced by the combination of augmented reality and augmented virtual reality. Augmented reality is a simple superposition of real information and virtual information. Virtual reality is the technical means to realize virtual information. Hybrid reality is to realize the real-time interaction between the real world and virtual objects in the emerging visualization environment.

The characteristics of hybrid reality technology are as follows:

  1. (1)

    Authenticity. Mixed reality technology can well integrate virtual and reality, create the feeling of being born in the environment, and make the experience produce good sensory experience.

  2. (2)

    Imaginative. In a realistic new environment, users use their imagination to construct an objective non-existent environment. Hybrid reality technology has a very large space for development. With the continuous development and improvement of computer, virtual reality and other technologies, hybrid reality will continue to be enriched in museum display technology. The development of science and technology will certainly create infinite possibilities for the development of hybrid reality technology in museums (Markopoulos et al. 2021; Almhana and Kessentini 2021).

  3. (3)

    Real time interaction. Hybrid reality technology combines virtual reality with traditional physical display, builds a bridge between virtual objects and real world, and establishes the relationship between users and objects. Users can shuttle freely between reality and virtual, changing the traditional static display state. Breaking the unidirectionality of information transmission to realize the real-time interaction and communication between people and objects.

Advances in sensor and processing performance have led to new areas of computer input from the environment. The interaction between the computer and the environment is efficient for understanding or perceiving the environment. Environmental inputs can capture a person's location in the world, such as head tracking, surfaces and boundaries. Today the combination of computer processing, human input and environmental input creates opportunities for a truly mixed reality experience. The motion of the physical world can be transformed into the motion of the digital world. Boundaries in the physical world can affect the application experience in the digital world. Without environmental input, user experience cannot be fused between physical and digital reality (Subhashini et al. 2020; Xiang and Zhifen 2005; Taojian 2016; Ting 2012). Figure 1 depicts the relationship between computer, environment and human after the integration, and mixed reality is the integration of the three technologies.

Fig. 1
figure 1

A diagram of the relationship between people, computers, and the environment

Full size image

MR is a further development of augmented reality technology, which introduces real scene information into virtual environment. MR sets up an interactive feedback information loop between virtual world, real world and users to enhance the sense of reality of user experience. The main feature of MR lies in its ability of spatial scanning and positioning and real-time operation. It can merge virtual objects into real space and achieve precise positioning, so as to achieve a virtual and real fusion visual environment.

3.2 Theory of multiple intelligences and theory of situational learning

In 1983, Harvard scholar Cardner, proposed the theory of multiple intelligences on the basis of various research achievements in comprehensive psychology and anthropology. Intelligence is a psychological potential, the ability to solve problems or make products in a particular cultural context of a society. The definition emphasizes the diversity of intelligence, the importance of environment and education to the development of intelligence (Cao and Zeng 2021; Yavuz and Bai 2020). He believes that everyone has at least eight basic intelligences, but each intelligence has individual differences due to its different performance and exertion. Even if one is not proficient in a certain field, one can develop one's intelligence to an appropriate level if one can be encouraged, guided and educated from an early age. Under the background that our country begins to advocate quality education and lifelong education, school education can no longer meet the needs of individual all-round development. In the mixed reality exhibition form design, the designer can design the corresponding explanation form according to the cultural background of the exhibits to make the audience react, so as to achieve the purpose of museum science education.

In the mid-1980s, the theory of situational learning was put forward. In 1987, Reznik’s “Learning in and Out of School” compared in-school learning with out-of-school learning. Learning in school context is purposeful, which pays more attention to the acquisition of knowledge and skills, while learning in life context lays more emphasis on students' use of situational reasoning to solve problems.

In 1991, Lev et al. proposed “legitimate marginal participation” through the study of practitioners, and proposed that learning is a series of practical activities in social life. It can be seen that situational learning is different from traditional learning. Traditional learning takes abstract conceptual knowledge and applies it to practice. The theory of situational learning regards abstract knowledge as a tool, and conceptual knowledge can only be fully understood in the application of practical activities. Individuals interact with others and the environment in social practice activities so as to improve their practical ability and socialization level. According to the relevant theoretical research of situational learning, the design of effective learning situation will be paid much attention to in the process of MR mixed realistic exhibit display form design (Liu and Lan 2021).

3.3 Introduction to hybrid reality equipment as the basis

The current experience of hybrid reality devices is the hollens developer version released by Microsoft, which is Microsoft’s first digital computer device free of cable restrictions. The device adopts windows 10 system, which is not restricted. Microsoft hololens has the characteristics of digital, high-definition lens, stereo and so on, which enables users to interact with digital content and interact with digital images in the surrounding real environment.

Figure 2 shows the disassembly effect of hololens equipment.

Fig. 2
figure 2

Hololes equipment disassembly drawing

Full size image

The gaze, the first form of input from Hololens, is the main form of mixed reality. The user can tell Hololens what they are looking at by staring at the cursor. In the real world, users often see an object they intend to interact with. The same is true of stares in Hololens. Hololens USES the position and direction of the user's head, rather than their eyes, to determine their gaze vector. You can think of this vector as a laser pointer coming directly from the front of the user's eye. As the user circles the room, the application can intersect the light with its own digital graph and spatial mapping grid to determine the virtual or real objects that the user might be viewing. On HoloLens, the interaction usually takes the target from the user's eye, rather than presenting or interacting directly from the position of the hand. Once the interaction begins, the relative motion of the hand can be used to control gestures, just like a manipulation or navigation gesture. Figure 3 shows the Hololens cursor schematic. In network transmission, due to bandwidth limitations, the amount of data should be reduced as much as possible.

Fig. 3
figure 3

Diagram of staring cursor

Full size image

SmoothLOD encoding generates a large amount of repeated vertex data after generating a linked list. In order to reduce the amount of repeated vertex data, the triangle faces of the model are represented by vertex index. Decoding is a real-time process, not a pre-processing process like encoding. When the encoded data is obtained from the network, it is necessary to immediately recover the triangular patch from the node data and draw it. Therefore, the decoding process requires very high time efficiency. Through analysis, the main time consumption in the decoding process is the process of modifying the triangle faces included in the updated list of the triangle face list represented by the current model. This process is received every time A node needs to be updated once the data is taken out, and it is necessary to traverse the triangular face linked list represented by the current model to find the triangular face that needs to be modified. This is actually a retrieval problem. We have improved the efficiency of decoding by using a hash table.

Speech input and recognition is one of the three main forms of HoloLens input. It allows users to manipulate digital graphs directly through voice commands without using gestures. The user is required to speak voice commands based on a digital “gaze” diagram. Voice input can be a natural way of communicating the user's intent. Voice is particularly suited to traversing complex interfaces because it allows the user to cut through complex nested menus with just one command (Li et al. 2019; Agostino et al. 2021).

In HoloLens, gestures allow users to interact naturally with digital graphs. HoloLens recognizes gestures by tracking the position of the hands visible to the device. HoloLens sees both hands when the gesture is ready (index finger pointing up) and pressed (index finger changing from pointing up to pinching). When the hands are in other positions, HoloLens ignores them.

Spatial mapping provides HoloLens with the ability to represent real-world surfaces in detail, allowing developers to create compelling mixed reality experiences. By combining the real world with the virtual world, applications can make digital graphs look real. By providing familiar real-world behavior and interactions, applications are more likely to conform to user expectations.

In Hololens, the two main object types used for spatial mapping are “spatial surface observer” and “spatial surface”. The application provides one or more boundary volumes to the spatial surface viewer to define the spatial regions where the application wants to receive spatially mapped data. For each of these boundary volumes, the spatial mapping will provide the application with a set of spatial surfaces. These volumes may be stationary, in a fixed position relative to the real world, or they may attach to the HoloLens, they may move, but they will not rotate as the HoloLens moves around the environment. Each space surface describes the real world surface in a small space, represented as a triangular mesh connected to the world locked space coordinate system. Figure 4 shows an example of a spatial scan using Hololens. As the environment changes, HoloLens will collect new data about the environment, so the spatial surface information will be in the state of emergence, disappearance and change.

Fig. 4
figure 4

Grid mapping model example

Full size image

3.4 Digital museum display design based on mixed reality

Since the industrial revolution, museums are open to the society in an all-round way. The educational function of museums has become increasingly prominent. People gradually regard museums as one of the ways to acquire knowledge. In order to attract more visitors, museums began to separate the collections used for display from the collections stored. Therefore, museum display came into being and developed. At the beginning of the 20th century, the British Natural Museum first adopted the standardized museum display cabinet, and improved the traditional exhibition form. On the basis of the original text, the British Natural Museum provided the exhibits with pictures, models, illustrations and other new interpretation forms. Since the 21st century, museums all over the world have developed unprecedentedly in terms of quantity and display forms. However, some museums still have many problems in the display of exhibits.

Museum exhibition is not a simple accumulation of exhibits, but a combination of all the exhibition contents to form a whole, and put them into the corresponding historical culture, so that the audience can better understand the cultural value of exhibits. At present, in order to meet the needs of modern audiences, museums also apply a lot of multimedia technologies, such as audio and video, scene synthesis, touch screen, etc.

  1. (1)

    Multimedia touch screen technology The audience can use their hands to move, rotate and other gestures to touch the screen, so that the audience can fully understand the exhibits information.

  2. (2)

    Intelligent display device Intelligent display relies on flat panel TV, which is used as electronic display board. When the audience is close to the display board, the display board will automatically demonstrate the display content. This kind of exhibition form mainly uses the dynamic display to increase the display interest.

  3. (3)

    Holographic projection The application of phantom imaging technology, the shooting image and simulation scene synthesis, let the audience experience the related story deduction in the simulation landscape. At present, holographic projection technology has been relatively mature and widely used.

  4. (4)

    Virtual reality scene Virtual reality is the use of computer technology through the simulation of three-dimensional scene display objects, for the audience to construct an immersive environment, so that the audience can get the experience and feeling of immersive from the multi sensory.

It can be divided into two modules: mobile phone and mobile phone. After the mobile terminal scans and identifies the cultural relics, the main functions of the cultural relics are displayed in the way of mixed reality (MR). The audience can not only understand the use function of the exhibits, but also enhance the interest. Users can also move, rotate, zoom in and zoom out the cultural relic model through the touch screen operation to realize multi angle observation (Fig. 5).

Fig. 5
figure 5

Functional recovery

Full size image

By scanning the original building blank and the remains of the site, the original appearance of the three-dimensional historic site is restored. The user’s search results are returned by comparing the similarity between the document vector and the query vector. Therefore, the search process is also the process of comparing the user’s query keywords with the documents, and the same is true for information retrieval in digital museums. But in general, the queries provided by ordinary users are descriptive keywords.

These keywords are not necessarily included in the target document, so that some documents that are important to the final result of the query are not retrieved. On the other hand, there are a large number of non-text audio and video files in digital museums, and user query keywords need to be compared with the labeled keywords of the audio and video files. To completely describe the information of a cultural relic work requires as many as 60 types of data items, which makes the metadata model more complicated. Generally, browsing or searching only needs the most basic information of cultural relics. Therefore, this paper proposes a metadata representation model combining simplified and traditional ones, which is to construct a simplified metadata representation based on the most basic information of cultural relics, that is, the core element set. The extended metadata model represents the complete information of cultural relics. The core element set includes: category, name, size, brief description of form, creator's name, identification information, era, date, collection unit, collection location, identification number. The core element set can be used in browsing and retrieval, making the actual operation more convenient (Fig. 6).

Fig. 6
figure 6

Restoration of historic sites

Full size image

Scan the cultural relics such as drums and bells on display, and users can click directly. At this time, a virtual hammer can be displayed in the client, which can beat the drum or bell, and reproduce its use scenario in the interaction (Fig. 7).

Fig. 7
figure 7

Perceptual interaction

Full size image

3.5 Construction of integrated data acquisition and analysis platform

In the design of embedded system, the underlying hardware plays a key role. It is very important to choose a high performance controller, which will bring great convenience to the overall design of the system. The ARM development board selected for the project is S5PV210 as the main control chip. S5PV210 is a microprocessor produced by Samsung based on the ARM Cortex-A8 kernel. Its main frequency reaches up to 1 GHz, it has a second-level cache, and it can realize the high-performance computing capacity of 2000DMIPS. The chip has high-end 3D graphics engine and 2D graphics engine inside, and supports 2D graphics and 3D graphics acceleration functions. In addition, the entire development board is equipped with 512 MB SDRAM as the system memory and 512 MB NandFlash as the system hard disk, as well as Ethernet interface, UART interface and 4 USB HOST2.0 interfaces.

This platform supports two data acquisition CARDS, but their data communication methods are different. One is the network communication mode based on TCP/IP protocol, the other is the communication mode based on USB protocol. Among them, the USB control chip CH376S used by the DATA acquisition card based on USB communication has no corresponding driver program in the Linux kernel. Therefore, before the development of platform applications, the CH376S driver in the Linux underlying driver should be designed and implemented first.

The whole application program is composed of several functional modules, including data acquisition and control module, touch screen calibration and test module, waveform playback/data analysis module, hot-plug monitoring module, system setting module and soft keyboard module. The data acquisition control is mainly used to invoke the data acquisition control module, which is used to set relevant parameters during data acquisition, such as acquisition channel, acquisition duration, acquisition frequency and so on. Through the module interface, the collection can be started and stopped, and the collected data can be uploaded and saved to the mounted USB disk.

The system setup control mainly calls the system setup module, which is used to set whether the current platform supports data acquisition card based on USB communication or data acquisition card based on TCP/IP protocol communication. Since the AD reference voltage of the two data acquisition CARDS is adjustable, the reference voltage value can also be set in the module interface to inform the relevant module of the current AD reference voltage of the data acquisition card. In addition, the module interface can be used to set the system date and time.

Screen calibration control is mainly to call the touch screen calibration test module. In the interface of this module, you can choose to call the touch screen calibration tool or the test tool through the button control. The touch screen calibration tool is mainly used to recalibrate the touch screen and update the configuration parameters in the root file system. The test tool is mainly to detect whether there is a large error in the accuracy of the touch screen.

Data acquisition and waveform display are two key tasks of the application program for the ultimate purpose of this platform. Data acquisition mainly controls the front-end data acquisition card to collect waveform data according to the setting of the current collection parameters, or the data can be uploaded and saved to the file according to the requirements. Waveform display is mainly used to restore and display the waveform according to the collection parameters recorded at the beginning of several bytes in the file and the collection data after that.

Aiming at the program design of acquisition and control part, the following realization methods are mainly introduced:

  1. (1)

    Gets the usage of the mounted external mobile memory. When viewing the storage space, the usage of acquisition card memory and external mobile memory (U disk) will be displayed in the pop-up interface. The memory information of the acquisition card is obtained by sending the corresponding command code to the acquisition card, while the information of the U disk is obtained by calling the statfs function.

  2. (2)

    The realization method of keeping the platform record and data acquisition card synchronized. Because of the non-volatile nature of the acquisition card in the collection and recording, that is, the acquisition card can still restore the previous acquisition record after the card is shut down and restarted. In order to ensure that the data can be uploaded accurately according to the upload command, the platform must be synchronized with the acquisition record of the acquisition card. When the platform is powered on, each acquisition information recorded in the temporary file is stored in the custom structure instance, and the address of the structure instance is added to the vector container corresponding to the acquisition channel. These vector containers corresponding to the acquisition channel are mainly used in data upload and information query functions.

  3. (3)

    Command and data transmission format. The command is sent to the data acquisition card by the platform application software, and the data is uploaded to the application program by the acquisition card. The data includes collection data, the usage data of the memory of the acquisition card, and the feedback data whether the command is effective. When uploading data, the platform application software does not know how much data will be uploaded. Therefore, in the process of data upload, the application software will not be able to determine when the data transmission ends.

The realization of the function of wave sliding in any direction. When in the non measurement and non playback mode, the waveform can be moved in any direction by manually swiping the screen. Second thread drawing implementation. When more data points are drawn at one time, drawing will also be a time-consuming process. In order to prevent the wave line display interface from freezing, the double thread mode is also used in the waveform display part. The main thread is mainly responsible for the interface operation, and the secondary thread is mainly responsible for drawing the waveform image and sending the image to be displayed to the main thread.

3.6 Designed machine learning method in data acquisition

Machine learning is a kind of technology that studies how computer simulates or realizes human's learning behavior in order to acquire new knowledge or skills and reorganize existing knowledge structure to improve its performance continuously. Machine learning method is mainly used to solve the problems of feature extraction, the relationship between the extracted features and the relationship between fragments. Through the feature extraction and training of the acquired data, we can find out the internal correlation, and then find out the relationship between fragments. Machine learning method can avoid artificial patching and shorten the restoration time of cultural relics.

Spectrum analysis is the process of analyzing and processing signals in frequency domain after transforming the time-domain signals into frequency-related functions through Fourier transform. In the computer system, all data are discrete quantities, so before the discrete Fourier Transform (DFT) is used for the spectral analysis of signal data, the Nyquist sampling theorem is used to obtain the discrete data.

Suppose the number n of the sequence x(n) is equal to a power of 2. If this condition is not satisfied, then the final complement of the sequence zero satisfies this condition. According to the parity of N, the original sequence x(n) is divided into the following two groups:

$$ x(2r) = x_{1} (r) $$
(1)
$$ x(2r + 1) = x_{2} (r) $$
(2)
$$ r = 0,1,...,N/2 - 1 $$
(3)

The DFT transform of the original sequence x(n) can be obtained by using the DFT transform of its odd–even sequence, which can be directly expressed by the butterfly flow (Fig. 8).

Fig. 8
figure 8

Butterfly computing flow

Full size image

In DFT transform, when the duration of the intercepted signal is not equal to the integral times of the measured signal period, the spectrum leakage phenomenon will occur, and the spectrum leakage will make the small amplitude signal components difficult to be identified. In order to be effective.

In order to suppress spectrum leakage, the transform sequence is usually windowed in time domain. The commonly used window functions are as follows:

$$ {\text{Rectangular window:}}\,W(n) = 1 $$
(4)
$$ {\text{Hanning window:}}\,W(n) = 0.5*\left( {1 - \cos \frac{2\pi n}{{N - 1}}} \right) $$
(5)
$$ {\text{Haiming window:}}\,W(n) = 0.54 - 0.46*\cos \frac{2\pi n}{{N - 1}} $$
(6)

4 Experiment and verification

The system selects different types of cultural relics for digital display. For example, ordinary 3D cultural relics are placed on the virtual display cabinet, and selected cultural relics and ancient paintings will be automatically generated on the scene wall of users. The main implementation steps are as follows:

  1. (1)

    The cultural relics are divided into two types: plane type and vertical type, which are set as the attribute parameters of cultural relics.

  2. (2)

    The distance between the nearest boundary coordinate of the plane generated by spatial scanning and the user’s hololens coordinate is sorted to find a more suitable projection distance, which is between 1 and 5 m.

  3. (3)

    Set the minimum area of the desired plate. If the area of the plate with the appropriate distance is less than the minimum area, replace it with the plate on the left and right of the distance sequence until the first suitable wall panel and floor slab are found.

  4. (4)

    The cultural relics with vertical attributes are generated on the ground and those with horizontal attributes are generated on the wall.

As shown in the figure below, on the left of the figure is the cultural relic intelligently generated on the ground, and on the right is the cultural relic and ancient painting intelligently generated on the wall (Fig. 9).

Fig. 9
figure 9

Intelligent generation of cultural relics on the ground and walls

Full size image

The intelligent red blue projection results of ancient paintings are shown in the following Fig. 10.

Fig. 10
figure 10

Intelligent red and blue projection of ancient paintings

Full size image

Scanning modeling uses three-dimensional scanning modeling technology to scan three-dimensional cultural relic models in real space, which automatically decomposes the captured subject into 49 faces for sequential capture and acquisition. Open the shooting function of the scanning application. When the viewfinder captures the two-dimensional coordinates in the image, the three-dimensional space coordinate system is generated, and the feature points of cultural relics can be collected (Figs. 11, 12, 13, 14 and 15).

Fig. 11
figure 11

Detection of ruler background by feature extraction

Full size image
Fig. 12
figure 12

Feature point acquisition of 3D objects

Full size image
Fig. 13
figure 13

Feature point acquisition test

Full size image
Fig. 14
figure 14

Scene structure configuration and parameter setting

Full size image
Fig. 15
figure 15

3D interactive product catalog demonstration system

Full size image

The number of feature points will increase to 546 after the 49 sections of cultural relics are fully scanned.

After the whole cultural relic is identified, the system automatically generates a.ob file, which will be used as the data set of cultural relics for subsequent use.

The results of scene structure configuration and parameter setting are as follows:

The results of 3D interactive product catalog demonstration system are as follows:

5 Conclusion and prospect

Statistics show that more and more people in China acquire knowledge by visiting museums, science and technology museums, historical and cultural monuments and other carriers with high cultural information content. Based on the mixed reality environment, the product display can greatly improve the two-way interaction between products and consumers. This seemingly “person-to-thing” display form is actually the interactive essence of “person-to-person” and the essence of product marketing. Although the mixed reality product display form has a broad prospect and huge market, it cannot completely replace the traditional substantive product display form. The application of hybrid reality technology is not to deny the traditional way of physical display, but to better play the function of product display, improve the speed of product information transmission, broaden the width of product information transmission, and enhance the accuracy and precision of product information transmission. This paper designs and implements a new intelligent digital museum system based on the hybrid reality technology. Besides, the software-driven data collection and analysis models are combined for the systematic performance improvement of the model. The comparison experiment has shown that the proposed model is efficient. In our future study, the proper integration of different models will be combined to comprehensively evaluate the performance.