Keywords

1 Introduction

The Mixed Reality (MR) learning approach of wearing transparent smart glasses creates a new vision of wearable experimental education technology. This innovative approach provides active access to a variety of additional information and is more conducive to physical interaction with reality as well as conceptual learning and mastery [1, 2]. The integration of mother and child nursing teaching with XR (VR/AR/MR) technology is an important trend in modern education. VR has the distinguishing features of pure virtualisation and immersion, while AR emphasises the overlaying of virtual data based on real-world information. MR, on the other hand, combines the advantages of both very well. Mobile MR, supplemented by flexibility, can bring a natural perspective but with a novel experience to the teaching of maternal and child care [3, 4].

Clinical simulation is rapidly becoming an important tool in training students who are pursuing careers in nursing as a viable complement or alternative to practicing with living patients. Laura K conducted an exploratory and descriptive study through a survey of nursing students in the Magdalena University practice laboratory using humanoid robots for nursing instruction. The results of the study showed that students in the project were positively motivated by the use of technology as a teaching strategy (specifically humanoid robots), suggesting a greater exploration of its use in educational nursing [7].Ijaz Ahmad presented a multimodal MRI interaction between real and virtual worlds. Previous related work has only focused on multimodal interactions between users on the real end and virtual beings, or visual and auditory interactions between users of VR experiences and users on the real end. As a result, they developed an MR game content that enables interaction between the real and virtual worlds by tilting the sitting position of the VR experience user according to the real side waiting and/or accompanying user’s hand position [5]. Jason J. Saleem conducted an in-depth investigation into the relationship between maternal education and infant health. This study focused on the differential effects of maternal education on infant health outcomes across locations based on income status by using birth weight and low birth weight as measures of infant health. A substantial non-linear effect of maternal high school completion on infant health outcomes was found, which was masked when maternal education was specified as a linear variable. More importantly, the relationship between mother’s education and infant health was concentrated in relatively deprived geographical areas. This is partly due to the fact that educated mothers living in poorer areas have substantially higher utilisation of health services compared to less educated mothers [6]. Therefore, it is relevant to study a realistic training system for maternal and child health care simulation based on MR virtual technology [7].

While most current research on maternal and infant health care training has been conducted from a single perspective of a professor, this paper combines contextual perception with the design of maternal and infant health care simulation training, exploring the interaction between the contextual needs of maternal and infant health care and the environment from a contextual perspective, providing a more detailed design direction for the design of maternal and infant health care simulation training systems that can enable users to autonomously perceive the situation of mothers and infants and provide them with This will enable the user to perceive the situation of the mother and baby autonomously and provide them with personalised health care guidance.

2 A Study of a Realistic Training System for Maternal and Child Health Care Based on MR Virtual Technology

2.1 Maternal and Infant Health Care Training Scenarios

The system’s realistic training context contains the conditions, life patterns and behavioural preferences of mothers and infants. In the design of the system, the various types of information in the context are fully considered, and the situation of the mother and infant object from which the user conducts the realistic training is taken into account to consider how to improve maternal and infant health care and precise service capabilities [8, 9].

The environmental context contains the physical and social environment, and involves many environmental factors in the process of maternal and child health care. The environmental context is considered in the maternal and infant health care simulation training system to provide design directions and strategies for the simulation training system based on the specific environmental elements mentioned above, and to enhance the maternal and infant health care capabilities of maternal and infant nursing students through the optimal integration of the environmental elements [10]. For example, the environmental scenario involves the residential and medical environment for mothers and infants [11].

Task contexts contain many health care behaviours for maternal and infant health care and the specific goals of these behaviours. Users should fully explore the opportunity points for experience optimization in each task context, improve the system task operation experience, and provide more complete and convenient health management services for the mother and infant subjects of the realistic training [12, 13].

2.2 Multi-Camera Collaborative Tracking Technology Based on MR Glasses

When users are wearing MR head-mounted displays for virtual experiments, the object with the Mark attached is facing the camera of the MR head-mounted display most of the time [14, 15]. However, the handheld Mark sometimes does not maintain a perfect orientation; on the other hand, some Marked objects placed on the table are themselves positioned with the Mark facing the other way, thus reducing the visual clutter caused by too many Marks appearing in the frontal view [16, 17]. Therefore, a multi-camera tracking feature would be helpful, and placing two user-facing auxiliary cameras on the desktop to track the back Mark would greatly improve the robustness and aesthetics of the maternal and child health care simulation training platform. In Eq. (1), f2 represents the satisfaction of the multi-camera collaboration feature and s2 indicates the degree of attribute adequacy. The feature satisfies the charm attribute.

$$ f_{2} = \beta e^{\lambda zsz} \begin{array}{*{20}c} {} & {\beta ,\lambda } \\ \end{array}_{2} \in R $$
(1)

For the desktop MR maternal and child health care simulation training platform, the user needs to access the phenomenon of real-time MR experiments through the monitor. Three additional auxiliary cameras (ordinary RGB cameras) are set up to help track and identify the poses of the Mark, which are then transposed according to the positional relationship with the main camera. Ultimately, registration is performed according to the maximum confidence level obtained by calculating Eq. (2), with p denoting the camera and b denoting the Mark.

$$ Con = \max (Con(p_{1} ,b),Con(p_{2} ,b),Con(p_{3} ,b)) $$
(2)

For the mobile MR multi-camera practical training platform, the MR glasses come with one camera as the primary camera and only two additional secondary cameras are required. The difficulty in implementing the multi-camera function actually lies in the fact that the acquired poses need to be transposed according to accurate pre-calibrated multi-camera data and the virtual object is finally displayed on the screen of the head-mounted display in the correct pose [18].

3 Design of a Realistic Training System for Maternal and Child Health Care Based on MR Virtual Technology

3.1 WiFi Module

MR virtual technology-based maternal and infant health care simulation training system, need to be in the computer virtual simulation computing end, and a number of MR glasses operators to communicate with each other, different from the desktop virtual experiment platform of the single host mode, this platform to join the MR glasses constitute MR glasses - PC host of the client --server dual device model. Therefore, the dual devices are used to transmit the positional information of the virtual objects using WiFi to communicate under the local area network, and the direction is that the PC host processes the data and sends it to the MR glasses end.

3.2 MR Scene Data Storage and Loading Process Design

The information stored in the MR scene includes the position pose of the roving detector itself, augmented reality display, terrain information and manual label information. The specific data storage and loading process is shown in Fig. 1.

(1) When the server obtains the scene information from the roving probe, it first needs to form a terrain height map based on the DEM terrain and add an augmented scenario design.

Fig. 1.
figure 1

Scenario storage and loading process

Full size image

(2) Once the Unity scene is built, it is divided by square areas and the terrain height map, non-terrain obstacles and augmented reality location coordinates, category and size for each block of the database are uploaded.

(3) When using MR glasses to load a particular area map, it can then be loaded according to its coordinates. In the MR glasses side you can add labeled objects, move object positions and upload all scene information according to the final scene data at the end of the operation.

4 Implementation of a Realistic Training System for Maternal and Child Health Care Based on MR Virtual Technology

4.1 Implementing Human-Computer Interaction

The system uses MRTK as a mixed reality development component, using it to enrich the controls while using basic operations and simplifying the development effort. WindowsMixedReality runs as an extensible framework on Unity, providing developers with the ability to exchange core components. The basic MRTK file is first configured to use its corresponding component functionality at runtime. Its basic services include input methods, spatial awareness, camera, remote, 3D boundaries, etc. Through gaze and Tap gestures, spatial objects containing interaction methods with framework components can be manipulated for interaction methods, manipulation processing and object boundaries.

The interaction operations of mixed reality teleoperation in this paper are built on the basis of MRTK, adding suitable functional components to specific functions.

For example, the virtual tagging function is implemented. The tagging function is a 3D tagging of objects of interest to mother and child care students in the scene. The first step is to retrieve the prefabricated 3D virtual tag by clicking on the UI, add the interaction component ManipulationHandler for the spatial dragging operation, the tag size and direction adjustment component BoundingBox and the spatial position movement synchronisation component UpdatePosition. The functionality of the tabs is achieved by adding basic components.

4.2 User Satisfaction Masurement

After the completion of the system hardware and software design solution, a certain number of users were required to evaluate the design solution in order to check whether it was feasible. Fifteen users were selected to participate in this user satisfaction evaluation, and the users involved in the evaluation were given a detailed explanation of the functions and usage of the system, and a Likert scale was created from the perspective of several evaluation indicators, each item being divided into 5 levels (-2, -1, 0, 1, 2), in the order of extremely dissatisfied, dissatisfied, satisfied, more satisfied and very satisfied, and the users were asked to score through the satisfaction evaluation form. The scoring results are shown in Table 1 respectively. The average value of user satisfaction scores for each indicator is above 1, which shows that users are more satisfied with the system design solution, as shown in Fig. 2.

Table 1. System Scoring Results
Full size table
Fig. 2.
figure 2

Customer Satisfaction

Full size image

5 Conclusions

Competence, knowledge and skills are both different and related to each other. Knowledge is the result, object and content of an individual’s mental activity, while competence is the mental characteristic that an individual displays in order to complete an activity; skill is an automatic way of activity that has been consolidated through practice. The realistic training system can develop students’ abilities, knowledge and skills. This paper proposes a design for a realistic training system for maternal and child health care based on MR virtual technology. In the process of system design, the situational tasks of maternal and infant health care are realistically reproduced, and the functions of system simulation are achieved, which proves the feasibility of simulation teaching as well as the broad application prospects. This study provides a theoretical basis for understanding students’ competence and exploring scientific and effective laboratory management, which has certain social and practical significance. However, due to the limited capacity of the researcher, this study inevitably has many shortcomings and the system needs to be optimised based on the results of user satisfaction evaluation.