1 Introduction

Active media technology has improved our daily lives and simplified the decision-making process. As the complexity of active media services has increased, there has been an increased focus on the importance of user interaction-based methods. One of the user-interaction research domains is Natural User Interface/Natural User eXperience (NUI/NUX) [1, 2]. By recognizing and utilizing users’ natural gestures and postures, various kinds of applications can be easily controlled. One of the advantages is the reduced effort required by users when learning how to utilize the applications.

Recently, various kinds of active media devices such as wearable devices [3], smart phones, and TVs have been developed and utilized. Given that users utilize multiple active media devices concurrently, the importance of the user interface (UI) of active media devices is paramount. In addition, from the perspective of the active media devices, they do not perform only for a single user but for multiple users. Therefore, it is very difficult for these devices to choose one user or a subset of users. As a result, the approach employed to match a user and active media devices is important to increase the performance of the active media devices. Intuitively, one of the active media devices can be selected depending on the distance between the user and the devices, which requires the detection of the user location.

In ubiquitous computing environments where active media devices are utilized, motion cameras [4], as well as vision-based and beacon-based location-recognition [5, 6] approaches are usually applied. A beacon is a low-cost piece of hardware that connects smart phones, tablets, and embedded devices via Bluetooth, and is devised for sending alerts or advertisements to users.

In this paper, we propose a control interface for active media devices based on beacons in indoor ubiquitous computing environments. By utilizing beacons, active media devices are matched to corresponding users for whom its services are provided. In the experiments, four access points are deployed and the distances are measured to each beacon possessed by the corresponding user. After measuring four distances, we use them to calculate the locations to one of the users.

The rest of this paper is comprised as follows. In Chapter 2, active media interface research and indoor location-recognition research is introduced. Chapter 3 proposes a framework of the active media control interface and then we perform experiments. Finally, Chapter 4 concludes the paper.

2 Related works

Given that the diverse kinds of devices are deployed and communicated each other to provide variety services, this chapter describes the requirements of active media selection interfaces. Before proposing an active media selection interface, active media applications and indoor location recognition research are introduced.

2.1 Requirements of active media selection interfaces

In active media environments, there are diverse kinds of active media devices. To enable the use of active media devices, we consider the following:

  • User location accuracy: To activate active media devices, user location information is required. It is very difficult to accurately obtain a user’s location in indoor environments. However, the accuracy of the user location is required for use of active media devices.

  • Selection accuracy: The selection and performance of active media devices depends on the user-location information. A variety of methods is used to choose active media devices that will have good performance. Some studies utilize the environmental information or additional information about the users to increase the selection accuracy.

  • Available space: Information regarding the recognized location needs to be delivered to a server or any other device that uses active media devices. Therefore, the amount of recognizable space should be considered.

  • Number of active devices: Depending on the method used, there is the limit to the number of possible active devices, which limits the effects and advantages of active media devices. In addition, there is increased interference between the signals of active media devices as the number of these devices is increased.

  • Interaction convenience: Multiple methods may be used to select an active media device. Nowadays, users have very high expectation of these systems. Therefore, the convenience afforded by the system is very important. In particular, the interaction with users should be considered carefully.

  • Signal revision: In indoor environments, signals related to the distances and locations are sometime incorrect and missed. Therefore, a signal revision approach is required to estimate and revise those values.

2.2 Active media applications

The interaction with users is one of the key considerations of active media applications. The following studies have been done to investigate media interfaces and experiences. IP++ is an application for interactive space, which has four components:smart floor, interactive walls, smart cubes, and an information canvas for ambient displays [7]. IP++ demonstrates how to develop an interactive media interface based on a conceptual framework that describes key elements of the conceptual framework. CityWall is a multi-touch-based screen device [8]. The interaction with the user is realized by recognizing multiple fingers, hands, gestures and postures based on vision systems to share photographed images between users. For signal revision, the hand-revision approach was introduced [9]. Leap motion is a hand-motion detection sensor. Because one hand is sometimes blocked by the other hand, an estimation of the hand is required. In addition, the measured signal has noise, which produces incorrect hand information. In this study, we propose a Bayesian probability-based signal-revision method, where noise is detected and revised based on probabilities.

Even though the following is not included in the requirements for active media selection interfaces, it should also be considered. For example, the assessment approach is introduced to determine a user’s behavior [10]. To interact with a user, it is very important to be able to assess that user’s behavior. The sequence of evaluating a user’s behavior is defined in advance by a tree. Whenever the user’s behaviors are evaluated, each node from the bottom node to the root node calculates its value at each instant in time. As a result, the root node value is an assessment value of a user’s behavior.

2.3 Indoor location recognition research

The global positioning system (GPS) is usually utilized to find current locations. However, GPS is not suitable for use in indoor environments. Therefore, alternative approaches are required.

The active badge location system is a very traditional study that is used to determine user locations [11]. Its application contains four functions: Find, With, Look, and Notify. The function Find is used to provide the current location of a user based on infrared (IR). The location information of active badges is then shared via the network infrastructure.

One of the widely utilized Wi-Fi-based positioning systems is finger printing [12–14]. Finger printing is an approach to determining a device’s location based on the received signal strength, given that the received signal strength varies depending on the indoor location. For example, received signal strength values are collected during the offline phase, and the device location is then estimated based on a comparison and analysis with the collected received signal strength values during the online phase [15]. The disadvantage of finger printing is the number of fingerprints that is required. As the space increases, the number of fingerprints also increases. One study applied the theory of compressive sensing to reduce the number of fingerprints required [15].

Given that received signal strength values are not predictable, and varies depending on the configuration of indoor environments, the direct use of this parameter is very risky. There have been some studies to reduce noise. For example, the peak distance is obtained by analyzing the measured distances, and then making adjustments to determine the average of the measured distances [16]. The results may be improved by applying a high and low peak distance. Given that received signal strength fluctuates, the ranges of the measured distances usually extend beyond the boundary of the indoor environment. Therefore, normalization methods are required for the measured distances. In one study, the author proposed a normalization method that revises the measured distances by considering the strength ratios of the measured distances [17]. Accurate distances can be utilized to calculate user locations [18].

3 Active media control framework

The configuration of active media control environments is variety. Therefore, this chapter describes the essential configuration for the environments. The processes to select one of the active media devices are then described, which is validated in a real environment

3.1 Concept of the active media control interface

In indoor ubiquitous computing environments, the active media environment involves the deployment and use of various devices, as shown in Fig. 1. For each device, the environmental information is shared through the ubiquitous environment. This section describes the concept and the environment of the proposed method.

Fig. 1
figure 1

Active media control environment

Full size image
Fig. 2
figure 2

Active media control framework

Full size image

In the proposed environment, we use at least four Access Points (APs) to determine the location of a beacon. Given that the shape of the proposed environment is rectangular, the use of four APs is sufficient to enable us to receive the signals from the beacon. To increase the accuracy of the beacon signals, the number of APs can be increased. APs are positioned on the ceiling of rooms because if the APs are deployed on the floor or on the walls, the beacon signals may be blocked. The location of APs should also consider the possible movement path of users in the environment. We assume that the locations of active media devices are obtained in advance.

3.2 Beacon distance revision processes

Active media devices are selected by the active media control framework, as shown in Fig. 2, in seven stages. The active media control framework consists of the initialization stage, measurement stage, peak-revision stage, distance-revision stage, location stage, utilization stage, and termination stage.

In the initialization stage, the environment used to control active media devices is configured as shown in Fig. 2, and is divided into active media device deployment, access point and reference beacon deployment, and target beacon deployment steps. The active media device deployment step deploys the active devices to be controlled considering the configuration of the environment. The ith active media device is defined by device \(d_{i}\). The access point and reference beacon deployment step deploys APs and reference beacons containing the positions of the active media devices. There are four APs: AP \(a_{fl}\) at the front left side, AP \(a_{fr}\) at the front right side, AP \(a_{bl}\) at the bottom left side, and AP \(a_{br}\) at the bottom right side. The reference beacons \(r_{fl}, r_{fr}, r_{bl}\), and \(r_{br}\) are also respectively deployed on each corresponding AP. A reference beacon [16] is used to determine the distance from an AP to a reference beacon (AP-reference distance), and is referenced to measure distances between APs and a target beacon (AP-target distance) in real-time. Distance \(d_{fr,bl}\) is the distance from AP \(a_{fr}\) to beacon \(r_{bl}\), while distance \(d_{fr,tg}\) is the distance from AP \(a_{fr}\) to a target beacon b. Each user holds a target beacon [16]. In the target beacon deployment step, a target beacon b is placed and moved.

Next, the measurement stage measures the AP-reference distances and the AP-target distance to revise the AP-target distance. This stage consists of a time increment, reference beacon distance measurement, target beacon distance measurement, reference beacon distance storing, and target beacon distance storing steps. The time increment step increases the time t by one, the reference beacon distance measurement step measures the AP-reference distances at time t, and the target beacon distance measurement step measures the AP-target distance at time t. The reference beacon distance-storing step stores the measured AP-reference distances, and the target beacon distance storing step stores the AP-target distances to be revised.

Next, the peak revision stage revises the AP-reference distances and the AP-target distance based on the peak-based distance revision approach [16]. The peak revision stage is divided into the following steps: reference beacon distance edge calculation, target beacon distance edge calculation, reference beacon distance peak revision, and target beacon distance peak revision steps. Given that there are no initial measured distances, the measured distances could not be revised. The distance revision stage is performed after checking the condition First Distance. The condition check, Peek Distance, confirms whether the current measured distance is the peak [16]. If the current measured distance is the peak, the reference beacon distance edge calculation step is performed. If it is not the peak, the reference beacon distance peak revision step is performed. The AP-target distance is also revised as the AP-reference distances by performing the target beacon distance edge calculation step and the target beacon distance peak revision step. In the proposed method, when peak distances are revised, the average distance is not set to the possible maximum distance as with the traditional revision approach [16]. The two-distance edge-calculation step sets a criterion distance by using the distance that corresponds to the distance edge e% from a zero distance among the distances within the \(\phi _{100-\gamma }\) % stored distances. The stored distances are cleared after each distance revision. The distance peak revision step replaces the current distance by a criterion distance when the current distance exceeds the criterion distance.

Next, the distance revision stage revises the AP-target distance based on the AP-reference distances. Beacon signals are very unstable in indoor environments. Therefore, the measured distances from beacons should be revised to utilize those distances. In this paper, we use the relative distance estimation method (ReDEM) [17] to revise the distance. The distance revision stage consists of the reference beacon min-max extraction step and target beacon ReDEM steps. The former returns the maximum and minimum AP-reference distances for each AP. For example, in Fig. 3, the minimum and maximum AP-reference distances from AP \(a_{fr}\) are the distance d\(_{fr,fr}\) and the distance \(d_{fr,bl}.\) The latter step revises the AP-target distances based on the maximum and minimum AP-reference distances for all APs that utilize ReDEM. The revised distance \(\textit{d}_{fr,tg}\) is denoted by the distance d’\(_{fr,tg}\).

Next, the location stage calculates the location of a target beacon b (target location) based on the revised AP-target distances. In this paper, the target beacon ReLEM step finds the location of the target beacon b considering the ratios of the revised AP-target distances. Next, the utilization stage utilizes the target location in order to estimate the locations of active media devices (device location). The location utilization step controls active media devices when the Euclidean distance between a target location and device locations is smaller than \(\gamma \). Next, the termination stage determines whether a target location is calculated again. The condition check, Exit step, goes to the time increment step in the measurement stage when a target location is again estimated.

Fig. 3
figure 3

Active media control environment

Full size image

3.3 Service scenario

An active media control server controls active media devices based on one target beacon and four reference (\(r_{fl}, r_{fr}, r_{bl}\), and \(r_{br})\) beacons through four APs (\(a_{fl}, a_{fr}, a_{bl}\), and \(a_{br})\) as shown in Fig. 4. All types of beacons radiate radio signals based on Tx while active media devices (d\(_{1}\), d\(_{2}\), ...) are activated and deactivated.

First, a target beacon is moved by a user (Step 1 in Fig. 4) and radiates the radio signals to denote it location (Step 2, 8 & 12 in Fig. 4). The radio signals are propagated without considering the state of APs and an active media control server. Any APs within a specific distance can receive the radio signals (Step 3 in Fig. 4). The received radio signals are transformed to distances by filtering its signals and by multiplying \(\frac{d_{x,y} -\phi _\gamma }{\phi _{100-\gamma } -\phi _\gamma }\) by (\(\beta \)-\(\alpha \)) where . The revised signals are transferred to the control server through APs (Step 4). Therefore, the only roll of APs is transforming radio signals and transferring distances to the control server to reduce the required AP performance given that the number of APs is increased as the ubiquitous computing environment is wider. The control server revises the AP-target distance based on the four AP-reference distances (Step 9 in Fig. 4). The AP-reference distances are also transferred to the control server through the APs (Step 5, 6, & 7 in Fig. 4). The AP-target distance is then utilized for calculating the location of the target beacon (Step 11 in Fig. 4). One of active media devices that is the nearest from the target beacon is selected by utilizing Euclidean distance (Step 13 in Fig. 4) and is activated by the control server (Step 15 & 16 in Fig. 4). The selected active media device is \(d_{j}\) where \(d_{i,x}\) and \(d_{i,y}\) are the location of active media device \(d_{i}\), \(b_{x}\) and \(b_{y}\) are the location of the target beacon, and j is \({\begin{array}{ll} {min} \\ i \\ \end{array} }\sqrt{(d_{i,x} -b_x )^2+(d_{i,y} -b_y )^2} \). The other active media devices where i is not j are deactivated for the user to concentrate on the selected active media device (Step 17 & 18 in Fig. 4). In real time, all processes are performed concurrently.

Fig. 4
figure 4

Sequence diagram of active media control

Full size image

3.4 Experiments and analysis

The configuration of active media devices, APs, and beacons vary depending on the active media environment. This chapter describes the experimental setting used to validate the proposed method. Then, we evaluate the revised distances that are utilized to recognize the locations of users. In the experiment, four APs were deployed and user location was estimated by using four reference beacons on each AP and one target beacon in the 4 m \(\times \) 4 m space, as shown in Fig. 5. The target beacon was moved within the space.

Figure 6 shows the maximum and minimum AP-reference distances and AP-target distances while the target beacon was moved. Figure 7 shows the revised distances. \(\gamma \) and e were set to 20 and 100, respectively. Distances that increased drastically were smaller compared to the measured distances.

Fig. 5
figure 5

There are four APs, four reference beacons, and one target beacon. a Configuration of the space. b Path of a target beacon

Full size image
Fig. 6
figure 6

Maximum and minimum reference beacons and target beacon from four APs. a Distances in AP \(\mathrm{a}_\mathrm{br}\). b Distances in AP \(\mathrm{a}_\mathrm{fr}\). c Distances in AP \(\mathrm{a}_\mathrm{fl}\). d Distances in AP \(\mathrm{a}_\mathrm{bl}\)

Full size image
Fig. 7
figure 7

Revised distances from four APs. a Revised distances in AP \(\mathrm{a}_\mathrm{br}\). b Revised distances in AP \(\mathrm{a}_\mathrm{fr}\). c Revised distances in AP \(\mathrm{a}_\mathrm{fl}\). d Revised distances in AP \(\mathrm{a}_\mathrm{bl}\)

Full size image

In Fig. 6a, for the range 25–27, the distance d\(_{br,fl}\) was 32.738. After revising the distance d\(_{br,fl}\) considering the peaks, the distance d’\(_{br,fl}\) was revised to 11.495, as shown for the corresponding range in Fig. 7a. However, when a large distance d\(_{br,fl}\) was measured for the range 20–27, the distance d’\(_{br,fl}\) was set to a high value, 32.738. Therefore, although the next distance d\(_{br,fl}\) was a high value of 21.518, distance d’\(_{br,fl}\) was not revised. Figure 6b shows the distances and revised distances when the measured distances are stable. There are small differences between the measured distances and revised distances. Figure 6c shows the case when the measured distances are unstable. For this case, Fig. 7c shows that all of the revised measured distances are lower than the original distances because of revisions to the measured distances, with the exception of the range 58 to 62, which shows the increased revised distances.

Figure 8 shows the revised target distance based on the maximum and minimum AP-reference distances and the AP-target distances. Figure 9 shows the activated active media devices according to the target locations. When the difference in the distance between the target locations and the device locations is smaller than 1.2 m, the active media devices were enabled.

Fig. 8
figure 8

Revised target distances

Full size image
Fig. 9
figure 9

Selected and executed active media devices based on the location of the target beacon

Full size image

In Fig. 9, although a subject was in front of the television for the range from 35 to 38, the television is not activated if the target distance is not revised. Further, although a subject reached the location of a smart phone in the time range from 26 to 29 in Fig. 9, it was estimated that the subject reached at time 31 to 34. When the subject moved, there was a delay between the subject’s actual arrival time and the time at which the active media devices were activated. We need to improve the target location for the range from 75 to 88 and the range from 154 to 170 in Fig. 9, where wrong active media devices were activated.

4 Conclusions

In this paper, we proposed an active media control interface using beacons in indoor ubiquitous computing environments. Given that a variety of active media devices are currently being deployed and utilized, the proposed method aims to enhance methods that are employed to increase the performance of the active media devices by selecting active media devices.

In the experiments, we used four APs and one target beacon. The four APs were deployed on the ceiling of a room, and the beacon was assigned to a subject. The distances that were measured from the beacon were revised by ReDEM, and were then utilized to calculate the location of the subject. By applying the proposed method, the location of the subject and was utilized it to select one of the active media devices.

Six factors were considered when selecting the active media devices: User location accuracy, selection accuracy, available space, number of active devices, interaction convenience, and signal revision. The available space obtained by the proposed method is in proportion to the maximum possible distances to the beacons. When the maximum distances within the space is larger than the maximum possible signal distance to the beacons, an AP should be deployed in the middle of the space. Compared to the finger printing method, the proposed method is more convenient given that there is no need for an additional configuration, and that no additional signal collection is required. Therefore, when the configuration of the active media environment changes, the proposed method can be applied by only adding and changing AP locations. From the perspective of the number of active devices, the proposed method does not require any additional settings or devices when adding new active media devices and new subjects. The only consideration is the interference of beacon signals. The convenience of the interaction realized by the proposed method is good in that no action is required by the subjects to activate active media devices. On the other hand, it is necessary for subjects to express their intention to active media environments to prevent the automatic enabling of active media devices. From the perspective of signal revision, the proposed method includes an algorithm to revise the measured distances, which increase the user location accuracy and selection accuracy. In the future, to enable commercial applications, we will perform additional research into methods to increase the location accuracy and selection accuracy.