An Advanced Tactile-Haptic Controller for Smart Home

Abstract

A smart home philosophy strongly relies on a technical principle of automatization almost every aspect of today’s way of living. Sole automation of does not only manage system processes in household, or living environment in general, but all adjacent systems and corresponding subsystems which helps to maintain system-in-whole in operational state. From control of environmental temperature, air conditioning, lighting, utility control, etc., all existing “smart” systems predominantly control actuation part of smart home. Very miniscule “smart” systems do manage and serve human-machine interaction part of it and, if are made “smart”, they rely on solutions that utilizes some sort of existing ICT technologies that are mainly optimized for communication and visual presentation, not for in-situ smart home interaction. This paper deals with the problem of development of a tactile-haptic solution for control of smart home specific subsystem and that utilize IoT platform, giving feel and user experience similar to traditional one in form of an advanced switch with regulation ability. Tactile commanding with haptic feedback is presented and applied in smart home temperature regulation. Finally, an IoT platform solution is developed, tactile-haptic relation investigated with optimal solution established, and its functionality is verified as well as is proven in real time environment.

1 Introduction

Systems that encompass philosophies of IoT and vast-connectivity of everything in modern way of living are growing rapidly and penetrates in everyday activities silently and inadvertly. Smart homes are such systems and they represent closest integration of living environments driven by technocratic society trends and accepted interaction norms. Smart systems started to emerge into focus of broad society mindset horizon since beginning of 70’s in last century, [1]. With appearance and development of complex digital circuitry, integrated onto small form factor or in form of SOC’s of all kinds, same time with rapid development of communication technologies, smart principles of everything” takes its place in everyday processes and social interactions. At the very beginning the focus was primarily directed toward development of the systems for actuation, to control system variables on a physical level, thus neglecting interaction user-side to the system. Such development leave user-system interaction at the very generalized, and for that time acceptable, forms of interaction that often requires even advanced knowledge in technical sciences or requires special training courses. Correspondingly, they became and are highly user-unfriendly and unpleasant for everyday use and intended handle, [2]. As years of technology development progresses, and actuator systems become more advanced and feature rich and functional, focus of development slowly shifts to the user-interaction side with primary goal to develop more user-friendly interfaces and mechanism for user-machine interaction. Many researchers, and companies that are oriented to sales based on end-user-experience, put lot of effort and their assets in investigation of more convenient methods for pleasant and, same time informative, interaction mechanisms of the end-user to the given technical system in manner to ensure pleasant experience-of-use to some extent that current technology enables, [3]. One of many technologies, methods, that inevitably arose among other, are technology of tactile-haptic feedback where user can “feel” state of controlled system in manner of haptic-caused stimulus. Such interaction causes strong and positive (pleasant) feedback of the end-user, making it more natural by the response and more convenient and easily to handle in general, [4]. Tactile-haptic mechanism was developed, in electronically controlled sense, since rise if integrated circuits, but the complexity of the such systems for that time prevents even more advancements in that field of development, [5]. With today’s state of technological development, a highly integrated systems, that are small and affordable enough there arose high thrive in its employment in everyday use and integration in way of living as is today. At the time when IoT strengthen in technologically oriented communities, especially among researchers, that tactile-haptic systems become more easily than ever achievable with high potential to became inseparable part of, so called, Smart Home systems what encircles modern societies in every aspect of its life.

This paper deals with and investigate, as well as develop and presents, an exemplary tactile-haptic system that is based on the IoT principles, and integrates it on everyday utilizing smart home concept for modern way of living. It conducts general justification of such-systems development by analyzing end-user qualitative assessment of its use, expressing aspects of its strength and niches for further development for further deeper integration into smart households and similar systems.

1.1 Paper Structure and Organization

The main premise of this paper is to present importance of tactile-haptic interaction in smart home environment with emphasis on usability and convenience, all that based on an IoT solution that fulfill all prerequisites for such system design and validation. A showcase example, test system respectively, was built and validated with given qualitative overview on integration into smart homes and pleasance of the end-user use. The paper is structured into five chapters as follows; In first chapter and introductory overview of such systems is done, with emphasis on importance of UI experience for smart home use and living. In second chapter a theoretical background is laid out where principles of operation and impact onto use experience is analyzed. In third chapter a detailed design, setup and functional integration is proposed with emphasis on hardware-software merge with application in real case of use. In fourth chapter a test and validation of the system functionality is conducted, and an analysis is laid out through perspective of end-user experience in their use. Last chapter is dedicated to the discussion and conclusion words with analysis of key aspects that have potential for further development and improvement.

2 Tactile-Haptic Systems in Smart Homes – An Overview and Theory of Operation

In this chapter an overview and theory of operation of tactile-haptic systems are laid out. It will analyze needs, convenience-of-use and theory of operation in sense of applicability to the technical systems adapted for end-user handling and ease of use. The most vital aspect of end-user ease of use and handling focus toward development of convenient and appropriate UI (user interface), as well as tactile-haptic feedback mechanism for real-state conditional stimuli in backpropagation toward user. After analysis of tactile-haptic mechanisms and their influence onto end-user experience, an investigation and theory of design is shifted toward utilization of IoT principles in such systems with emphasis on broad connectivity and connectivity-of-everything applied on Smart Homes.

2.1 Tactile-Haptic Systems in UI Interaction

In the dynamic landscape of contemporary web and mobile application development, the evolution of user interfaces has been marked by a notable departure from haptic and tactile interactions, [6]. Users, in this rapidly changing digital environment, have gradually relinquished their once tangible connection with interfaces, becoming increasingly dependent on on-screen controls. This fundamental shift has given rise to a somewhat limited and one-dimensional user experience, notably deficient in the sensory richness that accompanies physical touch and feedback, [7]. Recognizing this deficiency, there is a burgeoning interest in the resurgence of back haptic feedback within Human-Machine Interfaces (HMIs). Such a reintroduction promises to usher in a new era of user interaction, liberating individuals from the predictability and uniformity that have come to define contemporary applications, [8] (Fig. 1).

Fig. 1.

User – System interaction

When a user seeks to modify a selected value, they input the adjustment through the controller’s haptic knob interface. Subsequently, the system initiates an automated procedure and furnishes the user with feedback, encompassing both graphical and haptic modalities. In contrast to graphical feedback, which necessitates a confluence of user attention and technological acumen, the user accrues advantages from haptic feedback, characterized by its inherent naturalness, minimal attentional demands, and temporal efficiency.

2.2 Tactile-Haptic Controllers Based on IoT for Smart Homes

The widespread availability of low-cost, network-attached microprocessors and the adoption of open-source technologies have fueled the rapid growth of smart home appliances and devices, [9]. Virtually every major home appliance manufacturer has embraced Internet of Things (IoT) devices. However, this has resulted in market fragmentation, where each manufacturer employs its proprietary technology and control interface. Consequently, consumers find themselves constrained, locked into a particular ecosystem. To address this issue, the introduction of a tactile-haptic IoT controller, integrated with multiple systems through open-source technology, grants users the liberty to explore the entire market. This transition liberates users from the confines of proprietary applications that often compromise their data privacy. Tactile-haptic feedback controllers, situated atop the IoT platform, symbolically and literally restore control to the hands of users (Fig. 2).

Fig. 2.

Controller smart home integration

3 Tactile-Haptic IoT Based Interactive Controller

The Tactile-Haptic interactive controller seamlessly integrates a comprehensive suite of advanced hardware and software functionalities, concurrently facilitating open-source cloud integrations. The controller’s versatile architectural design engenders a diverse spectrum of potential usage scenarios, rendering it an inherently adaptable device. Noteworthy among its multifaceted capabilities is the emulation of various levels of knob detents, affording it the capacity to operate in a free-spinning mode, return to a central position, or execute simple on/off functions in accordance with user requirements. Additionally, the controller incorporates a nuanced knob pressure sensitivity feature, permitting the emulation of button presses, thereby enriching the spectrum of interaction possibilities available to users. Further enriching the user experience, the inclusion of an RGB LED backlight introduces an extra layer of feedback, serving as a visual conduit for relaying information and conveying status updates. Additionally, the controller incorporates an integrated ambient light sensor, enabling users to discern and adapt to the prevailing ambient lighting conditions within their surroundings. The foundation of this controller’s versatility is its inherent connectivity. With seamlessly integrated Wi-Fi and Bluetooth options, it harnesses the complete potential of the Internet of Things (IoT) framework, thus enabling a frictionless conduit for communication and integration with cloud-based services. This expansive connectivity paradigm empowers users with augmented functionality and the ability to seamlessly interface with a broader ecosystem of interconnected devices and services.

3.1 Design of the Hardware – Structural and Functional Design

The hardware design of the controller centers on a hollow shaft BLDC motor, which, when combined with FOC motor control, provides users with a software-controlled knob that boasts boundless capabilities. A circular LCD mounted on the front end offers users visual feedback on ongoing actions being processed. Real-time tracking of the knob’s position is facilitated through the use of a magnetic encoder. Furthermore, the incorporation of a pressure sensor introduces a third dimension of interaction, enhancing the device’s versatility.

3.1.1 Circuit Design

The hardware configuration comprises three main modules: input, output, and control. In the input module, you’ll find a magnetic encoder that connects magnetically to the motor shaft, enabling precise monitoring of rotational position. Additionally, a pressure gauge and an ambient light sensor are part of this module. In the output module, a BLDC motor with its driving circuitry is employed to translate user input into mechanical motion for knob control and feedback. There’s also a round LCD on the front panel and a backlight RGB LED ring to provide visual feedback and cues. The control module, functioning as the central processor, manages the hardware components and provides Wi-Fi and Bluetooth connectivity for seamless communication with external devices (Fig. 3).

Fig. 3.

Controller hardware design

3.1.2 Structural Design

The structural design of the controller incorporates a hollow shaft BLDC (Brushless Direct Current) motor, serving as a central component. This motor also serves as the conduit for routing LCD support and communication wires. An interesting feature is the mounting of the device’s knob directly onto the motor’s rotor, granting it the capability to spin freely and without any constraints (Fig. 4).

Fig. 4.

Controller mechanical design

The controller’s base is made from a rectangular-shaped PCB (printed circuit board), providing a solid foundation. On top, there’s a centered hole that allows a ring LED to shine light, creating a backlight for the knob. This design not only looks good but also adds functionality. Additionally, the controller can be easily mounted on a wall, making it convenient to access and view from different angles. This feature gives users flexibility in where they can place it, so it fits their needs and preferences (Fig. 5).

Fig. 5.

Controller housing design

The LCD at the front is shielded by a durable watch glass, ensuring its surface remains free from scratches and damage, even after prolonged use. This protective layer not only enhances the longevity of the display but also guarantees a consistently clear and pristine viewing experience for users.

3.2 Software Solution for UI Interaction and Smart Home Deep Integration

The software employed in the controllers can be categorized into two distinct functional blocks: internal functionality control and cloud and IoT integration. The internal functionality control component assumes responsibility for various essential tasks, including closed-loop knob position control, management of LCD and LED operations, and the collection of data from the light sensor. The control of knob position is achieved through the utilization of a magnetic encoder, which provides precise angular position information. By effectively controlling the BLDC motor and accurately reading its angular position, we can attain a servo motor with highly precise angular positioning capabilities. Furthermore, the incorporation of field-oriented motor control (FOC) within the closed-loop system affords remarkable flexibility in terms of angle adjustment. This versatility in angle information and rotational features grants users the ability to establish a variety of knob detent settings. These include a wide range of detent counts, the automatic return-to-center detent following a completed action, unrestricted free-spinning without detents, and straightforward on/off detent positions.

The pressure gauge within the system is designed to detect the force applied to the knob. When coupled with brief motor actions, this functionality allows the system to emulate button presses, thereby expanding the range of control features available to the user (Fig. 6).

Fig. 6.

Closed-loop angular error correction

The connectivity of these devices plays a pivotal role in offering users an extensive spectrum of customization and control possibilities. The operational aspects of the system have been meticulously abstracted to ensure the utmost flexibility and expansiveness in terms of user options. By integrating the standard Wi-Fi protocol, the device facilitates seamless connectivity to home automation systems or cloud platforms, such as Home Assistant. This integration opens up a wealth of possibilities for communication and interaction with other IoT (Internet of Things) devices. Moreover, the inclusion of an integrated Bluetooth stack further amplifies the potential for communication with a diverse range of IoT devices, underpinning the device’s commitment to delivering comprehensive connectivity solutions to its users.

4 System Test and Validation

In this chapter a testing and validation of the presented tactile-haptic system is done. The system is analyzed in whole, and qualitative assessment is given. The detailed explanation how testing and validation methodology was done is prerequisite for help to interpret results in right manner so; at the beginning an introductory of testing and validation methods are explained detailly continuing onto testing and validation followed by results analysis. The assessment of the system-in-whole functionality and usability is given at the very end.

4.1 Test and Validation Methodology

Due to the complexity of the presented system where qualitative factor is favored regarding quantitative one, test and validation procedures have to be detailly explained to understand results properly. Qualitative assessment solely relies onto end-user experience and feel-of-use. Several use case scenarios are defined and applied onto group of selected users that faced developed system, to use it according test scenarios, and express their experience-of-use through descriptive assessment of the system functionality.

The tests and validation are conducted onto group of 17 people, gender-equal, into controlled environment that simulates a smart home where use-case scenarios are applied.

• Use-case scenario #1: Individual use

  In this scenario, the smart haptic knob would be used by individuals or families in their own homes to control various aspects of their home automation system. This could include controlling lights, temperature, music, and other smart devices. The haptic feedback would provide a more intuitive and satisfying user experience, making it easier for users to control their home environment without having to look at a screen.

• Use-case scenario #2: Group use

  In this scenario, the smart haptic knob would be used by multiple people in a shared space, such as an office or a coworking space. The knob could be used to control shared devices, such as lights, heating, cooling, and a sound system. The haptic feedback would help to prevent users from accidentally changing settings, and it would also provide a more engaging and collaborative user experience.

• Use-case scenario #3: Hotel use

  In this scenario, the smart haptic knob would be used by guests in hotel rooms to control various aspects of their room environment, such as lights, temperature, and curtains. The haptic feedback would provide a more luxurious and intuitive user experience, making it easier for guests to control their room environment without having to fumble with unfamiliar controls.

Results produce qualitative assessment of general satisfaction of the system usage with emphasis on strong key-point of the system, as well as on those that are unsatisfactory. A general table of pleasance-of-use and ease-of-use is given, that summarizes all results and validatory indicators.

4.2 Results and Analysis of the Test and Validation

The tests and validation procedures are done according statements laid out in previous chapter. Use-case scenarios are applied on simulated smart home environment and test group of end-users gave their opinion as follows;

• Use-case scenario #1: Individual use

  Participants were invited to envision their own homes equipped with the smart haptic knob, seamlessly integrated into their living spaces. They were guided through scenarios where the device effortlessly controlled various aspects of their home environment, from adjusting lights and temperature to managing music and other smart devices. Through these questions, participants were encouraged to assess their perceived ease of adopting such a system in their own homes.

  Their experience and system quality assessment are summarized in following table, Table 1. Figure 7 also, more clearly, depict general satisfaction of the end-users with the emphasis on users ease of adoption.

  Table 1. Quality assessment and experience in tactile-haptic controller use.

  Fig. 7.

  

  Quality and experience-of-use index – Individual use.

• Use-case scenario #2: Group use

  Participants were invited to envision shared spaces, such as offices or coworking environments, equipped with the smart haptic knob, seamlessly integrated into their collaborative settings. They were guided through scenarios where the device effortlessly controlled various aspects of the shared environment, from adjusting lights and temperature to managing music and other shared devices. Through these questions, participants were encouraged to assess their perceived ease of adopting such a system in their shared spaces.

  Their experience and system quality assessment are summarized in following table, Table 2. Figure 8 also, more clearly, depict general satisfaction of the end-users with the emphasis on users ease of adoption.

  Table 2. Quality assessment and experience in tactile-haptic controller use.

  Fig. 8.

  

  Quality and experience-of-use index – Group use.

• Use-case scenario #3: Hotel use

  Participants were invited to envision hotel rooms equipped with the smart haptic knob, seamlessly integrated into their temporary accommodations. They were guided through scenarios where the device effortlessly controlled various aspects of the hotel room environment, from adjusting lights and temperature to managing curtains and other room amenities. Through these questions, participants were encouraged to assess their perceived ease of adopting such a system in hotel rooms.

  Their experience and system quality assessment are summarized in following table, Table 3. Figure 9 also, more clearly, depict general satisfaction of the end-users with the emphasis on users ease of adoption.

  Table 3. Quality assessment and experience in tactile-haptic controller use.

  Fig. 9.

  

  Quality and experience-of-use index – Hotel use.

In general, the test group of users express its general satisfaction with the proposed system solution for tactile-haptic controller based on IoT principle with application in smart homes environment. Their experience and system quality assessment are summarized in following table, Table 4. Figure 10 also, more clearly, depict general satisfaction of the end-users.

Table 4. Generalized quality assessment and experience in tactile-haptic controller use.

Fig. 10.

Quality and experience-of-use index – Generalized on all use-case scenarios.

Generally, test group of users express their satisfaction with proposed concept of tactile-haptic controllers for use in smart home environments and adjacent applications. It is reasonable to conclude that there is inevitable need for such systems that give haptic feedback on systems state that enables user to feel and inadvertly sense system condition, same time giving new dimension od smart home experience through IoT based vast-connectivity to the services that integrates on a even higher level of home automation. With such systems users are put into new perspective where advanced technologies are employed for everyday use but with high level of comfort and, almost natural, feel and ease of use. This all confirm need and justification of further development and deeper integration with systems that are compatible for IoT principles and philosophy of smart living.

5 Discussion and Conclusion

As technology continues to advance and automation becomes more prevalent, we must strive to restore a balance between digital control and physical interaction. The introduction of haptic interfaces, such as the smart haptic knob, offers a promising solution to this challenge, allowing users to regain a sense of control in their increasingly digitized environments [10]. The positive feedback and enthusiasm from participants in user testing highlight the potential of haptic interfaces to improve user experience, promote intuitive control, and reintroduce a sense of physicality in a world dominated by virtual interactions. By seamlessly integrating these intuitive control mechanisms into smart devices and environments, we can create a more harmonious and engaging human-computer interaction model that caters to both the convenience of digital control and the innate human desire for tangible experiences, [11].

The resurgence of haptic interfaces marks a significant shift in user interaction, providing a promising solution to the growing disconnect between individuals and their physical surroundings. As we progress in this era of technological advancements, it is crucial to prioritize the development of intuitive and engaging interfaces that bridge the digital and physical worlds, enabling users to regain control and reconnect with their increasingly digitized world.

In addition to the growing demand for haptic interfaces in home automation, the potential applications extend far beyond the confines of personal residences. The versatility and intuitive nature of haptic control mechanisms make them well-suited for a diverse range of environments, from hotels and hospitality settings to industrial and commercial applications. The market for tactile haptic smart home controllers is still in its early stages of development, with immense potential for growth and innovation. As the technology continues to mature and the demand for intuitive, user-friendly interfaces increases, we can expect to see a proliferation of haptic control solutions across a wide spectrum of applications, transforming the way we interact with our increasingly digitized world.