Android Robots as In-between Beings

Abstract

The Geminoid is an android robot based on an existing person and it can act as an avatar of the original person using a teleoperation system. The Telenoid is another android which is characterized by implementing a minimal design representation of a human. By this design, the Telenoid allows people to feel as if a spatially distant acquaintance is close-by. We created two artworks with the Geminoid through collaborations with artists. Firstly, we conceived the Android Theater. In Android Theatre human actors and androids shared the stage in a first play of its kind worldwide. The second work is an “Intelligent Mannequin”. Here the Geminoid was interacting with the visitors in a department store as an interactive mannequin. In this chapter, we give an overview of the Geminoid and the Telenoid, describing its appearance, teleoperation system and the concept of Android Science. We then focus on the artworks.

These keywords were added by machine and not by the authors. This process is experimental and the keywords may be updated as the learning algorithm improves.

Android Technology and Science

In this section, we describe the android technology of the Geminoid and the Telenoid including their control systems. We then outline the concept of Android Science.

Overview of a Geminoid

Why do we feel another person’s presence? How can this presence be captured, revived, and transmitted? To tackle these mysteries, we have developed a new artificial being, Geminoid. The word “Geminoid” comes from the Latin geminus meaning “twin” or “double” and postfix “oids” which means “similarity”. As the name suggests, the Geminoid is a robot that will work as a duplicate of an existing person. Because they are closely connected by network and sensor technology, the Geminoid not only appears but also behaves just like its source person.

Geminoid belongs to a new category of robots, which were originally planned to be test-beds for studying the individual nature of human beings. Whilst humanoid robots are good for studying the effectiveness of having a human-like body, and androids are used for seeking the general nature of humans, studies using Geminoid focus on investigating the nature of individuality. Geminoids allow us to examine personal aspects, such as presence or personality traits, tracing their origins and implementing them into robots. Differences among people enable us to distinguish individuals and they emerge from complex combinations of various elements, such as appearance, facial expression, or ways of speaking. We intuitively know this from our daily experience, but until now scientific ways to examine this complex interplay have been rather limited. By using Geminoid, we can systematically investigate the essentials of what makes a person an individual.

The first Geminoid prototype HI-1, created in 2006, was modeled on Dr. Hiroshi Ishiguro, Professor of Osaka University and ATR (Fig. 1). Since then numerous studies have been performed. Research with Geminoid takes two approaches: The first one follows the engineering approach that focus on aspect such as the development of an effective teleoperation interface and the generation of natural human-like motion. The second follows the cognitive modeling approach to study aspects of human nature, such as “human presence”. These two approaches in combination will eventually lead to both advanced robots that closely resemble humans and new insights on human nature.

Fig. 1

Android robot, Geminoid HI-1 (left Geminoid HI-2, right Creator of Geminoid)

Appearance of Geminoid

The appearance of Geminoid is based on an existing person and does not depend on the imagination of designers. Currently, two factors are considered: how Geminoid looks and how Geminoid moves. Similarity to the original person can be measured by comparing these two factors with those of the original. Also the existence of a real person analogous to the robot enables us to easily perform comparison studies. As HI-1 presented here is modeled after a researcher, we even have access to the source person’s most personal subjective impressions. These insights are especially important at the very first stage of a new field of study.

In creating the first Geminoid prototype HI-1, efforts were concentrated on making a robot that appears not just to resemble a living person, but also to be a copy of the original person. Silicone skin was molded using a cast taken from the original person; shape adjustments and skin textures were painted manually based on magnetic resonance imaging scans and photographs. Fifty pneumatic, i.e. air-pressure driven, actuators let the robot generate smooth and quiet movements, which are important attributes when interacting with humans. The allocation of actuators was used so that the resulting robot can effectively show the necessary movements for human interaction and also allow for the recreation of the original person’s personality traits. Of the 50 actuators, 13 are embedded in the face, 15 in the torso, and the remaining 22 move the arms and legs. The softness of the silicone skin and the compliant nature of the pneumatic actuators also provide safely while interacting with humans.

Teleoperation

So far several androids have been developed. Although these androids enabled us to conduct a variety of cognitive experiments, their functionality was still quite limited. The bottleneck in interaction with humans is an android’s inability to perform long-term conversation. Robots equipped with artificial intelligence cannot yet perform at a level comparable to that of adult humans and still respond in a simple manner. This heavily constrains research on human-robot interaction. Thus, our solution to this problem lies in combining androids with teleoperation technology. Using teleoperation we can immediately start researching and implementing high-level human interaction, shedding light on mysteries such as human presence (Fig. 2).

Fig. 2

The tele-operation console of Geminoid HI-2

The teleoperation system with which every Geminoid is equipped also allows us to tackle a more philosophical question: whether a human’s “mind” is separable from his or her “body”. In Geminoids, the operator (mind) can easily be exchanged, while the robot (body) remains the same. In addition, the strength of connection—that is how much information of which kind is exchanged between Geminoid’s body and an operator’s mind—can easily be reconfigured. This is especially important when taking a top-down approach that adds or deletes elements from a person to discover the “critical” elements that constitute a human’s character. Before the era of Geminoid, this research methodology was impossible.

Some operator movements are captured, converted and transmitted to drive Geminoid. Therese includes, for example, lip motions while speaking and head movements while looking around. The operator can also explicitly send commands for controlling android behavior using a simple graphical user interface. Several selected movements, such as nodding, opposing, or staring in a certain direction, can be triggered with a single mouse click. This relatively simple interface was used because the robot has 50 degrees of freedom, which make it one of the world’s most complex robots. This huge amount of actuators cannot be manipulated manually in real time. Thus, a simple, intuitive interface was conceived so that the operator can concentrate on the interaction itself and does not have to think much about how to drive the androids’ behavior. Despite its simplicity this interface enables the operator to generate natural humanlike motions for the robot, with the help of Geminoid management system.

The teleoperation system also maintains the state of interaction and generates autonomous movements for the robot, which are driven unconsciously in humans. With a robot’s appearance nearly matching that of a human, its behavior should also become suitably sophisticated to retain a “natural” look. A human never stops breathing or eye blinking, because these easily observable kinds of behavior are driven unconsciously by the autonomic nervous system. Most robots, however, lack these movements. Thus, to increase Geminoid’s naturalness, Geminoid management system emulates a human’s autonomic nervous system by automatically generating these micro-movements, depending on the state of interaction. When the android is “speaking” its micro-movements are different from those triggered when it is “listening” to others. These automatic robot motions, generated without an operator’s explicit orders, are merged with explicit operation commands from the teleoperation interface.

Telenoid

Humans cannot recognize others based on only one picture. We change clothes everyday, make our face up in a morning, hair grows day by day, and the face changes during the day. One picture does not represent the person. We humans, therefore, create the images of others by imagination. Imagination is also an important ability in communication. Language is an incomplete way to understand each other. We cannot transfer everything that we think through language. However we can feel as if we understand each other because imagination fills the missing information.

We expected that room for interpretation might maximize human imagination and that this can be applied to android design.

The Telenoid was designed to appear and to behave as a minimalistic human; at very first glance, one can easily recognize the Telenoid as a human while on the other hand the Telenoid appears to be both male and female, both old and young (Fig. 3). The Telenoid has 9 degrees of freedom (3 for the eyeballs, 1 for the mouth, 3 for the neck and 2 for the arms for giving a hug) and it is controlled by teleoperator using the same system as in the Geminoid. By this design, Telenoid allows people to feel as if a spatially distant acquaintance is close-by. In other words, the Telenoid’s minimal design maximizes the imagination of the person talking through the Telenoid. Moreover, the Telenoid’s soft and pleasant skin texture and the small body size (approx. 50 cm) allow one to enjoy hugging and having intimate communications with it.

Fig. 3

Android robot, Telenoid

In fact, some elderlies start weeping when they talked with someone through the Telenoid. They said, “He was very kind to me like my true family” or “He must be a best friend of mine”. This implies that, basically, the imagination works in a positive direction. In other words, the Telenoid’s minimal design generates room for interpretation, and then the user’s imagination fills in details and creates a good communication experience.

Android Science

If we could build an android that is very similar to a human, how can we distinguish a real human from an android? The answer is not trivial. While interacting with androids, we cannot see their internal mechanisms and thus we may simply believe that they are human.

We propose to use androids that behave similarly to humans for studying what it essentially means to “be human”, i.e. the mystery of human nature. Androids and Geminoids are artificial humans that allow us to investigate human nature by means of psychological and cognitive tests, which we conduct during interaction with people. This new approach for understanding humans is called Android Science.

Current robotics research builds upon the field of cognitive science, especially in the area of human-robot interaction. Robotics researchers try to adopt mechanisms underlying successful human-human interaction to create robots that people can easily communicate with. At the same time, cognitive scientists have begun to utilize robots. As the scientific understanding of complex, higher-level human functions steadily increases, expectations will rise for robots to function as easily controlled machines with communicative ability. However, the contribution from robotics to cognitive science has not been adequate because the appearance and behavior of current robots cannot be separately handled. Since traditional robots look quite mechanical and very different from human beings, their appearance strongly influences a human’s expectations. As a result, researchers cannot clarify whether a specific finding reflects the robot’s appearance, its movement, or a combination of both.

We expect to solve this problem using androids, which closely resemble humans in their appearance and behavior. To achieve this goal, an objective, quantitative means to measure the effect of appearance is required, which forms part of our research endeavor.

In summary, the motivation of Android Science is twofold: On the one hand, a major robotics issue in the construction of androids is the development of humanlike appearance, movements, and perception functions. On the other hand, cognitive scientists are aiming to gain insights into the processes leading to “conscious and unconscious recognition.” The goal of android science is to realize a humanlike robot and to find the essential factors for representing human likeness. How can we define human likeness? Further, how do we perceive human likeness? It is commonly assumed that humans have conscious and unconscious recognition. When we observe others, various brain areas are activated. Each of them matches sensory input with human models, thereby modulating our response behaviors. These unconscious processes let us, for example, treat an android as if it were a human partner in conversation, although we consciously recognize it as what it is: a robotic system with very humanlike appearance. This is a fundamental issue for both engineering and scientific approaches. It will be an evaluation criterion in android development and helpful for understanding the mechanisms of human brains that make us social and emotional creatures.

Android Theatre, “Good-Bye”

Can androids become more human than humans, if only for a split second, if they look, move and talk like real people? What does it mean to be human, if human beings feel that androids are as human as themselves? These questions are what we have on the “robot theater project”.

As we described above, since artificial intelligence technology has still not reached the level of human behavior, robots can only respond in quite a simple manner. This was a major obstacle in conducting research on human-robot interaction. With Geminoid’s teleoperation system, it is possible to avoid this problem, and conduct various kinds of research on the implementation of high-level human interaction, including the study of human presence.

Research using Geminoids follows two approaches. One is the engineering approach, such as the development of effective tele-operation interfaces or the generation of natural, human-like motion. The other approach focuses on cognitive aspects, investigating the sense of human presence. Through these two approaches we aim to create an advanced robot that is very similar to humans, and, at the same time, to discover the essence of human nature.

The collaborator, Oriza Hirata, and we have been co-developing a robot-human theater project, which combines theater with our research on the cohabitation of humans and robots. The creation process and presentation of the research data fuse to make the performance a groundbreaking collaboration of engineering, science and theater.

As an aspect of engineering, for example, this collaboration has the potential to make robots more natural and human-like. Androids can induce familiar feelings in humans because of their human-like appearance. However, android’s appearances may induce a negative feeling [1]. The unnatural sensation of interacting with humanoid robots is caused by tiny differences between androids and humans. In fact, 76 % of subjects cannot distinguish an android from a human after watching for less than two seconds [2]. Therefore, the negative feeling is induced after long-term exposure to an android robot and remains a central barrier to comfortable human-android interactions. According to several studies, harmony between a robot’s appearance and behavior alleviates the negative feeling in observers [3]. This consideration has led to some successfully implemented android behaviors that do not evoke the negative feeling [4, 5]. However, a methodology for building an android robot that is perceived as a human-like entity after long-term exposure has yet to be established. To tackle this issue, the robot theatre project, in which robot and humans act in a long-lasting stage production it is important to know how we can achieve the android which doesn’t cause uncanny feelings.

The history of stage art is replete with implicit knowledge for directing actors on the stage. Therefore, by collaborating with stage directors, we can acquire useful knowledge for humanizing a robot. Moreover, creating a stage play and presenting public performances enables large-scale evaluation of audiences’ impressions toward acting robots.

Stage plays are universal culture in all over the world from ancient ages. A large number of art works have been produced based on professional technique of stage directors and actors. A director focuses on a representing human behavior for improving his stage plays. Therefore stage directors may have important knowledge for developing robots representing human behavior.

Oriza Hirata, is a widely esteemed as a playwright and stage director, and has advocated what he calls the “Contemporary Colloquial Theatre Theory (CCTT)” [6]. Since CCTT replicates on stage the reality of everyday human activities, it is potentially applicable to designing robots with human behaviors. CCTT advocates precise, rather than ambiguous, instructions for actors. Actors are instructed when to alter their physical actions, such as utterances and body orientations. Such precise instructions are expected to be directly applicable to android robots. Therefore, creating the stage play with android based on the CCTT, it might be helpful for developing more human-like android.

Premiered in 2010, the android theater play “Good-bye” shows an android and a person communicating with each other at an unprecedented level (Fig. 4). This short piece is the latest achievement of our collaboration, which started in 2008. We, researcher, and the Oriza Hirata, an artist, have been working together on this project to present a rendition of human-robot interaction in the near future: robot and humans acting, talking and communicating with one another naturally.

Fig. 4

Android Theater, “Good-bye” (left Geminoid F, right human actor)

In “Good-bye”, Geminoid F (Fig. 5), a female version of Geminoid, is cast as an android calmly reading poems to a dying girl, played by a human actress. Their quiet conversation casts profound questions such as “What is life/death for a robot/human?” and “What does it mean to be a human/robot?”

Fig. 5

Android robot, Geminoid F (left Geminoid F, right original person)

The robot theater project does not seek to amaze people with advanced robots as shown at expositions. The aim is to show the presence of robots and how they interact with humans on stage, to provoke the audience to reflect about what it means to be human.

It is also a social experiment for robotics to know the cultural differences of how people perceive long-term exposure to an android. In fact, this performance was held in many countries such as Japan, China, Thailand, Austria, Germany, France, Australia, US, and so on. We asked the audience about their impression toward the android. The results are very important to capture different stereotypes of androids across the world.

Intelligent Mannequin

By creating the android theater, the teleoperated android could be perceived as a natural existence on the stage. As a next trial, we tried to create an autonomous android in a real world scenario that is “Intelligent Mannequin” (Fig. 6).

Fig. 6

Geminoid F inside the show window

A mannequin is an ordinary and familiar thing for us. We can easily find it everywhere in town. On the other hand, a mannequin is sometimes an uncanny. There is a dissonance between its human-like appearance and non-human-like communication ability. We, humans are forced to read the communication ability of an object from its appearance. A mannequin cannot move its body even though it has human-like appearance. Humans sometimes feel uncanniness toward an object, if its appearance does not meet our expectation of its communication ability. This is well known as the effect of the “uncanny valley”. The effect of the uncanny valley implies that if an object’s appearance becomes similar to humans beyond a certain point, humans suddenly experience a feeling of uncanniness. The uncanny valley is named so because the line representing how natural an object is draws a valley on the graph with the ordinate axis for naturalness and the abscissas axis for human-likeness.

A mannequin is used for advertising human’s outfits or accessories. It makes sense to use a mannequin for this purpose because these goods are designed for humans and the mannequin’s human-like appearance induces customers to imagine themselves with these goods. Why then not use actual humans instead of mannequins? Why is a mannequin used, despite being sometimes associated with uncannyness? One of the reasons is that there is an ethical resistance against using humans. We feel disgusted when a human is treated as if it were just an object. By contrast, androids that we have developed for this project, can talk and behave like humans by using several sensors, and yet androids are not recognized as humans—they are on a boundary between humans and objects. That’s why androids can play the role of an intelligent and human-like mannequin that does not stir up ethical resistance. This means that an android can play a special role in being an alternative entity to human and mannequin, because of its intermediate presence.

This experiment was a preliminary trial. Therefore we did not measure or survey visitor responses. However, we are sure that the visitors enjoyed the interaction with the android and they considered its existence natural. This suggests that the android could autonomously play the role of an intelligent mannequin.

Conclusion

Humans have envisioned autonomous machines for a long time. Numerous scientists and engineers have dreamt of building a machine that behaves and thinks autonomously. It is still a big challenge to pass the “Total Turing Test”. However, we believe that the development of android technology, even if slow, gives us a chance to meet this challenge.