Keywords

1 Introduction

Normally, a user interface provides a mechanism for individuals to access all the features and functionalities of their system. Without a user-oriented interface, these features and functionalities are typically inaccessible to the end user. The window-based user interface (also referred to as a graphical user interface or GUI), which is powerful for organizing the capabilities and resources available in mobile devices. It enables the user, incrementally explore and discover the capabilities of mobile devices like laptop and Mobile phone. It keeps everything in a convenient visual context, using helpful metaphors, like desktops and windows (see Fig. 1).

Fig. 1.
figure 1

Interface of Photoshop CS

Full size image

Unfortunately, current computer user interface and interaction have limited abilities to allow users to express themselves (Forlizzi and Battarbee 2004). Particularly, the GUI provides a menu hierarchy which is accessible via an instrument interaction (via a point device or mouse) the disadvantages for GUI is that the user is often has to repeat an endless selection cycles (using a mouse to click item) through a maze of GUI windows. As a result, the user must be able to work on a low level of interaction which means that the user is losing the ability to group a related series of basic actions into a higher level of interaction like effective communication in a semantic level (Beaudouin-Lafon 2004). Under this paradigm, there is no longer access or work with objects that are not visible or unknown to us. For instance, there are many tasks that a user must repeat it when using a GUI, such as opening certain files and activating certain controls. For such tasks, the GUI presents the user with a single logic set, implemented within the limited computer screen. In other words, current form of interactions which is to be used to carry out human computer interactions is imbalanced and incomplete. We need to rethink about the purpose, structure of interaction and also the users’ role. By investigating the human computer interaction, we can find out that the existing interfaces used in mobile equipment are mainly focusing on accessing users to complete a task or archive an object rather than supporting the user personalize his/her interactions. One example is the software entrepreneurs have developed “shortcut” utilities of various designs. Adobe Photoshop provides a shortcut to functions and processes such as opening applications and files, making menu selections, and performing multi-step operations. However, the user found it very difficult to remember all of key combination. In summary, over the years, a number of approaches and methods have been invented to deal with the problem of efficient interacting. Because of their inherent limitation, For example, instrument interactions are good at completing some fixed operation such as choosing items, dragging and moving file and so on but are unable to deal with user’s individual needs from the interaction. It is clear that the current trend to rely solely on window-based user interfaces, and object-oriented interaction design method has seriously constrained a user’s ability to utilize mobile device effectively.

In the following section, we provide a new type of interface that provides users with an efficient, convenient and natural way to create a personalized interface for mobile device (laptop) by utilizing human natural language. By doing so, the interface between the users and computers needs build up on a dynamic model, which represents individual user’s intentions and history in order to better serve the user’s needs and to eliminate unnecessary repetitive activities in an unobtrusive way.

2 Towards Personalized Interfaces

The Natural Language Processing based on an idea of concerning human communication as cooperative activity between participator (computer) and covers different aspects and levels of communication, applicable to various types of communicative activities (Allwood 1976, 2007; Allwood et al. 1992). It requires a user personalized interface to complete the conversation. To produce the user personalized interface designer need to consider how to create a conversation and make it more effective and sustainable. Mark’s theory of common place and language provides a basic pattern to construct a conversational interaction between human and electrical product (Monk 2009). It requires two essential premises: One is the computer interface need to shift from a static statement to a dynamic platform. That means interface should allow the user to perform any task at any time, irrespective of the application that is currently running. Second premise is the fundamental purpose of user’s interaction allows active user to understand the interaction and make the user able to organize the interface to improve their productivity by using computer. As every computer user has a unique pattern of interaction.

The user personalized interface is the result of conversational interaction between the user and the computer that can benefit the mobile device user from the following aspects: First, the personalized interface provides a way to generate effective human computer interaction by adopting the user’s individual characteristics. For example, the basic acticity (completing a task by following a predefined task flow) would been transformed into a semantically format (enter semantic interaction words) that are designed to allow users to gain access to understand their perspectives and support their specific needs rather than enforcing user to complete predefined tasks or objects. We argue that the human computer interaction should become to language-based activity, and the user can define it in their personal (experience), social (using environment) and cultural context. Second, personalized interface reduce the misunderstanding between the user and the computer by provides a dynamic conversational platform. In other words, the interface allows the user to interact with the computer in a personal way rather than being a static information board. The personalized interface makes the computer response more reasonable, on a system-wide basis, to the users’ input (when a user enters semantic interaction word) in any context (i.e. any level of application or operation system workplace).

Based on the above perspectives, via the domain specific interaction language and personalized interface the user can effectively control the operations of the computer.

3 Portable Drawing Application Prototype

Analogously to human-human communication, human computer interaction can also be viewed as enacting communicative acts by using a domain specific interaction language. By enhancing all of the richness and power inherent in a human language, we aim to establish a domain specific interaction language that allows the human computer interaction in a manner that is much closer to human’s natural way of interaction. In particular, a user is allowed to express his/her concept to control a particular system and build a special interface. The specific interface will be changed by the terminologies the user inputted expressing unique meanings to a specific interaction context and stage of the application.

In this section, we will create a portal drawing application prototype as an example to demonstrate how to allow a user to build a personalized interface by using natural language processing. The personalized interface is generated by using a domain specific interaction language, and the language specifies high-level user interactions in three levels: syntax level, semantic level and pragmatic level.

3.1 Syntax Level

In this level, we work on generation of the user interaction domain class together with their binding to the actual application functionality or usability by using program language (in this case I use Java and OWL). Interface is the structure of the visible (or audible) forms of language. The syntax rules or grammar of a language determine the basic elements (word, sound, etc.) and the ways in which they can be combined (according to functionality or usability).

For this reason, we make use of user linguistic input in interaction design specification to carry out textual interaction customization language and produce an interface. There are two types of interaction words: Script words and Action words. Script words are utterances that can be either ordinary content words or the function title, depending on the user’s intention in typing the word. Action words related to the action, which user want to operate with the system for a particular task or object. For example, to complete a task of drawing a line, a user can type a word “pen” to get pen pallet (or chose a pen from a tool bar) to draw a line; meanwhile the user allows to input a word “larger” in linguistic input window that will bring the pen size pallet on the workplace screen that access user choose a right size for the pen (see Fig. 2).

Fig. 2.
figure 2

Linguistic interaction bar

Full size image

3.2 Semantic Level

On semantic level, users may be able to explore the systematic relation between structures in a language and a space of potential meanings. It includes the definitions of individual elements of interface (icon, menu and function) and the meaning that is generated by combining them. In this stage, the end users are going to build an individual interactive semantic image, they need to go through internalization and an externalization processes. Different users may have different classification spaces, perceptive ability and purpose of interaction, so they may have different output for the same input. It is common, users may have different language competency to specify the interface.

The textual interaction customization language allows a user utilize a Semantic Interaction Word (SIW) to operate a computer in a highly efficient way. SIW allows a user to launch applications, navigate within applications and control application functions by using textual interaction customization language rather than dragging and clicking with a pointing device such as a mouse. The textual interaction customization language is personalized by each user. In addition, the textual interaction customization language also enables a new type of interaction including function utilize organization, information retrieval, and other services to be performed by using computer. When users go through SIW to complete a task, computer can save all information that is entered by the user, and the semantic structure of the user’s workflow together composes a textual interaction customization language (Fig. 3). The user can create a specific interface to match their unique interaction needs.

Fig. 3.
figure 3

A user personalized interface of portal drawing prototype

Full size image

3.3 Pragmatic Level

On this level, the user could systemize their interaction to complete their tasks or achieve a goal based on personal intention. The more confident the users are to modify the interface, the easier it is for the users to archive the goal they have. As a result, a concrete user semantic interface will be rendered based on how the user applying textual interaction customization language (see Fig. 4).

Fig. 4.
figure 4

The process of building a user personalized interface

Full size image

During the process of textual interaction, user personalized interface is guided by device profiles, application-specific style guides and the user’s particular requirements. The device profiles contain device specific constraints regarding the user interface capabilities like the screen size. The user’s preferences and application specific requirements guides influence the layout, selection and rendering of widgets.

4 Future Works

The future work focuses on building a Hi-Fi prototype to evaluate how the user personalized interface can optimize interaction between individual user and variety of portal devices. Moreover, we consider that the user personalized interface can be used in designing different domain specific portal device that can provide a more intuitive interaction in a manner that is simple, richer and natural than using traditional interaction design methods.

5 Conclusions

By building user personalized interface that enable us to produce most of interaction design work and making them available through natural language-based commands, and it can enhance portal device user’s productivity dramatically. Moreover, user personalized interface supports a more intuitive interaction than traditional interface by improving the standard graphical user interaction (GUI) in a manner that is simple, richer and natural.