Keywords

1 Introduction

In recent years, Affective Computing become a trending topic in Artificial Intelligence focusing, among others, on detection and extraction of affective information from different sources e.g., images, music, brain scans, text. Particularly studied is the analysis of text, written either in structured/semi-structured form or in natural language [27,28,29,30, 33]. Using Natural Language Processing (NLP) techniques, automated agents can gain the ability to process and analyze text at different levels of abstraction, exploiting the speed and computational power of modern systems.

Among various branches of Affective Computing, it is possible to identify, Sentiment Analysis (SA) and Emotion Recognition (ER):

  • Sentiment Analysis studies the polarity of a text, which can be positive, negative or neutral.

  • Emotion Recognition extracts and recognizes the emotional content, according to models of emotions accepted as State of the Art, e.g. the Ekman, Plutchik, and Lovheim models.

Several other models are present in literature and others will come, because the hype initiated at the end of last century didn’t still finish its effect on the scientific research [31]. Furthermore, the technical advent of new and affordable tools to measure aspects linked to emotional reactions lead to an interdisciplinary worldwide interest to Affective Computing and Emotion Recognition, so that both private and public research groups are studying emotions in different points of view. For our point of view focusing on textual information, different approaches can be followed to analyze text e.g., knowledge-based, statistical or hybrid approaches. Nowadays, companies already invest a lot of resources to fructify information on the sentiment expressed towards a target, e.g. political subjects, brands, a product with its associated reviews; the efficiency of a marketing campaign depends largely on the ability to detect ongoing trends and to foresee future customers’ needs.

1.1 Affective computing for marketing applications

Recommender Systems (RS) can use affective computing to implement and augment knowledge about the sentiment/opinion towards a product or a brand, helping sellers to automatically tailor their recommendations to users with a user-centered approach for a better personalization. Along with commercial applications, researchers are applying Affective Computing also to spam detection [23], and to build human-computer interaction systems based on natural language, both spoken [5, 32] and written [13, 14, 18,19,20,21, 26].

Another application of Emotion Recognition (ER) that is already on the market, is used to detect in real time the interest, involvement and emotions of a customer through video-based techniques. A simple classification of visual features captured with a camera, recognized and tracked while a user is looking, hearing or trying a product, gives sellers immediate feedback about their products and the connected customers’ feelings.

1.2 Novel Applications

Novels applications include to use knowledge of emotions as prosthesis knowledge for tutoring, or auto-evaluation of teachers and leaders, who can analyze their facial or written or spoken expressions in order to be aware of emotions and manage them as possible, with the aim of improving their social skills and avoid human misunderstanding of tone. For example the SEMO model of Semantic Emotion Recognition [1, 13] allow to detect emotions from text and thus to eventually correct it before sending it via email or posting on social networks [22].

1.3 A Translation-Based Approach

In this work, a translation-based approach to emotion recognition in texts written in “secondary” languages is presented, and the feasibility of such approach is discussed. For this purpose, we present a prototypical system using IBM Watson to extract emotions from Italian text by means of Bluemix Alchemy Language. Some preliminary results are shown and discussed in order to stress pros and cons of the approach. In the first section of this paper an overview of the software tools and platforms that perform our approach is presented. A detailed breakdown of the IBM Alchemy Language API is given, describing each function related to a complete process of emotion recognition and extraction. The second section shows an example of usage of the Sentiment and Emotion APIs, in conjunction with the Language translator module, used here to translate terms from Italian to English. A final discussion concludes the paper.

2 Platform and Tools

2.1 IBM Watson

IBM Watson is an AI system capable of answering questions posed in natural language. Developed within the DEEPQA IBM project, it improves the capability of artificial intelligences in understanding questions posed in natural language, providing relevant answers. Watson was firstly shown in public in 2011, when it competed against two human champions in the TV quiz Jeopardy!. The stress was put on the ability of the AI to understand questions and provide exact answers with speed comparable to humans: Watson, in fact, won the first prize in the game.

IBM engineers then focused on using Watson as a clinical decision support system, in partnership with the Memorial Sloan-Kettering Cancer Center, New York. The goal was to enable medical personnel to supply the system with a patient’s clinical situation, including symptoms and risk factors, and get a list of personalized recommendations, ranked by a confidence index. Tests shown that for particular applications (e.g. lung cancer diagnosis) Watson could dramatically improve the time needed for diagnosis and give relevant suggestions. More generally, the key feature of Watson in the medical field is the ability to process the huge amount of knowledge produced in medical literature on a daily basis, at a speed impossible for humans. Diagnoses can then be performed taking into account the whole literature, possibly leading to improved results, which will then be analyzed and used by human personnel.

After the initial presentation of Watson to the public, a variety of Artificial Intelligence applications have been introduced in the service, using machine learning techniques for rapid exploration of an enormous source of linked data [6, 7, 15]. Knowledge in several fields was enhanced, ranging from NLP to personality insights for annotating user profiles basing on the content of their messages, emails and tweets. In a future Internet-of-Things (IoT) vision, General Motors is planning to use Watson to improve location-aware services in their cars, and the Imperial College in London is focusing on crime prediction using Watson for pattern analysis. We are showing here how Watson can be used also for emotion recognition, in a simple application on text data [8, 17].

2.2 IBM Bluemix

IBM Watson tools are available on Bluemix, the IBM cloud computing PaaS (Platform-as-a-Service). Bluemix serves both as an infrastructure, offering the possibility to rent computing power, network appliances and storage space, and as a software platform, with runtime applications developed in different languages and technologies. Bluemix offers access to various services too; a list of all the available services can be found on the Bluemix website (an IBM account is needed).

2.3 Watson Developer Cloud

IBM Watson Developer Cloud offers services for helping users to build cognitive applications; it is the access point for developers to the Watson API, and the basis for the service of Affective Computing. A plethora of services are available on the platform, covering five main areas:

  1. 1.

    Language, recognizing 97 spoken languages;

  2. 2.

    Speech, with NLP functions;

  3. 3.

    Vision, with image recognition;

  4. 4.

    Data Insights for data analysis;

  5. 5.

    Embodied Cognition, including sentiment analysis and alchemy language on linked data.

It is possible to combine functions and libraries belonging to different areas, in order to build complex multimedia applications.

2.4 AlchemyLanguage

AlchemyLanguage is a set of services to build complex applications for textual NLP, with a high level of abstraction used in several applications of emotion recognition (i.e. see [16]. This service is part of the Language area in the Watson Developer Cloud. AlchemyLanguage is composed of two main modules:

  1. 1.

    AlchemyLanguage functions: a set of text analysis functions to extract semantic information from input text terms;

  2. 2.

    AlchemyData: a database indexing about 300 thousand news and blog articles everyday, in English. Users can create custom NLP queries that will can performed directly on the cloud database, without the need of heavy calculation on local client machines. Results will include the indexed articles, matching the query constraints and enriched with NLP information.

2.5 Sentiment Analysis and Emotion Recognition

Entity Recognition. The Entities function extracts entities from text. The user inputs a text, HTML document or URL pointing to a web page, and Watson returns a list of entities, using complex statistical algorithms and NLP technology in the form of a combination of multilingual support, linked data, context-sensitive entity disambiguation, comprehensive type support and quotations extraction. The API supports custom models that can be trained using Watson Developer Studio. Results can include additional content, e.g.:

  • linked data (active by default);

  • entity-level sentiment;

  • emotions associated to detected entities.

Supported languages for the function are: English, French, German, Italian, Portuguese, Russian, Spanish, and Swedish.

Sentiment on Document. The Sentiment function analyses the overall sentiment in a given text. The user inputs some text content, HTML documents or URL pointing to the information to analyze, and the system returns the associated sentiment, in the form of a triple:

  1. 1.

    a flag Mixed, with 1 meaning that the sentiment is not polarized (i.e. both positive and negative), and 0 meaning that the sentiment is polarized;

  2. 2.

    a weight in the \([-1;1]\) range representing the sentiment, with 0 meaning Neutral;

  3. 3.

    a polarity value among: positive, negative or neutral.

Supported languages are: English, French, Italian, German, Portuguese, Russian, and Spanish.

Sentiment on Specific Targets. The Targeted Sentiment function works like the precious, but at a different level of abstraction, associating a sentiment to user-specified targets, e.g. a product or a brand. Furthermore, other items can be analyzed as part of other calls, e.g. quotations, entities (Entities call), and keywords (Keywords call). Thanks to the strong NLP abilities of Watson, target options include also relational and directional functions (Relations call). Supported languages for these functions are: Arabic, English, French, German, Italian, Portuguese, Russian, and Spanish.

3 Experimental Use Case: Assigning Emotional Tags to Book Blurbs in Italian Language

In a previous work, the authors of this work tested if it is possible to classify books using emotional by means of classifying terms extracted not from the whole book, but from its presentation blurb (usually in the fourth page of the edition). Experiments has been carried on books written or translated in Italian language, and the final emotions has been compared to the emotional tags given by users of the social network Zazie. Zazie is an Italian product giving to users a place where to review the books they read, assigning them reviews and emotional tags to discover, from collective information, new books to choose, based on their capacity to raise emotions (e.g. happy, sad) or to be read while in particular moods (e.g. relaxed on the coach, distracted on the beach). In [12, 14] the authors tested the possibility of automatic classification of books associating to the words of blurbs to emotional tags extracted from WordNet Affect, which is part of the WordNet ontology. In this work, we implement a simple experimentation of what the IBM Alchemy API can do for the same task, using a simple example from the Zazie data set. In [9, 11] the Wikipedia semantic network is navigated by using a trace based approach [25] similar to ACO algorithms [2,3,4, 24] and several web-based proximity measures [10], it is worth investigating in future work if the same techniques and proximity measures can be used in the framework of emotion classification.

3.1 Experiment Design

As shown in the previous section, Alchemy APIs provide the user with methods to assign a sentiment/emotion tag. While sentiment can be extracted in a number of languages, including Italian, emotions can be instead recognized only in English. A translation from Italian to English will be therefore exploited using the Language Translator module as a preliminary action, then the result will be used with the Emotion Analysis module. As an example, we will carry out the Italian text of the blurb from the book Il turno di notte lo fanno le stelle by the author Erri De Luca:

figure a

3.2 NLP Preprocessing Phase

The Relations function extracts triplets (Subject, Action, Object) expressing relations. It is one of the most advanced and interesting tools available in the AlchemyLanguage API, and well demonstrates the power of Watson Data Mining AI in the field of NLP. The user inputs some text content, HTML documents or URLs pointing to the information to analyze, and the system returns a list of sentences along with their elements, performing the following steps, for which we provide a description including a tag “charged” if they are due to an additional payment in the IBM system:

  1. 1.

    Sentence Parsing: parses sentences into subject, action, object and location.

  2. 2.

    Keyword extraction (charged): extracts keywords information

  3. 3.

    Entity extraction (charged): extracts entities information.

  4. 4.

    Sentiment Analysis (charged): calculates sentiment at a relational level for subjects, objects and locations, and directional level for actions.

  5. 5.

    Verb Normalization: lemmatizes verbs, identifies the tense and detects an eventual negation.

By default, linked data content links are included in the response; users can optionally hide them.

Supported languages are English and Spanish. Italian is not supported, thus our application will require a preliminary translation.

3.3 Results on the Sentiment Analysis Task

In this section a Sentiment tag is assigned to the blurb, among positive, negative and neutral.

The following script queries the Sentiment Analysis API and prints the result, in the form of a JSON document, on screen.

figure b

The following JSON-style document shows the result of the call.

figure c

Results include some control information which is common to each API call, i.e.:

  • usage field, containing a link to the terms of usage of AlchemyAPI;

  • status field, indicating whether the request was successful or not;

  • totalTransactions field, indicating how many API calls were necessary to get all the required information. May be higher than one, for example in combined calls.

The sentiment information is embedded in the docSentiment object, which is composed of three elements:

  • A “Mixed” flag, with 1 meaning that the sentiment is both positive and negative, and 0 meaning that the sentiment is polarized

  • A weight in the \([-1;1]\) range representing the sentiment strength, with 0 meaning Neutral

  • A Sentiment tag that can be Positive, Negative or Neutral.

In our case, the API reported a Positive polarity for the text, with a score of 0.467522 and no mixed sentiment.

Note: the mixed attribute is not included in the result when its value is 0. We decided to include it for clarity purposes.

3.4 Translation and Emotion Analysis

In this section we will show the second step of our example, translating the blurb to the English language and calculating emotional scores.

The following script first queries the Language Translator service, prints the result on screen and uses it as input to the Emotion Analysis API.

figure d

The translation performed by the Language Translator service:

figure e

The JSON-style document in listing 2 shows the result of the call for Emotion Analysis. The emotion information is embedded in the docEmotions object, which contains five emotion-score pairs, as described in Chap. 2, section Emotion Analysis. In our case, the API reports joy as the prevalent emotion in the text, followed in turn by fear.

4 Results Discussion and Conclusion

In this work an overview of the AlchemyLanguage API was given and an example use case on the Zazie data set was shown. The main goal of this work was to check if IBM Watson Developer Cloud Services can be used for research purposes in the area of Emotion Recognition and Sentiment Analysis, both as ways to discover new information and ease processes and as benchmarks to compare against laboratory results. The answer is positive, although with some caveats:

figure f

  • First, even if IBM allows to launch the computation on their dedicated cloud platform, it could prove expensive in terms of time: there is a free daily/monthly quota for AlchemyLanguage, depending on the different methods for Language Translator; other services, which we only mentioned, have their own subscription plans. Those amounts are enough for small testing, but working on Big Data may require planning to optimize time: it was calculated that to perform Sentiment Analysis on all the blurbs included in the Zazie dataset without incurring in any charge approximately 10 days are needed, with the same time required for Emotion Analysis.

  • Building and using custom models for specific needs requires a consistent effort in terms of money. The free one time quota is 250.000 characters.

  • The quality of translation still needs to be improved. In the previously shown example, the system was not able to translate some words, which were left in the original language. This could lead to less precise results, especially in the case when the words

Nevertheless, IBM AlchemyLanguage, even in the limited free version, represents a good and easy way to access tools at the state of art of Natural Language Processing. It would be interesting to analyze all the services on Watson Developer Cloud and their applications, to test not only text-related but Image Recognition and Speech Recognition ones, to perform Affective Computing and Emotion Recognition from images and audio directly on the IBM cloud.