Interactive Evaluation of Pragmatic Features in Spoken Journalistic Texts

Abstract

The designed annotation tool intends to facilitate the evaluation of pragmatic features in spoken political and journalistic texts, in particular, interviews, live conversations in the Media and discussions in Parliament. The evaluation of pragmatic features focuses in the discourse component of spoken political and journalistic texts, in addition to implied information and connotative features. The present tool may be used by professional journalists and for training purposes, for journalists and other professionals.

1 Introduction

The designed interactive annotation tool targets to function as an aid to journalists and other professionals, intending to facilitate the evaluation of pragmatic features in spoken political and journalistic texts. The proposed tool may be used to process and to evaluate spoken texts such as interviews, live conversations in the Media, as well as discussions in Parliament.

The evaluation of pragmatic features focuses in the discourse component of spoken political and journalistic texts, in addition to implied information and connotative features, available in monolingual as well as in multilingual contexts. The design of the present tool is based on data and observations provided by professional journalists.

Transcriptions from two-party or multiple party discussions of spoken journalistic texts constitute the basis of the present data. These transcriptions, performed by professional journalists, are also compared to older data transcribed from spoken journalistic texts in European Union projects. The collected data included transcriptions from projects of graduate courses for journalists, in particular, the Program M.A in Quality Journalism and Digital Technologies, Danube University at Krems, Athena- Research and Innovation Center in Information, Communication and Knowledge Technologies, Athens - Institution of Promotion of Journalism Ath.Vas. Botsi, Athens.

Transcription and linguistic analysis was performed by two groups of 18–20 professional journalists for the academic years 2012–2013 and 2013–2014. Each journalist was assigned the evaluation of 4 interviews and performed a transcription of 15–20 min of each discussion, two in English (or any other natural language, for example, German) and two in Greek. Transcriptions from the following news broadcast networks were included in the files processed: BBC World News, CNN, AL Jazeera, Russia Today (RT) (“Crosstalk”), ZDF (Germany), ARD (Germany), available transcriptions from the German Parliament and Greek TV channels (ERT-NERIT (public television), the Parliament Channel (“Vouli”), Mega TV, SKAI TV, ALPHA). The speakers participating in the interviews or discussions were native speakers of British English or American English, German (Standard or a Dialect) and Modern Greek. English-speaking participants from countries as varied as India, Russia, Eastern Europe, the Middle East (Arab-speaking countries and Israel), Iran, Pakistan, P.R. China, Kenya and Nigeria were among the international speakers in the transcribed data.

Specifically, the professional journalists were assigned the task to transcribe interviews and two-party or multiparty discussions from national and international news networks, recording content, style and linguistic features, time assigned to each participant, turn-taking, interruptions, as well as gestures and other paralinguistic features. At the end of the analysis, the journalists provided an outline of the discourse structure of the interviews and the two-party or multiparty discussions.

2 Design and User Interaction

2.1 Overview and Design

The designed annotation tool targets (1) to provide the User-Journalist with the tracked indications of the topics handled in the interview or discussion and (2) to view the graphic pattern of the discourse structure of the interview or discussion, (3) to evaluate the discourse structure, (4) to allow the User to compare the discourse structure of conversations and interviews with similar topics or the same participants /participant and (5) to indicate the largest possible percentage of the points in the texts signalizing information with implied information and connotative features.

The interface of the annotation tool is designed to (a) to track the “local” topic discussed in a given segment of an interview or discussion or change of “local” topic in an interview or discussion and (b) annotate and highlight all the points possibly containing connotative features information, alerting the User to evaluate the parts of the text containing these expressions.

The designed tool allows the tracking of any change of topic or the same or a similar answer, as well as associations and generalizations related to the same topic. Since processing speed and the option of re-usability in multiple languages of the written and spoken political and journalistic texts constitutes a basic target of the proposed approach, strategies typically employed in the construction of Spoken Dialog Systems such as keyword processing in the form of topic detection are adapted in the present design.

Incoming texts to be processed constitute transcribed data from journalistic texts. The interactive annotation tool is designed to operate with most commercial transcription tools, some of which are available online. The designed tool may also be adapted to downloaded written texts from the internet (blogs, pdfs etc.) or scanned files from newspaper articles.

2.2 User Interaction

The steps in the interaction are initiated by the User activating the annotation tool when viewing journalistic texts online. The online journalistic texts are scanned by the proposed tool and the User is presented with an output comprising the online journalistic text with all the tracked topics, a graph of the text structure, as well as the instances of “marked” information with implied information and connotative features.

User interaction involves two fully automatic processes, one process assisted by System and one fully User-interactive process.

To help the User with the choice of local topics as a first step in the interaction, the indication of candidate topics (SELECT-TOPIC Process) is a process assisted by the System. The indication of the local topics (LOCAL-TOPIC Process) at each point in the interview or conversation is a fully User-interactive process, constituting the second step in the interaction. The SELECT-TOPIC Process is activated with the “Select Topic” command. The LOCAL-TOPIC Process may be divided into two stages, corresponding to the activation of two respective commands, “Identify Topic” and “Identify Relation” (Fig. 1).

Fig. 1.

Processes activated and respective commands

The list of the topics identified and tracked in the discussion or interview is presented to the User in chronological order.

The generation of the discourse structure (GEN-GRAPH Process) and the signalization of the points containing connotative features (CONN-FEATURE Process) are fully automatic processes. The GEN-GRAPH Process is activated with the “Show Structure” command. The separate CONN-FEATURE Process is activated with the “Show Possible Connotative Features” command.

The generation of the discourse structure (GEN-GRAPH Process) constitutes the final step of the interaction. With the activation of “Show Structure”, a graphic representation of the relation between the topics is presented, assisting the User to evaluate the flow of the conversation and the discourse structure of the discussion or interview.

2.3 The CONN-FEATURE Process: Signalizing Implied Information and Connotative Features

The signalization of “marked” information (CONN-FEATURE Process) is a fully automated process and is activated independently from the other three above-mentioned processes. The CONN-FEATURE Process signalizes all “connotatively marked” words and expressions based on the flouting of the Gricean Cooperativity Principle, especially in regard to the violation of the Maxims of Quality and Quantity [2, 4, 5].

These sets of expressions may be grouped into a finite set based on word stems or suffix type. Recognition on a word-stem or a suffix basis involves the detection of word-classes such as adjectives and adverbials or types of verbs, containing specific semantic features accessible with Wordnets and/or Selectional Restrictions [2].

Word categories whose connotative features are detected in the morphological level and whose semantic content is related to connotatively emotionally and socio-culturally “marked” elements are labelled as word groups with implicit connotative features. These word groups include the grammatical categories of verbs (or nominializations of verbs) containing semantic features (including implied connotations in language use) related to (i) mode (ii) malignant/benign action or (iii) emotional/ethical gravity, as well as nouns with suffixes producing diminutives, derivational suffixes resulting to a (ii) verbalization, (iii) an adjectivization or (iii) an additional nominalization of proper nouns (excluding derivational suffixes producing participles and actor thematic roles) [2].

3 Topic Tracking

3.1 The SELECT-TOPIC Process: Assisted Topic Selection

To assist the User in regard to the choice of topic, candidate topics consisting nouns are automatically signalized and listed with the SELECT-TOPIC Process. Pronouns and anaphoric expressions are not tracked. For the achievement of speed and efficiency, in the SELECT-TOPIC Process, the annotation module operates on keyword detection at morpheme-level or word-level. Only one topic can be set for each question or response.

Additionally, we note that selected topics may be subjected to Machine Translation. Selected topics may also be accessible as Universal Words, with the use of the Universal Networking Language (UNL) [10, 11, 13].

Finally, it should be considered that candidate topics corresponding to general subjects may receive additional signalization, also functioning as candidate topics for indicating the relation of “Generalization”, presented in following sections. The topics corresponding to general subjects may be related to a sublanguage-based data base or extracted from existing resources such as specialized lexica and corpora or Wordnets. Examples of general topics are “energy”, “financial crisis” or “biohazard” or a cluster of related word groups, for example, “war”, “battle”, “event”, “incident”, or “treaty”, “ally” and “side” [1].

3.2 The LOCAL-TOPIC Process: Topic Identification

From the list of available nouns chosen by the System, the User chooses the topic of the question or issue addressed by the interviewer or journalist-moderator and compares it to the topic of the answer or response.

With the activation of the “Identify Topic” command, topics are defined at a local level, in respect to the question asked or issue addressed by the interviewer or moderator, allowing the content of answers, responses and reactions to be checked in respect to the question asked or issue addressed.

Topics are treated as local variables. They are defined, registered and tracked.

Within the framework of questions/issues raised and respective answers, responses or reactions, the discourse structure is observed to be in some cases compatible to turn-taking in “push-to-talk conversations” [9], where there is a strict protocol in managing the interview or discussion and turn-taking. In other cases, discourse structure and turn-taking is partially compatible to the models of Sacks et al. 1974, Wilson and Wilson 2005 [8, 12], where each participant selects self.

A basic feature of the present approach is that the expert or world knowledge of the Users-Journalists helps identify the topic of each segment of the interview or discussion and also it helps determine the relation types between topics, due to the fact that in the domain of journalistic texts these relations cannot be strictly semantic and automatic processes may result to errors.

3.3 Relation Types

In the LOCAL-TOPIC Process, with the activation of the “Identify Relation” command, the User-Journalist indicates the type of relation between the topic of the question or issue addressed with the topic of the respective response or reaction. Relations between topics may of the following types: (1) Repetition, (2) Association (3) Generalization and (4) Topic Switch.

The User determines the relation type from the available relation types related to the corresponding tags. The relation of “Repetition” between selected topics receives the “REP” tag and involves the repetition of the same word or synonym. The relation of “Association” between selected topics receives the “ASOC” tag. Association may be defined by the User’s world knowledge or, if a defined sublanguage is used, the relation may be set by lexicon or Wordnet. The relation of “Generalization” between selected topics receives the “GEN” tag. We note that topics related to each other by a relation of “Generalization” can be easily defined within a sublanguage or a Wordnet. Finally, the relation of “Topic Switch” is used when the topic of a discussion or interview changes between selected topics without any evident semantic relations. The relation of “Topic Switch” receives the “SWITCH” tag.

4 The GEN-GRAPH Process: Graphic Representation

The final step of the interaction produces a graphic representation of the selected local topics in the discourse structure. The graphic representation has a “Generate Graph” and a “Generate Tree” option, since the generated structure may be depicted as a graph (“Generate Graph” option) or it may be depicted in a tree-like form, similar to a discourse tree [3, 6, 7] (“Generate Tree” option).

The generated graphic representation is based on the relations of the topics to each other, including distances from one word to another.

Distances between topics are defined as 1, 2 and 3. Distance 1 corresponds to the relation of “Repetition”, Distance 2, corresponds to the relation of “Association” and Distance 3 corresponds to the relation of “Generalization”.

The selected topics are attached to each other, either automatically by the System or interactively by the User, based on the set values of distances.

Depending on the type of graphic representation chosen, Distances 1, 2 and 3 are depicted as vertical lines from top to bottom, in the case of the generation of a tree-like structure, or they may be depicted as horizontal lines from left to right, in the case of the generation of a graph. The length of the lines depends on the type of distance. Thus, the shortest line corresponds to the relation of “Repetition”, related to Distance 1, while Distance 2, corresponding to the relation of “Association”, is represented as a longer line to the next word-node. Distance 3, corresponding to the relation of “Generalization”, is represented as the longest of three types of lines to the next word-node. The longest lines corresponding to the relation of “Generalization” may be modelled to form a curve (or a slight peak) to the next word-node, if a graph is generated or a node with a larger size and characteristic indication, if a tree-like structure is generated.

The fourth type of relation, “Topic Switch”, is depicted as a new, disconnected node generated to the right of the current point of the discourse structure, whether it is a tree or a graph. As separate nodes, topic switches may be connected as different branches of a tree structure, similar to discourse structures presented in tree-like forms, resembling discourse trees [3, 6, 7]. In a generated graph, topic switches may be depicted as breaks in the continuous flow of the graph.

Independently from the type of graphic representation chosen, the “Generate Graph” or the “Generate Tree” options, it should be noted that the overall shape of the generated graphic representation is dependent on the mostly occurring relation types in the discourse structure of the interview or discussion. For example, a generated graphic representation may have, in one case, many separate nodes (“Generate Tree” option) or peaks (“Generate Graph” option), in another case, a generated graphic representation may have a predominately linear structure, regardless of whether the “Generate Tree” option or the “Generate Graph” option is selected.

Furthermore, some types of linear structures may be modelled into circular structures, if the predominate relation is “Repetition” and in the graphic representation the last node is attached to the first node, corresponding to the same or a very similar (or synonymous) topic.

In the structures below, Fig. 2, corresponding to a discussion (with three participants) may be related to a predominately linear structure. On the other hand, Fig. 3, corresponding to an interview, may be related to a circular structure. In the transcribed discussions in Figs. 2 and 3, names of countries and national groups are withheld. The relation of “Generalization” is depicted in bold print.

Fig. 2.

Relations between topics generating a mostly linear structure

Fig. 3.

Relations between topics resulting to the modelling of a circular structure.

In Fig. 2, the topic of the conversation is “respecting laws of National State”, which is, in turn, associated to “Nationals”, “Nationals outside Country”, a “(definition) of minority group” and “(country’s) citizens” (Generalization). A topic shift occurs to the subject “Nationalist” and “Nationalism” which is, in turn, associated with the topics of a “country’s culture” and a “county’s civilization”. A third topic shift occurs towards the subject “immigrants” that is, in turn, associated with the general topics “people” (Generalization), “law-abiding household owners”, “(National) Law” and “violence” (Generalization).

On the other hand, in Fig. 3, the topic of the interview is “(country’s) people” which is always reoccurring in questions and responses. Repetition is the mostly occurring relation in the predominately linear discourse structure in Fig. 3 and may be modelled into a circular structure.

In the examples in Figs. 2, 3 and 4, Distance 1 (Repetition- REP) is depicted as “≫”, Distance 2 (Association- ASOC) as “=>” and Distance 3 (Generalization- GEN) is depicted as “==≫”. Topic switch (SWITCH) is depicted as “[>]”.

Fig. 4.

Relations between topics generating a tree-like structure with multiple branches or a graph with multiple peaks.

The example in Fig. 4 is a typical example resulting to the generation of a graphic representation of many separate nodes (“Generate Tree” option) or peaks (“Generate Graph” option).

The example in Fig. 4, corresponding to a transcript of an available online interview, contains a relatively high percentage of “Generalization” relations (depicted in bold print) affecting the overall shape of the generated graphic representation.

In the example in Fig. 4, the initial topics “poverty” and “family” (Generalization) are associated with the topic “African-American community”. Associated topics such as “children” and “school” are followed by multiple topic shifts to “Keystone pipeline”, “State Department” and “the Little Sisters of the Poor”. There is a topic shift to “health” (Generalization), a topic switch to the “IRS” and “Benghazi”, both related to the topics “issues” and “questions” (Generalization). General topics such as “criticism” and “policies” are connected to multiple topic shifts concerning the topics “President of the United States” associated with the topics “President Clinton”, “Richard Nixon” and “EPA”, “FDR”, which are, in turn, associated with “liberal and democrat”, “liberal and conservative” and “country” (Generalization). The topic “country”, a “Generalization” relation, is associated with “infrastructure”, in turn, associated with “2 trillion dollars worth”. The next topic shift concerns the topics “basic research”, “innovation edge” “space”, “internet” and “seventeen trillion dollar debt” which is connected to the next topic shift, “tax code” and “loopholes”. Another topic shift is “minimum wage” associated with “welfare” (Generalization) and “nanny state”. The last topic shift is “World War II”, associated with the topic “Veterans”, and connected by topic shift to the topic “middle class” (Generalization).

5 Conclusions and Further Research

The evaluation of pragmatic features in spoken political and journalistic texts is proposed to be assisted by an interactive annotation tool providing a graphic representation of the discourse structure and signalizing implied information and connotative features. The tool may be used to evaluate the content of interviews, live conversations in the Media and discussions in Parliament, enabling an overview of content and targeting to an objective evaluation of discourse structure and connotative elements.

The processes activated are combined with the decision-making process of the Journalist-User, allowing the combination of expert knowledge with automatic procedures.

Further research includes a full implementation of the designed tool, along with a comparison of the local topics chosen by different Users-Journalists, as well as a comparison of the effectiveness of graphic representation types, tree-like structures or graphs. Furthermore, the possibility of fully automatic topic tracking with the aid of a network or other form of resource based on a defined sublanguage is a subject of further investigation. Full implementation may provide concrete results of the proposed strategy’s success level and an insight to resolving any further issues.