You can not select more than 25 topics
Topics must start with a letter or number, can include dashes ('-') and can be up to 35 characters long.
2887 lines
125 KiB
2887 lines
125 KiB
<?xml version="1.0" encoding="UTF-8"?>
|
|
<!--OFFLINE
|
|
<!DOCTYPE html SYSTEM "DTD/xhtml1-strict.dtd">
|
|
OFFLINE-->
|
|
<!DOCTYPE html PUBLIC "-//W3C//DTD XHTML 1.0 Transitional//EN"
|
|
"http://www.w3.org/TR/xhtml1/DTD/xhtml1-transitional.dtd">
|
|
<html xmlns="http://www.w3.org/1999/xhtml" xml:lang="en" lang="en">
|
|
<head>
|
|
<title>Emotion Markup Language (EmotionML) 1.0</title>
|
|
<meta http-equiv="content-type" content="text/html; charset=UTF-8" />
|
|
<style type="text/css" xml:space="preserve">
|
|
/*<![CDATA[*/
|
|
.sample { border: thin black solid }
|
|
.boilerplate, .boilerplate-nocode { font-size: small; background: #ddd }
|
|
div.boilerplate, p.boilerplate, blockquote.boilerplate, div.boilerplate-nocode, p.boilerplate-nocode, blockquote.boilerplate-nocode { margin: 1em; border: thin black solid; padding: .25em }
|
|
div.source { margin: 1em }
|
|
.notetoeditor { color: red; background: white}
|
|
.comment { color: red; background: white}
|
|
|
|
div.issue {
|
|
border-width: thin;
|
|
border-style: solid;
|
|
border-color: maroon;
|
|
background-color: #FFEECC;
|
|
color: maroon;
|
|
width: 95%; padding: 0.5em; }
|
|
|
|
div.issue h4 {
|
|
margin-top: 0;
|
|
font-weight: bold;
|
|
color: maroon;
|
|
}
|
|
|
|
pre {
|
|
font-family: monospace;
|
|
font-weight: bold;
|
|
white-space: pre;
|
|
background-color: rgb(204,204,255);
|
|
padding: 0.5em;
|
|
border: none;
|
|
margin-left: 0;
|
|
width: 90%;
|
|
}
|
|
|
|
pre.xml {
|
|
font-family: "Lucida Sans Unicode", monospace;
|
|
background-color: rgb(204,204,255);
|
|
border: solid black thin;
|
|
}
|
|
code {
|
|
color: green;
|
|
font-family: monospace;
|
|
font-weight: bold;
|
|
}
|
|
|
|
table.defn th { background-color: rgb(220,220,255);
|
|
border-style: solid; border-color: black; border-width: thin }
|
|
table.defn td { background-color: rgb(230,230,255);
|
|
border-style: solid; border-color: black; border-width: thin }
|
|
|
|
|
|
/*]]>*/
|
|
|
|
</style>
|
|
<link rel="stylesheet" type="text/css"
|
|
href="http://www.w3.org/StyleSheets/TR/W3C-WD" />
|
|
</head>
|
|
|
|
<body>
|
|
|
|
<div class="head">
|
|
<p><a href="http://www.w3.org/"><img alt="W3C" height="48"
|
|
src="http://www.w3.org/Icons/w3c_home" width="72" /></a> </p>
|
|
|
|
<h1 style="clear:both" id="title">Emotion Markup Language (EmotionML) 1.0 </h1>
|
|
|
|
<h2 id="W3C-doctype">W3C Working Draft 7 April 2011 </h2>
|
|
<dl>
|
|
<dt>This version:</dt>
|
|
<dd><a
|
|
href="http://www.w3.org/TR/2011/WD-emotionml-20110407/">http://www.w3.org/TR/2011/WD-emotionml-20110407/</a>
|
|
</dd>
|
|
<dt>Latest version:</dt>
|
|
<dd><a
|
|
href="http://www.w3.org/TR/emotionml/">http://www.w3.org/TR/emotionml/</a>
|
|
</dd>
|
|
<dt>Previous version:</dt>
|
|
<dd><a
|
|
href="http://www.w3.org/TR/2010/WD-emotionml-20100729/">http://www.w3.org/TR/2010/WD-emotionml-20100729/</a>
|
|
</dd>
|
|
</dl>
|
|
<dl>
|
|
<dt>Editor:</dt>
|
|
<dd>Marc Schröder (DFKI GmbH)</dd>
|
|
<dt>Authors: </dt>
|
|
<dd><em>(in alphabetic order)</em></dd>
|
|
<dd>Paolo Baggia (Loquendo, S.p.A.)</dd>
|
|
<dd>Felix Burkhardt (Deutsche Telekom AG)</dd>
|
|
<dd>Catherine Pelachaud (Telecom ParisTech)</dd>
|
|
<dd>Christian Peter (Fraunhofer Gesellschaft)</dd>
|
|
<dd>Enrico Zovato (Loquendo, S.p.A.)</dd>
|
|
</dl>
|
|
|
|
<p class="copyright"><a href="http://www.w3.org/Consortium/Legal/ipr-notice#Copyright">Copyright</a> © 2011 <a href="http://www.w3.org/"><acronym title="World Wide Web Consortium">W3C</acronym></a><sup>®</sup> (<a href="http://www.csail.mit.edu/"><acronym title="Massachusetts Institute of Technology">MIT</acronym></a>, <a href="http://www.ercim.eu/"><acronym title="European Research Consortium for Informatics and Mathematics">ERCIM</acronym></a>, <a href="http://www.keio.ac.jp/">Keio</a>), All Rights Reserved. W3C <a href="http://www.w3.org/Consortium/Legal/ipr-notice#Legal_Disclaimer">liability</a>, <a href="http://www.w3.org/Consortium/Legal/ipr-notice#W3C_Trademarks">trademark</a> and <a href="http://www.w3.org/Consortium/Legal/copyright-documents">document use</a> rules apply.</p>
|
|
<hr />
|
|
</div>
|
|
|
|
<h2 id="abstract">Abstract</h2>
|
|
|
|
<p>As the web is becoming ubiquitous, interactive, and multimodal, technology
|
|
needs to deal increasingly with human factors, including emotions. The
|
|
specification of Emotion Markup Language 1.0 aims to strike a balance between
|
|
practical applicability and scientific well-foundedness. The language is
|
|
conceived as a "plug-in" language suitable for use in three different areas:
|
|
(1) manual annotation of data; (2) automatic recognition of emotion-related
|
|
states from user behavior; and (3) generation of emotion-related system
|
|
behavior.</p>
|
|
|
|
<h2 id="status">Status of this document</h2>
|
|
|
|
<p><em>This section describes the status of this document at the time of its
|
|
publication. Other documents may supersede this document. A list of current W3C
|
|
publications and the latest revision of this technical report can be found in
|
|
the <a href="http://www.w3.org/TR/">W3C technical reports index</a> at
|
|
http://www.w3.org/TR/.</em></p>
|
|
|
|
<p>This is a Last Call Working Draft of "Emotion Markup Language 1.0",
|
|
published on 7 April 2011. The W3C Membership and other interested parties are
|
|
invited to review the document and send comments to <a
|
|
href="mailto:www-multimodal@w3.org">www-multimodal@w3.org</a> (with <a
|
|
href="http://lists.w3.org/Archives/Public/www-multimodal/">public archive</a>)
|
|
until 7 June 2011.</p>
|
|
|
|
<p>This Last Call Working Draft has addressed all open issues from the previous
|
|
working draft, as well as the issues which were raised at the W3C workshop on
|
|
EmotionML. The changes compared to the previous Working Draft of 29 July 2010
|
|
are listed in <a href="#changelog">Appendix A</a>.</p>
|
|
|
|
<p>This document was developed by the <a
|
|
href="http://www.w3.org/2002/mmi/">Multimodal Interaction Working Group</a>.
|
|
The Working Group expects to advance this Working Draft to Recommendation
|
|
Status. </p>
|
|
|
|
<p>Publication as a Working Draft does not imply endorsement by the W3C
|
|
Membership. This is a draft document and may be updated, replaced or obsoleted
|
|
by other documents at any time. It is inappropriate to cite this document as
|
|
other than work in progress. </p>
|
|
|
|
<p>This document was produced by a group operating under the <a
|
|
href="http://www.w3.org/Consortium/Patent-Policy-20040205/">5 February 2004 W3C
|
|
Patent Policy</a>. W3C maintains a <a rel="disclosure"
|
|
href="http://www.w3.org/2004/01/pp-impl/34607/status">public list of any patent
|
|
disclosures</a> made in connection with the deliverables of the group; that
|
|
page also includes instructions for disclosing a patent. An individual who has
|
|
actual knowledge of a patent which the individual believes contains <a
|
|
href="http://www.w3.org/Consortium/Patent-Policy-20040205/#def-essential">Essential
|
|
Claim(s)</a> must disclose the information in accordance with <a
|
|
href="http://www.w3.org/Consortium/Patent-Policy-20040205/#sec-Disclosure">section
|
|
6 of the W3C Patent Policy</a>. </p>
|
|
|
|
<h2 id="conv">Conventions of this document</h2>
|
|
|
|
<p>The key words "MUST", "MUST NOT", "REQUIRED", "SHALL", "SHALL NOT",
|
|
"SHOULD", "SHOULD NOT", "RECOMMENDED", "MAY", and "OPTIONAL" in this document
|
|
are to be interpreted as described in [<a href="#ref-rfc2119">RFC2119</a>]. </p>
|
|
|
|
<p>The sections in the main body of this document are normative unless
|
|
otherwise specified. The appendices in this document are informative unless
|
|
otherwise indicated explicitly.</p>
|
|
|
|
<h2 id="toc">Table of Contents</h2>
|
|
|
|
<div class="toc">
|
|
<ul>
|
|
<li><a href="#s1">1 Introduction</a>
|
|
<ul>
|
|
<li><a href="#s1.1">1.1 Reasons for defining an Emotion Markup
|
|
Language</a></li>
|
|
<li><a href="#s1.2">1.2 The challenge of defining a generally usable
|
|
Emotion Markup Language</a></li>
|
|
<li><a href="#s1.3">1.3 Glossary of terms</a></li>
|
|
</ul>
|
|
</li>
|
|
<li><a href="#s2">2 Elements of Emotion Markup</a>
|
|
<ul>
|
|
<li><a href="#s2.1">2.1 Document structure</a>
|
|
<ul>
|
|
<li><a href="#s2.1.1">2.1.1 Document root: The
|
|
<code><emotionml></code> element</a></li>
|
|
<li><a href="#s2.1.2">2.1.2 A single emotion annotation: The
|
|
<code><emotion></code> element</a></li>
|
|
</ul>
|
|
</li>
|
|
<li><a href="#s2.2">2.2 Representations of emotions and related
|
|
states</a>
|
|
<ul>
|
|
<li><a href="#s2.2.1">2.2.1 The <code><category></code>
|
|
element</a></li>
|
|
<li><a href="#s2.2.2">2.2.2 The <code><dimension></code>
|
|
element</a></li>
|
|
<li><a href="#s2.2.3">2.2.3 The <code><appraisal></code>
|
|
element</a></li>
|
|
<li><a href="#s2.2.4">2.2.4 The <code><action-tendency></code>
|
|
element</a></li>
|
|
</ul>
|
|
</li>
|
|
<li><a href="#s2.3">2.3 Meta-information</a>
|
|
<ul>
|
|
<li><a href="#s2.3.1">2.3.1 The <code>confidence</code>
|
|
attribute</a></li>
|
|
<li><a href="#s2.3.2">2.3.2 The <code>expressed-through</code>
|
|
attribute</a></li>
|
|
<li><a href="#s2.3.3">2.3.3 The <code><info></code>
|
|
element</a></li>
|
|
</ul>
|
|
</li>
|
|
<li><a href="#s2.4">2.4 References and time</a>
|
|
<ul>
|
|
<li><a href="#s2.4.1">2.4.1 The <code><reference></code>
|
|
element</a></li>
|
|
<li><a href="#s2.4.2">2.4.2 Timestamps</a>
|
|
<ul>
|
|
<li><a href="#s2.4.2.1">2.4.2.1 Absolute time</a></li>
|
|
<li><a href="#s2.4.2.2">2.4.2.2 Duration</a></li>
|
|
<li><a href="#s2.4.2.3">2.4.2.3 Relative time</a></li>
|
|
<li><a href="#s2.4.2.4">2.4.2.4 Timing in media</a></li>
|
|
</ul>
|
|
</li>
|
|
</ul>
|
|
</li>
|
|
<li><a href="#s2.5">2.5 Scale values</a>
|
|
<ul>
|
|
<li><a href="#s2.5.1">2.5.1 The <code>value</code> attribute</a></li>
|
|
<li><a href="#s2.5.2">2.5.2 The <code><trace></code>
|
|
element</a></li>
|
|
</ul>
|
|
</li>
|
|
</ul>
|
|
</li>
|
|
<li><a href="#s3">3 Defining vocabularies for representing emotions</a>
|
|
<ul>
|
|
<li><a href="#s3.1">3.1 Mechanism for defining vocabularies</a>
|
|
<ul>
|
|
<li><a href="#s3.1.1">3.1.1 The <code><vocabulary></code>
|
|
element</a></li>
|
|
<li><a href="#s3.1.2">3.1.2 The <code><item></code>
|
|
element</a></li>
|
|
</ul>
|
|
</li>
|
|
</ul>
|
|
</li>
|
|
<li><a href="#s4">4 Conformance</a>
|
|
<ul>
|
|
<li><a href="#s4.1">4.1 EmotionML namespace</a></li>
|
|
<li><a href="#s4.2">4.2 Use with other namespaces</a></li>
|
|
<li><a href="#s4.3">4.3 Schema validation and processor validation of
|
|
EmotionML documents</a></li>
|
|
</ul>
|
|
</li>
|
|
<li><a href="#s5">5 Examples</a>
|
|
<ul>
|
|
<li><a href="#s5.1">5.1 Examples of emotion annotation</a>
|
|
<ul>
|
|
<li><a href="#s5.1.1">5.1.1 Manual annotation of emotional
|
|
material</a>
|
|
<ul>
|
|
<li><a href="#example_annotation_images">Annotation of static
|
|
images</a></li>
|
|
<li><a href="#example_annotation_videos">Annotation of
|
|
videos</a></li>
|
|
</ul>
|
|
</li>
|
|
<li><a href="#s5.1.2">5.1.2 Automatic recognition of emotions</a></li>
|
|
<li><a href="#s5.1.3">5.1.3 Generation of emotion-related system
|
|
behavior</a>
|
|
<ul>
|
|
<li><a href="#example_generation_mpeg4">Generation of facial
|
|
expressions in an MPEG-4 face model</a></li>
|
|
<li><a href="#example_generation_robot">Generation of robot
|
|
behavior</a></li>
|
|
</ul>
|
|
</li>
|
|
</ul>
|
|
</li>
|
|
<li><a href="#s5.2">5.2 Examples of possible use with other markup
|
|
languages</a>
|
|
<ul>
|
|
<li><a href="#s5.2.1">5.2.1 Use with EMMA</a></li>
|
|
<li><a href="#s5.2.2">5.2.2 Use with SSML</a></li>
|
|
<li><a href="#s5.2.3">5.2.3 Use with SMIL</a></li>
|
|
</ul>
|
|
</li>
|
|
</ul>
|
|
</li>
|
|
<li><a href="#s6">6 References</a>
|
|
<ul>
|
|
<li><a href="#s6.1">6.1 Normative references</a></li>
|
|
<li><a href="#s6.2">6.2 Informative references</a></li>
|
|
</ul>
|
|
</li>
|
|
<li><a href="#s7">7 Acknowledgments</a></li>
|
|
<li><a href="#changelog">Appendix A: Changes</a></li>
|
|
</ul>
|
|
</div>
|
|
|
|
<h2 id="s1">1 Introduction</h2>
|
|
|
|
<p><i>This section is informative.</i></p>
|
|
|
|
<p>Human emotions are increasingly understood to be a crucial aspect in
|
|
human-machine interactive systems. Especially for non-expert end users,
|
|
reactions to complex intelligent systems resemble social interactions,
|
|
involving feelings such as frustration, impatience, or helplessness if things
|
|
go wrong. Furthermore, technology is increasingly used to observe
|
|
human-to-human interactions, such as customer frustration monitoring in call
|
|
center applications. Dealing with these kinds of states in technological
|
|
systems requires a suitable representation, which should make the concepts and
|
|
descriptions developed in the affective sciences available for use in
|
|
technological contexts.</p>
|
|
|
|
<p>This report specifies Emotion Markup Language (EmotionML) 1.0, a markup
|
|
language designed to be usable in a broad variety of technological contexts
|
|
while reflecting concepts from the affective sciences. </p>
|
|
|
|
<h3 id="s1.1">1.1 Reasons for defining an Emotion Markup Language</h3>
|
|
|
|
<p>As for any standard format, the first and main goal of an EmotionML is
|
|
twofold: to allow a technological component to represent and process data, and
|
|
to enable interoperability between different technological components
|
|
processing the data.</p>
|
|
|
|
<p>Use cases for EmotionML can be grouped into three broad types:</p>
|
|
<ol>
|
|
<li>Manual annotation of material involving emotionality, such as annotation
|
|
of videos, of speech recordings, of faces, of texts, etc;</li>
|
|
<li>Automatic recognition of emotions from sensors, including physiological
|
|
sensors, speech recordings, facial expressions, etc., as well as from
|
|
multi-modal combinations of sensors;</li>
|
|
<li>Generation of emotion-related system responses, which may involve
|
|
reasoning about the emotional implications of events, emotional prosody in
|
|
synthetic speech, facial expressions and gestures of embodied agents or
|
|
robots, the choice of music and colors of lighting in a room, etc.</li>
|
|
</ol>
|
|
|
|
<p>Interactive systems are likely to involve both analysis and generation of
|
|
emotion-related behavior; furthermore, systems are likely to benefit from data
|
|
that was manually annotated, be it as training data or for rule-based
|
|
modeling. Therefore, it is desirable to propose a single EmotionML that can be
|
|
used in all three contexts.</p>
|
|
|
|
<p>Concrete examples of existing technology that could apply EmotionML
|
|
include:</p>
|
|
<ul>
|
|
<li>Opinion mining / sentiment analysis in Web 2.0, to automatically track
|
|
customer's attitude regarding a product across blogs;</li>
|
|
<li>Affective monitoring, such as ambient assisted living applications for
|
|
the elderly, fear detection for surveillance purposes, or using wearable
|
|
sensors to test customer satisfaction;</li>
|
|
<li>Character design and control for games and virtual worlds;</li>
|
|
<li>Social robots, such as guide robots engaging with visitors;</li>
|
|
<li>Expressive speech synthesis, generating synthetic speech with different
|
|
emotions, such as happy or sad, friendly or apologetic;</li>
|
|
<li>Emotion recognition (e.g., for spotting angry customers in speech dialog
|
|
systems);</li>
|
|
<li>Support for people with disabilities, such as educational programs for
|
|
people with autism.</li>
|
|
</ul>
|
|
|
|
<p>The Emotion Incubator Group has listed <a
|
|
href="http://www.w3.org/2005/Incubator/emotion/XGR-emotion/#AppendixUseCases">39
|
|
individual use cases for an EmotionML</a>.</p>
|
|
|
|
<p></p>
|
|
|
|
<p>A second reason for defining an EmotionML is the observation that ad hoc
|
|
attempts to deal with emotions and related states often lead people to make the
|
|
same mistakes that others have made before. The most typical mistake is to
|
|
model emotions as a small number of intense states such as anger, fear, joy,
|
|
and sadness; this choice is often made irrespective of the question whether
|
|
these states are the most appropriate for the intended application. Crucially,
|
|
the available alternatives that have been developed in the affective science
|
|
literature are not sufficiently known, resulting in dead-end situations after
|
|
the initial steps of work. Careful consideration of states to study and of
|
|
representations for describing them can help avoid such situations.</p>
|
|
|
|
<p>EmotionML makes scientific concepts of emotions practically applicable. This
|
|
can help potential users to identify the suitable representations for their
|
|
respective applications.</p>
|
|
|
|
<h3 id="s1.2">1.2 The challenge of defining a generally usable Emotion Markup
|
|
Language</h3>
|
|
|
|
<p>Any attempt to standardize the description of emotions using a finite set of
|
|
fixed descriptors is doomed to failure: even scientists cannot agree on the
|
|
number of relevant emotions, or on the names that should be given to them. Even
|
|
more basically, the list of emotion-related states that should be distinguished
|
|
varies depending on the application domain and the aspect of emotions to be
|
|
focused. Basically, the vocabulary needed depends on the context of use. On the
|
|
other hand, the basic structure of concepts is less controversial: it is
|
|
generally agreed that emotions involve triggers, appraisals, feelings,
|
|
expressive behavior including physiological changes, and action tendencies;
|
|
emotions in their entirety can be described in terms of categories or a small
|
|
number of dimensions; emotions have an intensity, and so on. For details, see
|
|
<a
|
|
href="http://www.w3.org/2005/Incubator/emotion/XGR-emotion/#ScientificDescriptions">Scientific
|
|
Descriptions of Emotions</a> in the Final Report of the <a
|
|
href="#ref-emotion-xg">Emotion Incubator Group</a>.</p>
|
|
|
|
<p>Given this lack of agreement on descriptors in the field, the only practical
|
|
way of defining an EmotionML is the definition of possible structural elements
|
|
and their valid child elements and attributes, but to allow users to "plug in"
|
|
vocabularies that they consider appropriate for their work. A separate W3C
|
|
Working Draft complements this specification to provide a central repository of
|
|
[<a href="#ref-emotion-voc">Vocabularies for EmotionML</a>] which can serve as
|
|
a starting point; where the vocabularies listed there seem inappropriate, users
|
|
can create their custom vocabularies.</p>
|
|
|
|
<p>An additional challenge lies in the aim to provide a generally usable
|
|
markup, as the requirements arising from the three different use cases
|
|
(annotation, recognition, and generation) are rather different. Whereas manual
|
|
annotation tends to require all the fine-grained distinctions considered in the
|
|
scientific literature, automatic recognition systems can usually distinguish
|
|
only a very small number of different states.</p>
|
|
|
|
<p>For the reasons outlined here, it is clear that there is an inevitable
|
|
tension between flexibility and interoperability, which need to be weighed in
|
|
the formulation of an EmotionML. The guiding principle in the following
|
|
specification has been to provide a choice only where it is needed, and to
|
|
propose reasonable default options for every choice.</p>
|
|
|
|
<h3 id="s1.3">1.3 Glossary of terms</h3>
|
|
|
|
<p>The terms related to emotions are not used consistently, neither in common
|
|
use nor in the scientific literature. The following glossary describes the
|
|
intended meaning of terms in this document.</p>
|
|
|
|
<p></p>
|
|
<dl>
|
|
<dt id="term-action-tendency">Action tendency</dt>
|
|
<dd>Emotions have a strong influence on the motivational state of a
|
|
subject. Emotion theory associates emotions to a small set of so-called
|
|
action tendencies, e.g. avoidance (relates to fear), rejecting (disgust)
|
|
etc. Action tendencies can be viewed as a link between the outcome of an
|
|
<a href="#term-appraisal">appraisal</a> process and actual actions. </dd>
|
|
<dt id="term-affect">Affect / Affective state</dt>
|
|
<dd>In the scientific literature, the term "affect" is often used as a
|
|
general term covering a range of phenomena called "affective states",
|
|
including emotions, moods, attitudes, etc. Proponents of the term
|
|
consider it to be more generic than "emotion", in the sense that it
|
|
covers both acute and long-term, specific and unspecific states. In this
|
|
report, the term "affect" is avoided so that the scope of the intended
|
|
markup language is more easily accessible to the non-expert; the term
|
|
"affective state" is used interchangeably with "<a
|
|
href="#term-emotion-related-state">emotion-related state</a>".</dd>
|
|
<dt id="term-appraisal">Appraisal</dt>
|
|
<dd>The term "appraisal" is used in the scientific literature to describe
|
|
the evaluation process leading to an emotional response. Triggered by an
|
|
"emotion-eliciting event", an individual carries out an automatic,
|
|
subjective assessment of the event, in order to determine the relevance
|
|
of the event to the individual. This assessment is carried out along a
|
|
number of "appraisal dimensions" such as the novelty, pleasantness or
|
|
goal conduciveness of the event. </dd>
|
|
<dt id="term-attitude">Attitude</dt>
|
|
<dd>In psychology, "attitude" is related to the global evaluation of an
|
|
object, such as a person, an object, oneself, etc. Attitude is considered
|
|
to include an emotional component, as well as cognition and behavior.
|
|
However, the term "attitude" is sometimes used with slightly different
|
|
meanings, such as speaking style ("he said it with a certain attitude")
|
|
or more generally personal lifestyle ("she has quite an attitude").
|
|
Because of this ambiguity, this specification avoids the term.</dd>
|
|
<dt id="term-emotion">Emotion</dt>
|
|
<dd>In this report, the term "emotion" is used in a very broad sense,
|
|
covering both intense and weak states, short and long term, with and
|
|
without event focus. This meaning is intended to reflect the
|
|
understanding of the term "emotion" by the general public. In the
|
|
scientific literature on emotion theories, the term "emotion" or
|
|
"fullblown emotion" refers to intense states with a strong focus on
|
|
current events, often in the context of the survival-benefiting function
|
|
of behavioral responses such as "fight or flight". This reading of the
|
|
term seems inappropriate for the vast majority of human-machine
|
|
interaction contexts, in which more subtle states dominate; therefore,
|
|
where this reading is intended, the term "<a
|
|
href="#term-fullblown-emotion">fullblown emotion</a>" is used in this
|
|
report.</dd>
|
|
<dt id="term-emotion-related-state">Emotion-related state</dt>
|
|
<dd>A cover term for the broad range of phenomena intended to be covered by
|
|
this specification. In the scientific literature, several kinds of
|
|
emotion-related or affective states are distinguished, see <a
|
|
href="http://www.w3.org/2005/Incubator/emotion/XGR-emotion/#EmotionRelatedStates">Emotions
|
|
and related states</a> in the final report of the <a
|
|
href="#ref-emotion-xg">Emotion Incubator Group</a>.</dd>
|
|
<dt id="term-emotion-dimensions">Emotion dimensions</dt>
|
|
<dd>A small number of continuous scales describing the most basic
|
|
properties of an emotion. Often three dimensions are used: valence
|
|
(sometimes named pleasure), arousal (or activity/activation), and potency
|
|
(sometimes called control, power or dominance). However, sometimes two,
|
|
or more than three dimensions are used.</dd>
|
|
<dt id="term-fullblown-emotion">Fullblown emotion</dt>
|
|
<dd>Intense states with a strong focus on current events, often in the
|
|
context of the survival-benefiting function of behavioral responses such
|
|
as "fight or flight".</dd>
|
|
</dl>
|
|
|
|
<div class="comment">
|
|
</div>
|
|
|
|
<h2 id="s2">2 Elements of Emotion Markup</h2>
|
|
|
|
<p>The following sections describe the syntax of the main elements of
|
|
EmotionML.</p>
|
|
|
|
<h3 id="s2.1">2.1 Document structure</h3>
|
|
|
|
<h4 id="s2.1.1">2.1.1 Document root: The <code><emotionml></code>
|
|
element</h4>
|
|
|
|
<table class="defn" cellspacing="0" cellpadding="5" width="98%"
|
|
summary="property definition">
|
|
<tbody>
|
|
<tr>
|
|
<th>Annotation</th>
|
|
<td><code><emotionml></code></td>
|
|
</tr>
|
|
<tr>
|
|
<th>Definition</th>
|
|
<td>The root element of an EmotionML document.</td>
|
|
</tr>
|
|
<tr>
|
|
<th>Children</th>
|
|
<td>The element MAY contain one or more <code><emotion></code>
|
|
elements. It MAY contain a single <a
|
|
href="#s2.3.3"><code><info></code></a> element. It MAY contain
|
|
one or more <a href="#s3.1.1"><code><vocabulary></code></a>
|
|
elements.</td>
|
|
</tr>
|
|
<tr>
|
|
<th>Attributes</th>
|
|
<td><ul>
|
|
<li><strong>Required</strong>:
|
|
<ul>
|
|
<li>Namespace declaration for EmotionML, see <a
|
|
href="#s4.1">EmotionML namespace</a>.</li>
|
|
<li><code>version</code> indicates the version of the
|
|
specification to be used for the document and MUST have the
|
|
value "1.0".</li>
|
|
</ul>
|
|
</li>
|
|
<li><strong>Optional</strong>:
|
|
<ul>
|
|
<li><code>category-set</code>, <code>dimension-set</code>,
|
|
<code>appraisal-set</code> and <code>action-tendency-set</code>
|
|
indicate default emotion vocabularies used in an EmotionML
|
|
document. Any of these attributes used at document level makes
|
|
optional the use of the same attribute in any
|
|
<code><emotion></code> elements within the document; the
|
|
document-level attribute determines the emotion vocabularies
|
|
used for any <code><emotion></code> elements for which
|
|
the respective attribute is not locally specified. The
|
|
attributes are of type <code>xsd:anyURI</code> and MUST point
|
|
to a definition of an emotion vocabulary as specified in <a
|
|
href="#s3">Defining vocabularies for representing
|
|
emotions</a>.</li>
|
|
</ul>
|
|
</li>
|
|
</ul>
|
|
</td>
|
|
</tr>
|
|
<tr>
|
|
<th>Occurrence</th>
|
|
<td>This is the root element -- it cannot occur as a child of any other
|
|
EmotionML element.</td>
|
|
</tr>
|
|
</tbody>
|
|
</table>
|
|
|
|
<p><code><emotionml></code> is the root element of a standalone EmotionML
|
|
document. It MAY contain a single <code><info></code> element, providing
|
|
document-level metadata.</p>
|
|
|
|
<p>The <code><emotionml></code> element MUST define the <a
|
|
href="#s4.1">EmotionML namespace</a>.</p>
|
|
|
|
<p>Standalone EmotionML documents usually serve one or both of the following
|
|
two purposes:</p>
|
|
<ol>
|
|
<li>to wrap a number of <a href="#s2.1.2"><code><emotion></code></a>
|
|
elements into a single document;</li>
|
|
<li>to define <a href="#s3">emotion vocabularies</a> for use with <a
|
|
href="#s2.1.2"><code><emotion></code></a> annotations in the same or
|
|
other documents.</li>
|
|
</ol>
|
|
|
|
<p>Example:</p>
|
|
<pre class="example"><emotionml version="1.0" xmlns="http://www.w3.org/2009/10/emotionml">
|
|
...
|
|
</emotionml></pre>
|
|
|
|
<p>or</p>
|
|
<pre class="example"><em:emotionml version="1.0" xmlns:em="http://www.w3.org/2009/10/emotionml">
|
|
...
|
|
</em:emotionml></pre>
|
|
|
|
<p></p>
|
|
|
|
<p>Note: One of the envisaged uses of EmotionML is to be used in the context of
|
|
other markup languages. In such cases, there will be no
|
|
<code><emotionml></code> root element, but <code><emotion></code>
|
|
elements will be used directly in other markup -- see <a href="#s5.2">Examples
|
|
of possible use with other markup languages</a>.</p>
|
|
|
|
<h4 id="s2.1.2">2.1.2 A single emotion annotation: The
|
|
<code><emotion></code> element</h4>
|
|
|
|
<table class="defn" cellspacing="0" cellpadding="5" width="98%"
|
|
summary="property definition">
|
|
<tbody>
|
|
<tr>
|
|
<th>Annotation</th>
|
|
<td><code><emotion></code></td>
|
|
</tr>
|
|
<tr>
|
|
<th>Definition</th>
|
|
<td>This element represents a single emotion annotation.</td>
|
|
</tr>
|
|
<tr>
|
|
<th>Children</th>
|
|
<td>All children are optional. However, at least one of
|
|
<code><category></code>, <code><dimension></code>,
|
|
<code><appraisal></code>, <code><action-tendency></code>
|
|
MUST occur.
|
|
|
|
<p>If present, the following child element can occur only once:
|
|
<code><info></code>.</p>
|
|
|
|
<p>If present, the following child elements may occur one or more
|
|
times: <code><category></code>; <code><dimension></code>;
|
|
<code><appraisal></code>; <code><action-tendency></code>;
|
|
<code><reference></code>.</p>
|
|
|
|
<p>There are no constraints on the combinations of children that are
|
|
allowed. There are no constraints on the order in which children
|
|
occur.</p>
|
|
</td>
|
|
</tr>
|
|
<tr>
|
|
<th>Attributes</th>
|
|
<td><ul>
|
|
<li><strong>Required</strong>:
|
|
<ul>
|
|
<li><code>category-set</code>, <code>dimension-set</code>,
|
|
<code>appraisal-set</code> and <code>action-tendency-set</code>
|
|
indicate the emotion vocabularies used in this
|
|
<code><emotion></code>. The attributes are of type
|
|
<code>xsd:anyURI</code> and MUST point to a definition of an
|
|
emotion vocabulary as specified in <a href="#s3">Defining
|
|
vocabularies for representing emotions</a>. The attributes are
|
|
required as follows:
|
|
<ul>
|
|
<li>if the <code><emotion></code> element has a child
|
|
element <code><category></code>, the
|
|
<code>category-set</code> attribute is required and must
|
|
point to the definition of a category vocabulary;</li>
|
|
<li>if the <code><emotion></code> element has a child
|
|
element <code><dimension></code>, the
|
|
<code>dimension-set</code> attribute is required and must
|
|
point to the definition of a dimension vocabulary;</li>
|
|
<li>if the <code><emotion></code> element has a child
|
|
element <code><appraisal></code>, the
|
|
<code>appraisal-set</code> attribute is required and must
|
|
point to the definition of an appraisal vocabulary;</li>
|
|
<li>if the <code><emotion></code> element has a child
|
|
element <code><action-tendency></code>, the
|
|
<code>action-tendency-set</code> attribute is required and
|
|
must point to the definition of an action tendency
|
|
vocabulary.</li>
|
|
</ul>
|
|
<p>The attribute values default to the values of any attributes
|
|
of same name on an enclosing <code><emotionml></code>
|
|
element. Any of these attributes used at document level makes
|
|
optional the use of the same attribute in any
|
|
<code><emotion></code> elements. </p>
|
|
</li>
|
|
<li><code>version</code> indicates the version of the
|
|
specification to be used for the <code><emotion></code>
|
|
and its descendants. Its value defaults to "1.0". </li>
|
|
</ul>
|
|
</li>
|
|
<li><strong>Optional</strong>:
|
|
<ul>
|
|
<li><code>id</code>, a unique identifier for the emotion, of type
|
|
<code>xsd:ID</code>.</li>
|
|
<li><a href="#s2.4.2.1"><code>start</code></a>, <a
|
|
href="#s2.4.2.1"><code>end</code></a>, <a
|
|
href="#s2.4.2.2"><code>duration</code></a>, <a
|
|
href="#s2.4.2.3"><code>time-ref-uri</code></a>, <a
|
|
href="#s2.4.2.3"><code>time-ref-anchor-point</code></a> and <a
|
|
href="#s2.4.2.3"><code>offset-to-start</code></a> provide
|
|
information about the times at which an emotion happened, as
|
|
defined in <a href="#s2.4.2">Timestamps</a>.</li>
|
|
<li><a href="#s2.3.2"><code>expressed-through</code></a>, the
|
|
modality, or list of modalities, through which the emotion is
|
|
expressed. </li>
|
|
</ul>
|
|
</li>
|
|
</ul>
|
|
</td>
|
|
</tr>
|
|
<tr>
|
|
<th>Occurrence</th>
|
|
<td>as a child of <code><emotionml></code>, or in any markup using
|
|
EmotionML.</td>
|
|
</tr>
|
|
</tbody>
|
|
</table>
|
|
|
|
<p>The <code><emotion></code> element represents an individual emotion
|
|
annotation. No matter how simple or complex its substructure is, it represents
|
|
a single statement about the emotional content of some annotated item. Where
|
|
several statements about the emotion in a certain context are to be made,
|
|
several <code><emotion></code> elements MUST be used. See <a
|
|
href="#s5.1">Examples of emotion annotation</a> for illustrations of this
|
|
issue.</p>
|
|
|
|
<p>An <code><emotion></code> element MAY have an <code>id</code>
|
|
attribute, allowing for a unique reference to the individual emotion
|
|
annotation. Since the <code><emotion></code> annotation is an atomic
|
|
statement about the emotion, it is inappropriate to refer to individual emotion
|
|
representations such as <code><category></code>,
|
|
<code><dimension></code>, <code><appraisal></code>,
|
|
<code><action-tendency></code> or their children directly. For this
|
|
reason, these elements do not allow for an <code>id</code> attribute.</p>
|
|
|
|
<p>Whereas it is possible to use <code><emotion></code> elements in a
|
|
standalone <code><emotionml></code> document, a typical use case is
|
|
expected to be embedding an <code><emotion></code> into some other markup
|
|
-- see <a href="#s5.2">Examples of possible use with other markup
|
|
languages</a>.</p>
|
|
|
|
<h3 id="s2.2">2.2 Representations of emotions and related states</h3>
|
|
|
|
<h4 id="s2.2.1">2.2.1 The <code><category></code> element</h4>
|
|
|
|
<table class="defn" cellspacing="0" cellpadding="5" width="98%"
|
|
summary="property definition">
|
|
<tbody>
|
|
<tr>
|
|
<th>Annotation</th>
|
|
<td><code><category></code></td>
|
|
</tr>
|
|
<tr>
|
|
<th>Definition</th>
|
|
<td>Description of an emotion or a related state using a category.</td>
|
|
</tr>
|
|
<tr>
|
|
<th>Children</th>
|
|
<td><a href="#s2.5.2"><code><trace></code></a>: A
|
|
<code><category></code> MAY contain either a <code>value</code>
|
|
attribute or a <code><trace></code> element.</td>
|
|
</tr>
|
|
<tr>
|
|
<th>Attributes</th>
|
|
<td><ul>
|
|
<li><strong>Required</strong>:
|
|
<ul>
|
|
<li><code>name</code>, the name of the category, which must be
|
|
contained in the set of categories identified in the enclosing
|
|
<a href="#s2.1.1"><code><emotionml></code></a> or <a
|
|
href="#s2.1.2"><code><emotion></code></a> element's
|
|
<code>category-set</code> attribute.</li>
|
|
</ul>
|
|
</li>
|
|
<li><strong>Optional</strong>:
|
|
<ul>
|
|
<li><a href="#s2.5.1"><code>value</code></a>: A
|
|
<code><category></code> MAY contain either a
|
|
<code>value</code> attribute or a <code><trace></code>
|
|
element.</li>
|
|
<li><a href="#s2.3.1"><code>confidence</code></a>, the
|
|
annotator's confidence that the annotation given for this
|
|
category is correct.</li>
|
|
</ul>
|
|
</li>
|
|
</ul>
|
|
</td>
|
|
</tr>
|
|
<tr>
|
|
<th>Occurrence</th>
|
|
<td>One or more <code><category></code> elements MAY occur as a
|
|
child of <code><emotion></code>. For any given category name in
|
|
the set, zero or one occurrence is allowed within an
|
|
<code><emotion></code> element.</td>
|
|
</tr>
|
|
</tbody>
|
|
</table>
|
|
|
|
<p><code><category></code> describes an <a
|
|
href="#term-emotion">emotion</a> or a <a
|
|
href="#term-emotion-related-state">related state</a> in terms of a category
|
|
name, given as the value of the <code>name</code> attribute. The name MUST
|
|
belong to a clearly-identified set of category names, which MUST be defined
|
|
according to <a href="#s3">Defining vocabularies for representing
|
|
emotions</a>.</p>
|
|
|
|
<p>The set of legal values of the <code>name</code> attribute is indicated in
|
|
the <code>category-set</code> attribute of the enclosing <a
|
|
href="#s2.1.1"><code><emotion></code></a> or <a
|
|
href="#s2.1.1"><code><emotionml></code></a> element. Different sets can
|
|
be used, depending on the requirements of the use case. In particular,
|
|
different types of <a href="#term-emotion-related-state">emotion-related</a> /
|
|
<a href="#term-affect">affective states</a> can be annotated by using
|
|
appropriate value sets. </p>
|
|
|
|
<p>The intensity of an emotion category MAY be specified as a <a
|
|
href="#s2.5">Scale value</a>, either as a static value in the <a
|
|
href="#s2.5.1"><code>value</code></a> attribute, or as a dynamic trace over
|
|
time using the <a href="#s2.5.2"><code><trace></code></a> element. </p>
|
|
|
|
<p>Examples:</p>
|
|
|
|
<p>In the following example, the emotion category "satisfaction" is being
|
|
annotated; it must be contained in the definition of the emotion category
|
|
vocabulary located at http://www.w3.org/TR/emotion-voc/xml#everyday-categories,
|
|
which is one of the category vocabularies provided in [<a
|
|
href="#ref-emotion-voc">Vocabularies for EmotionML</a>].</p>
|
|
<pre class="example"><emotion category-set="http://www.w3.org/TR/emotion-voc/xml#everyday-categories">
|
|
<category name="satisfaction"/>
|
|
</emotion></pre>
|
|
|
|
<p>The following is an annotation of an interpersonal stance "distant" which
|
|
must be defined in the custom category set at the URI given in the category-set
|
|
attribute: </p>
|
|
<pre class="example"><emotion category-set="http://www.example.com/custom/category/interpersonal-stances.xml">
|
|
<category name="distant"/>
|
|
</emotion></pre>
|
|
|
|
<p>In the following example, an emotion is described by several categories,
|
|
each being present with different values of intensity. The category set used is
|
|
the "big six" set described in [<a href="#ref-emotion-voc">Vocabularies for
|
|
EmotionML</a>].</p>
|
|
<pre class="example"><emotion category-set="http://www.w3.org/TR/emotion-voc/xml#big6">
|
|
<category name="sadness" value="0.3"/>
|
|
<category name="anger" value="0.8"/>
|
|
<category name="fear" value="0.3"/>
|
|
</emotion></pre>
|
|
|
|
<h4 id="s2.2.2">2.2.2 The <code><dimension></code> element</h4>
|
|
|
|
<table class="defn" cellspacing="0" cellpadding="5" width="98%"
|
|
summary="property definition">
|
|
<tbody>
|
|
<tr>
|
|
<th>Annotation</th>
|
|
<td><code><dimension></code></td>
|
|
</tr>
|
|
<tr>
|
|
<th>Definition</th>
|
|
<td>One or more <code><dimension></code> elements jointly describe
|
|
an emotion or a related state according to an emotion dimension
|
|
vocabulary.</td>
|
|
</tr>
|
|
<tr>
|
|
<th>Children</th>
|
|
<td><a href="#s2.5.2"><code><trace></code></a>: A
|
|
<code><dimension></code> MUST contain either a <code>value</code>
|
|
attribute or a <code><trace></code> element.</td>
|
|
</tr>
|
|
<tr>
|
|
<th>Attributes</th>
|
|
<td><ul>
|
|
<li><strong>Required</strong>:
|
|
<ul>
|
|
<li><code>name</code>, the name of the dimension, which must be
|
|
contained in the set of dimensions identified in the enclosing
|
|
<a href="#s2.1.1"><code><emotionml></code></a> or <a
|
|
href="#s2.1.2"><code><emotion></code></a> element's
|
|
<code>dimension-set</code> attribute.</li>
|
|
<li><a href="#s2.5.1"><code>value</code></a>: A
|
|
<code><dimension></code> MUST contain either a
|
|
<code>value</code> attribute or a <code><trace></code>
|
|
element.</li>
|
|
</ul>
|
|
</li>
|
|
<li><strong>Optional</strong>:
|
|
<ul>
|
|
<li><a href="#s2.3.1"><code>confidence</code></a>, the
|
|
annotator's confidence that the annotation given for this
|
|
dimension is correct.</li>
|
|
</ul>
|
|
</li>
|
|
</ul>
|
|
</td>
|
|
</tr>
|
|
<tr>
|
|
<th>Occurrence</th>
|
|
<td><code><dimension></code> elements occur as children of
|
|
<code><emotion></code>. For any given dimension name in the set,
|
|
zero or one occurrence is allowed within an
|
|
<code><emotion></code> element.</td>
|
|
</tr>
|
|
</tbody>
|
|
</table>
|
|
|
|
<p>One or more <code><dimension></code> elements jointly describe an <a
|
|
href="#term-emotion">emotion</a> or a <a
|
|
href="#term-emotion-related-state">related state</a> in terms of a set of <a
|
|
href="#term-emotion-dimensions">emotion dimensions</a>. The names of the
|
|
emotion dimensions MUST belong to a clearly-identified set of dimension names,
|
|
which MUST be defined according to <a href="#s3">Defining vocabularies for
|
|
representing emotions</a>.</p>
|
|
|
|
<p>The set of legal values of the <code>name</code> attribute is indicated in
|
|
the <code>dimension-set</code> attribute of the enclosing <a
|
|
href="#s2.1.1"><code><emotionml></code></a> or <a
|
|
href="#s2.1.2"><code><emotion></code></a> element. Different sets can be
|
|
used, depending on the requirements of the use case.</p>
|
|
|
|
<p>The position on an emotion dimension MUST be specified as a <a
|
|
href="#s2.5">Scale value</a>, either as a static value in the <a
|
|
href="#s2.5.1"><code>value</code></a> attribute, or as a dynamic trace over
|
|
time using the <a href="#s2.5.2"><code><trace></code></a> element. </p>
|
|
|
|
<p>Examples:</p>
|
|
|
|
<p>One of the most widespread sets of emotion dimensions used (sometimes by
|
|
different names) is the combination of valence, arousal and potency. The
|
|
following example is a state of rather low arousal, very positive valence, and
|
|
high potency -- in other words, a relaxed, positive state with a feeling of
|
|
being in control of the situation. The example uses the
|
|
Pleasure-Arousal-Dominance (PAD) vocabulary from [<a
|
|
href="#ref-emotion-voc">Vocabularies for EmotionML</a>]:</p>
|
|
<pre class="example"><emotion dimension-set="http://www.w3.org/TR/emotion-voc/xml#pad-dimensions">
|
|
<dimension name="arousal" value="0.3"/><!-- lower-than-average arousal -->
|
|
<dimension name="pleasure" value="0.9"/><!-- very high positive valence -->
|
|
<dimension name="dominance" value="0.8"/><!-- relatively high potency -->
|
|
</emotion></pre>
|
|
|
|
<p>In some use cases, custom sets of application-specific dimensions will be
|
|
required. The following example uses a custom set of dimensions, defining a
|
|
single dimension "friendliness".</p>
|
|
<pre class="example"><emotion dimension-set="http://www.example.com/custom/dimension/friendliness.xml">
|
|
<dimension name="friendliness" value="0.2"/><!-- a pretty unfriendly person -->
|
|
</emotion></pre>
|
|
|
|
<p>The usual way to represent the intensity of an emotion would be the
|
|
<code>value</code> attribute of a <code><category></code>. However, if
|
|
only the intensity of an emotion is annotated, but not its nature, this can be
|
|
done by using an "intensity" dimension. Thus, an emotional state's "strength"
|
|
or "intensity" can be described independently from categorical or dimensional
|
|
descriptions, as shown by the following example.</p>
|
|
<pre class="example"><emotion dimension-set="http://www.w3.org/TR/emotion-voc/xml#intensity-dimension">
|
|
<dimension name="intensity" value="0.2"/><!-- not in a strong emotional state -->
|
|
</emotion></pre>
|
|
|
|
<p></p>
|
|
|
|
<h4 id="s2.2.3">2.2.3 The <code><appraisal></code> element</h4>
|
|
|
|
<table class="defn" cellspacing="0" cellpadding="5" width="98%"
|
|
summary="property definition">
|
|
<tbody>
|
|
<tr>
|
|
<th>Annotation</th>
|
|
<td><code><appraisal></code></td>
|
|
</tr>
|
|
<tr>
|
|
<th>Definition</th>
|
|
<td>One or more <code><appraisal></code> elements jointly describe
|
|
an emotion or a related state according to an emotion appraisal
|
|
vocabulary.</td>
|
|
</tr>
|
|
<tr>
|
|
<th>Children</th>
|
|
<td><a href="#s2.5.2"><code><trace></code></a>: An
|
|
<code><appraisal></code> MAY contain either a <code>value</code>
|
|
attribute or a <code><trace></code> element. </td>
|
|
</tr>
|
|
<tr>
|
|
<th>Attributes</th>
|
|
<td><ul>
|
|
<li><strong>Required</strong>:
|
|
<ul>
|
|
<li><code>name</code>, the name of the appraisal, which must be
|
|
contained in the set of appraisals identified in the enclosing
|
|
<a href="#s2.1.1"><code><emotionml></code></a> or <a
|
|
href="#s2.1.2"><code><emotion></code></a> element's
|
|
<code>appraisal-set</code> attribute.</li>
|
|
</ul>
|
|
</li>
|
|
<li><strong>Optional</strong>:
|
|
<ul>
|
|
<li><a href="#s2.5.1"><code>value</code></a>: An
|
|
<code><appraisal></code> MAY contain either a
|
|
<code>value</code> attribute or a <code><trace></code>
|
|
element. </li>
|
|
<li><a href="#s2.3.1"><code>confidence</code></a>, the
|
|
annotator's confidence that the annotation given for this
|
|
appraisal is correct.</li>
|
|
</ul>
|
|
</li>
|
|
</ul>
|
|
</td>
|
|
</tr>
|
|
<tr>
|
|
<th>Occurrence</th>
|
|
<td><code><appraisal></code> elements occur as children of
|
|
<code><emotion></code>. For any given appraisal name in the set,
|
|
zero or one occurrence is allowed within an
|
|
<code><emotion></code> element.</td>
|
|
</tr>
|
|
</tbody>
|
|
</table>
|
|
|
|
<p>One or more <code><appraisal></code> elements jointly describe an <a
|
|
href="#term-emotion">emotion</a> or a <a
|
|
href="#term-emotion-related-state">related state</a> in terms of a set of <a
|
|
href="#term-appraisal">appraisals</a>. The names of the appraisals MUST belong
|
|
to a clearly-identified set of appraisal names, which MUST be defined according
|
|
to <a href="#s3">Defining vocabularies for representing emotions</a>.</p>
|
|
|
|
<p>The set of legal values of the <code>name</code> attribute is indicated in
|
|
the <code>appraisal-set</code> attribute of the enclosing
|
|
<code><emotion></code> element. Different sets can be used, depending on
|
|
the requirements of the use case.</p>
|
|
|
|
<p>The degree to which an appraisal is present MAY be specified as a <a
|
|
href="#s2.5">Scale value</a>, either as a static value in the <a
|
|
href="#s2.5.1"><code>value</code></a> attribute, or as a dynamic trace over
|
|
time using the <a href="#s2.5.2"><code><trace></code></a> element. </p>
|
|
|
|
<p>Examples:</p>
|
|
|
|
<p>One of the most widespread sets of emotion appraisals used is the appraisals
|
|
set proposed by Klaus Scherer, covering aspects of novelty, intrinsic
|
|
pleasantness, goal/need significance, coping potential, and norm/self
|
|
compatibility. Another very widespread set of emotion appraisals, used in
|
|
particular in computational models of emotion, is the OCC set of appraisals (<a
|
|
href="#ref-Ortony1988">Ortony et al., 1988</a>), which includes the
|
|
consequences of events for oneself or for others, the actions of others and the
|
|
perception of objects. Using Scherer's appraisals from [<a
|
|
href="#ref-emotion-voc">Vocabularies for EmotionML</a>], the following example
|
|
is a state arising from the evaluation of an unpredicted and quite unpleasant
|
|
event:</p>
|
|
<pre class="example"><emotion appraisal-set="http://www.w3.org/TR/emotion-voc/xml#scherer-appraisals">
|
|
<appraisal name="suddenness" value="0.8"/>
|
|
<appraisal name="intrinsic-pleasantness" value="0.2"/>
|
|
</emotion></pre>
|
|
|
|
<p>In some use cases, custom sets of application-specific appraisals will be
|
|
required. The following example uses a custom set of appraisals, defining the
|
|
single appraisal "likelihood".</p>
|
|
<pre class="example"><emotion appraisal-set="http://www.example.com/custom/appraisal/likelihood.xml">
|
|
<appraisal name="likelihood" value="0.8"/><!-- a very predictable event -->
|
|
</emotion></pre>
|
|
|
|
<p></p>
|
|
|
|
<h4 id="s2.2.4">2.2.4 The <code><action-tendency></code> element</h4>
|
|
|
|
<table class="defn" cellspacing="0" cellpadding="5" width="98%"
|
|
summary="property definition">
|
|
<tbody>
|
|
<tr>
|
|
<th>Annotation</th>
|
|
<td><code><action-tendency></code></td>
|
|
</tr>
|
|
<tr>
|
|
<th>Definition</th>
|
|
<td>One or more <code><action-tendency></code> elements jointly
|
|
describe an emotion or a related state according to an emotion action
|
|
tendency vocabulary.</td>
|
|
</tr>
|
|
<tr>
|
|
<th>Children</th>
|
|
<td><a href="#s2.5.2"><code><trace></code></a>: An
|
|
<code><action-tendency></code> MAY contain either a
|
|
<code>value</code> attribute or a <code><trace></code> element.
|
|
</td>
|
|
</tr>
|
|
<tr>
|
|
<th>Attributes</th>
|
|
<td><ul>
|
|
<li><strong>Required</strong>:
|
|
<ul>
|
|
<li><code>name</code>, the name of the action tendency, which
|
|
must be contained in the set of action tendencies identified in
|
|
the enclosing <a
|
|
href="#s2.1.1"><code><emotionml></code></a> or <a
|
|
href="#s2.1.2"><code><emotion></code></a> element's
|
|
<code>action-tendency-set</code> attribute.</li>
|
|
</ul>
|
|
</li>
|
|
<li><strong>Optional</strong>:
|
|
<ul>
|
|
<li><a href="#s2.5.1"><code>value</code></a>: An
|
|
<code><action-tendency></code> MAY contain either a
|
|
<code>value</code> attribute or a <code><trace></code>
|
|
element. </li>
|
|
<li><a href="#s2.3.1"><code>confidence</code></a>, the
|
|
annotator's confidence that the annotation given for this
|
|
action tendency is correct.</li>
|
|
</ul>
|
|
</li>
|
|
</ul>
|
|
</td>
|
|
</tr>
|
|
<tr>
|
|
<th>Occurrence</th>
|
|
<td><code><action-tendency></code> elements occur as children of
|
|
<code><emotion></code>. For any given action tendency name in the
|
|
set, zero or one occurrence is allowed within an
|
|
<code><emotion></code> element.</td>
|
|
</tr>
|
|
</tbody>
|
|
</table>
|
|
|
|
<p>One or more <code><action-tendency></code> elements jointly describe
|
|
an <a href="#term-emotion">emotion</a> or a <a
|
|
href="#term-emotion-related-state">related state</a> in terms of a set of <a
|
|
href="#term-action-tendency">action tendencies</a>. The names of the action
|
|
tendencies MUST belong to a clearly-identified set of action tendency names,
|
|
which MUST be defined according to <a href="#s3">Defining vocabularies for
|
|
representing emotions</a>.</p>
|
|
|
|
<p>The set of legal values of the <code>name</code> attribute is indicated in
|
|
the <code>action-tendency-set</code> attribute of the enclosing
|
|
<code><emotion></code> element. Different sets can be used, depending on
|
|
the requirements of the use case.</p>
|
|
|
|
<p>The degree to which an action tendency is present MAY be specified as a <a
|
|
href="#s2.5">Scale value</a>, either as a static value in the <a
|
|
href="#s2.5.1"><code>value</code></a> attribute, or as a dynamic trace over
|
|
time using the <a href="#s2.5.2"><code><trace></code></a> element. </p>
|
|
|
|
<p>Examples:</p>
|
|
|
|
<p>One well known use of action tendencies is by N. Frijda. This model uses a
|
|
number of action tendencies that are low level, diffuse behaviors from which
|
|
more concrete actions could be determined. It is provided in [<a
|
|
href="#ref-emotion-voc">Vocabularies for EmotionML</a>]. An example of someone
|
|
attempting to attract someone they like by being confident, strong and
|
|
attentive might look like this: </p>
|
|
<pre class="example"><emotion action-tendency-set="http://www.w3.org/TR/emotion-voc/xml#frijda-action-tendencies">
|
|
<action-tendency name="approach" value="0.7"/> <!-- get close -->
|
|
<action-tendency name="being-with" value="0.8"/> <!-- be happy -->
|
|
<action-tendency name="attending" value="0.7"/> <!-- pay attention -->
|
|
<action-tendency name="dominating" value="0.7"/> <!-- be assertive -->
|
|
</emotion></pre>
|
|
|
|
<p>In some use cases, custom sets of application-specific action tendencies
|
|
will be required. The following example shows control values for a robot who
|
|
works in a factory and uses a custom set of action-tendencies, defining example
|
|
actions for a robot. In the example, the robot has very low battery, so it
|
|
needs to get ready to charge its battery and stop its work of picking up
|
|
boxes.</p>
|
|
<pre class="example"><emotion action-tendency-set="http://www.example.com/custom/action/robot.xml">
|
|
<action-tendency name="charge-battery" value="0.9"/> <!-- need to charge battery soon -->
|
|
<action-tendency name="pickup-boxes" value="0.3"/> <!-- feeling tired, avoid work -->
|
|
</emotion></pre>
|
|
|
|
<p></p>
|
|
|
|
<h3 id="s2.3">2.3 Meta-information</h3>
|
|
|
|
<h4 id="s2.3.1">2.3.1 The <code>confidence</code> attribute</h4>
|
|
|
|
<table class="defn" cellspacing="0" cellpadding="5" width="98%"
|
|
summary="property definition">
|
|
<tbody>
|
|
<tr>
|
|
<th>Annotation</th>
|
|
<td><code>confidence</code></td>
|
|
</tr>
|
|
<tr>
|
|
<th>Definition</th>
|
|
<td>A representation of the degree of confidence or probability that a
|
|
certain element of the representation is correct. </td>
|
|
</tr>
|
|
<tr>
|
|
<th>Occurrence</th>
|
|
<td>An optional attribute of <a
|
|
href="#s2.2.1"><code><category></code></a>, <a
|
|
href="#s2.2.2"><code><dimension></code></a>, <a
|
|
href="#s2.2.3"><code><appraisal></code></a> and <a
|
|
href="#s2.2.4"><code><action-tendency></code></a> elements.</td>
|
|
</tr>
|
|
</tbody>
|
|
</table>
|
|
|
|
<p></p>
|
|
|
|
<p>Confidence MAY be indicated separately for each of the <a
|
|
href="#s2.2">Representations of emotions and related states</a>. For example,
|
|
the confidence that the <code><category></code> is assumed correctly is
|
|
independent from the confidence that the position on a dimension is correctly
|
|
indicated. </p>
|
|
|
|
<p>Rooted in the tradition of statistics a confidence is given in an interval
|
|
from 0 to 1, resembling a probability. Insofar, the confidence is a <a
|
|
href="#s2.5">Scale value</a>.</p>
|
|
|
|
<p>Examples:</p>
|
|
|
|
<p>In the following, one simple example is provided for each element that can
|
|
carry a <code>confidence</code> attribute. </p>
|
|
|
|
<p>The first example indicates a very high confidence that surprise is the
|
|
emotion to annotate.</p>
|
|
<pre class="example"><emotion category-set="http://www.w3.org/TR/emotion-voc/xml#big6">
|
|
<category name="surprise" confidence="0.95"/>
|
|
</emotion</pre>
|
|
|
|
<p>The next example illustrates using <code>confidence</code> to indicate that
|
|
the annotation of high arousal is probably correct, but the annotation of
|
|
slightly positive pleasure may or may not be correct.</p>
|
|
<pre class="example"><emotion dimension-set="http://www.w3.org/TR/emotion-voc/xml#pad-dimensions">
|
|
<dimension name="arousal" value="0.8" confidence="0.9"/>
|
|
<dimension name="pleasure" value="0.6" confidence="0.3"/>
|
|
</emotion></pre>
|
|
|
|
<p>Finally, an example for the case of intensity: A high confidence is given
|
|
that the emotion has a low intensity.</p>
|
|
<pre class="example"> <emotion dimension-set="http://www.w3.org/TR/emotion-voc/xml#intensity-dimension">
|
|
<dimension name="intensity" value="0.1" confidence="0.8"/>
|
|
</emotion></pre>
|
|
|
|
<p>Note that, as stated, obviously an emotional annotation can be a combination
|
|
of the above, as in the following example: the intensity of the emotion is
|
|
quite probably low, but if we have to guess, we would say the emotion is
|
|
boredom.</p>
|
|
<pre class="example"><emotion category-set="http://www.w3.org/TR/emotion-voc/xml#everyday-categories"
|
|
dimension-set="http://www.w3.org/TR/emotion-voc/xml#intensity-dimension">
|
|
<category name="bored" confidence="0.1"/>
|
|
<dimension name="intensity" value="0.1" confidence="0.8"/>
|
|
</emotion></pre>
|
|
|
|
<p></p>
|
|
|
|
<h4 id="s2.3.2">2.3.2 The <code>expressed-through</code> attribute</h4>
|
|
|
|
<table class="defn" cellspacing="0" cellpadding="5" width="98%"
|
|
summary="property definition">
|
|
<tbody>
|
|
<tr>
|
|
<th>Annotation</th>
|
|
<td><code>expressed-through</code></td>
|
|
</tr>
|
|
<tr>
|
|
<th>Definition</th>
|
|
<td>The modality, or list of modalities, through which the emotion is
|
|
expressed. An attribute of type <code>xsd:nmtokens</code> which
|
|
contains a space delimited set of values from an open set of values
|
|
including: {<code>gaze</code>, <code>face</code>, <code>head</code>,
|
|
<code>torso</code>, <code>gesture</code>, <code>leg</code>,
|
|
<code>voice</code>, <code>text</code>, <code>locomotion</code>,
|
|
<code>posture</code>, <code>physiology</code>, ...}.</td>
|
|
</tr>
|
|
<tr>
|
|
<th>Occurrence</th>
|
|
<td>An optional attribute of <a
|
|
href="#s2.1.2"><code><emotion></code></a> elements.</td>
|
|
</tr>
|
|
</tbody>
|
|
</table>
|
|
|
|
<p>The <code>expressed-through</code> attribute describes the modality through
|
|
which an emotion is produced, usually by a human being. It is not the technical
|
|
modality by which it was detected, e.g. "face" rather than "camera" and "voice"
|
|
rather than "microphone". The <code>expressed-through</code> attribute is
|
|
agnostic about the use case: when detecting emotion, it represents the modality
|
|
from which the emotion has been detected; when generating emotion-related
|
|
system behavior, it represents the modality through which the emotion is to be
|
|
expressed.</p>
|
|
|
|
<p>The list of values provided covers a broad range of modalities through which
|
|
emotions may be expressed. These values SHOULD be used if they are appropriate.
|
|
The list is an open set in order to allow for more fine-grained distinctions
|
|
such as "eyes" vs. "mouth" etc.</p>
|
|
|
|
<p>The <code>expressed-through</code> attribute is not specific about the
|
|
sensors used for observing the modality. These can be specified using the <a
|
|
href="#s2.3.3"><code><info></code></a> element, or by the
|
|
<code>emma:mode</code> attribute in an enclosing [<a href="#ref-EMMA">EMMA</a>]
|
|
document.</p>
|
|
|
|
<p>Example:</p>
|
|
|
|
<p>In the following example the emotion is expressed through the voice.</p>
|
|
<pre class="example"><emotion category-set="http://www.w3.org/TR/emotion-voc/xml#everyday-categories"
|
|
expressed-through="voice">
|
|
<category name="satisfied"/>
|
|
</emotion></pre>
|
|
|
|
<p>In case of multimodal expression of an emotion, a list of space separated
|
|
modalities can be indicated in the <code>expressed-through</code> attribute,
|
|
like in the following example in which the two values "face" and "voice" are
|
|
used.</p>
|
|
<pre class="example"><emotion category-set="http://www.w3.org/TR/emotion-voc/xml#everyday-categories"
|
|
expressed-through="face voice">
|
|
<category name="satisfied"/>
|
|
</emotion>></pre>
|
|
|
|
<p></p>
|
|
|
|
<p>See also the examples in sections <a href="#s5.1.2">5.1.2 Automatic
|
|
recognition of emotions</a>, <a href="#s5.1.3">5.1.3 Generation of
|
|
emotion-related system behavior</a> and <a href="#s5.2.3">5.2.3 Use with
|
|
SMIL</a>.</p>
|
|
|
|
<p></p>
|
|
|
|
<h4 id="s2.3.3">2.3.3 The <code><info></code> element</h4>
|
|
|
|
<table class="defn" cellspacing="0" cellpadding="5" width="98%"
|
|
summary="property definition">
|
|
<tbody>
|
|
<tr>
|
|
<th>Annotation</th>
|
|
<td><code><info></code></td>
|
|
</tr>
|
|
<tr>
|
|
<th>Definition</th>
|
|
<td>This element can be used to annotate arbitrary metadata.</td>
|
|
</tr>
|
|
<tr>
|
|
<th>Children</th>
|
|
<td>One or more elements in a different namespace than the <a
|
|
href="#s4.1">EmotionML namespace</a>, providing metadata.</td>
|
|
</tr>
|
|
<tr>
|
|
<th>Attributes</th>
|
|
<td><ul>
|
|
<li><strong>Optional</strong>:
|
|
<ul>
|
|
<li><code>id</code>, a unique identifier for the info element, of
|
|
type <code>xsd:ID</code>.</li>
|
|
</ul>
|
|
</li>
|
|
</ul>
|
|
</td>
|
|
</tr>
|
|
<tr>
|
|
<th>Occurence</th>
|
|
<td>A single <code><info></code> element MAY occur as a child of
|
|
the <code><emotionml</code>> root tag to indicate global
|
|
metadata, i.e. the annotations are valid for the document scope;
|
|
furthermore, a single <code><info></code> element MAY occur as a
|
|
child of each <code><emotion></code> element to indicate local
|
|
metadata that is only valid for that <code><emotion></code>
|
|
element.</td>
|
|
</tr>
|
|
</tbody>
|
|
</table>
|
|
|
|
<p>This element can contain arbitrary XML data in a different namespace (one
|
|
option could be [<a href="#ref-rdf">RDF</a>] data), either on a document global
|
|
level or on a local "per annotation element" level.</p>
|
|
|
|
<p>Several initiatives of standardizing metadata exist, such as [<a
|
|
href="#ref-imdi">IMDI</a>] and [<a href="#ref-clarin">CLARIN</a>]. Metadata may
|
|
contain information on a large spectrum of elements such as: location
|
|
description (continent, country, address), content type (e.g., genre, task,
|
|
modalities), session (title, a recording date, a group of participants); each
|
|
participant may be defined by her role in the session (e.g. annotator, filmer),
|
|
her name, her social family role, etc.</p>
|
|
|
|
<p>Examples:</p>
|
|
|
|
<p>In the following example, the automatic classification for an annotation
|
|
document was performed by a classifier based on Gaussian Mixture Models (GMM);
|
|
the speakers of the annotated elements were of different German origins.</p>
|
|
<pre class="example"><emotionml xmlns="http://www.w3.org/2009/10/emotionml"
|
|
xmlns:classifiers="http://www.example.com/meta/classify/"
|
|
xmlns:origin="http://www.example.com/meta/local/"
|
|
category-set="http://www.w3.org/TR/emotion-voc/xml#big6">
|
|
<info>
|
|
<classifiers:classifier classifiers:name="GMM"/>
|
|
</info>
|
|
|
|
<emotion>
|
|
<info><origin:localization value="bavarian"/></info>
|
|
<category name="happiness"/>
|
|
</emotion>
|
|
|
|
<emotion>
|
|
<info><origin:localization value="swabian"/></info>
|
|
<category name="sadness"/>
|
|
</emotion>
|
|
</emotionml></pre>
|
|
|
|
<p></p>
|
|
|
|
<p>The following example uses the IMDI metadata language to represent
|
|
information about the annotator who produced the emotion annotation in the
|
|
current document, in a global <code><info></code> element.</p>
|
|
<pre><emotionml xmlns="http://www.w3.org/2009/10/emotionml"
|
|
xmlns:imdi="http://www.mpi.nl/IMDI/Schema/IMDI">
|
|
<info>
|
|
<imdi:Actors>
|
|
<imdi:Actor>
|
|
<imdi:Role>Annotator</imdi:Role>
|
|
<imdi:Name>John</imdi:Name>
|
|
<imdi:FullName>John Smith Junior</imdi:FullName>
|
|
<imdi:Code>JS</imdi:Code>
|
|
<imdi:FamilySocialRole>Teacher</imdi:FamilySocialRole>
|
|
...
|
|
</imdi:Actor>
|
|
</imdi:Actors>
|
|
</info>
|
|
...
|
|
<emotion>...</emotion>
|
|
<emotion>...</emotion>
|
|
</emotionml> </pre>
|
|
|
|
<p></p>
|
|
|
|
<p>The following example illustrates how <code><info></code> can be used
|
|
for annotating information on sensors through which an affective signal has
|
|
been detected. In the global <code><info></code> section, the sensors
|
|
used in the particular scenario are specified. Apart from their ID, information
|
|
on the modality observed by this sensor is provided as well as information on
|
|
the confidence for that sensor. In this example, the modality "posture" is
|
|
observed by a camera and a chair equipped with pressure sensors. For some
|
|
reason it is decided that emotion estimates based on camera data should be
|
|
trusted more than those based on chair data. Within the
|
|
<code><emotion></code> elements, <code><info></code> is used to
|
|
specify which sensor has been used to calculate the actual emotion value.</p>
|
|
<pre><emotionml xmlns="http://www.w3.org/2009/10/emotionml"
|
|
xmlns:sensors="http://www.example.com/meta/sensors/"
|
|
category-set="http://www.w3.org/TR/emotion-voc/xml#everyday-categories">
|
|
|
|
<info>
|
|
<sensors:sensor id="camera1" confidence="0.9" expressed-through="posture"/>
|
|
<sensors:sensor id="chair" confidence="0.3" expressed-through="posture"/>
|
|
...
|
|
</info>
|
|
|
|
<emotion expressed-through="posture">
|
|
<info>
|
|
<sensors:sensor idref="camera1"/>
|
|
</info>
|
|
<category name="angry"/>
|
|
</emotion>
|
|
|
|
<emotion expressed-through="posture">
|
|
<info>
|
|
<sensors:sensor idref="chair"/>
|
|
</info>
|
|
<category name="neutral"/>
|
|
</emotion>
|
|
|
|
</emotionml></pre>
|
|
|
|
<p></p>
|
|
|
|
<h3 id="s2.4">2.4 References and time</h3>
|
|
|
|
<h4 id="s2.4.1">2.4.1 The <code><reference></code> element</h4>
|
|
|
|
<table class="defn" cellspacing="0" cellpadding="5" width="98%"
|
|
summary="property definition">
|
|
<tbody>
|
|
<tr>
|
|
<th>Annotation</th>
|
|
<td><code><reference></code></td>
|
|
</tr>
|
|
<tr>
|
|
<th>Definition</th>
|
|
<td>References may be used to relate the emotion annotation to the "rest
|
|
of the world", more specifically to the emotional expression, the
|
|
experiencing subject, the trigger, and the target of the emotion.</td>
|
|
</tr>
|
|
<tr>
|
|
<th>Children</th>
|
|
<td>None</td>
|
|
</tr>
|
|
<tr>
|
|
<th>Attributes</th>
|
|
<td><ul>
|
|
<li><strong>Required</strong>:
|
|
<ul>
|
|
<li><code>uri</code>, a URI identifying the actual reference
|
|
target. The URI MAY be extended by a media fragment, as
|
|
explained in section <a href="#s2.4.2.4">2.4.2.4</a>.</li>
|
|
</ul>
|
|
</li>
|
|
<li><strong>Optional</strong>:
|
|
<ul>
|
|
<li><code>role</code>, the type of relation between the emotion
|
|
and the external item referred to; one of
|
|
"<code>expressedBy</code>" (default),
|
|
"<code>experiencedBy</code>", "<code>triggeredBy</code>",
|
|
"<code>targetedAt</code>".</li>
|
|
<li><code>media-type</code>, an attribute of type
|
|
<code>xsd:string</code> holding the MIME type of the data that
|
|
the <code>uri</code> attribute points to.</li>
|
|
</ul>
|
|
</li>
|
|
</ul>
|
|
</td>
|
|
</tr>
|
|
<tr>
|
|
<th>Occurrence</th>
|
|
<td>Multiple <code><reference></code> elements MAY occur as
|
|
children of <code><emotion></code>.</td>
|
|
</tr>
|
|
</tbody>
|
|
</table>
|
|
|
|
<p>A <code><reference></code> element provides a link to media as a URI
|
|
[<a href="#ref-rfc3986">RFC3986</a>]. The semantics of references are described
|
|
by the <code>role</code> attribute which, if present, MUST have one of four
|
|
values:</p>
|
|
<ul>
|
|
<li>"<code>expressedBy</code>" indicates that the reference points to
|
|
observable behavior expressing the emotion. This is the default value if
|
|
the <code>role</code> attribute is not explicitly stated;</li>
|
|
<li>"<code>experiencedBy</code>" indicates that the reference points to the
|
|
subject experiencing the emotion;</li>
|
|
<li>"<code>triggeredBy</code>" indicates that the reference points to an
|
|
emotion-eliciting event that caused an emotion and/or related
|
|
appraisals;</li>
|
|
<li>"<code>targetedAt</code>" indicates that the reference points to an
|
|
object towards which an emotion-related action, or action tendency, is
|
|
directed.</li>
|
|
</ul>
|
|
|
|
<p>For reference targets representing a period of time, start and end time MAY
|
|
be denoted by using the media fragments syntax, as explained in section <a
|
|
href="#s2.4.2.4">2.4.2.4</a>.</p>
|
|
|
|
<p>The <code>media-type</code> attribute MAY be used to differentiate between
|
|
different media types such as audio, video, text, etc. </p>
|
|
|
|
<p>There is no restriction regarding the number of
|
|
<code><reference></code> elements that MAY occur as children of
|
|
<code><emotion></code>.</p>
|
|
|
|
<p>Examples:</p>
|
|
|
|
<p>The following example illustrates the reference to two different URIs having
|
|
a different <code>role</code> with respect to the emotion: one reference points
|
|
to the emotion's expression, a video clip showing a user expressing the
|
|
emotion; the other reference points to the trigger that caused the emotion, in
|
|
this case another video clip that was seen by the person who expressed the
|
|
emotion.</p>
|
|
|
|
<p></p>
|
|
<pre class="example"><emotion ... >
|
|
...
|
|
<reference uri="http://www.example.com/data/video/v1.avi?t=2,13" role="expressedBy"/>
|
|
<reference uri="http://www.example.com/events/e12.xml" role="triggeredBy"/>
|
|
</emotion></pre>
|
|
|
|
<p>Several references may follow as children of one
|
|
<code><emotion></code> tag, even having the same <code>role</code>; for
|
|
example, the following annotation refers to a portion of a video and to
|
|
physiological sensor data, both of which expressed the emotion:</p>
|
|
<pre class="example"><emotion ... >
|
|
...
|
|
<reference uri="http://www.example.com/data/video/v1.avi?t=2,13" role="expressedBy"/>
|
|
<reference uri="http://www.example.com/data/physio/ph7.txt" role="expressedBy"/>
|
|
</emotion></pre>
|
|
|
|
<p>It is possible to explicitly indicate the MIME type of the item that the
|
|
reference refers to:</p>
|
|
<pre class="example"><emotion ... >
|
|
...
|
|
<reference uri="http://www.example.com/data/video/v1.avi?t=2,13" media-type="video/mp4"/>
|
|
</emotion></pre>
|
|
|
|
<p></p>
|
|
|
|
<h4 id="s2.4.2">2.4.2 Timestamps</h4>
|
|
|
|
<h5 id="s2.4.2.1">2.4.2.1 Absolute time</h5>
|
|
|
|
<table class="defn" cellspacing="0" cellpadding="5" width="98%"
|
|
summary="property definition">
|
|
<tbody>
|
|
<tr>
|
|
<th>Annotation</th>
|
|
<td><code>start</code>, <code>end</code></td>
|
|
</tr>
|
|
<tr>
|
|
<th>Definition</th>
|
|
<td>Attributes to denote the starting and ending absolute times. They are
|
|
of type <code>xsd:nonNegativeInteger</code> and indicate the number of
|
|
milliseconds since 1 January 1970 00:00:00 GMT.</td>
|
|
</tr>
|
|
<tr>
|
|
<th>Occurrence</th>
|
|
<td>The attributes MAY occur inside an <code><emotion></code>
|
|
element.</td>
|
|
</tr>
|
|
</tbody>
|
|
</table>
|
|
|
|
<p><code>start</code> and <code>end</code> attributes denote the absolute
|
|
starting and ending times at which an <a href="#term-emotion">emotion</a> or <a
|
|
href="#term-emotion-related-state">related state</a> happened. This might be
|
|
used for example with an "emotional diary" application. </p>
|
|
|
|
<p>Examples:</p>
|
|
|
|
<p>In the following example, the emotion category "surprise" is annotated,
|
|
immediately followed by the category "happiness". The <code>start</code> and
|
|
<code>end</code> attributes specify for each <code>emotion</code> element the
|
|
absolute beginning and ending times.</p>
|
|
<pre class="example"><emotion category-set="http://www.w3.org/TR/emotion-voc/xml#big6"
|
|
start="1268647200" end="1268647330">
|
|
<category name="surprise"/>
|
|
</emotion>
|
|
<emotion category-set="http://www.w3.org/TR/emotion-voc/xml#big6"
|
|
start="1268647331" end="1268647400">
|
|
<category name="happiness"/>
|
|
</emotion></pre>
|
|
|
|
<p>The <code>end</code> value MUST be greater than or equal to the
|
|
<code>start</code> value.</p>
|
|
|
|
<p>The ECMAScript Date object's getTime() function is a way to determine the
|
|
absolute time.</p>
|
|
|
|
<h5 id="s2.4.2.2">2.4.2.2 Duration</h5>
|
|
|
|
<table class="defn" summary="property definition" width="98%" cellpadding="5"
|
|
cellspacing="0">
|
|
<tbody>
|
|
<tr>
|
|
<th>Annotation</th>
|
|
<th><code>duration</code></th>
|
|
</tr>
|
|
<tr>
|
|
<th>Definition</th>
|
|
<td>Attribute <span>of type <code>xsd:nonNegativeInteger</code></span>,
|
|
defaulting to zero. It specifies the duration of the event in
|
|
milliseconds.</td>
|
|
</tr>
|
|
<tr>
|
|
<th>Occurrence</th>
|
|
<td>This attribute MAY occur inside an <code><emotion></code>
|
|
element.</td>
|
|
</tr>
|
|
</tbody>
|
|
</table>
|
|
|
|
<p></p>
|
|
|
|
<p>The duration of an input in milliseconds MAY be specified with the
|
|
<code>duration</code> attribute. The <code>duration</code> attribute MAY be
|
|
used either in combination with the <code>start</code> or
|
|
<code>offset-to-start</code> attribute or independently.</p>
|
|
|
|
<p>A <code>start</code> or <code>offset-to-start</code> attribute together with
|
|
the <code>duration</code> attribute set to zero MAY be used to indicate a
|
|
single timestamp on a time axis.</p>
|
|
|
|
<p>Examples:</p>
|
|
|
|
<p>In the following example, the <code>start</code> and <code>duration</code>
|
|
of the emotion category "surprise" are annotated:</p>
|
|
<pre class="example"><emotion category-set="http://www.w3.org/TR/emotion-voc/xml#big6"
|
|
start="1268647200" duration="130">
|
|
<category name="surprise"/>
|
|
</emotion></pre>
|
|
|
|
<p></p>
|
|
|
|
<h5 id="s2.4.2.3">2.4.2.3 Relative time</h5>
|
|
|
|
<table class="defn" summary="property definition" width="98%" cellpadding="5"
|
|
cellspacing="0">
|
|
<tbody>
|
|
<tr>
|
|
<th>Annotation</th>
|
|
<th><code>time-ref-uri</code></th>
|
|
</tr>
|
|
<tr>
|
|
<th>Definition</th>
|
|
<td>Attribute of type <code>xsd:anyURI</code> indicating the URI used to
|
|
anchor the relative timestamp.</td>
|
|
</tr>
|
|
<tr>
|
|
<th>Occurrence</th>
|
|
<td>This attribute MAY occur inside an <code><emotion></code>
|
|
element.</td>
|
|
</tr>
|
|
<tr>
|
|
<th>Annotation</th>
|
|
<th><code>time-ref-anchor-point</code></th>
|
|
</tr>
|
|
<tr>
|
|
<th>Definition</th>
|
|
<td>Attribute with a value of <code>start</code> or <code>end</code>,
|
|
defaulting to <code>start</code>. It indicates whether to measure the
|
|
time from the start or end of the interval designated with
|
|
<code>time-ref-uri</code>.</td>
|
|
</tr>
|
|
<tr>
|
|
<th>Occurrence</th>
|
|
<td>This attribute MAY occur inside an <code><emotion></code>
|
|
element.</td>
|
|
</tr>
|
|
<tr>
|
|
<th>Annotation</th>
|
|
<th><code>offset-to-start</code></th>
|
|
</tr>
|
|
<tr>
|
|
<th>Definition</th>
|
|
<td>Attribute <span>of type <code>xsd:integer</code></span>, defaulting
|
|
to zero. It specifies the offset in milliseconds for the start of input
|
|
from the anchor point designated with
|
|
<span><code>time-ref-uri</code></span> and
|
|
<span><code>time-ref-anchor-point</code></span></td>
|
|
</tr>
|
|
<tr>
|
|
<th>Occurrence</th>
|
|
<td>This attribute MAY occur inside an <code><emotion></code>
|
|
element.</td>
|
|
</tr>
|
|
</tbody>
|
|
</table>
|
|
|
|
<p>Relative timestamps define the start of an input relative to the start or
|
|
end of a reference interval such as another input.</p>
|
|
|
|
<p></p>
|
|
|
|
<p>The reference interval is designated with <code>time-ref-uri</code>
|
|
attribute. This MAY be combined with <code>time-ref-anchor-point</code>
|
|
attribute to specify whether the anchor point is the start or end of this
|
|
interval. The start of an input relative to this anchor point is then specified
|
|
with <code>offset-to-start</code> attribute.</p>
|
|
|
|
<p>The <code>time-ref-uri</code> attribute can point to a custom-defined
|
|
timestamp or can be, for example, a session identifier.</p>
|
|
|
|
<p></p>
|
|
|
|
<p>Examples:</p>
|
|
|
|
<p>Here is an example where the emotion "surprise" occurs two seconds after the
|
|
reference time point:</p>
|
|
<pre class="example"><emotion category-set="http://www.w3.org/TR/emotion-voc/xml#big6"
|
|
time-ref-uri="#my_session_id" offset-to-start="2000">
|
|
<category name="surprise"/>
|
|
</emotion></pre>
|
|
|
|
<p></p>
|
|
|
|
<h5 id="s2.4.2.4">2.4.2.4 Timing in media</h5>
|
|
|
|
<table class="defn" cellspacing="0" cellpadding="5" width="98%"
|
|
summary="property definition">
|
|
<tbody>
|
|
<tr>
|
|
<th>Annotation</th>
|
|
<td><code>URI fragment: t</code></td>
|
|
</tr>
|
|
<tr>
|
|
<th>Definition</th>
|
|
<td>Attributes to denote start and endpoint of an annotation in a media
|
|
stream. Allowed values must be conform with the Media Fragments
|
|
Specification [<a href="#ref-MediaFragments">Media Fragments</a>]</td>
|
|
</tr>
|
|
<tr>
|
|
<th>Occurence</th>
|
|
<td>The URI fragment MAY occur in the <code>uri</code> attribute of a
|
|
<code><reference></code> element.</td>
|
|
</tr>
|
|
</tbody>
|
|
</table>
|
|
|
|
<p>Temporal clipping is denoted by the name t, and specified as an interval
|
|
with a begin time and an end time. Either or both may be omitted, with the
|
|
begin time defaulting to 0 seconds and the end time defaulting to the duration
|
|
of the source media. The interval is half-open: the begin time is considered
|
|
part of the interval whereas the end time is considered to be the first time
|
|
point that is not part of the interval. If a single number only is given, this
|
|
is the begin time.</p>
|
|
|
|
<p></p>
|
|
|
|
<p>Temporal clipping can be specified either as Normal Play Time (npt) [<a
|
|
href="http://www.ietf.org/rfc/rfc2326.txt">RFC 2326</a>], as SMPTE timecodes,
|
|
[<a href="#ref-SMPTE">SMPTE</a>], or as real-world clock time (clock) [<a
|
|
href="#ref-rfc2326">RFC 2326</a>]. Begin and end times are always specified in
|
|
the same format. The format is specified by name, followed by a colon (:), with
|
|
npt: being the default.</p>
|
|
|
|
<p></p>
|
|
|
|
<p>Examples:</p>
|
|
|
|
<p>In the following example, the emotion category "happiness" is displayed in
|
|
an audio file called "myAudio.wav" from the 3rd to the 9th second.</p>
|
|
<pre class="example"><emotion category-set="http://www.w3.org/TR/emotion-voc/xml#big6">
|
|
<category name="happiness"/>
|
|
<reference uri="myAudio.wav#t=3,9"/>
|
|
</emotion></pre>
|
|
|
|
<p>In the following example, the emotion category "happiness" is displayed in a
|
|
video file called "myVideo.avi" in SMPTE values, resulting in the time interval
|
|
[120,121.5).</p>
|
|
<pre class="example"><emotion category-set="http://www.w3.org/TR/emotion-voc/xml#big6">
|
|
<category name="happiness"/>
|
|
<reference uri="myVideo.avi#t=smpte-30:0:02:00,0:02:01:15"/>
|
|
</emotion></pre>
|
|
|
|
<p></p>
|
|
|
|
<p>A last example states this in a video file in real-world clock time code, as
|
|
a 1 min interval on 26th Jul 2009 from 11hrs, 19min, 1sec.</p>
|
|
<pre class="example"><emotion category-set="http://www.w3.org/TR/emotion-voc/xml#big6">
|
|
<category name="happiness"/>
|
|
<reference uri="myVideo.avi#t=clock:2009-07-26T11:19:01Z,2009-07-26T11:20:01Z"/>
|
|
</emotion></pre>
|
|
|
|
<p></p>
|
|
|
|
<h3 id="s2.5">2.5 Scale values</h3>
|
|
|
|
<p>Scale values are needed to represent content in <a
|
|
href="#s2.2.1"><code><category></code></a>, <a
|
|
href="#s2.2.2"><code><dimension></code></a>, <a
|
|
href="#s2.2.3"><code><appraisal></code></a> and <a
|
|
href="#s2.2.4"><code><action-tendency></code></a> elements, as well as in
|
|
<a href="#s2.3.1"><code>confidence</code></a>.</p>
|
|
|
|
<p>Representations of scale values can be static or dynamic. A static, constant
|
|
scale value is represented using the <code>value</code> attribute; for dynamic
|
|
values, their evolution over time is expressed using the
|
|
<code><trace></code> element.</p>
|
|
|
|
<h4 id="s2.5.1">2.5.1 The <code>value</code> attribute</h4>
|
|
|
|
<table class="defn" cellspacing="0" cellpadding="5" width="98%"
|
|
summary="property definition">
|
|
<tbody>
|
|
<tr>
|
|
<th>Annotation</th>
|
|
<td><code>value</code></td>
|
|
</tr>
|
|
<tr>
|
|
<th>Definition</th>
|
|
<td>Representation of a static scale value.</td>
|
|
</tr>
|
|
<tr>
|
|
<th>Occurrence</th>
|
|
<td>The <a href="#s2.2.2"><code><dimension></code></a> element MUST
|
|
contain either a <code>value</code> attribute or a
|
|
<code><trace></code> element; <a
|
|
href="#s2.2.1"><code><category></code></a>, <a
|
|
href="#s2.2.3"><code><appraisal></code></a> and <a
|
|
href="#s2.2.4"><code><action-tendency></code></a> MAY contain
|
|
either a <code>value</code> attribute or a <code><trace></code>
|
|
element.</td>
|
|
</tr>
|
|
</tbody>
|
|
</table>
|
|
|
|
<p>The <code>value</code> attribute represents a static scale value of the
|
|
enclosing element.</p>
|
|
|
|
<p>Conceptually, a scale can represent concepts that vary from "nothing" to "a
|
|
lot" (unipolar scales), or concepts that vary between two opposites, from "very
|
|
negative" to "very positive" (bipolar scales). Both are represented in
|
|
EmotionML using floating point values from the interval [0;1]. The min and max
|
|
values of the scale SHOULD be interpreted as the extreme values, for both
|
|
unipolar and bipolar scales. For example in a <code><category></code>, a
|
|
<code>value="0"</code> SHOULD be interpreted to mean absolutely no emotion
|
|
(emotionless); a <code>value="1.0"</code> SHOULD be interpreted to mean emotion
|
|
at maximum intensity (pure uncontrolled emotion). For bipolar scales, such as
|
|
the valence dimension, a value of 0 represents the most negative possible
|
|
value, whereas a value of 1 represents the most positive value possible. The
|
|
neutral middle point of the scale is at 0.5.</p>
|
|
|
|
<p>Here are several examples for the usage of scales with EmotionML. </p>
|
|
<pre class="example"><emotion dimension-set="http://www.w3.org/TR/emotion-voc/xml#fsre-dimensions">
|
|
<dimension name="arousal" value="0.4"/> <!-- a bit less than average arousal -->
|
|
<dimension name="valence" value="0.6"/> <!-- a bit above average valence -->
|
|
</emotion>
|
|
|
|
<emotion category-set="http://www.w3.org/TR/emotion-voc/xml#everyday-categories">
|
|
<category name="angry" value="0.5"/> <!-- anger at medium intensity -->
|
|
</emotion>
|
|
|
|
<emotion appraisal-set="http://www.w3.org/TR/emotion-voc/xml#scherer-appraisals">
|
|
<appraisal name="suddenness" value="0.9"/> <!-- appraisal as a very sudden event -->
|
|
</emotion>
|
|
|
|
<emotion action-tendency-set="http://www.w3.org/TR/emotion-voc/xml#frijda-action-tendencies">
|
|
<action-tendency name="approach" value="0.3"/> <!-- a rather weak tendency to approach -->
|
|
</emotion></pre>
|
|
|
|
<p>Further examples of the <code>value</code> attribute can be found in the
|
|
context of the <a href="#s2.2.1"><code><category></code></a>, <a
|
|
href="#s2.2.2"><code><dimension></code></a>, <a
|
|
href="#s2.2.3"><code><appraisal></code></a> and <a
|
|
href="#s2.2.4"><code><action-tendency></code></a> elements.</p>
|
|
|
|
<h4 id="s2.5.2">2.5.2 The <code><trace></code> element</h4>
|
|
|
|
<table class="defn" cellspacing="0" cellpadding="5" width="98%"
|
|
summary="property definition">
|
|
<tbody>
|
|
<tr>
|
|
<th>Annotation</th>
|
|
<td><code><trace></code></td>
|
|
</tr>
|
|
<tr>
|
|
<th>Definition</th>
|
|
<td>Representation of the time evolution of a dynamic scale value.</td>
|
|
</tr>
|
|
<tr>
|
|
<th>Children</th>
|
|
<td>None</td>
|
|
</tr>
|
|
<tr>
|
|
<th>Attributes</th>
|
|
<td><ul>
|
|
<li><strong>Required</strong>:
|
|
<ul>
|
|
<li><code>freq</code>, a sampling frequency in Hz.</li>
|
|
<li><code>samples</code>, a space-separated list of numeric scale
|
|
values from the interval [0;1] representing the scale value of
|
|
the enclosing element as it changes over time.</li>
|
|
</ul>
|
|
</li>
|
|
</ul>
|
|
</td>
|
|
</tr>
|
|
<tr>
|
|
<th>Occurrence</th>
|
|
<td><p>The <a href="#s2.2.2"><code><dimension></code></a> element
|
|
MUST contain either a <code>value</code> attribute or a
|
|
<code><trace></code> element; <a
|
|
href="#s2.2.1"><code><category></code></a>, <a
|
|
href="#s2.2.3"><code><appraisal></code></a> and <a
|
|
href="#s2.2.4"><code><action-tendency></code></a> MAY contain
|
|
either a <code>value</code> attribute or a <code><trace></code>
|
|
element.</p>
|
|
</td>
|
|
</tr>
|
|
</tbody>
|
|
</table>
|
|
|
|
<p>A <code><trace></code> element represents the time course of a scale
|
|
value.</p>
|
|
|
|
<p>The <code>freq</code> attribute indicates the sampling frequency at which
|
|
the values listed in the <code>samples</code> attribute are given. </p>
|
|
|
|
<p>NOTE: The <code><trace></code> representation requires a periodic
|
|
sampling of values. In order to represent values that are sampled
|
|
aperiodically, separate <code><emotion></code> annotations with
|
|
appropriate timing information and individual <code>value</code> attributes may
|
|
be used.</p>
|
|
|
|
<p>Examples:</p>
|
|
|
|
<p>The following example illustrates the use of a trace to represent an episode
|
|
of fear during which the emotion's intensity is rising, first gradually, then
|
|
quickly to a very high value. Values are taken at a sampling frequency of 10
|
|
Hz, i.e. one value every 100 ms.</p>
|
|
<pre class="example"><emotion category-set="http://www.w3.org/TR/emotion-voc/xml#big6">
|
|
<category name="fear">
|
|
<trace freq="10Hz"
|
|
samples="0.1 0.1 0.15 0.2 0.2 0.25 0.25 0.25 0.3 0.3 0.35 0.5 0.7 0.8 0.85 0.85"/>
|
|
</category>
|
|
</emotion></pre>
|
|
|
|
<p>The following example combines a trace of the appraisal "suddenness" with a
|
|
global confidence that the values represent the facts properly. There is a
|
|
sudden peak of suddenness; the annotator is reasonably certain that the
|
|
annotation is correct:</p>
|
|
<pre class="example"><emotion appraisal-set="http://www.w3.org/TR/emotion-voc/xml#scherer-appraisals">
|
|
<appraisal name="suddenness" confidence="0.75">
|
|
<trace freq="10Hz" samples="0.1 0.1 0.1 0.1 0.1 0.7 0.8 0.8 0.8 0.8 0.4 0.2 0.1 0.1 0.1"/>
|
|
</appraisal>
|
|
</emotion></pre>
|
|
|
|
<p></p>
|
|
|
|
<h2 id="s3">3 Defining vocabularies for representing emotions</h2>
|
|
|
|
<p>EmotionML markup MUST refer to one or more vocabularies to be used for
|
|
representing emotion-related states, as specified in the context of the <a
|
|
href="#s2.1.1"><code><emotionml></code></a> and <a
|
|
href="#s2.1.2"><code><emotion></code></a> elements. Due to the lack of
|
|
agreement in the community, the EmotionML specification does not preview a
|
|
single default set which should apply if no set is indicated. Instead, the user
|
|
MUST explicitly state the set of descriptor names used.</p>
|
|
|
|
<p>The document [<a href="#ref-emotion-voc">Vocabularies for EmotionML</a>]
|
|
provides a number of emotion vocabularies which are likely to be of general
|
|
interest. In order to promote interoperability, users SHOULD verify if one of
|
|
the vocabularies defined in that document is suitable for their application. If
|
|
that is not the case, users can define their own custom vocabularies as defined
|
|
in the present section.</p>
|
|
|
|
<h3 id="s3.1">3.1 Mechanism for defining vocabularies</h3>
|
|
|
|
<p>The syntax for defining emotion vocabularies is based on the element
|
|
<code><vocabulary></code> and its child <code><item></code>. </p>
|
|
|
|
<h4 id="s3.1.1">3.1.1 The <code><vocabulary></code> element</h4>
|
|
|
|
<table class="defn" cellspacing="0" summary="property definition"
|
|
cellpadding="5" width="98%">
|
|
<tbody>
|
|
<tr>
|
|
<th>Annotation</th>
|
|
<td><code><vocabulary></code></td>
|
|
</tr>
|
|
<tr>
|
|
<th>Definition</th>
|
|
<td>Contains the definition of an emotion vocabulary.</td>
|
|
</tr>
|
|
<tr>
|
|
<th>Children</th>
|
|
<td>A <code><vocabulary></code> element MUST contain one ore more
|
|
<a href="#s3.1.2"><code><item></code></a> elements. A
|
|
<code><vocabulary></code> element MAY contain a single <a
|
|
href="#s2.3.3"><code><info></code></a> element, providing
|
|
arbitrary metadata about the vocabulary itself. </td>
|
|
</tr>
|
|
<tr>
|
|
<th>Attributes</th>
|
|
<td><ul>
|
|
<li><strong>Required</strong>:
|
|
<ul>
|
|
<li><code>type</code>, MUST be one of "<code>category</code>",
|
|
"<code>dimension</code>", "<code>appraisal</code>" or
|
|
"<code>action-tendency</code>". </li>
|
|
<li><code>id</code>, an unique vocabulary identifier of type
|
|
<code>xsd:ID</code>. </li>
|
|
</ul>
|
|
</li>
|
|
</ul>
|
|
</td>
|
|
</tr>
|
|
<tr>
|
|
<th>Occurrence</th>
|
|
<td><p>One or more <code><vocabulary></code> elements MAY occur as
|
|
direct children of an <a
|
|
href="#s2.1.1"><code><emotionml></code></a> element. </p>
|
|
</td>
|
|
</tr>
|
|
</tbody>
|
|
</table>
|
|
|
|
<p>Vocabulary definitions, when present, occur as direct children of the
|
|
document root element <a href="#s2.1.1"><code><emotionml></code></a>. It
|
|
is possible to refer to a vocabulary defined in the same or in a separate
|
|
EmotionML document, through URIs specified by the values of the attributes
|
|
<code>category-set</code>, <code>dimension-set</code>,
|
|
<code>appraisal-set</code> and <code>action-tendency-set</code> of the <a
|
|
href="#s2.1.2"><code><emotion></code></a> element.</p>
|
|
|
|
<p>The value of the <code>type</code> attribute explicitly states whether the
|
|
vocabulary represents category names, dimension elements, appraisal elements or
|
|
action tendency elements. </p>
|
|
|
|
<h4 id="s3.1.2">3.1.2 The <code><item></code> element</h4>
|
|
|
|
<table class="defn" cellspacing="0" summary="property definition"
|
|
cellpadding="5" width="98%">
|
|
<tbody>
|
|
<tr>
|
|
<th>Annotation</th>
|
|
<td><code><item></code></td>
|
|
</tr>
|
|
<tr>
|
|
<th>Definition</th>
|
|
<td>Represents the definition of one vocabulary item, associated with a
|
|
value which can be used in the "name" attribute of <a
|
|
href="#s2.2.1"><code><category></code></a>, <a
|
|
href="#s2.2.2"><code><dimension></code></a>, <a
|
|
href="#s2.2.3"><code><appraisal></code></a> or <a
|
|
href="#s2.2.4"><code><action-tendency></code></a> (depending on
|
|
the type of vocabulary being defined).</td>
|
|
</tr>
|
|
<tr>
|
|
<th>Children</th>
|
|
<td>An <code><item></code> element MAY contain a single <a
|
|
href="#s2.3.3"><code><info></code></a> element, providing
|
|
arbitrary metadata about the vocabulary item.</td>
|
|
</tr>
|
|
<tr>
|
|
<th>Attributes</th>
|
|
<td><ul>
|
|
<li><strong>Required</strong>:
|
|
<ul>
|
|
<li><code>name</code>: a name for the item, used to refer to this
|
|
item. An <code><item></code> MUST NOT have the same name
|
|
as any other <code><item></code> within the same
|
|
<code><vocabulary></code>.</li>
|
|
</ul>
|
|
</li>
|
|
</ul>
|
|
</td>
|
|
</tr>
|
|
<tr>
|
|
<th>Occurrence</th>
|
|
<td>One or more <code><item></code> elements occur as direct
|
|
children of a <a href="#s3.1.1"><code><vocabulary></code></a>
|
|
element. </td>
|
|
</tr>
|
|
</tbody>
|
|
</table>
|
|
|
|
<p>An <code><item></code> represents the definition of one vocabulary
|
|
item. A <a href="#s3.1.1"><code><vocabulary></code></a> MUST contain at
|
|
least one <code><item></code> element. </p>
|
|
|
|
<p></p>
|
|
|
|
<p>Examples:</p>
|
|
|
|
<p>In the following example, three vocabularies are wrapped into a single
|
|
EmotionML document. Their <code>id</code> attributes are: "big6",
|
|
"fsre-dimensions" and "frijda-subset". They are used to represent categories,
|
|
dimensions and action tendencies respectively. The first
|
|
<code><emotion></code> element specifies the emotion vocabularies used
|
|
through the attributes <code>category-set</code> and
|
|
<code>action-tendency-set</code>, while the second <code><emotion></code>
|
|
element uses the attribute <code>dimension-set</code>. </p>
|
|
<pre><emotionml version="1.0" xmlns="http://www.w3.org/2009/10/emotionml">
|
|
|
|
<!-- Vocabulary definitions -->
|
|
|
|
<vocabulary type="category" id="big6">
|
|
<item name="anger"/>
|
|
<item name="disgust"/>
|
|
<item name="fear"/>
|
|
<item name="happiness"/>
|
|
<item name="sadness"/>
|
|
<item name="surprise"/>
|
|
</vocabulary>
|
|
|
|
<vocabulary type="dimension" id="fsre-dimensions">
|
|
<item name="valence"/>
|
|
<item name="potency"/>
|
|
<item name="arousal"/>
|
|
<item name="unpredictability"/>
|
|
</vocabulary>
|
|
|
|
<vocabulary type="action-tendency" id="frijda-subset">
|
|
<item name="approach"/>
|
|
<item name="avoidance"/>
|
|
<item name="rejecting"/>
|
|
</vocabulary>
|
|
|
|
<!-- Emotion elements -->
|
|
|
|
<emotion category-set="#big6" action-tendency-set="#frijda-subset">
|
|
<category name="fear"/>
|
|
<action-tendency name="approach" value="0.0"/>
|
|
<action-tendency name="avoidance" value="0.9"/>
|
|
</emotion>
|
|
|
|
<emotion dimension-set="#fsre-dimensions">
|
|
<dimension name="arousal" value="0.3"/>
|
|
</emotion>
|
|
|
|
</emotionml></pre>
|
|
|
|
<p></p>
|
|
|
|
<h2 id="s4">4 Conformance</h2>
|
|
|
|
<h3 id="s4.1">4.1 EmotionML namespace</h3>
|
|
|
|
<p>The EmotionML namespace is "http://www.w3.org/2009/10/emotionml". All
|
|
EmotionML elements MUST use this namespace.</p>
|
|
|
|
<h3 id="s4.2">4.2 Use with other namespaces</h3>
|
|
|
|
<p>The EmotionML namespace is intended to be used with other XML namespaces as
|
|
per the Namespaces in XML Recommendation (1.0 [<a
|
|
href="#ref-XMLNS-10">XML-NS10</a>] or 1.1 [<a
|
|
href="#ref-XMLNS-11">XML-NS11</a>], depending on the version of XML being
|
|
used).</p>
|
|
|
|
<h3 id="s4.3">4.3 Schema validation and processor validation of EmotionML
|
|
documents</h3>
|
|
|
|
<p>The EmotionML schema is designed to validate the structural integrity of an
|
|
EmotionML document or document fragment, but cannot verify whether the emotion
|
|
descriptors used in the <code>name</code> attribute of
|
|
<code><category></code>, <code><dimension></code>,
|
|
<code><appraisal></code> and <code><action-tendency</code>> are
|
|
consistent with the vocabularies indicated in the respective
|
|
<code>category-set</code>, <code>dimension-set</code>,
|
|
<code>appraisal-set</code> and <code>action-tendency-set</code> attributes.</p>
|
|
|
|
<p>It is the responsibility of an EmotionML processor to verify that the use of
|
|
descriptor names and values is consistent with the vocabulary definition.</p>
|
|
|
|
<h2 id="s5">5 Examples</h2>
|
|
|
|
<p><i>This section is informative.</i></p>
|
|
|
|
<h3 id="s5.1">5.1 Examples of emotion annotation</h3>
|
|
|
|
<h4 id="s5.1.1">5.1.1 Manual annotation of emotional material</h4>
|
|
|
|
<h5 id="example_annotation_images">Annotation of static images</h5>
|
|
|
|
<p>An image gets annotated with several emotion categories at the same time,
|
|
but different intensities.</p>
|
|
<pre class="example"><emotionml xmlns="http://www.w3.org/2009/10/emotionml"
|
|
xmlns:meta="http://www.example.com/metadata"
|
|
category-set="http://www.example.com/custom/hall-matsumoto-emotions.xml">
|
|
<info>
|
|
<meta:media-type>image</meta:media-type>
|
|
<meta:media-id>disgust</meta:media-id>
|
|
<meta:media-set>JACFEE-database</meta:media-set>
|
|
<meta:doc>Example adapted from (Hall & Matsumoto 2004) http://www.davidmatsumoto.info/Articles/2004_hall_and_matsumoto.pdf
|
|
</meta:doc>
|
|
</info>
|
|
|
|
<emotion>
|
|
<category name="Disgust" value="0.82"/>
|
|
<category name="Contempt" value="0.35"/>
|
|
<category name="Anger" value="0.12"/>
|
|
<category name="Surprise" value="0.53"/>
|
|
</emotion>
|
|
</emotionml></pre>
|
|
|
|
<h5 id="example_annotation_videos">Annotation of videos</h5>
|
|
|
|
<p>Example 1: Annotation of a whole video: several emotions are annotated with
|
|
different intensities.</p>
|
|
<pre class="example"><emotionml xmlns="http://www.w3.org/2009/10/emotionml"
|
|
xmlns:meta="http://www.example.com/metadata"
|
|
category-set="http://www.example.com/custom/humaine-database-labels.xml">
|
|
<info>
|
|
<meta:media-type>video</meta:media-type>
|
|
<meta:media-name>ed1_4</meta:media-name>
|
|
<meta:media-set>humaine database</meta:media-set>
|
|
<meta:coder-set>JM-AB-UH</meta:coder-set>
|
|
</info>
|
|
<emotion>
|
|
<category name="Amusement" value="0.52"/>
|
|
<category name="Irritation" value="0.63"/>
|
|
<category name="Relaxed" value="0.02"/>
|
|
<category name="Frustration" value="0.87"/>
|
|
<category name="Calm" value="0.21"/>
|
|
<category name="Friendliness" value="0.28"/>
|
|
</emotion>
|
|
</emotionml></pre>
|
|
|
|
<p>Example 2: Annotation of a video segment, where two emotions are annotated
|
|
for overlapping but not identical timespans.</p>
|
|
<pre class="example"><emotionml xmlns="http://www.w3.org/2009/10/emotionml"
|
|
xmlns:meta="http://www.example.com/metadata"
|
|
category-set="http://www.example.com/custom/emotv-labels.xml">
|
|
<info>
|
|
<meta:media-type>video</meta:media-type>
|
|
<meta:media-name>ext-03</meta:media-name>
|
|
<meta:media-set>EmoTV</meta:media-set>
|
|
<meta:coder>4</meta:coder>
|
|
</info>
|
|
|
|
<emotion>
|
|
<category name="irritation" value="0.46"/>
|
|
<reference uri="file:ext03.avi?t=3.24,15.4">
|
|
</emotion>
|
|
<emotion>
|
|
<category name="despair" value="0.48"/>
|
|
<reference uri="file:ext03.avi?t=5.15,17.9"/>
|
|
</emotion>
|
|
</emotionml></pre>
|
|
|
|
<h4 id="s5.1.2">5.1.2 Automatic recognition of emotions</h4>
|
|
|
|
<p>This example shows how automatically annotated data from three affective
|
|
sensor devices might be stored or communicated.</p>
|
|
|
|
<p>It shows an excerpt of an episode experienced on 23 November 2001 from 14:36
|
|
onwards (absolute start time is 1006526160 milliseconds since 1 January 1970
|
|
00:00:00 GMT). Each device detects an emotion, but at slightly different times
|
|
and for different durations.</p>
|
|
|
|
<p>The next entry of observed emotions occurs about 6 minutes later (absolute
|
|
start time is 1006526520 milliseconds since 1 January 1970 00:00:00 GMT). Only
|
|
the physiology sensor has detected a short glimpse of anger, for the visual and
|
|
IR camera it was below their individual threshold so no entry from them.</p>
|
|
|
|
<p>For simplicity, all devices use categorical annotations and the same set of
|
|
categories. Obviously it would be possible, and even likely, that different
|
|
devices from different manufacturers provide their data annotated with
|
|
different emotion sets.</p>
|
|
<pre class="example"><emotionml xmlns="http://www.w3.org/2009/10/emotionml"
|
|
category-set="http://www.w3.org/TR/emotion-voc/xml#everyday-categories">
|
|
...
|
|
<emotion start="1006526160" expressed-through="face">
|
|
<!--the first modality detects excitement.
|
|
It is a camera observing the face. A URI to the database
|
|
is provided to access the video stream.-->
|
|
<category name="excited"/>
|
|
<reference uri="http://www.example.com/facedb#t=26,98"/>
|
|
</emotion>
|
|
|
|
<emotion start="1006526160" expressed-through="facial-skin-color">
|
|
<!--the second modality detects anger. It is an IR camera
|
|
observing the face. A URI to the database
|
|
is provided to access the video stream.-->
|
|
<category name="angry"/>
|
|
<reference uri="http://www.example.com/skindb#t=23,108"/>
|
|
</emotion>
|
|
|
|
<emotion start="1006526160" expressed-through="physiology">
|
|
<!--the third modality detects excitement again. It is a
|
|
wearable device monitoring physiological changes in the
|
|
body. A URI to the database
|
|
is provided to access the data stream.-->
|
|
<category name="excited"/>
|
|
<reference uri="http://www.example.com/physiodb#t=19,101"/>
|
|
</emotion>
|
|
|
|
<emotion start="1006526520" expressed-through="physiology">
|
|
<category name="angry"/>
|
|
<reference uri="http://www.example.com/physiodb2#t=2,6"/>
|
|
</emotion>
|
|
...
|
|
</emotionml></pre>
|
|
|
|
<p></p>
|
|
|
|
<p>Note that handling of complex emotions is not explicitly specified. This
|
|
example assumes that parallel occurrences of emotions will be determined on the
|
|
time stamp.</p>
|
|
|
|
<h4 id="s5.1.3">5.1.3 Generation of emotion-related system behavior</h4>
|
|
|
|
<h5 id="example_generation_mpeg4">Generation of facial expressions in an MPEG-4
|
|
face model</h5>
|
|
|
|
<p>The MPEG-4 standard offers 68 parameters, called Facial Animation Parameters
|
|
FAPs, to animate a 3D facial model. 66 of these parameters correspond to low
|
|
level parameters. These parameters act on the facial feature points defining a
|
|
3D facial model. They specify how these feature points are displaced. They
|
|
simulate muscular contraction. On the other hand, two FAPs, namely FAP1 and
|
|
FAP2, refer respectively to viseme and expression. FAP2 corresponds to one of
|
|
the six basic facial expressions (anger, disgust, fear, happiness, sadness and
|
|
surprise). The expressions associated to the six emotions are defined by
|
|
textual descriptions [<a href="#ref-Ostermann2002">Ostermann, 2002</a>].</p>
|
|
|
|
<p>In emotion theory, the idea of mixing emotions to create new emotions is
|
|
disputed. For the purposes of facial expression modeling, however, it is
|
|
possible to simulate different emotions as linear combinations of the six basic
|
|
facial expressions. MPEG-4 allows the linear combination of any two of these
|
|
expressions: emotion_1 * intensity_1 + emotion_2 * intensity_2. For example,
|
|
[<a href="#ref-Raouzaiou2005">Raouzaiou et al., 2005</a>] found that the
|
|
expressions of depression and guilt can be obtained by combinations of fear and
|
|
sadness with different intensities, while the expression of suspicion is
|
|
obtained by combining anger and disgust.</p>
|
|
|
|
<p>In EmotionML it is possible to represent the emotional input to an MPEG-4
|
|
based facial animation system using multiple <code><category></code>
|
|
elements, for example as follows.</p>
|
|
<pre class="example"><emotion xmlns="http://www.w3.org/2009/10/emotionml"
|
|
category-set="http://www.w3.org/TR/emotion-voc/xml#big6">
|
|
<!-- attempt to express suspicion as a combination of anger and disgust -->
|
|
<category name="anger" value="0.5"/>
|
|
<category name="disgust" value="0.3"/>
|
|
</emotion></pre>
|
|
|
|
<h5 id="example_generation_robot">Generation of robot behavior</h5>
|
|
|
|
<p>The following example describes various aspects of an emotionally competent
|
|
robot whose battery is nearly empty. The robot is in a global state of high
|
|
arousal, negative pleasure and low dominance, i.e. a negative state of distress
|
|
paired with some urgency but quite limited power to influence the situation. It
|
|
has a tendency to seek a recharge and to avoid picking up boxes. However,
|
|
sensor data displays an unexpected obstacle on the way to the charging station.
|
|
This triggers planning of expressive behavior of frowning. The annotations are
|
|
grouped into a stand-alone EmotionML document here; in the real world, the
|
|
various aspects would more likely be embedded into different specialized markup
|
|
in various parts of the Robot architecture.</p>
|
|
<pre class="example"><emotionml xmlns="http://www.w3.org/2009/10/emotionml"
|
|
xmlns:meta="http://www.example.com/metadata">
|
|
<info>
|
|
<meta:name>robbie the robot example</meta:name>
|
|
</info>
|
|
|
|
<!-- Robot's current global state configuration: negative, active, powerless -->
|
|
<emotion dimension-set="http://www.w3.org/TR/emotion-voc/xml#pad-dimensions">
|
|
<dimension name="pleasure" value="0.2"/>
|
|
<dimension name="arousal" value="0.8"/>
|
|
<dimension name="dominance" value="0.3"/>
|
|
</emotion>
|
|
|
|
<!-- Robot's action tendencies: want to recharge -->
|
|
<emotion action-tendency-set="http://www.example.com/custom/action/robot.xml">
|
|
<action-tendency name="charge-battery" value="0.9"/>
|
|
<action-tendency name="seek-shelter" value="0.7"/>
|
|
<action-tendency name="pickup-boxes" value="0.1"/>
|
|
</emotion>
|
|
|
|
<!-- Appraised value of incoming event: obstacle detected, appraised as novel and unpleasant -->
|
|
<emotion appraisal-set="http://www.w3.org/TR/emotion-voc/xml#scherer-appraisals">
|
|
<appraisal name="suddenness" value="0.8" confidence="0.4"/>
|
|
<appraisal name="intrinsic-pleasantness" value="0.2" confidence="0.8"/>
|
|
<reference role="triggeredBy" uri="file:scannerdata.xml#obstacle27"/>
|
|
</emotion>
|
|
|
|
<!-- Robot's planned facial gestures: will frown -->
|
|
<emotion category-set="http://www.example.com/custom/robot-emotions.xml"
|
|
expressed-through="face">
|
|
<category name="frustration"/>
|
|
<reference role="expressedBy" uri="file:behavior-repository.xml#frown"/>
|
|
</emotion>
|
|
</emotionml></pre>
|
|
|
|
<h3 id="s5.2">5.2 Examples of possible use with other markup languages</h3>
|
|
|
|
<p>One intended use of EmotionML is as a plug-in for existing markup languages.
|
|
For compatibility with text-annotating markup languages such as <a
|
|
href="#ref-SSML">SSML</a>, EmotionML avoids the use of text nodes. All
|
|
EmotionML information is encoded in element and attribute structures.</p>
|
|
|
|
<p>This section illustrates the concept using three existing W3C markup
|
|
languages: <a href="#ref-EMMA">EMMA</a>, <a href="#ref-SSML">SSML</a>, and <a
|
|
href="#ref-SMIL">SMIL</a>.</p>
|
|
|
|
<h4 id="s5.2.1">5.2.1 Use with EMMA</h4>
|
|
|
|
<p>EMMA is made for representing arbitrary analysis results; one of them could
|
|
be the emotional state. The following example represents an analysis of a
|
|
non-verbal vocalization; its emotion is described as a low-intensity state,
|
|
maybe boredom.</p>
|
|
<pre class="example"><emma:emma version="1.0" xmlns:emma="http://www.w3.org/2003/04/emma"
|
|
xmlns="http://www.w3.org/2009/10/emotionml">
|
|
<emma:interpretation emma:start="12457990" emma:end="12457995" emma:mode="voice" emma:verbal="false">
|
|
|
|
<emotion category-set="http://www.w3.org/TR/emotion-voc/xml#everyday-categories">
|
|
<category name="bored" value="0.1" confidence="0.1"/>
|
|
</emotion>
|
|
|
|
</emma:interpretation>
|
|
</emma:emma></pre>
|
|
|
|
<p></p>
|
|
|
|
<p>In the folllowing example, the EMMA <code><emma:derivation></code>
|
|
element is used to represent multiple emotion interpretations associated with
|
|
audio and video media sources. The first and the third interpretations specify
|
|
the same emotion category, "content", while the result of the second one is
|
|
"amused". The consolidated emotion is the result of some processing made on the
|
|
interpretations included in the derivation element. In this case it is
|
|
"content", which is the most frequent category within the available
|
|
interpretations.</p>
|
|
<pre class="example"><emma:emma version="1.0" xmlns:emma="http://www.w3.org/2003/04/emma" xmlns="http://www.w3.org/2009/10/emotionml">
|
|
|
|
<emma:derivation>
|
|
|
|
<emma:interpretation id="text1" emma:start="12457960" emma:end="12457995" emma:mode="voice"
|
|
emma:verbal="true" emma:signal="http://example.com/signals/emo123.wav"
|
|
emma:process="http://example.com/text_analysis.xml">
|
|
<emma:literal>I feel happy</emma:literal>
|
|
<emotion category-set="http://www.w3.org/TR/emotion-voc/xml#everyday-categories">
|
|
<category name="content" value="0.7" confidence="0.7"/>
|
|
</emotion>
|
|
</emma:interpretation>
|
|
|
|
<emma:interpretation id="voice1" emma:start="12457960" emma:end="12457995" emma:mode="voice"
|
|
emma:verbal="false" emma:signal="http://example.com/signals/emo123.wav"
|
|
emma:process="http://example.com/voice_analysis.xml">
|
|
<emotion category-set="http://www.w3.org/TR/emotion-voc/xml#everyday-categories">
|
|
<category name="amused" value="0.4" confidence="0.5"/>
|
|
</emotion>
|
|
</emma:interpretation>
|
|
|
|
<emma:interpretation id="video1" emma:start="12457980" emma:end="12458000" emma:mode="video"
|
|
emma:verbal="false" emma:signal="http://example.com/signals/emo123.mpg"
|
|
emma:process="http://example.com/video_analysis.xml">
|
|
<emotion category-set="http://www.w3.org/TR/emotion-voc/xml#everyday-categories">
|
|
<category name="content" value="0.5" confidence="0.7"/>
|
|
</emotion>
|
|
</emma:interpretation>
|
|
|
|
</emma:derivation>
|
|
|
|
|
|
<emma:interpretation id="multimodal1" emma:start="12457960" emma:end="12458000"
|
|
emma:medium="acoustic visual" emma:mode="voice video">
|
|
<emma:derived-from resource="#text1" composite="true"/>
|
|
<emma:derived-from resource="#voice1" composite="true"/>
|
|
<emma:derived-from resource="#video1" composite="true"/>
|
|
<emotion category-set="http://www.w3.org/TR/emotion-voc/xml#everyday-categories">
|
|
<category name="content" value="0.6" confidence="0.7"/>
|
|
</emotion>
|
|
</emma:interpretation>
|
|
|
|
</emma:emma></pre>
|
|
|
|
<h4 id="s5.2.2">5.2.2 Use with SSML</h4>
|
|
|
|
<p>Two options for using EmotionML with SSML can be illustrated.</p>
|
|
|
|
<p>First, it is possible with [<a href="#ref-SSML1.1">SSML 1.1</a>] to use
|
|
arbitrary markup belonging to a different namespace anywhere in an SSML
|
|
document; only SSML processors that support the markup would take it into
|
|
account. Therefore, it is possible to insert EmotionML below, for example, an
|
|
<code><s></code> element representing a sentence; the intended meaning is
|
|
that the enclosing sentence should be spoken with the given emotion, in this
|
|
case a moderately worried tone of voice:</p>
|
|
<pre class="example"><?xml version="1.0"?>
|
|
<speak version="1.1" xmlns="http://www.w3.org/2001/10/synthesis"
|
|
xmlns:emo="http://www.w3.org/2009/10/emotionml"
|
|
xml:lang="en-US">
|
|
<s>
|
|
<emo:emotion category-set="http://www.w3.org/TR/emotion-voc/xml#everyday-categories">
|
|
<emo:category name="worried" value="0.4"/>
|
|
</emo:emotion>
|
|
|
|
Do you need help?
|
|
</s>
|
|
</speak></pre>
|
|
|
|
<p>Second, a future version of SSML could explicitly preview the annotation of
|
|
paralinguistic information, which could fill the gap between the
|
|
extralinguistic, speaker-constant settings of the <code><voice></code>
|
|
tag and the linguistic elements such as <code><s></code>,
|
|
<code><emphasis></code>, <code><say-as></code> etc. The following
|
|
example assumes that there is a <code><style></code> tag for
|
|
paralinguistic information in a future version of SSML. The style could embed
|
|
an <code><emotion></code>, as follows:</p>
|
|
<pre class="example"><?xml version="1.0"?>
|
|
<speak version="x.y" xmlns="http://www.w3.org/2001/10/synthesis"
|
|
xmlns:emo="http://www.w3.org/2009/10/emotionml"
|
|
xml:lang="en-US">
|
|
<s>
|
|
<style>
|
|
<emo:emotion category-set="http://www.w3.org/TR/emotion-voc/xml#everyday-categories">
|
|
<emo:category name="worried" value="0.4"/>
|
|
</emo:emotion>
|
|
|
|
Do you need help?
|
|
</style>
|
|
</s>
|
|
</speak></pre>
|
|
|
|
<p>Alternatively, the <code><style></code> could refer to a previously
|
|
defined <code><emotion></code>, for example:</p>
|
|
<pre class="example"><?xml version="1.0"?>
|
|
<speak version="x.y" xmlns="http://www.w3.org/2001/10/synthesis"
|
|
xmlns:emo="http://www.w3.org/2009/10/emotionml"
|
|
xml:lang="en-US">
|
|
<emo:emotion category-set="http://www.w3.org/TR/emotion-voc/xml#everyday-categories"
|
|
id="somewhatWorried">
|
|
<emo:category name="worried" value="0.4"/>
|
|
</emo:emotion>
|
|
|
|
<s>
|
|
<style ref="#somewhatWorried">
|
|
Do you need help?
|
|
</style>
|
|
</s>
|
|
</speak></pre>
|
|
|
|
<h4 id="s5.2.3">5.2.3 Use with SMIL</h4>
|
|
|
|
<p>Using EmotionML for the use case of generating system behavior requires
|
|
elements of scheduling and surface form realization which are not part of
|
|
EmotionML. Necessarily, this use case relies on other languages to provide the
|
|
needed functionality. This is in line with the aim of EmotionML to serve as a
|
|
specialized plug-in language.</p>
|
|
|
|
<p>This example illustrates the idea in terms of a simplified version of a
|
|
storytelling application. A virtual agent tells a story using voice and facial
|
|
animation. The expression in face and voice is influenced by the rendering
|
|
engine in terms of EmotionML. The engine in this example uses SMIL [<a
|
|
href="#ref-SMIL">SMIL</a>] for defining the temporal relation between events;
|
|
EmotionML is used via SMIL's generic <code><ref></code> element. In
|
|
general it is the engine which knows how to render the emotion in the virtual
|
|
agent's expressive capabilities. To override this, the second
|
|
<code><emotion></code> contains an explicit request to realize the
|
|
emotional expression using both face and voice modalities.</p>
|
|
|
|
<p>ridinghood.smil:</p>
|
|
<pre class="example"><smil xmlns="http://www.w3.org/ns/SMIL" version="3.0">
|
|
<head> ... </head>
|
|
<body>
|
|
<par duration="8s">
|
|
<img src="file:forest.jpg"/>
|
|
<smileText>The little girl was enjoying the walk in the forest.</smileText>
|
|
<ref src="file:ridinghood.emotionml#emotion1"/>
|
|
</par>
|
|
<par duration="5s">
|
|
<img src="file:wolf.jpg"/>
|
|
<smileText>Suddenly a dark shadow appeared in front of her.</smileText>
|
|
<ref src="file:ridinghood.emotionml#emotion2"/>
|
|
</par>
|
|
|
|
</body>
|
|
</smil></pre>
|
|
|
|
<p></p>
|
|
|
|
<p>ridinghood.emotionml:</p>
|
|
<pre class="example"><emotionml xmlns="http://www.w3.org/2009/10/emotionml"
|
|
category-set="http://www.w3.org/TR/emotion-voc/xml#everyday-categories"
|
|
appraisal-set="http://www.w3.org/TR/emotion-voc/xml#scherer-appraisals">
|
|
|
|
<emotion id="emotion1">
|
|
<category name="content" value="0.7"/>
|
|
</emotion>
|
|
|
|
<emotion id="emotion2" expressed-through="face voice">
|
|
<category name="afraid" value="0.9"/>
|
|
<appraisal name="suddenness" value="0.9"/>
|
|
<appraisal name="intrinsic-pleasantness" value="0.1"/>
|
|
</emotion>
|
|
</emotionml></pre>
|
|
|
|
<p></p>
|
|
|
|
<p>Similar principles for decoupling emotion markup from the temporal
|
|
organization of generating system behavior can be applied using other
|
|
representations, including interactive setups.</p>
|
|
|
|
<p></p>
|
|
|
|
<h2 id="s6">6 References</h2>
|
|
|
|
<h3 id="s6.1">6.1 Normative references</h3>
|
|
<dl>
|
|
<dt id="ref-EMMA">EMMA</dt>
|
|
<dd><a href="http://www.w3.org/TR/2004/REC-speech-synthesis-20040907/"
|
|
shape="rect">EMMA: Extensible MultiModal Annotation markup language
|
|
version 1.0</a>, Michael Johnston et al., Editors. World Wide Web
|
|
Consortium, 11 December 2007. </dd>
|
|
<dt id="ref-MediaFragments">Media Fragments URI</dt>
|
|
<dd><a href="http://www.w3.org/TR/2011/WD-media-frags-20110317/">Media
|
|
Fragments URI 1.0</a>, Raphaël Troncy et al., Editors. World Wide Web
|
|
Consortium, W3C Working Draft 17 March 2011.</dd>
|
|
<dt id="ref-rdf">RDF</dt>
|
|
<dd><a href="http://www.w3.org/TR/rdf-syntax-grammar/">RDF/XML Syntax
|
|
Specification (Revised)</a>, Dave Beckett, Editor. World Wide Web
|
|
Consortium, W3C Recommendation 10 February 2004.</dd>
|
|
<dt id="ref-rfc2119">RFC2119</dt>
|
|
<dd><a href="http://www.ietf.org/rfc/rfc2119.txt">Key words for use in RFCs
|
|
to Indicate Requirement Levels</a>, S. Bradner, Editor. IETF RFC 2119,
|
|
March 1997.</dd>
|
|
<dt><a id="ref-rfc2326">RFC 2326</a></dt>
|
|
<dd><a href="http://www.ietf.org/rfc/rfc2326.txt">Real Time Streaming
|
|
Protocol (RTSP)</a>, H. Schulzrinne et al., Editors. IETF RFC 2326, April
|
|
1998.</dd>
|
|
<dt id="ref-rfc3986">RFC3986</dt>
|
|
<dd><a href="http://www.ietf.org/rfc/rfc3986.txt">Uniform Resource
|
|
Identifier (URI): Generic Syntax</a>, T. Berners-Lee et al., Editors.
|
|
IETF RFC 3986, January 2005.</dd>
|
|
<dt id="ref-SMIL">SMIL</dt>
|
|
<dd><a href="http://www.w3.org/TR/SMIL3/" shape="rect">Synchronized
|
|
Multimedia Integration Language (SMIL) Version 3.0</a>, Dick Bulterman et
|
|
al., Editors. W3C Recommendation, 1 December 2008.</dd>
|
|
<dt><a id="ref-SMPTE">SMPTE</a></dt>
|
|
<dd>SMPTE RP 136 Time and Control Codes for 24, 25 or 30 Frame-Per-Second
|
|
Motion-Picture Systems.</dd>
|
|
<dt id="ref-SSML">SSML</dt>
|
|
<dd><a href="http://www.w3.org/TR/2004/REC-speech-synthesis-20040907/"
|
|
shape="rect">Speech Synthesis Markup Language (SSML) Version 1.0</a>,
|
|
Daniel C. Burnett, et al., Editors. World Wide Web Consortium, W3C
|
|
Recommendation, 7 September 2004. </dd>
|
|
<dt id="ref-SSML1.1">SSML 1.1</dt>
|
|
<dd><a href="http://www.w3.org/TR/2010/REC-speech-synthesis11-20100907/"
|
|
shape="rect">Speech Synthesis Markup Language (SSML) Version 1.1</a>,
|
|
Daniel C. Burnett, et al., Editors. World Wide Web Consortium, W3C
|
|
Recommendation, 7 September 2010. </dd>
|
|
<dt id="ref-XMLNS-10">XML-NS10</dt>
|
|
<dd><a href="http://www.w3.org/TR/2006/REC-xml-names-20060816/">Namespaces
|
|
in XML 1.0</a>, Tim Bray et al., Editors. World Wide Web Consortium, W3C
|
|
Recommendation, 16 August 2006.</dd>
|
|
<dt id="ref-XMLNS-11">XML-NS11</dt>
|
|
<dd><a href="http://www.w3.org/TR/xml-names11/">Namespaces in XML 1.1</a>,
|
|
Tim Bray et al., Editors. World Wide Web Consortium, W3C Recommendation,
|
|
2006.</dd>
|
|
<dt id="ref-xml-schema">XML Schema</dt>
|
|
<dd><a href="http://www.w3.org/TR/xmlschema-1/">XML Schema Part 1:
|
|
Structures Second Edition</a>, Henry S. Thompson et al., Editors. World
|
|
Wide Web Consortium, W3C Recommendation, 2004.</dd>
|
|
</dl>
|
|
|
|
<h3 id="s6.2">6.2 Informative references</h3>
|
|
<dl>
|
|
<dt id="ref-clarin">CLARIN</dt>
|
|
<dd><a href="http://www.clarin.eu/files/wg2-4-metadata-doc-v5.pdf">CLARIN
|
|
Metadata Infrastructure for Language Resources and Technology</a>,Version
|
|
5, D. Broeder et al., Editors. Common Language Resources and Technology
|
|
Infrastructure Report, 4 February 2009.</dd>
|
|
<dt id="ref-emotion-xg">Emotion Incubator Group</dt>
|
|
<dd><a href="http://www.w3.org/2005/Incubator/emotion/XGR-emotion/">W3C
|
|
Emotion Incubator Group</a>, M. Schröder, E. Zovato, H. Pirker, C.
|
|
Peter, F. Burkhardt, Editors. Final Report of the Emotion Incubator Group
|
|
at the World Wide Web Consortium, 10 July 2007. </dd>
|
|
<dt id="ref-emotionml-xg">Emotion Markup Language Incubator Group</dt>
|
|
<dd><a
|
|
href="http://www.w3.org/2005/Incubator/emotion/XGR-emotionml/">Elements
|
|
of an EmotionML 1.0</a>, M. Schröder, Editor. Final Report of the
|
|
Emotion Markup Language Incubator Group at the World Wide Web Consortium,
|
|
20 November 2008. </dd>
|
|
<dt id="ref-emotionml-req">EmotionML Requirements</dt>
|
|
<dd><a
|
|
href="http://www.w3.org/2005/Incubator/emotion/XGR-requirements-20080513/">Emotion
|
|
Markup Language: Requirements with Priorities</a>. F. Burkhardt and M.
|
|
Schröder. W3C Incubator Group Report, 13 May 2008. </dd>
|
|
<dt id="ref-imdi">IMDI</dt>
|
|
<dd><a href="http://www.mpi.nl/corpus/manuals/manual-imdi-editor.pdf">IMDI
|
|
Editor version 3.2</a>, B. Hellwig and D. van Uytvanck. ISLE Metadata
|
|
Initiative Report, 19 June 2007.</dd>
|
|
<dt id="ref-Ortony1988">Ortony et al., 1988</dt>
|
|
<dd>Ortony, A., Clore, G. L., & Collins, A. (1988). The Cognitive
|
|
Structure of Emotion. Cambridge, UK: Cambridge University Press.</dd>
|
|
<dt id="ref-Ostermann2002">Ostermann, 2002</dt>
|
|
<dd>Ostermann, J. (2002). Face Animation in MPEG-4. In: MPEG-4 Facial
|
|
Animation - The Standard Implementation and Applications (I.S. Pandzic
|
|
and R. Forchheimer, eds.), pp. 17-55. England: Wiley.</dd>
|
|
<dt id="ref-Raouzaiou2005">Raouzaiou et al., 2005</dt>
|
|
<dd>Raouzaiou, A., Spyrou, E., Karpouzis, K. and Kollias, S. (2005).
|
|
Emotion Synthesis: an Intermediate Expressions’ Generator System in the
|
|
MPEG-4 Framework. International Workshop VLBV05, 15-16 September 2005,
|
|
Sardinia, Italy.</dd>
|
|
<dt id="ref-emotion-voc">Vocabularies for EmotionML</dt>
|
|
<dd><a
|
|
href="http://www.w3.org/TR/2011/WD-emotion-voc-20110407/">Vocabularies
|
|
for EmotionML</a>. M. Schröder and C. Pelachaud, Editors. W3C Working
|
|
Draft, 7 April 2011. </dd>
|
|
</dl>
|
|
|
|
<h2 id="s7">7 Acknowledgments</h2>
|
|
|
|
<p>The authors wish to acknowledge the contributions by all members of the
|
|
Multimodal Interaction Working Group, the Emotion Markup Language Incubator
|
|
Group and the Emotion Incubator Group, as well as the participants to the W3C
|
|
Workshop on EmotionML, in particular the following persons (in alphabetic
|
|
order):</p>
|
|
<ul>
|
|
<li>Kazuyuki Ashimura, W3C</li>
|
|
<li>Andrew Breen, Nuance Communications</li>
|
|
<li>Roddy Cowie, Queen's University Belfast</li>
|
|
<li>Deborah Dahl, Conversational Technologies</li>
|
|
<li>Sarah Jane Delany, Dublin Institute of Technology</li>
|
|
<li>Dylan Evans, University College Cork</li>
|
|
<li>Nestor Garay Vitoria, University of the Basque Country</li>
|
|
<li>Alain Giboin, INRIA Sophia Antipolis</li>
|
|
<li>Bill Jarrold, SRI International</li>
|
|
<li>Michael Johnston, AT&T</li>
|
|
<li>Kostas Karpouzis, Image, Video and Multimedia Systems Lab (IVML-NTUA)</li>
|
|
<li>Myriam Lamolle, University of Paris VIII</li>
|
|
<li>Tim Llewellyn, nViso</li>
|
|
<li>Jean-Claude Martin, CNRS</li>
|
|
<li>Alessandro Oltramari, CNR</li>
|
|
<li>Hannes Pirker, Austrian Research Institute for Artificial
|
|
Intelligence</li>
|
|
<li>Björn Schuller, Technische Universität München</li>
|
|
<li>Jianhua Tao, Chinese Academy of Sciences</li>
|
|
<li>Ian Wilson, Emotion AI</li>
|
|
<li>Gill Windall, University of Greenwich</li>
|
|
<li>Idoia Zearreta, University of the Basque Country</li>
|
|
</ul>
|
|
<hr />
|
|
|
|
<h2 id="changelog">Appendix A: Changes</h2>
|
|
|
|
<p><em>This section is informative.</em></p>
|
|
|
|
<h3 id="changes-current">Changes in the current Working Draft</h3>
|
|
|
|
<p>This section summarizes the main changes since the <a
|
|
href="http://www.w3.org/TR/2010/WD-emotionml-20100729/">previous working draft
|
|
of 29 July 2010</a>.</p>
|
|
<ul>
|
|
<li>Changes specific to EmotionML
|
|
<ul>
|
|
<li>The document now distinguishes between normative and non-normative
|
|
sections.</li>
|
|
<li>A <a href="#s3.1">mechanism for defining emotion vocabularies</a> was
|
|
specified.</li>
|
|
<li>The definitions of emotion vocabularies were moved from the
|
|
specification into a dedicated W3C Working Draft [<a
|
|
href="#ref-emotion-voc">Vocabularies for EmotionML</a>].</li>
|
|
<li>The <a href="#s2.2.1"><code><category></code></a> element was
|
|
harmonized with the other emotion descriptors to allow a
|
|
<code>value</code> attribute or a <code><trace></code> child
|
|
element indicating the intensity of that category. Multiple
|
|
<code><category></code> elements are now allowed within a single
|
|
<code><emotion></code> to reflect the possible co-presence of
|
|
these categories. The <code><intensity></code> element was
|
|
removed since the usual use is now covered by the <code>value</code>
|
|
attribute in <code><category></code>.</li>
|
|
<li>The specification of scale values through the <code>value</code>
|
|
attribute or the <code><trace></code> child element was made
|
|
optional for <a href="#s2.2.3"><code><appraisal></code></a> and
|
|
<a href="#s2.2.4"><code><action-tendency></code></a> elements, in
|
|
order to allow for the possibility to merely represent the fact that a
|
|
certain appraisal or action tendency is present, irrespective of its
|
|
intensity.</li>
|
|
<li>A mechanism for indicating <a
|
|
href="#s2.4.2.2"><code>duration</code></a> and <a
|
|
href="#s2.4.2.3">relative timestamps</a> was added.</li>
|
|
<li>More examples were added to illustrate possible uses of the <a
|
|
href="#s2.3.3"><code><info></code></a> element for representing
|
|
metadata.</li>
|
|
<li>An <a href="#example_generation_mpeg4">example</a> was added to
|
|
illustrate the use of EmotionML in the context of predicting mixed
|
|
emotions in an MPEG-4-based facial animation model.</li>
|
|
<li>An <a href="#s5.2.1">example</a> was added to illustrate the use of
|
|
EMMA for representing the derivation of a consolidated emotion from
|
|
individual emotion observations.</li>
|
|
<li>All examples were updated to be consistent with [<a
|
|
href="#ref-emotion-voc">Vocabularies for EmotionML</a>] where
|
|
appropriate.</li>
|
|
</ul>
|
|
</li>
|
|
</ul>
|
|
<ul>
|
|
<li>Consistency issues with other W3C specifications
|
|
<ul>
|
|
<li>To avoid confusion with the <code>emma:mode</code> attribute in [<a
|
|
href="#ref-EMMA">EMMA</a>], the <code>modality</code> attribute was
|
|
renamed to <a href="#s2.3.2"><code>expressed-through</code></a>.</li>
|
|
</ul>
|
|
</li>
|
|
</ul>
|
|
|
|
<p></p>
|
|
|
|
<h3 id="changes-wd2">Changes in Working Draft 2 (29 July 2010)</h3>
|
|
<ul>
|
|
<li>Changes specific to EmotionML
|
|
<ul>
|
|
<li>A mechanism for pointing to emotion vocabularies was agreed. An
|
|
emotion vocabulary is now identified by a URI in the attributes
|
|
category-set, dimension-set, appraisal-set and action-tendency-set of
|
|
<a href="#s2.1.2"><emotion></a> or, in the sense of a
|
|
document-wide default, of <a href="#s2.1.1"><emotionml></a>. The
|
|
consistency of an EmotionML annotation with the indicated vocabulary
|
|
must be verified by an EmotionML processor; it cannot be verified using
|
|
Schema validation. The <a href="#s4.3">section on validation</a> was
|
|
updated accordingly.</li>
|
|
<li>A collection of <a href="#s3.1">emotion vocabularies</a> was compiled
|
|
which may be useful defaults for many users. The list is incomplete and
|
|
not fully developed, but is already published in this form to elicit
|
|
feedback.</li>
|
|
<li>The notion of <a href="#s2.5">Scale values</a> was simplified to only
|
|
allow for continuous values in the range [0;1].</li>
|
|
<li>The notion of a confidence-trace was dropped for simplicity.</li>
|
|
</ul>
|
|
</li>
|
|
</ul>
|
|
<ul>
|
|
<li>Consistency issues with other W3C specifications
|
|
<ul>
|
|
<li>The syntax for representing dimensions, appraisals and action
|
|
tendencies was changed to be more in line with the expectation that
|
|
user-defined strings figure in attribute values rather than element or
|
|
attribute names. The specification now defines <a
|
|
href="#s2.2.2"><code><dimension></code></a>, <a
|
|
href="#s2.2.3"><code><appraisal></code></a> and <a
|
|
href="#s2.2.4"><code><action-tendency></code></a> elements with a
|
|
<code>name</code> attribute.</li>
|
|
<li>The <a href="#s2.4.2">representation of time</a> was synchronized
|
|
with <a href="#ref-EMMA">EMMA</a>: the specification now uses
|
|
<code>start</code> and <code>end</code> attributes to represent
|
|
absolute time, and <a href="#ref-MediaFragments">Media Fragment
|
|
URIs</a> to refer to portions of media files.</li>
|
|
<li>Metadata is now represented using an <code><a
|
|
href="#s2.3.3"><info></a></code> element, in synchrony with <a
|
|
href="#ref-EMMA">EMMA</a>.</li>
|
|
<li>The <code><link></code> element was renamed to <a
|
|
href="#s2.4.1"><code><reference></code></a> to avoid a name clash
|
|
with the <code><link></code> element in HTML, which has a
|
|
different scope and syntax.</li>
|
|
</ul>
|
|
</li>
|
|
</ul>
|
|
|
|
<p></p>
|
|
|
|
<p></p>
|
|
</body>
|
|
</html>
|