SynAnim - Synthetic Animation Engine
Methodology
SynAnim was developed to support both emotions and actions. Happy and Sad are examples of emotions. Waving and Nodding are examples of actions. Emotions and actions compliment each other and create specific feelings for users. For instance, a digital human might happily wave hello at the beginning of a conversation and sadly wave goodbye at the end. When the emotion changes, it can affect the action. Typically this is through a measure of intensity. When a digital human is happy and waving, the intensity of the waving action is increased compared to when the digital human is sad and waving.
Prerequisites
In order to use inline tagging, the following prerequisites must be met:
- Using a supported TTS system. The only supported TTS engine as of April 20, 2023 is Microsoft Azure Text to Speech
- Azure TTS supports "bookmark" tags within it's SSML specification. UneeQ relies on bookmark tag support in order to process the animation commands as well as SSML (that may or may not exist) properly.
- SynAnim-capable Digital Human - as of April 20, 2023 UneeQ must set a SynAnim-capable Digital Human manually for each customer persona. SynAnim Digital Humans are not currently selectable within our Creator Portal
- Dynamic camera support
Emotion Tagging
SynAnim uses an XML-style inline tagging solution. Example string with emotional markup:
As in the example above, you can use multiple emotion tags within a single response, and they are self-closing tags, so there is no need to wrap the tag around a word or sentence. Simply insert the tag at the beginning or at any point in the string for the emotion to begin.
Supported Emotions
The emotion value determines the absolute value of emotion for the digital human.
For instance, using joy at strong will cause the digital human to exhibit strong joy behaviour, regardless of what emotion they were previously expressing.
The Emotion Component of Synanim handles secondary and opposing emotions and ensures a reasonable emotional behaviour. You can use multiple emotion tags within a single response, but using tags back-to-back may not yield a clear result. It is recommended that you consider changing emotional states on the boundaries of the character's speech using punctuation like a period or a comma as the point at which their emotional state might transition.
Emotions decay over time and the speed of this is calculated based on the strength of the emotion and parameters within the Personality (these parameters are currently only available for customisation by the UneeQ team).
Key | Value |
joy | Valid values (String): “weak”, “normal”, “strong” |
trust | Valid values (String): “weak”, “normal”, “strong” |
fear | Valid values (String): “weak”, “normal”, “strong” |
surprise | Valid values (String): “weak”, “normal”, “strong” |
sadness | Valid values (String): “weak”, “normal”, “strong” |
disgust | Valid values (String): “weak”, “normal”, “strong” |
anger | Valid values (String): “weak”, “normal”, “strong” |
anticipation | Valid values (String): “weak”, “normal”, “strong” |
Camera Controls
The camera controls enable a conversational designer to change the camera zoom and pan during dialog. These camera changes will persist until another dialog node is encountered with another camera setting.
The amount of zoom and pan are currently presets, however it is on our roadmap to allow for variable zoom and pan amounts.
Zooms and pans are additive. That is, it is possible to zoom into the face and pan right at the same time.
Tag | Description |
<uneeq:camera_face /> | Zoom into a face close up |
<uneeq:camera_shoulders /> | Zoom to show the chest up |
<uneeq:camera_torso /> | Zoom to show the hips up |
<uneeq:camera_fullbody /> | Zoom to show from feet to head |
<uneeq:camera_left /> | Pan the camera left |
<uneeq:camera_center /> | Pan the camera back to center |
<uneeq:camera_right /> | Pan the camera to the right |
Camera Usage
Tags are to be applied as self closing SSML tags within the dialog transcript and can be placed anywhere within the dialog. One or more tags can be used in one response, however they cannot be consecutive (known bug as at Dec 7, 2022).
Example |
|
Zooming in on my face.<uneeq:camera_face /> | Valid - tag at end |
Just show my torso <uneeq:camera_torso />please. <uneeq:camera_left /> | Valid - two tags |
<uneeq:camera_fullbody />Zooming right out! | Valid - tag at beginning |
<uneeq:camera_face /><uneeq:camera_left />Look to my right. | Invalid - consecutive tags |
<uneeq:camera_face>Zooming in on my face! | Invalid - Tag does not have “/” |
Actions and Postures
Using inline tags, a conversational designer is able to trigger specific action behaviours and posture changes.
Each Personality may have different sets of actions and postures - as specific actions and postures may be authored for a Personality to better express the uniqueness of that Personality.
The format of the tag is as follows:
The specific list of postures and actions are to be provided per Personality by the UneeQ team.
Supported postures and actions
This is the list of postures and actions. These are manually driven and will only be triggered when used in the dialog. These are in addition to the automated gesture system.
Tag | Description |
<uneeq:action_shrug /> | Shrug gesture |
<uneeq:action_understandnod /> | Nod in understanding gesture |
<uneeq:action_wavingcalm /> | A calm wave |
<uneeq:action_confused /> | A confused animation |
<uneeq:action_disappointed /> | A disappointed animation |
<uneeq:action_understandnod /> | Nodding in understanding |
<uneeq:action_headaffirmdown /> | A downward nod in affirmation |
<uneeq:action_headaffirmup /> | An upward nod in affirmation |
<uneeq:action_headshakeslow /> | A slow shake of the head left and right (disagreement / discouragement) |
<uneeq:action_headshakemedium /> | A more energetic version of the head shake action |
<uneeq:action_headshakefast /> | The most energetic version of the head shake action |