Thesis Project Form

The form of an action, i.e., how an action is performed, conveys important information about the displayer’s attitude toward the interaction partner. Humans may perform the same gesture in different ways, e.g., neutrally, vigorously, gently, and rudely. This fundamental aspect of the action has been named vitality form by Stern (2010).

References:
Stern, D.N. (2010). Forms of vitality exploring dynamic experience in psychology,arts, psychotherapy, and development. Oxford University Press.
Suresh A. (2020). Modelling Interpersonal Stances for the iCub Robot, Emaro Master Thesis, University of Genoa

The main aim of this thesis is to develop a new large multimodal dataset to study vitality. The collected data will be used to build novel models for automatic vitality forms recognition. Such models can be used, for example, in human-robot interaction to allow the artificial agent (e.g., humanoid robot) to recognize and interpret the social attitudes of its human interaction partners. The previously created models (e.g., Suresh 2020) were trained on acted data by a very small number of actors. The current thesis will address these limitations as well as explore different sensor types to collect data about “how an action is performed”.

Consequently, particular focus will be given to collecting the multimodal data of humans’ gestures conveying different vitality forms in an ecological setting. For this purpose, several techniques (e.g., retrospection, imagination, mood-inducing sentences, role-playing, etc.) will be used to induce specific states/attitudes in the participants and collect naturalistic data.

The dataset will consist of the performances of several participants collected with multiple sensors such as motion capture systems, video cameras, wearable accelerometers, and tactile sensors.

In the second part, the collected data will be analyzed. It is expected that the student will develop models for vitality forms recognition, using the state-of-the-art techniques of machine learning. These models can be considered a baseline for this specific dataset, and the reference for eventual future improvements.

The student is expected to carry out the following tasks:
1. Short survey on data collection techniques in affective/social computing
2. Design of data collection protocol
3. Data collection
4. Data analysis and development of machine learning models for vitality forms classification
5. Writing thesis report

The student will have the opportunity to develop innovative solutions and, consequently, be involved in scientific publications. The thesis will be developed in - collaboration with CONTACT Unit (Istituto Italiano di Tecnologia). The student will work with an international team of researchers of various backgrounds (cognitive robotics, affective computing, and neuroscience).
The data collection will be performed in IIT offices in Erzelli (Genova).