Affective teaching: learning more effectively from empathic robots

PDF version | Permalink

Ginevra Castellano, Iolanda Leite, Ana Paiva, and Peter W. McOwan

9 January 2012

Children perceive robots as more helpful when the robot reacts to emotion by displaying empathic behaviour.

In recent years robots have increasingly been viewed as social entities to be integrated in our daily lives.1 Research has shown that robots used in applications requiring direct interaction with human users—for example robot companions—benefit from the integration of socially intelligent behaviour.2, 3 Current prototypes of robot companions lack important capabilities that impede human users' efforts to bond with them. The ability to display social, affective behaviour is one of the main requirements for an artificial companion to engage in a natural interaction with humans. Robotic companions can represent a valuable resource in many different contexts: they can participate directly in therapy for the elderly or disabled, or assist caregivers.4 Additionally, robots can be used as personal assistants, employed to entertain and motivate users (‘edutainment’ purposes) or in the entertainment industry.

Robots need social perception abilities, which include affect sensitivity (i.e., the ability to recognize people's emotions and affective states5), to behave in a socially intelligent manner.6 Affect-sensitive robots are more likely to engage with human users and maintain a believable interaction with them over extended periods of time. An affect-sensitive companion robot, for example, would try to ensure that the user is interested in maintaining the interaction and act empathically towards them if they are sad or not willing to engage. However, such a robot would not disturb the person by trying to engage them in some activity not approached.

User interacting with the iCat robot.

In the LIving with Robots and intEractive Companions (LIREC) project, we study companions that interact with people over extended periods. The goal of LIREC is not to build robot companions that replace human contact, but rather to design companions that fulfil their tasks and interact with people in a socially and emotionally acceptable manner. For example, one of the interaction scenarios we investigated includes a robot acting as a game companion for young children. This social robot, the Philips iCat, plays educational games (such as chess) with children using an electronic chessboard and helps them improve their skills (see Figure 1). While playing with the iCat, children receive feedback about their moves on the chessboard through the robot's facial expressions, which are generated by a system influenced by the state of the game.7

Previous user studies conducted in the iCat scenario have suggested the need for integrating empathic behaviours in the robot.8 Previous work also showed that children's behaviour can be used to predict their affective state.9, 10 Taking these findings into account, we developed an empathic robot model that contains two main parts: an affect detection system that recognizes whether the children are experiencing a positive or negative feeling, and an action selection mechanism that chooses the best empathic response to display at a certain moment.11 The main idea behind the empathic model is to allow the robot to establish an affective loop with the children, by recognizing their emotions and responding to them in an empathic way (see Figure 2). The robot's affect detector, based on Support Vector Machines (SVMs), processes real-time behavioural and contextual features—such as smile and eye gaze—as well as the state of the game and the game evolution. SVMs use a specific learning process to automatically analyse and classify data. The affect detector generates a probability value for each of the following valence conditions: positive, neutral and negative. When the user is not experiencing a positive feeling, the iCat employs an empathic strategy to increase the user's enjoyment (e.g., it provides encouraging comments or offers help).

Affect sensitivity and empathic reactions as components of the affective loop.

Initial experiments with primary school children showed that children perceived the robot as more engaging and helpful when it reacted to their emotions by displaying empathic behaviour. Our future work will include long-term studies where other affective states such as interest and engagement are detected by the robot. The empathic model will be further developed to include personalized empathic strategies.

In summary, we claim that social perceptive abilities are a necessary requirement for allowing a robot to behave in a socially intelligent manner. Models for empathic behaviour under development can build on some of these abilities, such as affect sensitivity, to create personalized companions that are capable of establishing a successful interaction with users over extended periods of time.


Ginevra Castellano
School of Electronic, Electrical and Computer Engineering University of Birmingham

Ginevra Castellano is a researcher at the Human Computer Interaction Centre. Her research interests include context-based social perception, affect recognition from the face and body, closing the affective loop in virtual agents and robots and evaluation of affective interaction.

Iolanda Leite
Instituto Superior Tecnico (IST) Technical University of Lisbon

Iolanda Leite is a PhD candidate at IST and a researcher at the Intelligent Agents and Synthetic Characters Group (GAIPS) of the Institute of Systems and Computer Engineering Research and Development in Lisbon, where she works on the LIREC project. Her research interests are focused on the role of empathy and adaptation in long-term interaction with social robots.

Ana Paiva
Instituto Superior Tecnico (IST) Technical University of Lisbon

Ana Paiva is a professor in the Department of Computer Science and Engineering. She is also the group leader of GAIPS. Her scientific interests are in autonomous agents, embodied conversational agents and robots and multi-agent simulation systems.

Peter W. McOwan
School of Electronic Engineering and Computer Science Queen Mary, University of London

Peter McOwan is a professor and the coordinator of LIREC. His research interests are in visual perception, mathematical models for visual processing, cognitive science and biologically inspired hardware and software. He is also active in science outreach.

  1. F. Tanaka, A. Cicourel and J. R. Movellan, Socialization between toddlers and robots at an early childhood education center, Proc. Nat'l. Acad. Sci. USA 194, pp. 17954-17958, 2007.

  2. C. Breazeal, Role of expressive behaviour for robots that learn from people, Philos. Trans. R. Soc. B 364, pp. 3527-3538, 2009.

  3. K. Dautenhahn, Socially intelligent robots: dimensions of human-robot interaction, Philos. Trans. R. Soc. B 362, pp. 679-704, 2007.

  4. B. Robins, P. Dickerson, P. Stribling and K. Dautenhahn, Robot-mediated joint attention in children with autism: a case study in robot-human interaction, Interaction Studies 5, pp. 161-198, 2004.

  5. Z. Zeng, M. Pantic, G. I. Roisman and T. S. Huang, A survey of affect recognition methods: audio, visual, and spontaneous expressions, IEEE Trans. Pattern Anal. Mach. Intelli. 31, pp. 39-58, 2009.

  6. G. Castellano and C. Peters, Socially perceptive robots: challenges and concerns, Interaction Studies 11, pp. 201-207, 2010.

  7. I. Leite, A. Pereira, C. Martinho and A. Paiva, Are emotional robots more fun to play with?, IEEE Int'l. Symp. Robot and Human Interactive Commun., pp. 77-82, 2008.

  8. I. Leite, C. Martinho, A. Pereira and A. Paiva, As time goes by: long-term evaluation of social presence in robotic companions, IEEE Int'l. Symp. Robot and Human Interactive Commun., pp. 669-674, 2009.

  9. G. Castellano, A. Pereira, I. Leite, A. Paiva and P. W. McOwan, Detecting user engagement with a robot companion using task and social interaction-based features, Proc. Int'l. Conf. Multimodal Interfaces, pp. 119-126, 2009.

  10. J. Sanghvi, G. Castellano, I. Leite, A. Pereira, P. W. McOwan and A. Paiva, Automatic analysis of affective postures and body motion to detect engagement with a game companion, IEEE Proc. Int'l. Conf. Human-Robot Interact., 2011.

  11. I. Leite, G. Castellano, A. Pereira, C. Martinho and A. Paiva, Modelling empathic behaviour in a robotic game companion for children: An ethnographic study in real-world settings, Submitted..

DOI:  10.2417/3201112.003948