Here we present data that include subjective reports, electrodermal activity changes and behavioural data corresponding to hand tapping behaviour from individuals interacting with a sonic interactive surface. This interactive surface delivers surface tapping sounds corresponding to three different applied strength levels during tapping. Results showed that altering the sound changes motor behaviour, emotion, and surface perception.The data in this collection are part of The Hearing Body project, a project investigating how the manipulation of action sounds may alter the mental representation of one's body and the related emotional state and body behaviour. Other data collections part of The Hearing Body project have been deposited (Please see Related resources section below). All 4 parts are experimental data, but they are data from different studies. Part 1 and 2 contain subjective reports and behavioural data, and Part 3 and 4 contain subjective reports, behavioural data and data on electrodermal activity changes. More information on the system and measures used can be found in the related papers: (1) Tajadura-Jiménez, A., Bianchi-Berthouze, N., Furfaro, E., Bevilacqua, F. (2015). Sonification of virtual and real surface tapping: evaluation of behavior changes, surface perception and emotional indices. IEEE MultiMedia, 22(1), 48-57 (Special issue on Interactive Sonification). (2) Furfaro, E., Berthouze, N., Bevilacqua, F., Tajadura-Jiménez, A. (2013). Sonification of surface tapping: Influences on behaviour, emotion and surface perception. Interactive Sonification Workshop (ISon 2013), Bielefeld, 9th-10th December, 2013.The mental representation we have of our body is essential for successful interaction with the environment. This representation is not fixed, but is continuously updated in response to the available sensory information. While previous studies have highlighted the role of vision, touch and proprioception in constructing the body-representation in the brain, the role of auditory information remains largely unknown. Interestingly, the sounds that accompany almost every bodily movement are highly rich in information about the body and the space immediately surrounding it. For instance, the sounds produced when tapping on a surface inform us about the length and strength of our arm. This project will investigate how auditory information generated by our bodies updates our body-representation. A series of psychological experiments will explore how altering self-produced sounds in real-time changes different body-representations, including the representation of the space surrounding the body, the potential actions that we can perform and the emotional states linked to our body capabilities. This multidisciplinary and innovative research project will provide novel insights into the nature of body-representations and, ultimately, guide the design of audio-based applications that can improve body-image, self-esteem, movement patterns and social interactions to support wellbeing and rehabilitation for people with movement impairments.

We looked at changes in emotional action-related responses by quantifying subjective and physiological emotion-related changes. For this purpose, we included several scales in a questionnaire. First, we used seven-point Likert scales to assess the perceived physical strength, the ability to complete the task, and the aggressiveness felt when tapping on the surface. Second, we quantified the subjective mental effort by asking participants to indicate the stress felt while tapping on a vertical analog scale. Third, we quantified participants’ emotional valence, dominance, and arousal by using the three nine-item graphic scales of the self-assessment manikin. Arousal was further quantified by looking at the physiological changes recorded by the GSR biosensor. We also looked at alterations in the way of interacting with the surface by quantifying the changes in the movement dynamics. For this purpose, we used the logged accelerometer and piezo data. Finally, we assessed the perceived surface physical quality of hardness with a seven-point Likert scale.