Here we present data that include subjective reports and behavioural data corresponding to finger touch behaviour from individuals interacting with a sonic interactive surface. We designed and tested a prototype that dynamically alters the texture-related sound feedback based on touch behavior, as in natural surface touch interactions. Data from this user study showed that the frequency of the sound feedback alters texture perception (coldness and material type) and touch behavior (velocity and pressure). The data in this collection are part of The Hearing Body project, a project investigating how the manipulation of action sounds may alter the mental representation of one's body and the related emotional state and body behaviour. Other data collections part of The Hearing Body project have been deposited (Please see Related Resources section below). All 4 parts are experimental data, but they are data from different studies. Part 1 and 2 contain subjective reports and behavioural data, and Part 3 and 4 contain subjective reports, behavioural data and data on electrodermal activity changes. More information on the system and measures used can be found in the related paper: Tajadura-Jiménez, A., Liu, B., Berthouze, N., Bevilacqua, F. (2014) Using sound in multi-touch interfaces to change materiality and touch behavior. Proceedings of the 8th Nordic Conference on Human-Computer Interaction: Fun, Fast, Foundational, Pages 199-202, ACM (see Related resources). The mental representation we have of our body is essential for successful interaction with the environment. This representation is not fixed, but is continuously updated in response to the available sensory information. While previous studies have highlighted the role of vision, touch and proprioception in constructing the body-representation in the brain, the role of auditory information remains largely unknown. Interestingly, the sounds that accompany almost every bodily movement are highly rich in information about the body and the space immediately surrounding it. For instance, the sounds produced when tapping on a surface inform us about the length and strength of our arm. This project will investigate how auditory information generated by our bodies updates our body-representation. A series of psychological experiments will explore how altering self-produced sounds in real-time changes different body-representations, including the representation of the space surrounding the body, the potential actions that we can perform and the emotional states linked to our body capabilities. This multidisciplinary and innovative research project will provide novel insights into the nature of body-representations and, ultimately, guide the design of audio-based applications that can improve body-image, self-esteem, movement patterns and social interactions to support wellbeing and rehabilitation for people with movement impairments.

We looked at changes in texture perception, by using a questionnaire assessing the perceived roughness, coldness, hardness and material of the surface. We also looked at alterations in the way of interacting with the surface by quantifying the changes in the finger movement dynamics, in particular, finger pressure and velocity.