MultiCo-Hub: a corpus of multimodal enrichments with motion-trajectory annotation

PID

MultiCo-Hub is a multimodal dataset including 11 zipped subsets (henceforth: sessions) of time-aligned audio, video and motion-capture–derived BVH data, together with multi-layered Annotation Pro files (ANTx) extended with automatically extracted motion-trajectory layers.

The dataset includes a dedicated training session demonstrating body movement (TESM_001).

The video and audio files included in remaining 10 sessions are derived from the MultiCo corpus (http://hdl.handle.net/11321/942). The original MultiCo sessions were enriched by means of: - full audio, video and BVH streams synchronization to enhance precise multimodal analysis; - motion-capture (BVH) data normalization, conversion, and integration directly into the annotation files as layers describing trajectories of selected body parts (positions, speeds, gesture-space coordinates).

Furthermore, for each session, the corpus provides a composite multi-view video file showing all four camera angles simultaneously. This makes the dataset easier to inspect and substantially more accessible for users working on standard-performance computers.

MultiCo-Hub offers a compact, ready-to-use resource for research and education in the areas of speech–gesture coordination, gesture space, temporal properties of movement, communicative alignment of interlocutors, and multimodal interaction.

Export to common formats (TextGrid, EAF, CSV, etc.) is supported via Annotation Pro, facilitating downstream statistical analysis, visualization and interoperability.

The MultiCo-Hub set also served as input for developing a set of R and C# applications and scripts that support the analysis and visualization of gesture space, temporal movement properties, and communicative alignment in dialogue.

Identifier
PID http://hdl.handle.net/11321/964
Related Identifier https://clarin-pl.eu/dspace/bitstream/handle/11321/964/TESM_001.zip?sequence=1&isAllowed=y
Related Identifier https://clarin-pl.eu/dspace/bitstream/handle/11321/964/ASMB_003.zip?sequence=13&isAllowed=y
Related Identifier https://clarin-pl.eu/dspace/bitstream/handle/11321/964/MJAB_003.zip?sequence=2&isAllowed=y
Related Identifier https://clarin-pl.eu/dspace/bitstream/handle/11321/964/JSKZ_003.zip?sequence=11&isAllowed=y
Related Identifier https://clarin-pl.eu/dspace/bitstream/handle/11321/964/JUPS_003.zip?sequence=8&isAllowed=y
Related Identifier https://clarin-pl.eu/dspace/bitstream/handle/11321/964/JWJS_003.zip?sequence=9&isAllowed=y
Related Identifier https://clarin-pl.eu/dspace/bitstream/handle/11321/964/KKWC_003.zip?sequence=5&isAllowed=y
Related Identifier https://clarin-pl.eu/dspace/bitstream/handle/11321/964/LCMT_003.zip?sequence=7&isAllowed=y
Related Identifier https://clarin-pl.eu/dspace/bitstream/handle/11321/964/MMMP_003.zip?sequence=3&isAllowed=y
Related Identifier https://clarin-pl.eu/dspace/bitstream/handle/11321/964/OKWS_003.zip?sequence=12&isAllowed=y
Related Identifier https://clarin-pl.eu/dspace/bitstream/handle/11321/964/TSRT_003.zip?sequence=4&isAllowed=y
Metadata Access https://clarin-pl.eu/oai/request?verb=GetRecord&metadataPrefix=oai_dc&identifier=oai:clarin-pl.eu:11321/964
Provenance
Creator Klessa, Katarzyna; Karpiński, Maciej; Jarmołowicz-Nowikow, Ewa; Sawicka-Stępińska, Brygida; Klessa, Wojciech
Publisher Adam Mickiewicz University, Poznan
Publication Year 2025
Rights Creative Commons - Attribution-NonCommercial-NoDerivatives 4.0 International (CC BY-NC-ND 4.0); PUB; https://creativecommons.org/licenses/by-nc-nd/4.0/
OpenAccess true
Contact clarin-pl(at)pwr.edu.pl
Representation
Language Polish
Resource Type corpus
Format text/plain; charset=utf-8; application/zip; downloadable_files_count: 11
Discipline Linguistics