Refiner processing in visual narrative grammar

DOI

Linguistic syntax has often been claimed as uniquely complex due to features like anaphoric relations and distance dependencies. However, visual narratives of sequential images, like those in comics, have been argued to use sequencing mechanisms analogous to those in language. These narrative structures include “refiner” panels that “zoom in” on the contents of another panel. Similar to anaphora in language, refiners indexically connect inexplicit referential information in one unit (refiner, pronoun) to a more informative “antecedent” elsewhere in the discourse. Also like in language, refiners can follow their antecedents (anaphoric) or precede them (cataphoric), along with having either proximal or distant connections. We here explore the constraints on visual narrative refiners created by modulating these features of order and distance. Experiment 1 examined participants’ preferences for where refiners are placed in a sequence using a force-choice test, which revealed that refiners are preferred to follow their antecedents and have proximal distances from them. Experiment 2 then showed that distance dependencies lead to slower self-paced viewing times. Finally, measurements of event-related brain potentials (ERPs) in Experiment 3 revealed that these patterns evoke similar brain responses as those observed to referential dependencies in language (i.e., N400, LAN, Nref). Across all three studies, the constraints and (neuro)cognitive responses to refiners parallel those shown to anaphora in language, suggesting domain-general constraints on the sequencing of referential dependencies.

Some files are also present in their preferred file format. Preferred formats are file formats of which DANS – based on international agreements – is confident that they will offer the best long-term guarantees in terms of usability, accessibility and sustainability. For more information on preferred file formats, see https://dans.knaw.nl/en/file-formats/.

Identifier
DOI https://doi.org/10.34894/BVDLZX
Metadata Access https://dataverse.nl/oai?verb=GetRecord&metadataPrefix=oai_datacite&identifier=doi:10.34894/BVDLZX
Provenance
Creator Cohn, Neil ORCID logo; Schilperoord, Joost ORCID logo; Foulsham, Tom ORCID logo; Van Middelaar, Lincy
Publisher DataverseNL
Contributor Cohn, Neil; Tilburg University; DataverseNL
Publication Year 2023
Rights CC-BY-4.0; info:eu-repo/semantics/openAccess; http://creativecommons.org/licenses/by/4.0
OpenAccess true
Contact Cohn, Neil (Tilburg University)
Representation
Resource Type Experimental data; Dataset
Format application/pdf; type/x-r-syntax; text/plain; application/vnd.oasis.opendocument.spreadsheet; application/vnd.openxmlformats-officedocument.spreadsheetml.sheet; application/vnd.openxmlformats-officedocument.wordprocessingml.document
Size 181092; 6352; 413934; 6022; 3656009; 187537; 142715; 186679; 142352; 25232; 316916
Version 1.0
Discipline Agriculture, Forestry, Horticulture, Aquaculture; Agriculture, Forestry, Horticulture, Aquaculture and Veterinary Medicine; Humanities; Life Sciences; Linguistics; Social Sciences; Social and Behavioural Sciences; Soil Sciences