105 Alignment and Agreement of Event Segmentation Relative to Expert Annotation of Naturalistic Experiences from a First-Person Perspective

Faculty Mentor: Cory Inman (Psychology, University of Utah)

Autobiographical memories are constantly formed as individuals navigate through the world and have firsthand experiences. Current laboratory-based memory tasks, however, do not accurately emulate this fluid, first-person experience, as memories are formed in complex, real-world environments. Existing research indicates that individuals chunk their perceived continuous experience into discrete events, a process known as event segmentation, which contributes to the later consolidation of the sequence of events in memory (Kurby & Zacks, 2008). Event Segmentation Theory (EST) proposes that event models are formed in working memory—the component of memory in Baddeley and Hitch’s (1974) model that temporarily holds a limited amount of task-relevant information for immediate use. These models represent the current event and its perceptual details, along with relevant semantic and episodic memory, contributing to predictions about what will occur next (Güler et al., 2024; Kurby & Zacks, 2008). When a prediction error occurs and the model can no longer accurately predict what will happen next due to a contextual shift (e.g. a turn or a doorway), an event boundary might occur. Previous studies have had participants indicate where they believe an event boundary has occurred while viewing a film, a task known as event segmentation (Kurby & Zacks, 2008). These studies have yielded results that demonstrate widespread agreement among individuals on when an event boundary occurs, but are often conducted by having participants view edited videos of actors from a third-person perspective. The current thesis instead used recordings filmed in a first-person perspective of people navigating around a real-world environment around a college campus to analyze event segmentation agreement and alignment of a continuous, first-person experience that is more indicative of our everyday experience.

The data analyzed for this study were archival data previously collected by the INMAN Lab. The study participants consist of 430 undergraduate students from the University of Utah who completed an event segmentation task. The participants were shown three out of 38 first-person videos of five epilepsy patients with implanted intracranial EEG (iEEG) devices navigating around the University of California, Los Angeles (UCLA) campus.

We hypothesized that segmentation agreement and alignment would be the largest and most immediate for highly salient events (e.g., physical shifts in context) and events where what happens next is the most unpredictable (e.g., unexpected social interactions). To gauge alignment, participant responses were compared to when in time trained researchers identified a contextual shift to have occurred. It was found that physical events had the strongest agreement and closest alignment; however, unpredictable events, which involved non-physical spatial changes, did not reach the same level of agreement and alignment as physical events. This pattern may be due to events involving more internal changes in state that event segmenters were not privy to, in comparison to more highly salient physical changes, such as moving from indoors to outdoors or slowing down. Additionally, physical event boundaries may exhibit the highest agreement due to changes in neural processing triggered by changes in the participant’s spatial context (Brunec et al., 2018; Jacobs et al., 2013; Miller et al., 2015). Drawing on Event Segmentation Theory, these findings can be applied to everyday life. According to this theory, physical event boundaries should reset neural processing related to remembering the prior experience and beginning a new memory for upcoming experiences In memories for these experiences, information presented or experienced around these boundaries should demonstrate increased recollection or stronger memory (Kurby & Zacks, 2008; Swallow et al., 2009).

The current study also examined individual-level segmentation agreement amongst the undergraduate participants and the patients themselves who filmed the navigation videos and later completed event segmentation tasks. The “surprise index” can be used to analyze how well the event boundaries identified by an individual agree with events identified by the overall group (Sasmita & Swallow, 2022). The surprise index was calculated to examine whether the patients who completed the walks perceive event boundaries differently from the undergraduate participants who were unfamiliar with the route. We found significant overlap between the responses of patients and undergraduates, with minor differences, suggesting that similar but not identical event segmentation processes are used when viewing novel experiences and familiar, past lived experiences. With regards to past Event Segmentation Theory research, this finding does not fully invalidate the ecological validity of having participants perform event segmentation tasks on other people’s experiences, as it supports that there is one underlying event segmentation process that underlies the same core types of events, independent of past experience. However, these findings raise questions about how exactly the segmentation patterns differ depending on past experience and novelty. Overall, this study characterized the perceived structure of experience, as measured by event segmentation during first-person navigation around a real-world environment, and opens the door to novel naturalistic paradigms for testing the influence of event boundaries on the neural processing of autobiographical memories.

Bibliography