Our actions shape our everyday experience: what we experience, how we perceive and remember it, is deeply affected by how we interact with the world. Performing an action to deliver a stimulus engages neurophysiological processes which are reflected in the modulation of sensory and pupil responses. In this study, we hypothesized that these processes shape memory encoding, parsing the experience by grouping self- and externally-generated stimuli into differentiated events. Participants encoded sound sequences, in which either the first or last few sounds were self-generated and the rest externally-generated. We tested recall of the sequential order of sounds that had originated from the same (within event) or different sources (across events). Memory performance was not higher for within event sounds, suggesting that the memory representation was not structured by actions. However, during encoding, we replicated the well-known electrophysiological response attenuation, together with increased pupil dilation for self-generated sounds. Moreover, we found that at the boundary between events, physiological responses to the first sound originating from the new source were determined by the direction of the source switch. The results suggest that introducing actions, acts as a stronger contextual shift than removing them, despite not directly contributing to memory performance. The findings contribute to our understanding of how interacting with sensory input shapes our experiences, by addressing the unexplored relationships between action effects on sensory responses, pupil dilation and memory encoding, and discarding a meaningful contribution of low-level neurophysiological mechanisms associated to action execution in the modulation of memory.