Word-to-meaning connection: a comparison between auditory and visual modalities in the first and second language

Abstract

The present study examined the link between words and their semantic representations in two modalities, namely, the visual modality vs. the auditory modality. It compared the reaction times (RTs) of native vs. advanced-level non- native speakers of English on a word-picture matching task. The goal was to examine in which of the two modalities an L2 word can more quickly activate the target semantic representation. The participants were presented with one word at a time, each followed by either a matching or mismatched image. The task consisted in using a key on the keyboard to respond as quickly as possible to indicate whether or not the presented image matched the previously presented image. Word stimuli were presented either in auditory format (using a headset) or visually (a string of letters on the screen). The time taken from the word recognition phase to the matching image represents the time that it takes the perceptual system to activate the sematic representation of a given word. The results show that the link between an L1 word and its semantic representation is not significantly different in these two modalities. However, there is a statistically significant difference between the two modalities when a word is presented in the L2 (RT after the auditory modality was slower). The observed result might be due to the difficulty in matching an auditory-presented L2 word to its L2 phonological representation. This factor would cause a slower activation of the related semantic representation compared to the visually-presented words.