“Metaphor in pictures and multimodal representations.” In: Raymond W. Gibbs, Jr. (ed.), The Cambridge Handbook of Metaphor and Thought, 462-482. Cambridge: Cambridge University Press.
Abstract: In this chapter I will outline the moderately well understood phenomenon of pictorial metaphor and demonstrate the validity of the hitherto untheorized concept “multimodal metaphor.” As a first approximation, I will define multimodal metaphors as metaphors in which target, source, and/or mappable features are represented or suggested by at least two different sign systems (one of which may be language) or modes of perception. Multimodal metaphors can be delineated best by first describing pictorial metaphors, the type of non-verbal metaphor that has attracted most scholarly attention. How exactly the construal and impact of pictorial and multimodal metaphors differ from their verbal counterparts is largely unexplored territory, but the following factors appear to play a role. First, the pictorial or multimodal nature of target and source means that they are apprehended differently from their verbal counterparts: pictures, sounds, and gestures have a perceptual immediacy that is lacking in language. One dimension of this perceptual immediacy is a high degree of specificity. Uncle Bert’s happiness is not just upward-directed, it is upward-directed in precisely the way depicted in the aforementioned scene, with a certain speed of ascension, a certain instability once the characters are up, etc. Second, pictorial and multimodal representations have different, medium-determined ways of cueing the similarity between target and source than language has. Third, inasmuch as sounds and pictures are more easily recognized transnationally than (unfamiliar) languages, pictorial and multimodal metaphors allow for greater cross-cultural access than verbal ones. Fourth, pictorial and multimodal source domains probably have a stronger emotional appeal than verbal ones.
Download article: http://dare.uva.nl/en/record/309344