Title: Multimodal realisation of conceptual structures on websitesSøren Vigild Poulsen
In this talk I will present a cognitive perspective on the analysis of websites as multimodal texts. In doing so, I try to show how the analytical model for websites by Martinec & van Leeuwen (2008) can be further developed, and how cognitive semantics can contribute to greater adequacy in multimodal website analysis.
Martinec & van Leeuwen argue that websites as multimodal text are structured by underlying semantic patterns that combine images, text, sound, typography and other modes into meaningful text units. An analysis of websites must therefore first examine the underlying patterns, and second the realization of these patterns into an interface.
In order to get a better understanding of the organizing principles of the patterns and how they can be realised multimodally in the website interface, I propose that the patterns are to be understood as conceptual structures, and that these structures and their projection are always manifested by the use of multiple semiotic modes (Hurtienne & Blessing 2007, Forceville & Urios-Aparisi 2009). By combining social semiotic multimodality theory and cognitive semantics, it becomes possible to map out the conceptual structures of a website in greater detail as well as to pinpoint how different semiotic modes contribute to realising parts of these structures.
After a short presentation of the theory, I will demonstrate how it can be applied in an analysis of a website. Finally, I will discuss some of the advantages and challenges of combining a multimodal framework and cognitive theory.References:
Forceville, Charles J. & Eduardo Urios-Aparisi (eds.)(2009). Multimodal Metaphor. Berlin/New York: Mouton de Gruyter.
Hurtienne, Jörn & Luciënne Blessing (2007). Design for Intuitive Use – Testing image schema theory for user interface design. In ICED 07 Paris, 16th International Conference on Engineering Design, Proceedings of the conference (pp. 1-12). Paris: Ecole Centrale.
Martinec, Radan & Theo van Leeuwen (2008). The Language of new media design –Theory and practice. London/New York:Routledge.