Image retrieved from Taylor, S.V., Leung, C.B. Multimodal Literacy and Social Interaction: Young Children’s Literacy Learning. Early Childhood Educ J 48, 1–10 (2020).
Multimodal literacy goes beyond traditional understanding and teaching. This phenomenon includes other modes of communication such as audio, video, sounds, gestures, body movements, and posture. The advent of new communication practices brought by the internet, globalization, and geopolitical shifts forces us to experiment new ways of researching, analysing, interpreting, and understanding social and cultural shifts.
In today's age of new media, we can no longer think of traditional literacy in the way we used to, where we were limited to thinking of all the practices of reading, writing and seeing as separate from each other. Although these semiotic modes have existed for a long time, the Internet provides a space where they are all mixed and combined to create a unique and complete narrative. People do not just read anymore; we are presented with a mix of images and information that requires a new understanding of these elements.
What is excluded from multimodality is a narrow understanding of communication that solely relies on one mode, such as written text or verbal language. Multimodality recognizes the complexity and diversity of human expression and emphasizes the importance of considering various modes of understanding and interpreting communication (Campano, et al, 2020).
Multimodal narratives are not only more engaging but also more inclusive, as they require less effort in the process of obtaining information and allow different senses to understand a concept or theme, providing a space where people with disabilities also have the chance to understand, thus facilitating the entire learning process. Multimodality is intended as a new way of expressing and creating knowledge, allowing for more creativity, increased accessibility,and realisation of deeper and cross-professional collaboration.
However, when we create such narratives and use those methods, we must consider that each medium comes with its characteristics and affordances that shape the meaning and contribute to the overall experience for the reader. Thus, as Marshall McLuhan stated in 1964, "The medium is the message," and different forms of communication can set up particular understandings. When using an image, its representation and understanding, unlike written text, are non-linear and instantaneous, and without a suitable caption, its meaning can be distorted and even lead to issues of racism or social issues. Therefore, we must consider that any practice involving multimodality can be in itself a sensory and synesthetic experience involving different senses which gives rise to an active transformative practice ( Campano, et al, 2020).
Through multimodal narratives, audiences can engage more deeply with the content. The synergy created between different modalities and senses allows for an embodied experience of reading that generates new levels of understanding and interpretation. The space where the digital materiality of objects and our bodies come together becomes a place of interpretive activity in which interpretation is intended as a generative practice that creates new knowledge ( Drucker, 2013).