Multimodal Storytelling: Enhancing Kermit's Story in Voyager

The Kermit Voyager Edition brings together various multimodal elements to tell a rich, interactive story of Kermit’s cultural and historical impact. Multimodal storytelling explores how meaning is created through a combination of communication modes such as language, images, gestures, music, and sound (Kress, 2010). This theory is important in understanding how this edition works, as it incorporates multiple forms of media to provide a deeper and more immersive experience for the viewer.

Almost all communication today is inherently multimodal, integrating various modes of expression to convey meaning. For example, spoken discourse combines language with intonation, gestures, and facial expressions, while written discourse merges language with typography, layout, and visuals (Van Leeuwen, 2015). In virtual environments, particularly in multimedia settings like Voyager, these modes come together dynamically to form a complete experience. For instance, Kermit's voice in the audio introduction and other elements contributes to his friendly and familiar image, while the interaction between Kermit's 3D model and other elements in the Voyager 3D viewer creates a playful experience for the viewer. This contrasts sharply with the conventions of academic writing. In academic writing, journals have followed similar visual conventions for more than 300 years, even though the ways they are produced and shared have changed. While they play a key role in sharing research, they are rarely known for visually engaging or inventive designs. Still, scholars express a clear interest in improving the reading experience, especially on digital platforms (Barness & Papaelias, 2021). In this edition, combining written texts with visual elements makes academic content easier to access and understand. These modes give the edition a unique character that brings academic research to life.

Modes of Communication in the Kermit Voyager Edition.

Reflecting on Kermit's evolution across various media such as music, TV, film, and social media, we can present him in a way that mirrors the media forms that have solidified his place as a cultural icon. Raptis et al. (2021) argue that multimodal technologies in cultural heritage applications combine visual and auditory elements to create immersive experiences, and the Kermit Voyager Edition does precisely this. The ability to click, rotate, or listen to him invites a closer engagement than traditional video viewing. This works particularly well because Kermit, as a character, is known for breaking the fourth wall, directly addressing the audience and drawing attention to the constructed nature of his world (Culp, 2011). Moreover, the multimodality of this edition reflects Jim Henson’s original intention of using Kermit and the Muppets to promote not only entertainment but also emotional and academic growth in young audiences (Falk, 2012). The edition brings together different modes of communication to engage viewers both intellectually and emotionally.

Balancing the different modes required careful consideration. We needed to decide how and when to combine text, media, or additional 3D models so that viewers could understand why certain elements were included without feeling overwhelmed or distracted. One challenge was obtaining images, as Kermit is part of a highly commercialized franchise, which limited our options due to copyright restrictions. As a result, we relied more on YouTube videos for media integration. It also raised the question of how to maintain a coherent narrative without fragmenting the viewer's attention. While the edition creates an enriched experience for users who engage with all its interactive features, some viewers may not fully explore these functions, potentially missing out on parts of the narrative.

Overall, the edition shows that combining text, visuals, and interactivity can make academic research more accessible, engaging, and meaningful to a broader audience.