Meet Melisa Allela, Storyteller and Interactive Media Design Maven
Re-imagining the Traditional Forms of Storytelling in Africa
by Melisa Allela
Human beings are storytelling animals; this is how we make sense of the world. In a rapidly changing world, how do we leverage the capabilities of emerging technologies to preserve African oral traditions of storytelling? Infusion of Artificial Intelligence with immersive technologies such as Virtual Reality (VR), which enable social interaction between users and animated virtual characters, are already revolutionizing the ways in which computational systems are used to entertain, educate, and train humans. My research on storytelling through experimental animation explores the use of animation techniques and emerging technologies as a medium for relaying works of African oral literature (“orature”).
We communicate through stories, but also use stories to entertain, educate and inform. Presentation of events within a story can take the form of either of the three main types of narrative structures. The first is a linear narrative, in which events are presented in a chronological order and the user has no control over the story and plot development. The second is a nonlinear narrative, where the plot does not conform to a direct causality pattern and the user may have limited control over the plot progression. The third is an interactive narrative, where the plot is either linear or nonlinear and is driven by the user’s interaction. In the field of animation, interactive narratives exist at the convergence of film and gaming technologies. This allows for interaction between users and virtual agents, and in doing so, mimics the experience of a face-to-face encounter between storyteller and audience.
Works of African orature, owing to their performative and participatory nature, cannot be adequately documented in textual format or via linear media such video recordings (combinations of animated or live action footage, text, and audio). To address this, I study and create animation systems that bring together interactive narratives, virtual reality, and animated virtual humans, and thus have the potential to relay the unique features of African oral storytelling in a digital medium. These systems should exhibit narrative intelligence, which means that they are capable of restructuring the plot and can react to the user’s narrative experience. In addition, the virtual characters must be believable with human-like qualities. These are virtual humans that can emote, communicate, and read the user’s verbal and non-verbal behaviour in real-time. This is no simple task, and requires a cross-disciplinary approach traversing computer graphics, artificial intelligence (AI), human-computer Interaction (HCI), and animation. Some key technical challenges include contextual kinesics (generating relevant gestures and facial expressions) of an autonomous animated character, synchronising speech and non-verbal behaviours, and lastly, natural language processing to enable human-like dialogue and verbal interaction between a user and the virtual character.
My research employs a studio-based research design, which is commonly used by practitioners in the creative industries. This involves a balance of both theory and practice, and will culminate in an exhibition of works. The resulting work will be an immersive and interactive animated adaptation of Ugandan poet Okot p’Bitek’s narrative poem, Song of Lawino. Its symbolic and affective properties, as well as rich figurative imagery, provide narrative content and visual reference for interpretation in the interactive animated format.
Through this research project, I seek to make a conceptual and technical contribution to the use of emerging technologies in reimagining traditional forms of storytelling in Africa. Virtual Reality is already widely used for cultural preservation of historical sites and archaeological artefacts in the emerging field of Virtual Heritage. However, preservation of intangible culture is still an overlooked aspect of cultural heritage preservation in VR. UNESCO’s Convention for the Safeguarding of the Intangible Cultural Heritage emphasizes the “importance of the intangible cultural heritage as a mainspring of cultural diversity and a guarantee of sustainable development.” Thus, my research also contributes to advancing these broader global goals by leveraging modern technologies towards the preservation of our intangible cultural heritage.
ABOUT THE AUTHOR
Melisa Allela is an Assistant Lecturer and 2nd year doctoral student at the Technical University of Kenya, enrolled in the Design and Creative Media department. Her research topic is “Storytelling Through Experimental Animation: Challenging Traditional Frameworks of Representation.” She is interested in the convergence of experimental animation and emerging technologies that push the boundaries of artistic and scientific practice in Interactive Media. She looks forward to the tipping point moment for women in tech and their increased participation in the fields of Animation and Interactive Media production.