Visual artists, performing artists and literary artists have been inspiring one another since
time immemorial. Simonides of Keos (c. 556-468 BC) confirmed this in his “poema pictura
loquens” – “a poem is a talking picture” or in a more modern expression, “a picture is worth
a thousand words”. Jonckheere (1989) calls this an ancient relationship, and Vanbrussel
(1972) holds the opinion that imitating one art form to create another art form is a form of
translation, transposition, contemplation, and discovery of artistic impressions, showing
parallel interpretations rather than physical comparison.
Kress and Van Leeuwen (2001) define multimodality as “[t]he use of several semiotic
modes in the design of a semiotic product or event”. It is this kind of research (see Kress &
Van Leeuwen (2001), Jewitt (2009) and O’Halloran & Smith (2010) in the field of multimodal
studies in the social semantic tradition that forms the nucleus of multimodality). O’Halloran
and Smith (2010) note that cross-pollination between disciplines increased during the 21st
century, and that interdisciplinary and transdisciplinary cooperation are at the core of research
and social challenges. Jewitt (2013) adds to this by defining multimodality as “an interdisciplinary approach that understands communication and representation to be more than
about language”. The poetry film as a transposed verbal text to a multimodal text can be
viewed as a hybrid, transdisciplinary and multimodal artform, combining poetry and film.
Cook (2017) emphasised the fact that poetry film is an intertwined entity of word, sound, and
visual image. It is an attempt to transpose and transform a poem to become a new artwork
that makes the poem more accessible to people who are not necessarily open to the written
word and will in effect attract a larger audience to a genre that usually has a limited market.
Animated poetry film is a goldmine for discovering, amongst others, multimodal metaphors,
particularly because it uses a vast variety of creative modes of meaning-making modes. It is
well-known that conventional researchers of the Conceptual Metaphor Theory (CMT) limit
their studies mainly to the conceptual metaphor as a verbal-linguistic expression. Little to no
attention is paid to the nonverbal manifestations of metaphors as such (Jacobs et al., 2013:
490). Forceville (1996, 2002, 2006, 2008, 2009) and other researchers (Carrol, 1994; Cienki
& Müller, 2006; Zbikowski, 2008; Koller, 2009; El Refaie, 2009; Urios-Aparisi, 2009 among
others) use the CMT as a conceptual framework for the research of multimodal metaphors.
Forceville (2002) defines multimodal metaphor as follows: “A phenomenon that is experienced
as a unified object or gestalt is represented in its entirety in such a manner that it resembles
another object or gestalt even without contextual cues.” Various source- and target domains
in verbal language are used, as well as domains in nonverbal communication, sounds and
music to irrefutably form a complex network through which meaning is created.
This article explores poetry film as a multimodal translation/transposition of a poem text
into an animated poetry film with specific reference to Jac and Wessel Hamman’s poetry film
by DJ Opperman, “Stad in die Mis”, and the extent to which multimodal metaphors are present
in the transposed version.
Visuele kunstenaars, uitvoerende kunstenaars en literêre kunstenaars het mekaar oor die eeue
heen geïnspireer. Jonckheere (1989) noem dit ’n oeroue verbintenis en Vanbrussel (1972) is
van mening dat die produk wat geskep word vanweë die inspirasie as ’n tipe vertaling,
transponering en ondersoek na dieper betekenis beskou moet word. Die poësiefilm as ’n vorm
van ’n vertaling van die woordteks na ’n beeldteks, kan benader word as ’n hibriede, transdissiplinêre en multimodale kunsvorm wat die poësie en film met mekaar verbind. In die
besonder geanimeerde poësiefilms bied ’n vrugbare ontdekkingsveld vir die verkenning van
multimodale metafore. Die draai van die millennium het ’n toename in die belangstelling in
die ondersoek van multimodale kommunikasie met betrekking tot die onderskeie semiotiese
modusse teweeggebring. Forceville (2015) is van mening dat animasiefilms ideaal is vir die
ondersoek van beeldskemas wat op multimodale metafore betrekking het. Die keuse van die
poësiefilm astoepassingsdomein stel in die vooruitsig dat kennis ontwikkel word met betrekking
tot multimodale metafore in Jac en Wessel Hamman se poësiefilm van DJ Opperman se “Stad
in die mis”.
