Research Article |
Corresponding author: Elena Loginova ( lenlogiya@gmail.com ) Academic editor: Maria Kiose
© 2022 Elena Loginova.
This is an open access article distributed under the terms of the Creative Commons Attribution License (CC BY-NC-ND 4.0), which permits to copy and distribute the article for non-commercial purposes, provided that the article is not altered or modified and the original author and source are credited.
Citation:
Loginova E (2022) Meaning construal through multimodal clusters in the theatrical discourse. Languages and Modalities 2: 27-36. https://doi.org/10.3897/lamo.2.78572
|
The paper presents the results of an empirical study of monomodal and multimodal data collected from the plays by Russian playwrights of the second half of the 20th century (A. Vampilov, A. Kazantsev, N. Sadur) and the theatrical performances based on these plays. The main aim is to find out what semiotic modalities are more frequently involved in the process of recurrent semiosis and, as such, in the multimodal meaning construal. Following the classical as well as contemporary works in the field explored, the research sets out to elaborate the cognitive-semiotic approach to the analysis of the theatre-specific means of multimodal meaning construal to get insights into dynamicity, embodiment and figurativity of artistic expression typical of theatrical performativity and inaccessible to other arts. The data are explored by means of a complex methodology that comprises quantitative and qualitative methods with the diagraph analysis as a key analytical tool which serves to recognize meaningful correlations within and across modalities resulting from mappings between dialogically juxtaposed independent utterances. This paper demonstrates that the involvement of nonverbal resources is mainly manifested in the use of hand and head gestures, facial movements and body repositioning. Next to the quantitative results, the research shows that co-occurring conceptualizations through verbal and nonverbal components, gestures in particular, can trigger off the amplification of meaning or bring about the emergence of new conceptual projections.
diagraph, meaning construal, multimodality, resonance, theatrical discourse
I started to analyze plays of British, American and Russian playwrights several years ago sharing the view that in this kind of discourse we deal with mediated perception, a secondary process derived from and based on the primary experience of direct perception, according to J. Gibson’s “ecological” approach to cognition [
These two potential focuses of cognitive-semiotic attention account for what can be termed as the dynamism and transformability of the sign, when it happens to be a modification in the secondary material vehicle and/or the mental concept, i.e., a change from the meaning expressed in one mode to the meaning expressed in another mode. In this paper, the term mode which stands for “a semiotic resource for making meaning” [
In this research my task is primarily of quantitative nature. It boils down to the analysis of Russian plays of the second half of the 20th century (A. Vampilov, A. Kazantsev, N. Sadur) and their theatrical interpretations in order to specify what semiotic modes are more frequently involved in the process of recurrent semiosis. The multimodal data are explored with the help of a statistically verified multimodal analysis [O’
The paper is organized as follows: as a starting point I present a brief overview of the main areas of contribution to the present-day multimodal studies. I revise the issue of semiotic modality relying on the notion of theatrical codes in the works by E. Fisher-Lichte and K. Elam. Special attention is given to gestural modality and the types of gestures offered by A. Cienki and C. Műller. Given the theoretical orientations, I formulate the objectives and working hypotheses. After that, I describe the material, methods and tools used in the research. Then I present a quantitative and qualitative discussion of the samples selected for the analysis, zooming in on the major findings which refer to the theoretical framework outlined. Finally, I discuss the implications of the study in light of the multimodal meaning construal in human communication and in the arts as communication and also present some suggestions for further research.
Recent research in a range of disciplines, including cognitive psychology, linguistics, semiotics and poetics, has focused on the ways in which people pair verbal language with nonverbal expressions, be it in comics, cartoons, memes, demotivators, advertisements, face-to-face interaction, and more. Recognition of multimodal meaning construction mechanisms in spoken discourse is the task tackled by scholars representing different fields of inquiry such as manual co-speech gestures in various types of discourse [
The pioneering works which radically changed the prospects for the scientific analysis of the theatrical performance include books and articles by the Prague School semioticians: J. Mukaŕovskĵ, O. Zich, P. Bogatyrev, R. Jakobson, etc. Their ideas of the theatrical sign as a culturally and socially determined unit of meaning which points beyond the denotative basis to the implicitly represented content (cf., Bogatyrev’s “a sign of a sign and not the sign of a material thing” [Semiotics of Art. Prague School Contributions 1976: 33]) were elaborated by K. Stanislavsky, V. Meyerhold, A. Tairov, J. Grotowski and other theatre directors, who acknowledged the power of actual visual perception. They argued that the actor’s work does not happen without a sense of the body. The processes by which actors move the language of the plays off the pages and into their bodily actions, so that it can be shared with the audience, are rooted in the physical and the sensual, as much as they are in words. And it is exactly for this reason К. Stanislavsky described a successful performance as “the artistic embodiment of inner emotional experience” [
Embodied construal of meaning linked to multimodal behavior has defined а theatrical performance since the origins of theatre in prehistoric rituals. Characters, be they gods, people or animals, have always been represented through the bodies of actors with puppet theatre as an exception [
In this paper, I intend to show that co-existing heterogeneous signs of a theatrical performance converge to enrich the meaning projections and achieve the pragmatic effect required. I proceed from the assumption that a play and its theatrical interpretation can be considered parts of a unique communicative space: the original communicative situation and its multimodal “derivative” respectively. The list of modalities involved in the process of meaning-making in the multimodal theatrical performance includes, but is not reduced to, facial expression, gesture and other movements related to body parts, stage decoration and stage conception, costume, hairstyle, music, lighting (see also the classifications of theatrical codes in [
Sounds | acoustic | transient | space-related |
Music | |||
Linguistic signs | actor-related | ||
Paralinguistic signs | visual | ||
Mimic signs | |||
Gestural signs | |||
Proxemic signs | |||
Mask | long-lasting | ||
Hair | |||
Costume | |||
Stage conception | space-related | ||
Stage decoration | |||
Props | |||
Lighting |
The multimodal derivational discourse draws simultaneously upon cognitive processes from across the spectrum given, involving visual and auditory perception together with verbal cognition. So, one and the same concept can be represented by verbal means (in the source discourse) and multimodal clusters of verbal and non-verbal means (in the discourse derivative). What results is a performance founded on the cultural norms of society at large and on secondary regulative rules, peculiar to the theatre as a type of multimodal discourse opposed to all other types both in terms of ontological status of the creative work it involves and with regard to the conditions for its production and interpretation.
The special nature of theatrical signs that stems from the special ontological status of the performance allows for yet another approach to be taken into consideration. Multimodality on stage is an adaptive process which results in the strengthening of meaning projections between the semiotic resources used. By simultaneously or successively representing complementary and supplementary information in various modalities, resonance arises to highlight or enhance the content of the original communicative situation. This phenomenon cannot be attributed to any sign in isolation. It is a property of discourse relations between verbal or verbal and nonverbal signs, gestures in particular [
What I intend to show with my analysis is that meaning construal in the theatrical discourse depends largely on resonance as a kind of interactive alignment which accounts for pragmatic effects and, as such, for communicative efficiency. The analysis concentrates on resonating gestures, particularly actors’ manual movements (iconic or metaphorical) linked to their speech.
For the purposes of the research, I use the classifications of speech-linked gestures offered by A. Cienki and C. Műller. Their classifications include referential gestures which are subdivided into deictic (pointing/touching) and representational gestures. The representation gestures are further specified according to the mode of representation: enacting, embodying, holding/touching, tracing, molding gestures. The other types encompass pragmatic gestures which show one’s stance on the issue under discussion; adaptors (self-adaptors and object-adaptors as non-speech-motivated movements) and beats [
Generally, it may be a problem for researchers to divide up speech-linked gestures into separate groups. It is so because different pragmatic gestures (e.g., the throwing away gesture to express a negative stance or the palm-up open hand used when presenting an idea) might have the same metaphor in their core being based on mimetic schemas or image-schemas [
The situation becomes more complicated when the communicative context analyzed is that of a secondary modelling system in which different functions of gestures can be realized in parallel to allow richer forms of expression. Furthermore, as a rule, gestures in the arts as communication are not characterized by immediacy, nor do actors on stage engage in spontaneous talk. Every verbal and nonverbal sign are the result of rehearsal so that certain images gain intensity and become appealing. How strictly bodily actions in theatrical discourse are regulated depends upon the strength of the conventions at work. E.g., in the classical Chinese and Japanese Noh theatres, the verbal and nonverbal behavior is strictly predetermined. In conventional Western theatre the secondary order embodiment is less tightly constrained with the primary goal for actors to make their choices emotionally charged. Theatrical gestures are regarded as showing body eloquence. Traditionally, they were classified into rhetorical gestures based on antique oratory (motions of the hand and fingers) and gestures-affects used to express surprise, disgust, remorse, censure, etc. The detailed description of these groups, with 24 codified gestures in each, was given in 1646 by J. Bulwer in his treatise on gesture which appeared to be the first book dedicated to the “natural language of the hand” [
Given this stance, the objectives I have set myself are twofold. On the one hand, I am going to find out what modalities are more frequently realized in a theatrical performance simultaneously with speech and under the guidance of a single communicative intention. On the other hand, the focus of my attention is on the multimodal cluster – speech and gesture – to reveal the functions of gestural component in meaning-making and create a gestural profile for gestures used on stage.
I hypothesize that: a) the gestural modality together with facial movements would be the most active in multimodal meaning construal in the theatrical discourse; b) the pragmatic and representational gestures would prevail in the theatrical discourse; c) gestures in the theatrical discourse would complement the content expressed by speech or represent some new content which can be in supplementary or contrary relations with the semantics of the verbal utterance.
The objectives and the hypotheses account for the design of the research and the methods used.
For case-study 1 the source discourse (SD) was A. Vampilov’s “The Elder Son” (1965). I also turned to 22 theatrical performances based on this play staged by the theatres in Moscow, Irkutsk, Nizhniy Novgorod, Perm, Kostroma, Tyumen, Velikie Luki, and more. They all were retrieved from The International on-line Festival “The Elder Son — 55” (November, 2020). The total duration of the discourses derivatives (DDs) is 3110 minutes.
The preliminary step included the analysis of the SD: searching for recurrent cues produced by the same character or different characters, and recurrent communicative situations. The total number included 147 instances of the monomodal data. The next step was to find out whether in the DDs: a) the recurrent cue(s) is(are) modified; b) nonverbal modality(s) accompanying the verbal one is(are) used. Given these tasks, I used the modelling method accompanied by the parametric analysis [Iriskhanova, Cienki 2018;
At first, a list of multimodal parameters was compiled. The list included the verbal modality and also those nonverbal modalities of a theatrical performance that interact with speech, i.e., reflect what is simultaneously produced verbally (gestures, facial movements, body movements, actions, music and/or sound effects, light effects). Each parameter was given a number from 1 to 9. The entire list includes the following parameters (P): Р₁ – verbal component unchanged; Р₂ – verbal component modified; Р₃ – gestural modality (hand gestures) added; Р₄ – head gestures and facial movements added; Р₅ – movements of the legs and/or body movements added (e.g., crossing or uncrossing the arms, legs; changing the seating position; etc.); Р₆ – body movements which indicate a change of place (proxemic signs) added; Р₇ – characters’ actions (also with props) added; Р₈ – music and/or sound effects added; Р₉ – light effects added.
Then the numerical indicator of activity for each parameter in the DD was calculated. In the table below I give the activity indicators on the 22 theatrical performances (DD) supplied with charts (Table
As the next step, I calculated the average activity indicator and the range of activity indicators for each parameter. Normally, the number of the DD, in which the activity of this or that parameter is above the average indicator, and the number of the DD, in which the activity of this or that parameter is below the average indicator, do not exhibit any significant differences. The results are given in the following table (Table
The next step was to create a model of the distributed parameter activity. Schematically, it is presented in the following volume chart (Diagram 1).
Given the need for more insight into the convergence of gesture and speech in the theatrical performance, I proceed with a more detailed research of gesture types used in the theatrical discourse.
General statistics on multimodal parameter activity in the 22 discourse-derivatives.
DD1 | DD2 | DD3 | DD4 | DD5 | DD6 | DD7 | DD8 | DD9 | DD10 | DD11 | DD12 | DD13 | DD14 | DD15 | DD16 | DD17 | DD18 | DD19 | DD20 | DD21 | DD22 | ||
---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|
P1 | 26 | 22 | 28 | 27 | 30 | 32 | 24 | 29 | 33 | 29 | 21 | 32 | 28 | 25 | 23 | 21 | 32 | 24 | 26 | 27 | 29 | 31 | |
P2 | 74 | 78 | 72 | 73 | 70 | 68 | 76 | 71 | 67 | 71 | 79 | 68 | 72 | 75 | 77 | 79 | 68 | 76 | 74 | 73 | 71 | 69 | |
P3 | 68 | 61 | 66 | 70 | 62 | 59 | 71 | 60 | 74 | 76 | 65 | 70 | 66 | 62 | 68 | 72 | 66 | 70 | 62 | 60 | 72 | 68 | |
P4 | 66 | 67 | 62 | 68 | 70 | 62 | 70 | 69 | 65 | 62 | 60 | 70 | 54 | 59 | 70 | 66 | 58 | 72 | 58 | 61 | 69 | 66 | |
P5 | 45 | 48 | 43 | 47 | 41 | 48 | 48 | 49 | 49 | 51 | 41 | 48 | 50 | 37 | 53 | 40 | 45 | 48 | 52 | 41 | 43 | 44 | |
P6 | 48 | 46 | 50 | 40 | 44 | 52 | 52 | 46 | 53 | 44 | 40 | 55 | 46 | 51 | 54 | 42 | 54 | 49 | 52 | 56 | 48 | 46 | |
P7 | 6 | 5 | 5 | 4 | 2 | 4 | 4 | 2 | 3 | 2 | 4 | 2 | 3 | 5 | 4 | 3 | 6 | 2 | 3 | 5 | 2 | 4 | |
P8 | 3 | 2 | 4 | 3 | 3 | 2 | 2 | 3 | 4 | 3 | 2 | 4 | 3 | 2 | 0 | 3 | 2 | 3 | 2 | 1 | 2 | 3 | |
P9 | 2 | 1 | 2 | 3 | 2 | 2 | 1 | 0 | 1 | 2 | 3 | 1 | 2 | 2 | 3 | 3 | 2 | 1 | 3 | 3 | 1 | 2 |
General statistics on the average activity indicators and the range of activity indicators for each parameter.
The average activity indicator | The range of activity indicators | The activity indicator exceeding the average value of the parameter | |||
---|---|---|---|---|---|
min | max | range | |||
P1 | 27.2 | 21 | 33 | 12 | 11 |
P2 | 72.8 | 67 | 79 | 12 | 11 |
P3 | 66.7 | 59 | 76 | 17 | 11 |
P4 | 64.7 | 54 | 72 | 18 | 12 |
P5 | 46.0 | 37 | 53 | 16 | 12 |
P6 | 48.5 | 40 | 56 | 16 | 11 |
P7 | 3.6 | 2 | 6 | 4 | 12 |
P8 | 2.5 | 0 | 4 | 4 | 12 |
P9 | 1.9 | 0 | 3 | 3 | 15 |
The monomodal dataset used in case-study 2 consists of four SDs – the plays by the 20th century Russian playwrights: A. Vampilov’s “Duck Hunt” (1967); N. Sadur’s “The Odd Woman” (1982) and “Go” (1983); A. Kazantsev’s “The Fleeing Wanderers” (1996). The multimodal dataset comprises six DDs, namely theatrical performances staged by the Moscow Arts Theatre, the Mossovet State Academic Theatre, the Drama School of Yaroslavl State Theatre Institute, Primorsky Drama Theater, “Magic lantern” and “On the chairs” theatres; with a total duration of 620 minutes.
At the first step of the analysis, I selected all the indicators for gestures given in the stage directions. The set of samples comprises 35 indications of representational, pragmatic and deictic (pointing) gestures. There were no indicators for discourse-structuring gestures in the plays analyzed. Then I resorted to the video data to see whether the gestures were realized in the performances or not. To fulfill the task, I used the diagraph analysis suggested by John W. Du Bois in his study of spoken dialogues. The key unit of the analysis is the diagraph recognized as a coupling structure when one speaker (addressee) forms an utterance based on the immediately co-present utterance of an interlocutor (addresser) so that affinities across utterances occur. The following exchange provides an illustration of what Du Bois calls a diagraph [
1 | Joanne. | it | ’s | kind of | like | ˆyou | Ken | . |
2 | Ken. | that | ’s | not at ˆall | like | me | Joanne | . |
To facilitate the analysis of similarities and differences in gestures across the SD and the DD I reformatted the diagraph in order to display the mappings both in rows and columns emerging in dialogic juxtaposition of a monomodal utterance in the SD and its multimodal counterpart in the DD. The multimodal diagraph consists of two parallel sets of rows for the SD and the DD respectively; and also contains a separate column for nonverbal modalities which interact with speech.
The quantitative results obtained with the help of the diagraph analysis are the following: 11 gestures (31.5%) were realized in the DD. The same percentage was for gestures which were modified (the change of gesture type). 13 gestures (37%) were not realized in the DD. The data did not exhibit any correlations between the type of gesture and its usage or non-use in the DD.
The next step of analysis deals with the study of gestures which were added in the DD to cooperate with speech in organizing a performance. The observational method employed made it possible to acquire a dataset of 2577 gestures synchronized with speech or following it. Using the Sampling Analysis tool in MS Excel 100 gestures were selected that were subject to further examination and statistical processing. The quantitative results are distributed as follows: the most frequent are the pragmatic and representative gestures (42% and 31% respectively); then come the deictic gestures (24%); the discourse-structuring gestures are characterized by the lowest frequency (3%).
The final step of the analysis engaged a modelling method based on a number of parameters identified due to the previous studies of gestures as a dynamic system [The multimodal dimensions of the discourse 2021]. They are as follows: regular, intentional, affective, cumulative, synthetic, complementary, additive, contrary. Schematically the multi-vector model for gestures is presented in the diagram below (Diagram 2).
In this section of the paper I discuss the results of the empirical study to see whether they confirm or refute the hypotheses made.
The first hypothesis was that gestural modality together with facial movements would be the most active in multimodal meaning construal in the theatrical discourse. This hypothesis was partially confirmed. The results obtained show that the activity indicators are really the highest for P₃ and Р₄: hand and head gestures and facial movements such as eyebrows raised, wrinkles across the forehead, eyelids wide open, and more. At the same time, Р₆ (body movements) and Р₂ (verbal component modified) also have relatively high activity indicators. The ways of verbal modification include the repetition or omissions of words or phrases; or enlarging the cue. The following samples illustrate this observation.
In sample 1 the actor repeats the concluding phrase of the cue twice, adding a representational gesture (the stretched index finger hold) and a mimic sign (squinted eyes) which resonate with the speech, making the content salient. The image extracted from the video data (Irkutsk Academic Drama Theater named for N. P. Okhlopkov) and also the arrows in the diagraph explicate the correlations between the verbal and nonverbal components when the information in one modality enhances the information in another modality.
Sample 2 shows that the verbal component added and the gestures synchronized with it indicate a new conceptual projection – the necessity to lie in order to survive in life – contributing to the source utterance content (Moscow Drama Theater “Sphere”).
Verbal | Nonverbal | |||||
SD | 1. Busygin. | We have to | invent | something | … | - |
2. | We have to | tell a lie | , | - | ||
3. <…> | We have to | frighten or | - | |||
pity | . | - | ||||
DD | 1. Busygin. | We have to | , | |||
2. | have to | invent | something | … | a pragmatic gesture | |
3. | We have to | tell a lie | ↔ | , | a pragmatic gesture | |
4. <…> | We have to | frighten or | a representational gesture | |||
5. | pity | ↔ | . | a representational gesture | ||
6. | Learn to | lie | . | |||
7. | Learn | ↔ | ! | discourse-structuring gestures | ||
8. | Learn | ! | (head nods) | |||
The original text and the script (abridged): | ||||||
SD | DD | |||||
Бусыгин. Надо выдумать что-то такое… <…> Надо соврать как следует, только тогда тебе поверят и посочувствуют. Их надо напугать или разжалобить. | Бусыгин. Надо, надо, соврать как следует, только тогда тебе поверят и посочувствуют. Их надо напугать или разжалобить. Так что учись врать. Учись! Учись! |
Sometimes a new conceptual projection can be activated with the help of nonverbal modalities only, e.g., body repositioning or body movements and poses that signal some implied meaning. The sample that can illustrate this observation is the final scene set by the actors as an allusion to “The return of the prodigal son”, a famous painting by Rembrandt. The resonating scene setting (The theater company “Free Stage”) emphasizes the role of nonverbal modalities in the process of recurrent semiosis which links the second order embodiment, salience and resonance.
Summarizing the findings, we see that whereas the arts as communication rely on multimodal interaction for meaning construction, the highlighting function is not restricted to speech-accompanying hand and head gestures and facial movements. But being mimetic and characterized by continuously varying forms, exactly these modalities have much in common with prosody in their potential for semantic nuancing [
The second hypothesis was that pragmatic and representational gestures would prevail in the DD due to the ontological nature of a theatrical performance. This hypothesis was confirmed. Besides, I wanted to check whether gestures indicated in the stage directions in the SDs were realized in the DDs. The results of the analysis reveal an approximately equal distribution of the gestures into three groups: gestures that are contextualized, concretized (or modified) and non-used in the DDs. The following samples from A. Vampilov’s “Duck Hunt” and the theatrical performance staged by the Moscow Arts Theatre illustrate the contextualization and concretization of the gestures prescribed by the playwright.
Verbal | Non-verbal | ||||
SD | Valeria. | Oh, Vadim Andreevich | I will | … | theatrically |
I will pray for you | . | ||||
DD | Valeria. | Oh, Vadim Andreevich | I will | … | a representational gesture: praying hands |
pray for you | . | In Bulwer’s classification this gesture is described as Oro | |||
I will pray for you | . | ||||
The original text and the script: | |||||
SD | DD | ||||
Валерия (театрально). О, Вадим Андреич! Я готова… Я готова на Вас молиться. | Валерия. О, Вадим Андреич! Я готова просто… (жест «мольба») молиться. На Вас молиться. |
Verbal | Non-verbal | ||||
SD | Galina. | Have you ever killed anything | at least a small, this small bird | ? | shows the size of a bird using the finger |
Kuzakov. | He can’t get into this one | . | shows the size of a bird using both hands | ||
DD | Galina. | Have you ever killed anything | at least a small, this small bird | ? | a representational gesture: with the thumb and index fingers shows the size of the bird |
Kuzakov. | He can’t get into this one | . | a representational gesture: stretches both hands up and spreads them to the side to show the size of a bird | ||
The original text and the script: | |||||
SD | DD | ||||
Галина. Ну скажи, убил ли ты что-нибудь хоть раз? Ну хотя бы маленькую, ну хоть вот такую (показывает на пальце) птичку? | Галина. Ну скажи, ну признайся, признайся! Убил ли ты что-нибудь хоть раз? Ну хотя бы такую, ну вот такую (большим и указательным пальцами показывает размеры птички) маленькую птичку? | ||||
Кузаков. Ну что ты ему показываешь? Он в такую (показывает обеими руками) не попадает. | Кузаков. Ну что ты ему показываешь? Он в такую (вытягивает обе руки вверх и разводит в стороны) попасть не может. |
As the main result, the analysis showed that 73% of the gestures used in the DDs are those with pragmatic and representational functions. This induces us to ask whether representational gestures (and also discourse-structuring gestures) in the theatrical performance as a second modelling system are deprived of pragmatic function or do they demonstrate different functions (e.g., pragmatic and representational) which complement each other?
While A. Kendon speaks of two main functions of co-speech gestures, namely substantial and pragmatic gestures [
Consider the following extract from A. Kazantsev’s “The Fleeing Wanderers” staged by the Mossovet Theatre: During this time, many have burst, rolled up, left, and so on… (The original text: Многие за это время лопнули, закатились, уехали и так далее…)
The verbs in the cue are synchronized with a hand gesture (hand swing upward). It is a metaphoric gesture which concurrently fulfills the discourse-structuring function (Fig. 1a–c). It would be possible to use the same type of gesture choosing the opposite direction. But it is the upward movement that accentuates the meaning invested in the multimodal cluster, i.e., to vanish into thin air, disappear without a trace. Thus, the gesture renders the speaker’s attitude which mirrors the intent that the stage director and the actor want to express being the co-authors of the theatrical performance.
The third assumption was that gestures in the DD can complement the content expressed by speech or represent some new content which can be in supplementary or contrary relations with the semantics of the cue. This hypothesis was also confirmed. Gestures and speech in a theatrical performance can contextualize the same image (samples 3–4); the gestural modality can add expressiveness (samples 1–2); or invest some new conceptual projections into the image represented. The additive function of gestures can be illustrated by the following sample from N. Sadur’s “The Odd Woman” staged by “On the chairs” theatre.
Verbal | Nonverbal | |||||||
SD | Woman | 1. I | ’m | running away | ||||
2.You | ’re | catching me up | ||||||
3. | If you catch me | there will be paradise | ||||||
4. | If you don’t catch me | there will be the end of the world | ||||||
DD | Woman | 1. I | ’m | running away | deictic and representational gestures (acting out the scene) | |||
2.You | ’re | catching me up | ||||||
3. | If you catch me | there will be paradise | representational (metaphoric) gestures | |||||
4. | If you don’t catch me | there will be the end of the world |
According to the dialogic syntax structure [
The original text and the script are the same (abridged):
Баба. <…> Я – убегаю. Ты – догоняешь. Поймаешь – рай, не поймаешь – конец всему свету.
One more observation is that the SD usually has verbal triggers that make it possible to activate the gestural modality or some other semiotic modalities. E.g., the description of a character’s behavior given in a stage direction can account for a certain costume, hairstyle, facial expression and also gait. Some linguistic units can suggest acting out the scene with the help of gestures-illustrators. E.g., phrases like “this kind” or “like this”, intensifiers and textual ellipsis trigger nonverbal completion of an object or an event. Manual action verbs usually co-occur with iconic gestures, e.g., to dial, to switch on, etc. Deictic words in a character’s speech can be accompanied by а pointing gesture on stage or, on the contrary, а pointing gesture can replace the verbal phrase and, as such, represent a nonverbal utterance.
So, the interplay of speech and co-speech gestures which emerges in everyday communication displays specific features and functions when realized in the arts as communication, namely in a theatrical performance in which gestures can simultaneously implement several communicative attitudes assuming a multiplicity of interpretations. As was confirmed, despite the context-dependent, polysemous nature of most hand and head movements the actors employ, it is possible to identify the pragmatic function as a core common to many particular uses.
The results of the multimodal analysis show that one can distinguish two major trajectories of inferencing the original communicative situation: complementary and supplementary. The former means that the co-occurring conceptualizations through verbal and gestural components in the performance can result in the salience of the content and resonance, the pragmatic aspect being preserved. The latter means the emergence of new conceptual projections accompanied by both resonance and the modulation of the pragmatic aspect. Thus, resonance in the arts as communication involves intended additional meaning and is linked to symbolic cognition enfolded with mimetic cognition.
Summing up, the research focus was on the exploration of speech-linked modalities that take part in the multimodal meaning construal in the theatrical discourse. The study has proved the efficiency of the diagraph analysis in organizing information from different modalities and in making the mapping relations explicit. Together with the statistical procedures this analytical tool made it possible to solve the main objectives of the research and confirm the hypotheses set. As was shown, the application of this methodology is relevant for estimating the activity and functions of different modalities in the meaning-making process, which means that it can be applied for the analysis of other types of heterogeneous discourse with the aim of revealing the construal process specificity.
Viewed from the perspective of the dialogic syntax, the original content (a play) and its multimodal derivative (a performance) are engaged in the “pragma-dialogue” [
Being relatively young, the multimodal theory of meaning in different types of discourse needs a more fine-grained exploration before it can answer all the questions that are asked of it. The current research serves as a necessary prelude to a more in-depth study of gestural modality in the theatrical discourse so as to specify the semiotic functions and also variation of gestural patterns as the pivotal semiotic resources of the theatre; and to an overarching research of resonance as a phenomenon that can help us enhance our understanding of embodied meaning construal through different modalities, especially in the arts as communication.