Corresponding author: Maíra Avelar ( mairavelar@uesb.edu.br ) Academic editor: Olga Iriskhanova
© 2021 Maíra Avelar, Beatriz Graça.
This is an open access article distributed under the terms of the Creative Commons Attribution License (CC BY-NC-ND 4.0), which permits to copy and distribute the article for non-commercial purposes, provided that the article is not altered or modified and the original author and source are credited.
Citation:
Avelar M, Graça B (2021) On counterfactuality: a multimodal approach to (apparent) contradictions between positive statements and gestures of negation. Languages and Modalities 1: 109-120. https://doi.org/10.3897/lamo.1.68236
|
Relying on discussions about recurrent gestures and gestures of negation, in this paper, we aim to demonstrate how the apparent contradiction between negative gestural utterances co-occurring with positive spoken utterances can be explained with the concepts of counterfactuality and epistemic stances, developed in the Mental Spaces Theory framework. To illustrate how gestures of negation can be analyzed as a case of multiple blends and be metaphorically interpreted, we chose three examples of co-occurrences of a positive verbal and negative gestural utterance. Specifically for the discussion proposed here, we selected three videos from the Brazilian TV show “What the hell is this story, Porchat?” (“Que história é essa, Porchat?”). To analyze the data we used the Linguistic Annotation System for Gestures (LASG) and focused on gestural forms and functions, as well as their semantic relation with the speech. The results showed that in all three videos gestures perform a metacommunicative function. Thus, they can be categorized as pragmatic and discursive gestures, realizing specific performative or operational functions. The sweeping away gestures found in two occurrences work on the discursive level to emphasize implicit counterfactuality of the verbal utterance. The throwing away gesture, found in one occurrence, works on the pragmatic level, also to dismiss the positive possibility created by the verbal utterance. In both cases the gesture operates to prevent any possibility of creating an alternative positive mental space, also demonstrating the implicit counterfactuality of the positive verbal utterances.
recurrent gestures, conceptual blending, counterfactuality, epistemic stances, negation, multimodality
Based on the project “Towards a Grammar of Gesture”, developed for gestures in German, we started the project “Towards a Grammar of Gestures for Brazilian Portuguese” several years ago. In this paper, we present a specific issue that emerged from the construction of the Repertoire of Brazilian Portuguese Gestures of Negation. In most cases, these gestures co-occur with a negative spoken utterance. However, we found some examples with a positive utterance and a gesture of negation. In order to explain this apparent contradiction, we used the Mental Spaces (
The paper is organized as follows: in the “Theoretical background”, we consider gestures of negation (Bressem & Müller, 2014;
Before approaching our object of analysis, gestures of negation, it is worth mentioning that we understand gestures in
Furthermore, as indicated by
In the present study we focus on the Recurrent Gestures, since gestures of negation can be placed in this segment of the continuum. As part of a conventionalization process, recurrent gestures can be described as intermediate (i.e., lying between lexical and grammatical processes) and hybrid gestural forms. According to
In terms of basic functions they perform, recurrent gestures can be grouped into referential, performative or pragmatic, modal and parsing gestures. It is pointed out by
In the same line of reasoning,
The idea of grouping gestures into families was initially proposed by
Depending on the orientation of the palm,
Based on the analysis of representational gestures and Gesture Families initially proposed by
Expanding on the ideas introduced by
As Bressem and Müller (2014) indicate, the Away Family is semantically connected by the themes of Rejection, Refusal, Negative Assessment and Negation. According to the authors, what unifies the Away Family is not the shape, position or movement of the hands, but rather the act or effect of moving or keeping unwanted objects, entities, ideas or actions away from the immediate gestural space. This effect can be considered as the motivation for gesture forms of negation. The Away Family consists of four recurrent gestures: sweeping away, holding away, brushing away and throwing away gestures (Bressem & Müller, 2014).
In the theoretical framework of Mental Spaces, negation is understood as a phenomenon that sets up counterfactual scenarios (
In
According to
From this perspective, negation is understood as a primary example of alternativity, since negative expressions set up two Mental Spaces instead of one: the negative space described in the sentence and its positive alternative (
Thus, when negation is used, there are at least two alternatives to which the speaker, as well as his interlocutor(s), can align epistemically (
In contrast, in the sentence “I didn’t buy a car. There was no room for it in the garage” (
Conceptual Blending Theory provides a theoretical framework to deal with how conceptualizations can happen on a human scale. The theory proposes four mental spaces that are projected and interact with each other (
By performing a holding away gesture, the speaker creates with his open hand and palm facing away from his body, a metaphoric barrier between him and the rejected topic, and keeps the space around him free from what he qualifies as unwanted. This occurrence, according to the Mental Space Theory, involves multiple integration operations, as illustrated in Figure
In the diagram we present iconic mappings that allow us to understand the speaker’s open hand as the representation of a barrier. In other words, in Input 1 the hand creates a gestural form and the iconic motivation to enact the construction of a barrier that corresponds to Input 2. The correspondence between these two inputs from the generic space results in a blended space A. The emergent frame of this blend, compressed by similarity between input spaces 1 and 2, results in blended space A “hand as a barrier”. In blended space B, which is metaphoric, the source and target domains correspond to input 3 – in fact, to input 2 of blended space A. Thus, blended spaces A and B share one input space that corresponds to the iconic motivation of the gesture. Specifically, in blended space B this input (Input 3) corresponds to the source domain of the metaphor.
In addition, the second input of this blended space (Input 4) corresponds to the verbal content or target domain of the metaphor. Thus, in blended space B the emergence of the blended structure is also a result from compression by similarity which creates the frame of metaphorical interpretation of the barrier as rejection. From these two operations, the two blends (A and B) now function as inputs for the creation of a third blended space – blended space C – in which the open-hand with the palm held away expresses the interruption or rejection of an action or idea.
In general, such gestures co-occur with negative utterances – with the words or prefixes of negation described earlier. In this sense, the third blended space can work as an input space, containing a metaphoric visual utterance of negation, whereas the second input space, with the space builder “no”, would work as a negative mental space evoked by the sentence and its positive counterpart that is negated or rejected. By saying “I don’t need a tribute”, in this specific context of Figure
The examples selected for analysis were collected from a larger database (
Regarding the methodological procedures adopted to analyze and annotate the occurrences of gestures of negation in multimodal data of the Brazilian Portuguese, we used the Linguistic Annotation System for Gestures (LASG), developed by
Keywords |
---|
No (Não) |
Neither (Nem) |
Never (Nunca) |
Nobody (Ninguém) |
Never* (Jamais) |
None (Nenhuma) |
Without (Sem) |
Nothing (Nada) |
Impossible (Impossível) |
Deactivate (Desativar) |
The MGA suggests that the meaning of a gesture emerges from the interaction between its shape, its sequential position and its insertion in a context of use (
The Linguistic Annotation System for Gestures (Bressem, Ladewig &
To narrow down the parameters of form, the authors of the LASG establish four form parameters established for describing sign languages: (1) handshape, such as: open or closed hand, extended or bended index finger; (2) palm-orientation, such as pronated, supinated, horizontal, vertical or diagonal; (3) movement, such as up, down, to the right, to the left, towards and away from the body; and (4) position in the gesture space, such as self-touching gesture (on the speaker’s own body), proximal, medium and distal from the speaker.
According to the authors (
Regarding the parameters of function, the authors’ proposition is the following: (1) referential gestures that depict concrete objects or abstract entities, ideas, relations or actions; (2) discursive gestures that structure the accompanying verbal utterance, for example by marking emphasis; (3) pragmatic gestures that enact a speech act, such as dismissing an idea (
To perform the gestural form and function analysis, we created tiers in ELAN (Wittenburg et al., 2008) following the annotation procedures proposed by the system: at first, in order not to be biased by the verbal content, the sound of the video was turned off and the gesture stroke form was described; next, the sound was turned on and, after the orthographic transcription using the GAT2 (2009) system was made, gesture modes of representation were categorized. Finally, gestures were analyzed along with the linguistic context with which they co-occurred, and the gestural functions were identified.
The annotation categories used in the research are presented in Table
Annotation of the data in ELAN according to the parameters suggested in the LASG.
In this section we present an analysis of three occurrences of positive verbal utterances along with negative gestural utterances. All of them were collected from the YouTube channel of the talk-show "What the hell is this story, Porchat?" (“Que história é essa, Porchat?”
The first occurrence consists of a sweeping away gesture, performed by actress Taís Araújo when responding positively to a question about criticism she received during her career.
According to the representation proposed in the Blending Space Theory, the verbal and gestural input spaces, as well as the blended space, can be represented in the following way:
The actress states, with a positive assertive stance (
In the second example host Fábio Porchat asks a question to Marília Gabriela, a famous Brazilian TV interviewer, and performs a sweeping away gesture combined with the statement that her shows “always” had the same format. The example is presented in Figure
By using the word “always” combined with the sweeping away gesture, the host configures a counterfactual alternative mental space, since the multimodal construction conveys a negative sense, that is, the fact that Marília Gabriela’s shows have never had a different format. In that sense, in the counterfactual mental space, there is a positive counterpart, denied by the gesture, which indicates the possibility of Marília Gabriela’s programs having other formats different from the formats already known. Thus, it is possible to assert that the meaning of the gesture remains the same, despite what is being said. That is, the sweeping away gesture, in this occurrence, is still a gesture of negation with an operational function, even co-occurring with a positive utterance, because it operates at the discursive level and not on the structure of the spoken utterance; the speaker implicitly builds a counterfactual space, referring to other possible formats of the show presented by Marília Gabriela and using the gesture eliminates these possibilities (
In this occurrence, the representation proposed by the Blending Space Theory, the verbal and gestural input spaces, as well as the blended space can be represented as in Figure
The following occurrence is taken from another episode of the program and consists of a story told by an audience participant about a supernatural experience she had as a child.
In this episode, the speaker states that people do not usually believe her when she tells this type of story. In doing so, she enacts the type of reaction that people usually have in situations like that and performs the throwing away gesture with a performative function. In this example, analyzed in terms of the Blending Theory, the spaces of verbal and gestural input, as well as the blended space can be represented as in Figure
The gesture in question reveals a negative epistemic stance conveyed in the speaker’s reported speech and provides a frame of interpretation for the verbal utterance, manifesting the illocutionary force (
As it was shown by the analysis, recurrent gestures, in general, are gestures employed in a metacommunicative way that operate on the speaker’s utterance. They can serve an operational, a parsing, a performative or a modal function. When they perform an operational function, these gestures act as an operator in relation to the spoken utterance (
When they perform modal and performative functions, recurrent gestures provide an interpretation frame for a certain excerpt of the spoken discourse and may also reveal an attitude or stance of the speaker in relation to what is being said or done. An example of this is the quotation mark gesture, commonly used to highlight a portion of the spoken utterance (
In general, the study has confirmed that recurrent gestures convey semantic and pragmatic information and can work as discursive markers or even as grammatical and lexical morphemes (
In addition, the Conceptual Blending Theory was shown to be a productive model to approach the apparent contradiction between positive verbal utterances and negative gestural utterances, since the notions of counterfactuality and alternativity demonstrate that gestures of negation can perform operational functions along with positive statements, working as intensifiers of the verbal utterance. They can also carry out performative functions, when co-occurring with verbal utterances that reveal epistemic stances of negative assessment.
In both kinds of gestures discussed here, the gestures perform pragmatic and discursive functions, operating in a metacommunicative way. In sum, recurrent gestures convey conventionalized gestural patterns that are very important to the meaning of multimodal constructions.
The majority of this research was conducted with the support of the Coordination for Improvement for Higher Education (CAPES Foundation, Brazil). We are grateful for the grant awarded to us (Code 001).