Multimodal classroom discourse analysis of TCSL teachers based on ELAN

: With the advent of the information age, a lot of diversified language symbols such as discourse, text, pictures, body language, audio and video are widely used in teaching. Therefore, the construction of discourse meaning has the characteristics of multimodality. Multimodal discourse analysis theory provides a new perspective for the research of international Chinese classroom teaching. Based on ELAN6.0, this paper analyses the multimodal discourse of four TCSL teachers, so as to reveal the modal selection strategies of excellent international Chinese teachers, and based on this, attempts to summarize the teaching mode suitable for international Chinese classroom teaching.


Introduction
Since entering the multimedia era, multimodal discourse has begun to occupy all fields of social communication, and it has also had a great impact on second language teaching. [1] Multimodal teaching was first practiced in English teaching and achieved good results, and it has strong applicability to TCSL. In order to clearly understand the current use of multimodality in the international Chinese classroom, we use ELAN6.0 video analysis software to mark, analyze and count the classroom teaching of Chinese teachers. Through a combination of quantitative and qualitative methods, this paper discusses the multimodal classroom discourse of Chinese teachers, and analyzes the modal invocation and coordination of different teaching links, in order to provide reference for international Chinese teachers to use multimodality and multimedia effectively.

Brief description of relevant studies
In the 1990s, there was a boom in multimodal discourse research in western countries, and its landmark achievement was Reading Images: the Grammar of Visual Design, published by Kress and van Leeuven in 1996. [2] The New London Group put forward the concept of multi-literacy in 1996, and proposed that teachers should make full use of images, audio, multimedia and other teaching modalities to stimulate students' multiple senses in language learning, in order to overcome the shortcomings of traditional literacy methods and improve students' literacy skills [3] . Subsequently, more and more scholars have begun to pay attention to the relationship between multimodality and language teaching. For example, Royce, T. (2002) studied the complementarity of different symbols in multimodal discourse and the synergy of multimodality in second language classroom teaching [4] . Jewitt (2006) proposed that classroom teaching should be a combination of multiple modalities. [5] .
Compared with some countries, the study of multimodal discourse in China started relatively late. Chinese multimodal discourse analysis began with Zhanzi Li (2003), which introduced Kress & van Leeuwen's "visual grammar" in detail and aroused the attention and discussion of multimodal discourse analysis in China academic community. [6] The paper introduced Kress & van Leeuwen's "visual grammar" in detail, which aroused attention and discussion on multimodal discourse analysis in China. A large number of scholars, including Yongsheng Zhu, Zhuanglin Hu, Yueguo Gu, Delu Zhang, Shulan Dai, Qinhong Wei, Dezheng Feng, etc., have discussed the origins, definition, and analysis framework of multimodality. [7] As a leading figure in multimodal discourse research in China, Delu Zhang's (2015) monograph Multimodal Discourse Analysis Theory and Foreign Language Teaching comprehensively discusses the application and design of multimodal discourse analysis theory in foreign language teaching [8] . It has attracted wide attention from the linguistic and educational communities. In addition, a succession of scholars ( Xiuli Liu, 2013; Jinlan Zhu, Xinren Chen, 2015, Jianhua Ning, 2019, etc.) have focused on the effectiveness of teachers' multimodal discourse in foreign language classrooms and put forward feasible suggestions for better multimodal teaching [9] .
China research on multimodal teaching is mainly focused on English teaching, on the contrary, there are few multimodal researches on teaching Chinese as a second language. The existing research results mainly discuss the application of multimodal teaching mode in teaching Chinese as a second language from the micro level, mainly discussing the application in teaching language elements such as vocabulary teaching, grammar teaching or culture class, reading and writing class and audio-visual courses class, such as Shujing Si (2013) [10] , Yanyan Jiang, Dengqiao Li(2016) [11] , Xianjun Pan (2019) [12] etc. The above studies are mainly qualitative descriptions. It is worth mentioning that since 2019, there has been a trend from quantitative descriptions to quantitative analysis using ELAN software in related researches on Chinese multimodal teaching (eg Jinhua Tian, 2019 [13] ; Yanyan Jiang, Bin Liu , 2020 [7] ; Shan Wang, Junyu Liu , 2020 [14] ).
In conclusion, the research on multimodal classroom discourse of Chinese teachers is still relatively few. On the one hand, there are more applied discussions and fewer theoretical ones; on the other hand, there are more descriptive qualitative studies and fewer statistical empirical studies using tools. This provides us with research directions and research spaces.

Research corpus
In this study, four classroom videos of international Chinese language teaching were selected as the research corpus. Video 1 and Video 2 are the classroom recordings of outstanding international Chinese teachers from Beijing Language and Culture University, and we identified the teachers as skilled teachers; Video 3 and Video 4 are the video recordings of the winners of the fourth "Chinlingo Award" National Comprehensive Chinese Education Skills Competition. All four videos are based on the teaching of language points, and the overall effect of the classroom is good, which is more exemplary and representative.

Research methodology
The study was mainly based on a combination of quantitative research and qualitative analysis. The research tool is ELAN 6.0 video analysis software (download at http://www.latmpi.eu/tools/elan/elan-old-versions). The software is mainly used for the analysis of multimodality, which can be accurate to 0.01 seconds, can be looped for easy segmentation, labeling, and also for location finding, transcription, replacement, and segmentation statistical analysis.

Research content
The research content is to explore the modal invocation and coordination of Chinese as a second language teaching in different teaching stages. Firstly, use ELAN video analysis software to mark and count the multi-modal usage of different teaching links. Secondly, compare and analyze the modal usage of teachers and contestants in different teaching links. Thirdly, the modal combination scheme applicable to each teaching session of Chinese language point teaching is summarized.

Analysis of multimodal teaching
The relationship between multimodality and the teaching process is a three-dimensional relationship of "multifaceted integration", that is to say, the teaching process is realized in the simultaneous operation of multiple modalities. [15] Modality can be categorized from different perspectives, and this study focuses on three main modalities in the Chinese teaching classroom: auditory, visual and body posture modalities.

Auditory modality
Auditory modality refers to the acquisition of symbolic resources by students through hearing in classroom teaching, that is, the media perceived by stimulating the learners' auditory nerves [16] . The teacher's discourse is mainly analyzed here. 2% Analysis of the data shows that both skilled teachers' classroom discourse accounted for about 45%, while both contestants' classroom discourse accounted for about 65%. We know that sound is the most direct tool for expressing meaning. It not only has a large amount of information, but also spreads quickly. Therefore, auditory modality accounts for a large proportion of the classroom. But generally speaking, the proportion of time between teachers' explanation and students' practice should be no less than 60% in the "comprehensive class" and no less than 70% in the "oral class"." [17] Therefore, this data shows to a large extent that the contestants' classes did not give students enough time to practice, and did not well implement the principle of "intensive lectures and more practice". In language teaching, teachers should make their explanations more concise and provide students with opportunities to express themselves.

Visual modality
Visual modality refers to the acquisition of symbolic resources by students in classroom teaching, that is, the media perceived by stimulating the learners' optic nerves. [16] The visual modality types and quantity statistics of this study are shown in the following table. 1.71% In this study, the visual modalities were divided into five categories: blackboard writing and text presented on PPT, PPT pictures (referring to static pictures presented on PPT), objects (pictures and objects displayed by teachers during explanations and drills), and videos. From the annotated data, the number of PPT text and drawings is the largest among the four teaching videos, and teachers seldom write on the blackboard, which proves that the classroom is no longer the traditional "textbook + blackboard + teacher speaking and students listening" model. Instead, they use a variety of modalities to fully engage students' senses in the classroom. In video 2, the teacher explained the structure of the word "Ba" by writing on the blackboard and using pictures to support the explanation, without choosing PPT presentation. After observation and analysis, it is found that it is necessary for teachers to write on the blackboard in class when explaining the sentence structure. On the one hand, it helps students' understanding. On the other hand, it can deepen students' memory. Although in the age of multimedia, teachers cannot rely on multimedia completely, and on-site board writing often helps to deepen students' understanding and memory.

Gesture language
Gesture language runs through almost every teaching link of teaching Chinese as a second language. On the one hand, it helps teachers to convey information more accurately, and on the other hand, it helps students to better receive information. Norris (2009) [18] classified gesture language into four types: Iconic gesture, Metaphoric gesture, Deictic gesture and Beat gesture. The relevant video annotation data statistics are as follows.
After statistical analysis, it is found that teachers used the most deictic gestures in the classroom, mainly in the following ways: (1) when the teacher pointing to pictures and texts on the PPT; (2) when the teacher asking a certain student to answer question; (3) when the teacher points to a specific object to signal the students to say the corresponding sentence. For example, in video 2, the teacher pointed to a map, pen, and other objects, and asked the students to say the corresponding "ba" sentence after doing the relevant demonstration. Deictic gestures can be used in the classroom to make the teacher's instructions more clear and to attract students' attention. Metaphoric gestures are mainly used for teachers to explain language points and issue instructions for group activities, and to explain more abstract concepts. For example, in video 1, students were asked to read the text in different roles, and then switched roles to read again. At this time, the teacher made a gesture of crossed hands to imply "switch roles after reading." At the end of the group exercise, the teacher reminded the students verbally and made a pause gesture to make the instruction more visual. In Video 2, the teacher in order to guide the students to say the target phrase "Put your wallet on ......", the teacher held the wallet in his right hand, and the index finger of the left hand pointed to the head, implying "think about it". Iconic gestures are mainly used when teachers explain vocabulary and explain a specific action, which can play a certain role in demonstrating, reducing the difficulty of students' understanding and facilitating their comprehension and memory. For example, in video 2, the teacher patted the blackboard with his palm when expressing the meaning of the verb "paste" in "paste a photo", and used the corresponding gesture to simulate the action of making a phone call when explaining the text.
Beat gestures are mainly used when teachers lead students to read sentences, and by waving rhythmic gestures help guide students to follow the teacher's rhythm and give them some emotional support.
To sum up, gesture language is a symbolic resource for multimodal classroom teaching, and it is a teaching language for teachers to convey teaching information. The reasonable use of gesture language can make the message conveyed by teachers clearer and more explicit. In addition, teachers should take into account the differences in different cultures when using gesture language, so as not to cause misunderstanding to students.

Eye contact
In classroom teaching, if teachers can make good use of eye contact with students, they can not only convey certain information to students, but also observe students' reception of teaching information through their eyes, so as to make corresponding adjustments to the teaching progress. Eye contact between teachers and students can be roughly divided into three types: Gaze -Students (Teachers look at students), Gaze-PPT (Teachers look at PPT text or pictures), Gaze -Environment (Teachers look at classrooms, props, etc.). .15% It can be clearly seen from the data that teachers use the "gaze -students" eye contact to account for the largest proportion, almost all around 70%. Since there was no student practice in Video 3, teachers mainly relied on PPT to explain and practice, so teachers spent more time on "gaze -PPT" than "gaze -students". Through comparative analysis, it is found that teachers' eye resource allocation in classroom teaching is quite reasonable. Apart from viewing courseware and other things needing individual attention, teachers mostly focus their eyes on students, this will help teachers to have a good grasp of the overall situation in the classroom and also help to increase the interaction between teachers and students.
To further analyze the interaction effects of eye contact between teachers and students, we subdivided "gaze -students" into "gaze -whole class" and "gaze -individual" in Video 1 and Video 2.". Through the analysis of the annotation, it is found that the proportion of "gaze -whole class" is 51.39%, "gaze -individual" is 22.38% in video 1, and the proportion of "gaze -whole class" is 60.45%, the "gaze -individual" proportion is 9.03 % in video 2. The comparison revealed that teachers mainly directed their gaze to the whole class throughout the classroom, while "gazeindividual" eye contact occurred mainly when teachers asked questions to individual students.
The eye contact is mutual, and teachers should actively observe students' eyes while letting them read their own eyes, so that they can understand students' psychological and emotional changes through their eyes in time. Teachers should also learn to use encouraging eyes to give students timely positive feedback. For example, when students cannot answer a question, teachers should give them an encouraging and trusting look and guide them to say the correct answer, which on the one hand can help the students to relieve the tension, on the other hand, students can continue to keep the interest and confidence in learning Chinese.

Coordination of modalities in different stages
Through annotation and analysis of the corpus, we analyzed the use of modalities at different teaching stages and the effect of different modal combinations, and suggested the combination of modalities at different teaching stages according to the "effective principle" and "economic principle".
The "effective principle" means that any modality should be chosen in order to achieve the best teaching effect and avoid using a modality ineffectively or with a negative effect equal to or greater than the positive effect. [8] For example, using multiple modalities to present the same meaning can strengthen memory, but it can also distract students' attention, and if this negative effect is too large, the effect may be negative or zero on the whole, so there is no need to use such a combination of modalities.
The "economy principle" is to achieve the goal at the minimum cost, that is to say, under the premise of a certain effect, the fewer and simpler modes to choose the better. [8] For example, announcing the beginning of the class is a task that can be accomplished by spoken words, so there is no need to use texts and images.

The beginning of the class ceremony
The main modality at this stage is the auditory modality, mainly teacher speech. At this stage of teaching, it is suggested that teachers use only the auditor modality, or they can add some gestures such as a wave or a slight bow to enhance the intimacy with the students. According to the "economic principle", there is no need to use other modalities.

The introduction of teaching content
This stage is a combination of auditory, visual, and posture modalities, that is, the teacher's words, the use of PowerPoint and pictures, and when necessary, the teacher will use body gestures to assist students' understanding. For example , in the language point teaching of "Ba" sentence in Video 2, the teacher used physical objects to do actions, and led out the "Ba" sentence by means of guiding questions and interrogative questions: "The teacher put the passport on the table", and so on. This process is complemented by auditory, visual and gestural modalities, making it easier for students to understand. In Video 4, the teacher showed pictures of different kinds of tea on the PPT and asked students "Do you like black tea or green tea?" This led to the language point "or". The analysis shows that the teacher's main modality at this stage is the auditory modality, while the visual and gestural modalities play a reinforcing role.

Language point explanation
This stage is the main part of the course, mainly explained by the teacher, involving auditory modality, visual modality and posture modality. For example, in Video 1, the teacher used PPT pictures to show different styles of dresses and different types of clothes, and then used the question-and-answer method to elicit language points "...how about" and "...beautiful" and "...not beautiful". In Video 2, the teacher showed the example sentences and wrote underneath them the format of the "Ba" sentence: S + Ba + O + V in/on + place, and explained the semantic meaning of the sentence. In order to highlight the key points, the teacher used blue font for "Ba", green font for verbs, and dashed lines for subjects and objects, which enriched the visual modality to a certain extent. The teacher's verbal explanation, together with the text on the PPT and the teacher's appropriate body gestures, enabled the students to further understand the expressions of the "Ba" sentence.
In general, in this teaching stage, teachers mainly explain knowledge points through words, and consciously change fonts and font colors to highlight key points, flexibly use physical objects to attract students' attention, and cooperate with body gestures to express information when necessary. At this stage, teachers should pay attention to the "effective principle" of modality selection to avoid distracting students' attention by using too many modalities.

Language point practice
This stage further strengthens and consolidates the learned language points, which is a key link in the classroom. Auditory modalities, visual modalities, and posture modalities are all involved. For example, in Video 1, the teacher used PPT drawings, cards, and objects (coats and cups) to simulate real situations, described the specific requirements in words, asked students to practice the activities in small groups, and invited them to the podium to demonstrate, which will be the effective combination of the various modes, so as to fully mobilize the various senses of students. In Video 3, the teacher designed different types of practice questions to help students practice language points more effectively, such as describing sentences through PPT animations and filling in the blanks according to the table as instructed by the teacher, which fully mobilized students' multiple senses and made them understand the points more deeply. In this link, teachers should pay special attention to handing over the initiative of the classroom to the students, so that students more output expression, follow the principle of "intensive lectures and more exercises", and give students more opportunities to practice on the basis of good input.

Assigning homework
This stage is dominated by auditory modality and visual modality, supplemented by posture modality. All four teachers typed the assignment requirements on the PPT, and dictated the assignment requirements while pointing to the text on the PPT. In Video 1, the teacher marked the keywords "writing", "listening", "preview" and "asking" in red fonts in the homework requirements, which played a prominent role. When necessary, teachers can also use pictures or diagrams to help students understand and remember.

The closing ceremony
It is a combination of auditory and body posture modalities, in which the teacher verbally announces the end of the class, accompanied by a bow or a wave of the hand. According to the "economic principle", it is not necessary to use PowerPoint, text, pictures at this stage.

Conclusion
After quantitative and qualitative research on the classroom discourse presented in the Chinese teachers' teaching videos, we suggest the following modality selection and coordination for different teaching stages according to the "economic principle" and "effective principle". In the beginning and ending ceremony of the classroom, according to the "economic principle", it is recommended that teachers mainly use auditory modality, supplemented by certain body posture to reflect interpersonal meaning. In the introduction stage, auditory modality and visual modality are mainly used, and intuitive means (objects, Pictures, etc.) are introduced, which makes it easier to stimulate students' vision and attract students' attention. At the same time, it can also make teachers' language more visual and intuitive, making the classroom more lively and interesting. In the stage of knowledge point explanation, the main modality is auditory modality, the visual modality and posture modality are supplementary. At this stage, teachers should follow the principle of "intensive lectures and more practice", and at the same time, teachers should pay attention to the rational use of "gaze -whole class" and "gaze -individual" eye contact. In the language point practice stage, auditory modality, visual modality, and posture modality should be involved as much as possible. On the one hand, it can increase the diversity of classroom forms, and on the other hand, it can fully mobilize students' multiple senses to participate in the classroom, so as to get a better practice effect. In the assignment stage, the auditory modality is the main modality and the posture modality is supplementary. The teacher dictates the homework requirements, and uses words and pictures as the auxiliary to help students understand and remember. Of course, the choice of modality is related to the teacher's teaching ideology, teaching purpose and teaching style, and there are some differences in the choice of modality due to different lesson types and teaching objectives. In addition, the quantity of the corpus used in this study is limited, and a larger multimodal corpus should be built to deepen the study in the future.