ASSESSING L2 CHINESE LISTENING USING AUTHENTICATED SPOKEN TEXTS

Abstract

This dissertation study explored the feasibility of using authenticated spoken texts to test L2 Chinese listening comprehension. The spoken texts used in the study were created using an “authenticating” technique, in which scripted spoken Chinese texts were infused with characteristics of real-world, unscripted spoken Chinese. In the first part of the study, 101 “naïve” L1 Chinese speakers reported their attitudes toward various characteristics of the scripted spoken texts and the authenticated spoken texts in a survey questionnaire. Comparisons of the L1 Chinese participants’ attitudes showed that they believed that the authenticated spoken texts were more authentic-sounding than the scripted texts in every measure. This indicates that the authenticity of scripted spoken texts normally used in L2 listening tests can be enhanced by using the text authentication technique. In the second part of the study, L2 listeners’ test performance and listening and test-taking processes during the post-test were examined. For the quantitative data, 184 intermediate and advanced adult L2 Chinese learners completed two listening tests (one pre-test assessing their listening proficiency and one post-test) and questionnaires tapping into their use of listening strategies and test-taking strategies. For the qualitative data, stimulated verbal recalls and verbal reports were employed to elicit another 16 L2 listeners’ verbalizations of their listening and test-taking processes. The comparison of the test scores showed that the listening test using authenticated spoken text as listening input was significantly more difficult than the same test using scripted spoken texts. The comparisons of the questionnaire results showed that the test-takers did not differ in their use of listening strategies and test-taking strategies. Aligned with the quantitative results, the verbal recalls showed that the two groups, despite hearing spoken texts with different degrees of scriptedness, did not adjust their use of listening strategies and test-taking strategies. However, the authenticated spoken texts elicited a greater reliance on lower-level processes compared to the scripted spoken texts. The last part of the study involving the same 16 L2 qualitative participants investigated text-related sources of bottom-up listening difficulties of the two experimental groups through a series of diagnostic listening procedures using L2 oral and written (typing) repetition tasks. It was found that, first, compared to the scripted group, the authenticated group had more phonological decoding difficulties due to connected speech; second, even though the authenticated group had more word segmentation difficulties, this type of listening difficulty seemed to be less severe for both groups compared to the difficulty of phonological decoding; and third, other features commonly found in unscripted spoken Chinese such as filled pauses and false starts did not seem to greatly impede participants’ bottom-up listening processes. The results from the three parts of the study and my experience creating authenticated spoken texts suggest that L2 Chinese test developers should include at least some authenticated spoken texts in their listening tests, because (a) they are more “authentic-sounding”; (b) high-proficiency L2 listeners with extensive Chinese second language learning experiences find them harder to comprehend than scripted texts; and (c) they elicit cognitive processes that better represent the processes in real-world listening. Additional practical and theoretical implications pertinent to test development and L2 Chinese listening instruction are also provided.