【作者】 沈蕾；

【导师】 王海啸；

【作者基本信息】 南京大学 ， 英语语言文学， 2010， 博士

【摘要】 大学英语四、六级考试(CET)从1987年丌始实施,至今已成为国内规模最大、影响范围最广的外语考试之一。测试界认为,越是高风险的考试,对考试的效度验证要求越高。因此,运用合适的效度理念和分析技术对四、六级考试进行研究,具有理论价值和现实意义。复合式听写是自1997年丌始CET听力部分增加的新题型,目前对它的研究大多集中在教学后效和评分信度等方面,相关的效度研究数量较少,研究方法也趋于单一。测试构念的建立、实验研究、有关测试中相关因素对测试难度和区分度的研究以及考生答题过程中的心理过程的分析都是效度研究的重要方法,但是目前的复合式听写研究很少涉及。现有的效度研究主要局限于相关系数分析。相关分析是效度研究的主要方法之一,但是完全根据简单的相关系数难以得出令人信服的结论。研究中关于这一听力题型是否有效测试了考生的听力,以及测试了哪些听力技能或者其他语言技能和策略技能,目前还没有定论。因此本文的目的是运用多种研究方法,从多个维度检测复合式听写的效度。本文首先通过相关的文献综述和复合式听写的内容分析提出了假设性的复合式听写能力构念,该构念包括语言能力和策略能力两个方面,然后从多个层面收集证据来验证这一构念的合理程度,最后根据发现成果对这一假设性的听写能力构念进行了修改和完善。本文对国内某重点大学166名非英语专业新生进行了一系列的测试,并且对这些测试分数用SPSS分析软件进行了多角度的定量和定性统计分析。定量分析包括对考试成绩的相关性数据分析、因子分析、回归分析和差异显著性检验。这些分析探索了不同的测试方法(包括不同的输入方式和不同的答题方式)对考生答题效果的影响；复合式听写题项和大学英语四、六级考试中其他题项(包括写作、阅读、听力、完形、翻译)的相关关系；听写填空中(填空单词和填空句子)不同的语法特征(词汇、语音和句法)以及考生的听力水平这两个双重因素对考生的听写成绩所产生的影响。除了上述定量分析,本文还采用“内省法”作为研究手段,对9位考生的“即刻回顾”口述报告进行了定性分析,分析了考生答题的心理过程,以揭示考生在听写过程中是通过什么途径、以什么方式以及经过了哪些步骤最终获取答案的。分析结果从另一个侧面对CET复合式听写的效度进行了检验。经过定量和定性分析,具体发现如下：1.复合式听写只在一定程度上考核了考生理解篇章基本内容的能力。听写的第二部分(即填写句子部分)的成绩依赖于对篇章基本内容的理解,但是第一部分(即填写单词部分)的成绩和考生理解篇章基本内容的能力没有直接关联。第一部分的成绩和考生理解单词的能力有关。2.复合式听写考核了考生总体听力能力。听力理解和复合式听写的两个部分属于共同的因子,听力理解的成绩也是对复合式听写成绩具有预测力的唯一因素。复合式听写的构念中不应该包括阅读能力或写作能力。3.听写段落中所填写的单词或句子的大部分语法特征对测试难度和区分度产生了影响。这些特征包括：词汇(词频、词形变化和词块)、语音(发音变化如连读、吞音等)以及句法(句子长度和句法复杂程度。)4.考生在复合式听写过程中确实遇到了一系列问题,并且使用了策略来处理所遇到的问题：(1)考生遇到了语言上的问题,包括对基本内容的理解、重要细节的理解和语法特征的理解和运用。他们同时也遇到了情感上的问题。中低水平考生在复合式听写中容易产生焦虑,影响到他们听力能力的正常发挥。(2)成绩高的考生在解决语言问题时能更多使用有效的听力策略(包括认知和元认知策略。)考生也都尝试了与听力理解无关的应试技巧,但是这些技巧的运用整体而言并没有有效地提高他们的复合式听写成绩。上述研究发现表明：复合式听写这一听力题型是有效的听力测试,整体而言考核了所预期的听力技能和策略技能。复合式听写中考生不能以应试技巧代替以听力理解为目的的听力技能和听力策略,即他们之间没有互补关系。当然这一听力测试题型并非完美无缺,主要缺陷在于其第一部分(即填写单词部分)所选择的填空单词中词汇类型过于单一,不能有效地影响测试的难易度,也缺乏区分度。本研究的发现对复合式听写测试的设计提供了参考：选择填空单词时,需要考虑到不同单词的语法特征：词长方面,应避免选择短词,特别是发音和拼写高度吻合的短词;词频和词形变化对低水平考生的影响要大于对高水平考生的影响；语音变化和词块都具有很强的区分力,但是语音变化会增加听写难度,而词块会降低听写难度；建议选词时增加词汇类型。在选择填空句子时,需要考虑到句子的长度、句子结构的复杂程度以及和上下文的意义和结构上的关联,因为这些特征都会影响到听力测试的难度。同时,本论文的研究成果在测试理论、研究方法和教学层面上都有一定的启示意义,为听写能力构念研究和其他测试效度研究提供了参考,对外语听力教学提供了启示。更多还原

【Abstract】 This dissertation reports a validation study of the compound dictation, a new item of the listening comprehension part of the College English Test (CET), a high-stakes test administered on a nationwide scale by the National College English Testing Committee.Research to date has hardly answered the question about what the compound dictation really measures. Most previous validation studies of the compound dictation were confined to the evidence of correlation coefficients. Little effort was directed either to the establishment of the test construct, or to the empirical item investigation or experimental research. While investigations into the test-taking processes contribute to a comprehensive understanding of test validity, relevant research was very limited and not systematic enough.The chief task of the present study, therefore, was to look for further evidence concerning the validity of the compound dictation. It was believed that through various types of data collected from a wide range of perspectives, the study could provide a solid basis for the validity of this test component.The dissertation proposed a hypothetical construct of the CET compound dictation by reviewing the current literature concerning the nature of listening and by establishing a descriptive framework of the test. This hypothetical construct was then tested in a study with166first-year non-English-major participants from a key university in China.A number of validity evidence was collected and a series of test score data and test performance data were analyzed with both quantitative and qualitative approaches. Quantitative approaches include T-tests, factor analysis, regression technique and repeated measures ANOVA. Retrospective verbal Protocol analysis was adopted as a major qualitative approach. These statistics procedures were applied to the analysis of the student performance data so as to measure whether the CET compound dictation measures the comprehension of dictation passages, whether it measures the overall ability of listening comprehension, how different grammatical factors affect the degree of difficulty and discrimination of the CET compound dictation, and what problems test-takers encounter and what strategies they use to cope with these problems.The study yielded the following findings:1. The CET compound dictation measures learners’comprehension of the basic content of the dictation passage only to a certain extent. The scores of the revised dicto-comp part, the second part of the CET dictation, do depend on the comprehension of the basic content. However, the scores of the spot dictation part, the first part of the dictation, do not depend on basic content comprehension; they have to do with learners’ comprehension of individual words.2. The CET compound dictation does measure learners’overall ability of listening comprehension. CET listening comprehension scores and scores of the two parts of the compound dictation belong to the same underlying factor, and among scores of various components of CET, only the listening comprehension scores are a significant predictor of the compound dictation scores. Neither reading nor writing abilities should be considered aspects of the construct of the CET compound dictation.3. A number of grammatical features of the missing portions (words, clauses or sentences) in the CET compound dictation may affect the degree of difficulty and discrimination of the test. They are certain features of vocabulary, phonology and syntax. For instance, frequency, sound variations, lexical bundles and inflectional features of the words affect the performance and serve to differentiate the students; length and syntactic complexity of the clauses or sentences also affect the difficulty level of the test.Type of words does not affect scores of the test in general or in different proficiency groups. Length of words does not discriminate the high-proficiency-level test-takers.4. When learners are completing the CET compound dictation, they do encounter many problems and employ many strategies in an attempt to cope with these problems.(1) Learners encounter problems of understanding basic meaning and specific ideas and understanding formal features (spelling, morphology, phonology, and syntax), and they also encounter affective problems. Anxiety especially among average and low proficiency students was found to be serious and affect their performance.(2) Successful listeners are much more able to handle the linguistic problems by utilizing learner strategies and test-management strategies (i.e., strategies for responding meaningfully to the test items and tasks), both matacognitive and cognitive, than their less successful peers in the compound dictation task. Learners also use test-wiseness strategies (i.e., strategies for using knowledge of test formats and other peripheral information to answer test items without going through the expected linguistic and cognitive processes), but the use of these strategies does not contribute to the successful completion of the test.These findings have important implications for the CET compound dictation. The compound dictation, as an item of the listening comprehension part of the CET, is in general a valid listening test, in the sense that it measures learners’overall ability of listening comprehension, and that test-wiseness strategies cannot be used in place of the authentic listening skills and contributory cognitive and matacognitive strategies defined by the construct.The findings are informative to the designers of the compound dictation in reference to the selection of the missing portions of the dictation passage. For the first part (individual words), different grammatical features should be taken into account and treated in different ways. For instance, short words, especially those with simple and stable relationships between sound and spelling should be avoided. Caution should be observed when sound variations and lexical bundles are considered, as the former make the listening more difficult, and the latter, easier. A selection of wider range of words including both concrete and abstract words is advised. For the second part (clauses or sentences), the length, the syntactic complexity and the close relationship with the adjacent context should be taken into account.The study also has theoretical, methodological and pedagogical implications. Theoretically, it proposed a construct of the CET compound dictation in an attempt to contribute to a better understanding of its test domain. Methodologically, it collected a combination of validity evidence from a variety of perspectives with both quantitative and qualitative methods. Pedagogically, teachers are well informed of the particular difficulties that are likely to be encountered by the learners during listening and can accordingly better facilitate them coping with their listening problems.更多还原