Using Corpora for Language Testing and Assessment

Abstract：Corpora have been recognized as an important tool in language testing and assessment， and research on the construction of corpora have been growing rapidly for the last twenty years. This paper explores the development and uses of corpora in language testing and assessment. Besides playing an important role in designing and validating tests， corpora have made it possible to rate automatically and detect plagiarism in tests. With the advances of theory and technology， there lies a promising future for the field of corpora-informed language testing and assessment.

Key words：corpora；language testing and assessment；automatic rating

1. Introduction

Language testing and assessment is referred to the measurement of the language mastery and proficiency of individuals in a variety of contexts and for a range of purposes， assessing language knowledge， performance or application （Barker， 2010： 633）. Since the early 1990s the term ‘corpus has been subjected to various types of linguistic analysis （Stubbs， 2004）. Most modern corpora are featured by their computer-readability using specially designed programs， in which way， linguistic data can be identified， sorted and analyzed. In the early 1990s， corpora and corpus linguistic tools played a relatively minor role in pedagogy assessment （Taylor & Barker， 241）. In the language assessment community， Charles Alderson was among the first to signal a potential role for corpora （Alderson， 1996）. While acknowledging the impact corpora began to have in linguistic analysis， he noted that they had yet to find an application in assessing language learning and proficiency.

2. Development and benefits of corpora in language testing and assessment

One of the first direct involvements of corpora in the field of language testing occurred in the 1990s. Pioneering work took place in Belgium. In 1993 in the UK， the Cambridge Learner Corpus （CLC） was set up. A domain-specific corpus that informs language test development is the TOEFL 2000 Spoken and Written Academic Language Corpus was built by Educational Testing Services （ETS） to investigate university level language skills.

In the mid-1990s， Charles Alderson （1996） signaled a potential role for corpora in language assessment including test writing， construction， scoring and score reporting. When it comes to test content， both native speaker and learner corpora play a useful role in creating and validating test materials. Learner corpora， by identifying typical errors， show us what learners of a language can do at certain levels of proficiency. In test design， native corpora， by providing authentic texts or model texts are used by language testers to ensure that all parts of a test are valid and reliable. Corpora can also help establish text readability and assess a texts coherence and cohesion. In relation to test scoring and rating， native speaker corpora can be regarded as a useful reference point against which test-takers responses are compared. Research on automatic rating and scoring began to take off in the 1960s. Since then researchers have been working hard to develop sophisticated systems for the automatic evaluation of speaking and writing. In 1999 ETS introduced Electronic Essay Rater for scoring essays. Evidence has shown that test-retest reliability of E-rater （0.6） is higher than that of manual scoring （0.5） （Attali & Burstein， 2005）.

3. Prospects of corpora in language testing

Though currently corpora have limitations and potential risks concerning language testing and assessment， the field of corpora use in language testing has been growing for years and with the development of technology and corpora building， there lies a promising future ahead for corpora. The spread of computer-based testing worldwide （e.g.， internet-based TOEFL and computer-based IELTS） should permit easier corpus-building in the future. Improvements in the digital recording and storing of soundfiles will make the collection of spoken corpora more straightforward， thus increasing their availability for language testers. Corpora can also assist in detecting cheating and malpractice with the development of software programs， thus playing an important role in international exams. Moreover， field-specific reference corpora are likely to assume greater relevance as language testing becomes increasingly domain-focused， e.g.， for professions such as law， accountancy， aviation， as well as the traditional academic and business fields.

4. Conclusion

Corpora are recognized as important and useful in the application of language testing and assessment. Corpora play a role in helping to decide on the format and content， securing authenticity of the texts and making it possible for automatic rating of writing and speaking. Over the past twenty years， the field of corpora-informed language testing and assessment has been opening up and growing rapidly. To make corpora to the fullest use， more endeavors are to be made to contribute to linguistic research and language testing.

【Reference】

[1] Alderson， J. C. Do Corpora Have a Role in Language Assessment？[A]. in J. A. Thomas and M. H. Short （eds） Using Corpora for Language Research. London： Longman， 1996：248-59.

[2] Attali Y and Burstein J. Automated essay scoring for e-rater v. 2.0[R]. Report Number： RR-04-45， Educating Testing Service， Princeton， NJ， 2005.

[3] Barker F. How can corpora be used in language testing？[A]. in The Routledge Handbook of Corpus Linguistics.London： Routledge， 2010：633-645.

[4] Stubbs， M. Language corpora [A]. in A. Davies and C. Elder （eds.）， The Handbook of Applied Linguistics， Oxford： Blackwell， 2004.

[5] Taylor， L and Barker， F. Using Corpora for Language Assessment[A]. in E. Shohamy and N. H. Hornberger （eds）， Encyclopedia of Language and Education， 2nd Edition， Volume 7. New York： Springer， 2008：241-254.

【作者簡介】

沈忠良（1992—），男，浙江大學碩士研究生，研究方向：英語語言文學。