Language Assessment Literacy and the Influential Factors:Evidence from a Survey of Middle School English Teachers in Chongqing

Southwest University,Chongqing,China Email:iammaggie@126.com

[Abstract]The study investigates the level of language assessment literacy of middle school English teachers in Chongqing and relationship between the influential factors based on a structural equation model.Results shows that teachers’language assessment literacy is underdeveloped and the influential factors interact mutually.It concludes with suggestions to improve language assessment literacy from the government,universities and teachers.

[Keywords]language assessment literacy;influential factors;English teachers;survey

Introduction

Assessing the performance of students is one of the most imperative content of teachers’job and assessment literacy plays an essential role in the process of teachers’professional development and students’learning.In 1991 Stiggin initiated the conception“assessment literacy”.In 2001,Brindley had a research on assessment literacy from the aspect of language testing.Later,Inbar-Lourie(2008)came up with the conception of language assessment literacy(henceforth,LAL).However,it was not until 2009 that LAL was discussed directly by Taylor,which officially aroused scholars’attention to LAL.The past 10 years have seen increasing interest in preparing English teachers to have language assessment literacy.

This study investigates the LAL level of middle school English teachers in Chongqing and its relations with teachers’background information(education degree,teaching experience,English proficiency,school type,school location and class size),as well as the relations among LAL,training experience and attitude based on a structural equation model.

Literature Review

There is no agreed-on comprehension of the definition of LAL,so this paper follows the definition of Chinese scholar Lin because it is conducted in Chinese educational setting,which refers to the teachers“have a more comprehensive and profound understanding of the language and language learning; have basic knowledge,skills and ability of designing,developing or evaluating language classroom assessment,be familiar with the process of language assessment,be aware of the principles and concepts behind the practice of language assessment,can enable learners to actively participate in the process of language assessment,can use appropriate feedback to effectively enable learners to set and achieve learning goals,and have a good understanding of the role and function of language assessment in a specific teaching environment”(2019,p.13)

Most previous studies primarily focus on how the conception LAL could be concisely defined(Inbar-Lourie,2008,2013; Fulcher,2012; Malone,2013; Taylor,2013; Lin,2016),what the LAL level of teachers is(Plake et al.,1993;Mertler,2003;Mertler&Campbell,2005;Zhen,2009;Zhao,2014;He,2015;Lin,2016;Xu et al.,2016),what knowledge should be involved in training(Hasselgreen et al.,2004;Guerin,2011;Vogt&Tsagari,2014;Shao,2015),how teachers’LAL could be professionally developed(Popham,2009; Malone,2013; Scarino,2013; Lam,2015),how the textbooks and courses could be designed(Brown & Bailey,2008; Davies,2008; Jin,2010; Jeong,2013)and the extent to which various stakeholders need differential LAL in a broader sociocultural setting(O’Loughlin,2011,2013;Pill&Harding,2013;Taylor,2013).

The previous studies have made great contribution to LAL research development,however,there have been some deficiencies of them and some efforts can be made to enrich and perfect LAL research.First,the subjects of research are not diverse.Main subjects of previous studies are college teachers and there are few studies on English teachers in middle schools in China.Second,some contents of LAL questionnaires abroad are not propitious to Chinese assessment setting.Third,the research of LAL diverges from LAL training experience and LAL attitude.Therefore,a mixed-method is adopted in the present study where the level was measured by questionnaires and the data was analyzed by SPSS and AMOS were adopted for quantitative data analysis and a follow-up interviews have been scarcely done to explore the underlying reasons behind teachers’LAL conditions.

Methodology

Two questions in the study are addressed:What is the level of LAL? What are the relations among LAL,LAL training experience and LAL attitude?

Two hundred and twenty-seven teachers with different education degree in different schools in Chongqing were chosen as research subjects.Table 1 shows the detailed information.

Table 1:Basic information of the subjects

A questionnaire and an interview protocol were used as research instruments.The questionnaire in this study is a revised Chinese version of Classroom Assessment Literacy Inventory(CALI)(Mertler,2003),which is based on seven standards:teachers should be skilled in(1)choosing assessment methods;(2)developing assessment methods;(3)administering,scoring and interpreting the results assessment；(4)using assessment results when making decisions;(5)developing valid grading procedures;(6)communicating assessment results;(7)recognizing unethical,illegal practices.The questionnaire consists of two parts.The first part contains 35 items(5 items per standard).The second part contains personal background information,LAL training experience and LAL attitude,which holds a total of 13 items.After a pilot study,the revised questionnaire shows high reliability with α= 0.92.Therefore,this questionnaire can truly and effectively evaluate the general condition of middle school English teachers’LAL.Ten teachers with different background information were selected as respondents at the follow-up semi-structured interview,aiming at eliciting respondents’opinions of choosing and developing assessment methods,administering and interpreting the results,using assessment results for decision-making,communicating assessment results,recognizing unethical assessment,LAL training experience and LAL attitude.

Results and Discussion

The current level of teachers’LAL

The mean of teachers’total score of LALis 17.31,the standard deviation is 3.085,the maximum is 26,and the minimum is 11.It is obvious that the average score is low,and teachers answer nearly 17 out of 35 items correctly,which is less than half of the total items.The difference between maximum possible score and minimum possible score is 15.

Table 2:Descriptive statistics of each standard and total score

Out of the 7 standards,the highest score for teachers was found on Standard 7-recognizing unethical,illegal,and otherwise inappropriate assessment methods(M=3.72; Max=5),while the lowest score for teachers was found on Standard 6-communicating assessment results(M=1.78).

Out of the 35 items,only Item 1(Standard 1),Item 33(Standard 7),34(the highest,94.3%,Standard 7)were answered correctly by 90% or more of the teachers.Item 7(the lowest,10.6%,Standard 2),Item 18(Standard 4),Item 27 and 28(Standard 6)were answered correctly by 25%or fewer of the teachers.

From the above analysis,it can be concluded that the general condition of teachers’LAL is underdeveloped,which parallels the results of earlier studies(Plake,1993;Mertler,2005;He,2015)but scores lower than them(in the study of Plake,M=23; Metler,M=22; He,M=18).In the follow-up interview,when inquired about their conceptual comprehension of LAL,most respondent teachers stated that they were not acquainted with this term and some even indicated they had not heard of this term before.Even a teacher identified“language assessment”with“ examination”.

Teachers performed best on Standard 7- recognizing unethical,illegal,and otherwise inappropriate assessment methods.Most teachers indicated that it was a basic principle to be ethical as a teacher and treated every student equally,which resulted in teachers’fairness and ethics in assessing students.Additionally,a respondent said that compared with other items which need professional knowledge to finish,the ethical items can be accomplished in line accordance with their morality.

The lowest score for teachers was found on Standard 6- communicating assessment results,which paralleled Plake’s study(1999).The reason for the poor performance is that the examination-oriented environment does not equip teachers with adequate opportunities to communicate assessment results to students,parents and schools that only focus attention on scores and rankings in a test-driven culture.A teacher expressed that the prominence of scores makes her doubt whether there is a need to communicate assessment results to them in view of the absence of attention to comprehensive assessment results.As a matter of fact,teachers are spilt into two parts in the“incompatible cultures”,with one requiring their obedience to the regulations established by the related societies(testing culture),while the other highlighting the application of alternative assessments to foster students’learning(Inbar-Lourie,2008).Consequently,when communicating assessment results to related parties,teachers are inactive to consider students’social,economic,cultural and other background factors,introduce methods to evaluate students’progress,and interpret assessment results of grade,region and nation.

The Relationship of LAL,LAL training experience and LAL attitude Based on Structural Equation Model

BecauseLAL training experience and LAL attitude are latent variables,and LAL is observed variable,a structural equation model is constructed to explore the relations among LAL,LAL training experience,and LAL attitude.In the study,teachers’LAL training experience is composed of five aspects:school training,working training,project researching,self-reading and examination designing,and LAL attitude consists of two aspects:attitude to LAL and self-evaluation of LAL.Based on the data from the questionnaire and the hypothesis(training experience affects LAL,attitude affects LAL,training experience and attitude affects each other),the structural relationships among variables are drawn by AMOS Graphics after model specification,model estimation,model testing,model modification.The model illuminates the extent to which LAL training experience and LAL attitude can have a sensible role to play in shaping teachers’LAL,as well as their mutual relationships.According to the effect values in the model,it can be concluded that LAL training experience has a high effect on LAL(0.80),LAL attitude has a negative effect on LAL(-0.96),LAL training experience and LAL attitude have a mutual effect(0.37),LAL attitude can affect LAL positively through LAL training experience.

Figure:Structural Equation Model of The Relationship of LAL,Training Experience and Attitude

Implications and Conclusion

The research results show that the status quo of teachers’LAL in Chongqing is underdeveloped.Five underlying reasons for the underdeveloped performance on LAL from the follow-up interview include:inferior course to develop LAL,insufficient training in work,an examination-oriented culture,heavy workload and language proficiency problem.Additionally,LAL training experience has a positive effect on LAL while LAL attitude affects LAL negatively.LAL training experience and LAL attitude have a mutual effect,and LAL attitude can affect LAL positively via LAL training experience.

The promotion of teachers’LAL is not only the matter of teachers themselves,but also the matter of government and the matter of universities or institutes.First,government need to initiate to administer LAL as part of teachers’qualification condition and launches compulsory LAL benchmarks for pre-service teachers as one of requirements for teacher qualification certificate.A more normative and scientific teacher qualification certificate system needs to be established.Secondly,in universities or institutes professional courses of LAL should be set and some improvements for the course need to be made for the development of teachers’LAL.Based upon the survey findings,LAL training,particularly in the form of university-based course,plays a significant role in shaping teachers’LAL.However,training experience has not prepared teachers sufficiently for practical teaching.The availability of course time should be guaranteed and LAL courses should be set as a compulsory course rather than electives.Second,the theory-practice gap should be bridged by merging assessment knowledge and skills into pedagogical content knowledge and teaching practice(Inbar-Lourie,2013).The teaching content needs to be more contextualized and the course can be delivered in a more learner-centered way.Practical content should be prioritized in language assessment training.Apart from professional courses,work training should also be offered by schools as many teachers are learning on-the-job with the combination of LAL knowledge,classroom practice and teaching experience.Seminars,conferences,text-based materials,and workshops can be provided to teachers for their access to up-to-date assessment theories and principles under a sufficient schedule for teachers’training.Thirdly,teachers are encouraged to cultivate their LAL by self-study.As a good addition for school and work training,more self-reading and research project can be helpful to foster teachers’LAL.Teachers should integrate theory with practice by applying what they have learned to their teaching practice.According to Popham(2011),it is essential that teachers implement LAL strategies under the circumstance of classroom teaching and large-scale public examinations for the reason that teachers and students are inclined to attach great importance to these high-stakes assessments in the real teaching setting.A learning community is recommended to be constructed where a group of teachers are teamed up to try innovative ideas and meet regularly to share specific professional development experience on specific goals(Arter,2001).In this learning community,teachers share not only what they have learned from books,but also what they have acquired from class.Successful experience and strategies can be shared for promoting teachers’learning,and difficulties and reasons behind them can be discussed for a sound solution(Arter&Busick,2001).