A FACETS Analysis of Rater Characteristics and Rater Bias in Measuring L2 Writing Performance

• 발행기관: 한국영어어문교육학회
• 저자명: 신유선(Yousun Shin)
• 간행물 정보: 『영어어문교육』영어어문교육 제16권 제1호, 123~142쪽, 전체 20쪽
• 주제분류: 사회과학 > 교육학
• 파일형태: PDF
• 발행일자: 2010.03.30

영문 초록

The present study used multi-faceted Rasch measurement to explore the characteristics and bias patterns of non-native raters when they scored L2 writing tasks. Three raters scored 254 writing tasks written by Korean university students on two topics adapted from the TOEFL Test of Written English (TWE). The written products were assessed using a five-category rating scale (Content, Organization, Language in Use, Grammar, and Mechanics). The raters only showed a difference in severity with regard to rating categories but not in task types. Overall, the raters scored Grammar most harshly and Organization most leniently. The results also indicated several bias patterns of ratings with regard to the rating categories and task types. In rater-task bias interactions, each rater showed recurring bias patterns in their rating between two writing tasks. Analysis of rater-category bias interaction showed that the three raters revealed biased patterns across all the rating categories though they were relatively consistent in their rating. The study has implications for the importance of rater training and task selection in L2 writing assessment.

목차

Abstract

I. INTRODUCTION

II. LITERATURE REVIEW

III. METHODOLOGY

IV. RESULTS AND DISCUSSION

V. CONCLUSION

REFERENCES

키워드

bias analysis L2 writing tasks task types rater variability rater training effects