Score Reliability of a Test Composed of Passage-Based Testlets - A Generalizability Theory Perspective

• 영문명: Score Reliability of a Test Composed of Passage-Based Testlets
• 발행기관: 한국영어교과교육학회
• 저자명: Lee Yong-Won
• 간행물 정보: 『영어교과교육』영어교과교육 제2권 제2호, 1~29쪽, 전체 29쪽
• 주제분류: 어문학 > 영어와문학
• 파일형태: PDF
• 발행일자: 2003.09.01

영문 초록

  The purpose of a study reported in this paper is to investigate the impact of local item dependence (LID) in passage-based testlets on the score reliability of an EFL (English as a Foreign Language) reading comprehension test from the perspective of generalizability (G) theory. Definitions and causes of LID in passage-based testlets are reviewed within the frameworks of classical test theory (CTT) and item response theory (IRT) along with 3 reliability estimation procedures for testlet-based tests. Scored item responses of 1,072 Korean high school students to a 40-item, 10-passage EFL reading comprehension test were analyzed with simulated data generated for a no LID situation for a 40-item test. It was found that item alpha (α<SUB>I</SUB>) coefficients (p x I) were significantly higher than both passage alpha (α<SUB>p</SUB>) coefficients (p x T) and generalizability (Eρ²) coefficients [p x (I:H)] in the real data, whereas all of these 3 coefficients were very close for different test lengths in the simulated data. Subsequent analyses of fixed test length scenarios have also revealed that score reliability could be maximized by increasing the number of passages rather than the number of items nested within passages. The findings of the study are also discussed in terms of the impact of LID on score reliability and its implications for test development.

목차

Abstract

Ⅰ. Introduction

Ⅱ. Reliability Estimation for Passage-Based Tests

Ⅲ. Methods

Ⅳ. Results

Ⅴ. Summary and Discussion

Ⅵ. Conclusions and Avenues for Future Research

Author Note

References

키워드

Score Reliability