Enhancing a Korean Part-of-Speech Tagger Based on a Maximum Entropy Model

• 발행기관: 한국자료분석학회
• 저자명: 나동열(Dong-Yul Ra) 조민희(Min Hee Cho) 김윤식(Yunsik Kim)
• 간행물 정보: 『Journal of The Korean Data Analysis Society (JKDAS)』Vol.9 No.4, 1623~1638쪽, 전체 16쪽
• 주제분류: 자연과학 > 통계학
• 파일형태: PDF
• 발행일자: 2007.08.30

영문 초록

The most core task for Korean text processing is to recognize the lexical morphemes in sentences and to determine their part of speeches. This task is called the part of speech tagging. We present several effective schemes to enhance Korean part-of-speech tagging systems that are based on a Maximum Entropy model. We employ two levels of tags, the inner and outer tags. A probability of a morpheme sequence is computed to augment the probability of the Maximum Entropy model. Special feature functions are employed to exploit co-occurrence of multiple lexical items, which seems effective for lexical ambiguity resolution. Experimental results demonstrate usefulness of these schemes.

목차

1. Introduction

2. Background

3. Exploiting Two Level Tags

4. Probability for a Morpheme Sequence

5. Using Co-occurrence of Lexical Items

6. Experimental Results and Discussion

7. Related Works

8. Conclusion

References

키워드

Korean text processing part of speech tagging maximum entropy model morphemes