Korean Text to Gloss: Self-Supervised Learning approach

• 영문명: Korean Text to Gloss: Self-Supervised Learning approach
• 발행기관: 한국스마트미디어학회
• 저자명: Thanh-Vu Dang 김진영 유광현 김지용 박영환 이칠우
• 간행물 정보: 『스마트미디어저널』Vol12, No.1, 32~46쪽, 전체 15쪽
• 주제분류: 공학 > 컴퓨터학
• 파일형태: PDF
• 발행일자: 2023.02.28

영문 초록

Natural Language Processing (NLP) has grown tremendously in recent years. Typically, bilingual, and multilingual translation models have been deployed widely in machine translation and gained vast attention from the research community. On the contrary, few studies have focused on translating between spoken and sign languages, especially non-English languages. Prior works on Sign Language Translation (SLT) have shown that a mid-level sign gloss representation enhances translation performance. Therefore, this study presents a new large-scale Korean sign language dataset, the Museum-Commentary Korean Sign Gloss (MCKSG) dataset, including 3828 pairs of Korean sentences and their corresponding sign glosses used in Museum-Commentary contexts. In addition, we propose a translation framework based on self-supervised learning, where the pretext task is a text-to-text from a Korean sentence to its back-translation versions, then the pre-trained network will be fine-tuned on the MCKSG dataset. Using self-supervised learning help to overcome the drawback of a shortage of sign language data. Through experimental results, our proposed model outperforms a baseline BERT model by 6.22%.

목차

I. INTRODUCTION

II. RELATED WORK

III. METHOD

IV. Experimental results

V. CONCLUSIONS

REFERENCES

키워드

Sign Language Production Neural Machine Translation Korean Corpus