Second Language Acquisition of Neural Language Models

Authors

Miyu Oba, Nara Institute of Science and Technology
Tatsuki Kuribayashi, Mohamed bin Zayed University of Artificial IntelligenceFollow
Hiroki Ouchi, Nara Institute of Science and Technology
Taro Watanabe, Nara Institute of Science and Technology

Document Type

Conference Proceeding

Publication Title

Proceedings of the Annual Meeting of the Association for Computational Linguistics

Abstract

With the success of neural language models (LMs), their language acquisition has gained much attention. This work sheds light on the second language (L2) acquisition of LMs, while previous work has typically explored their first language (L1) acquisition. Specifically, we trained bilingual LMs with a scenario similar to human L2 acquisition and analyzed their cross-lingual transfer from linguistic perspectives. Our exploratory experiments demonstrated that the L1 pretraining accelerated their linguistic generalization in L2, and language transfer configurations (e.g., the L1 choice, and presence of parallel texts) substantially affected their generalizations. These clarify their (non-)human-like L2 acquisition in particular aspects.

First Page

13557

Last Page

13572

Publication Date

1-1-2023

Recommended Citation

M. Oba et al., "Second Language Acquisition of Neural Language Models," Proceedings of the Annual Meeting of the Association for Computational Linguistics, pp. 13557 - 13572, Jan 2023.