REFeREE: A REference-FREE Model-Based Metric for Text Simplification
Document Type
Conference Proceeding
Publication Title
2024 Joint International Conference on Computational Linguistics, Language Resources and Evaluation, LREC-COLING 2024 - Main Conference Proceedings
Abstract
Text simplification lacks a universal standard of quality, and annotated reference simplifications are scarce and costly. We propose to alleviate such limitations by introducing REFeREE, a reference-free model-based metric with a 3-stage curriculum. REFeREE leverages an arbitrarily scalable pretraining stage and can be applied to any quality standard as long as a small number of human annotations are available. Our experiments show that our metric outperforms existing reference-based metrics in predicting overall ratings and reaches competitive and consistent performance in predicting specific ratings while requiring no reference simplifications at inference time.
First Page
13740
Last Page
13753
Publication Date
1-1-2024
Keywords
model-based evaluation, text simplification
Recommended Citation
Y. Huang and E. Kochmar, "REFeREE: A REference-FREE Model-Based Metric for Text Simplification," 2024 Joint International Conference on Computational Linguistics, Language Resources and Evaluation, LREC-COLING 2024 - Main Conference Proceedings, pp. 13740 - 13753, Jan 2024.