DocScript: Document-Level Script Event Prediction

Authors

Puneet Mathur, University of Maryland, College Park
Rajiv Jain, Adobe Inc.
Vlad Morariu, Adobe Inc.
Aparna Garimella, Adobe Inc.
Franck Dernoncourt, Adobe Inc.
Jiuxiang Gu, Adobe Inc.
Ramit Sawhney, Mohamed Bin Zayed University of Artificial IntelligenceFollow
Preslav Nakov, Mohamed Bin Zayed University of Artificial IntelligenceFollow

Document Type

Conference Proceeding

Publication Title

2024 Joint International Conference on Computational Linguistics, Language Resources and Evaluation, LREC-COLING 2024 - Main Conference Proceedings

Abstract

We present a novel task of document-level script event prediction, which aims to predict the next event given a candidate list of narrative events in long-form documents. To enable this, we introduce DocSEP, a challenging dataset in two new domains - contractual documents and Wikipedia articles, where timeline events may be paragraphs apart and may require multi-hop temporal and causal reasoning. We benchmark existing baselines and present a novel architecture called DocScript to learn sequential ordering between events at the document scale. Our experimental results on the DocSEP dataset demonstrate that learning longer-range dependencies between events is a key challenge and show that contemporary LLMs such as ChatGPT and FlanT5 struggle to solve this task, indicating their lack of reasoning abilities for understanding causal relationships and temporal sequences within long texts.

First Page

5140

Last Page

5155

Publication Date

1-1-2024

Keywords

document-level extraction, information extraction, script event prediction

Recommended Citation

P. Mathur et al., "DocScript: Document-Level Script Event Prediction," 2024 Joint International Conference on Computational Linguistics, Language Resources and Evaluation, LREC-COLING 2024 - Main Conference Proceedings, pp. 5140 - 5155, Jan 2024.