A divide-and-conquer approach to neural natural language generation from structured data

Dethlefs, Nina; Schoene, Annika; Cuayáhuitl, Heriberto

Abstract

Current approaches that generate text from linked data for complex real-world domains can face problems including rich and sparse vocabularies as well as learning from examples of long varied sequences. In this article, we propose a novel divide-and-conquer approach that automatically induces a hierarchy of “generation spaces” from a dataset of semantic concepts and texts. Generation spaces are based on a notion of similarity of partial knowledge graphs that represent the domain and feed into a hierarchy of sequence-to-sequence or memory-to-sequence learners for concept-to-text generation. An advantage of our approach is that learning models are exposed to the most relevant examples during training which can avoid bias towards majority samples. We evaluate our approach on two common benchmark datasets and compare our hierarchical approach against a flat learning setup. We also conduct a comparison between sequence-to-sequence and memory-to-sequence learning models. Experiments show that our hierarchical approach overcomes issues of data sparsity and learns robust lexico-syntactic patterns, consistently outperforming flat baselines and previous work by up to 30%. We also find that while memory-to-sequence models can outperform sequence-to-sequence models in some cases, the latter are generally more stable in their performance and represent a safer overall choice.

Citation

Dethlefs, N., Schoene, A., & Cuayáhuitl, H. (2021). A divide-and-conquer approach to neural natural language generation from structured data. Neurocomputing, 433, 300-309. https://doi.org/10.1016/j.neucom.2020.12.083

• Journal Article Type: Article
• Acceptance Date: Dec 14, 2020
• Online Publication Date: Jan 5, 2021
• Publication Date: Apr 14, 2021
• Deposit Date: Feb 1, 2021
• Publicly Available Date: Jan 6, 2022
• Journal: Neurocomputing
• Print ISSN: 0925-2312
• Publisher: Elsevier
• Peer Reviewed: Peer Reviewed
• Volume: 433
• Pages: 300-309
• DOI: https://doi.org/10.1016/j.neucom.2020.12.083
• Keywords: Neural networks; Artificial intelligence; Natural language processing
• Public URL: https://hull-repository.worktribe.com/output/3709268
• Publisher URL: https://www.sciencedirect.com/science/article/abs/pii/S0925231220319950?via%3Dihub