This portal has been archived. Explore the next generation of this technology.
PaLM: A Hybrid Parser and Language Model
lib:e6cfb02ed704d5a8 (v1.0.0)
Authors: Hao Peng,Roy Schwartz,Noah A. Smith
Where published: IJCNLP 2019 11
ArXiv: 1909.02134
Document: PDF DOI
Artifact development version: GitHub
Abstract URL: https://arxiv.org/abs/1909.02134v1
We present PaLM, a hybrid parser and neural language model. Building on an RNN language model, PaLM adds an attention layer over text spans in the left context. An unsupervised constituency parser can be derived from its attention weights, using a greedy decoding algorithm. We evaluate PaLM on language modeling, and empirically show that it outperforms strong baselines. If syntactic annotations are available, the attention component can be trained in a supervised manner, providing syntactically-informed representations of the context, and further improving language modeling performance.
Where published: IJCNLP 2019 11
ArXiv: 1909.02134
Document: PDF DOI
Artifact development version: GitHub
Abstract URL: https://arxiv.org/abs/1909.02134v1
We present PaLM, a hybrid parser and neural language model. Building on an RNN language model, PaLM adds an attention layer over text spans in the left context. An unsupervised constituency parser can be derived from its attention weights, using a greedy decoding algorithm. We evaluate PaLM on language modeling, and empirically show that it outperforms strong baselines. If syntactic annotations are available, the attention component can be trained in a supervised manner, providing syntactically-informed representations of the context, and further improving language modeling performance.
Relevant initiatives
Related knowledge about this paper
Reproduced results (crowd-benchmarking and competitions)
Artifact and reproducibility checklists
- ACM/cTuning artifact appendix and reproducibility checklist
- NeurIPS reproducibility checklist
- SIGPLAN's checklist for empirical evaluation
- Collective Knowledge (organizing research projects based on FAIR principles)