Check the preview of 2nd version of this platform
being developed by the open MLCommons taskforce on automation and reproducibility
as a free, open-source and technology-agnostic on-prem platform.
BERT has a Mouth, and It Must Speak: BERT as a Markov Random Field Language Model
lib:f97e6d2432caaf7f (v1.0.0)
Authors: Alex Wang,Kyunghyun Cho
Where published: WS 2019 6
ArXiv: 1902.04094
Document: PDF DOI
Artifact development version: GitHub
Abstract URL: http://arxiv.org/abs/1902.04094v2
We show that BERT (Devlin et al., 2018) is a Markov random field language model. This formulation gives way to a natural procedure to sample sentences from BERT. We generate from BERT and find that it can produce high-quality, fluent generations. Compared to the generations of a traditional left-to-right language model, BERT generates sentences that are more diverse but of slightly worse quality.
Where published: WS 2019 6
ArXiv: 1902.04094
Document: PDF DOI
Artifact development version: GitHub
Abstract URL: http://arxiv.org/abs/1902.04094v2
We show that BERT (Devlin et al., 2018) is a Markov random field language model. This formulation gives way to a natural procedure to sample sentences from BERT. We generate from BERT and find that it can produce high-quality, fluent generations. Compared to the generations of a traditional left-to-right language model, BERT generates sentences that are more diverse but of slightly worse quality.
Relevant initiatives
Related knowledge about this paper
Reproduced results (crowd-benchmarking and competitions)
Artifact and reproducibility checklists
- ACM/cTuning artifact appendix and reproducibility checklist
- NeurIPS reproducibility checklist
- SIGPLAN's checklist for empirical evaluation
- Collective Knowledge (organizing research projects based on FAIR principles)