We are very excited to join forces with MLCommons and OctoML.ai! Contact Grigori Fursin for more details!

BERT has a Mouth, and It Must Speak: BERT as a Markov Random Field Language Model

lib:f97e6d2432caaf7f (v1.0.0)

Vote to reproduce this paper and share portable workflows   1 
Authors: Alex Wang,Kyunghyun Cho
Where published: WS 2019 6
ArXiv: 1902.04094
Document:  PDF  DOI 
Artifact development version: GitHub
Abstract URL: http://arxiv.org/abs/1902.04094v2


We show that BERT (Devlin et al., 2018) is a Markov random field language model. This formulation gives way to a natural procedure to sample sentences from BERT. We generate from BERT and find that it can produce high-quality, fluent generations. Compared to the generations of a traditional left-to-right language model, BERT generates sentences that are more diverse but of slightly worse quality.

Relevant initiatives  

Related knowledge about this paper Reproduced results (crowd-benchmarking and competitions) Artifact and reproducibility checklists Common formats for research projects and shared artifacts Reproducibility initiatives

Comments  

Please log in to add your comments!
If you notice any inapropriate content that should not be here, please report us as soon as possible and we will try to remove it within 48 hours!