[1911.00536] DialoGPT: Large-Scale Generative Pre-training for Conversational Response Generation
In future, we will investigate how to detect and control toxic generation, and leverage reinforcement learning to further improve the relevance of the generated responses and prevent the model from generating egregious responses.
Abstract: We present a large, tunable neural conversational response generation model,
DialoGPT (dialogue generative pre-trained transformer). Trained on 147M
conversation-like exchanges extracted from Reddit comment chains over a period
spanning from 2005 through 2017, DialoGPT extends the Hugging Face PyTorch
transformer to attain a performance close to human both in terms of automatic
and human evaluation in single-turn dialogue settings. We show that
conversational systems that leverage DialoGPT generate more relevant,
contentful and context-consistent responses than strong baseline systems. The
pre-trained model and training pipeline are publicly released to facilitate
research into neural response generation and the development of more
intelligent open-domain dialogue systems.
‹Figure 1: Generated response can surpass human response in automatic metrics. Example responses are from Gupta et al. (2019) (DSTC-7 Dialogue Generation Challenge)›
[1906.02738v1] Conversing by Reading: Contentful Neural Conversation with On-demand Machine Reading[1906.02738] Conversing by Reading: Contentful Neural Conversation with On-demand Machine Reading[1909.05361] Structuring Latent Spaces for Stylized Response Generation