Chatbots often respond to human input with incorrect or nonsensical answers. Why not enable them to search for helpful information?
What's new: Mojtaba Komeili, Kurt Shuster, and Jason Weston at Facebook devised a chatbot that taps knowledge from the internet to generate correct, timely conversational responses.
Key insight: A chatbot typically knows only what it has learned from its training set. Faced with a subject about which it lacks information, it can only make up an answer. If it can query a search engine, it can gather information it may lack.
How it works: The chatbot comprised two BART models. To train and test the system, the authors built a dataset of roughly 10,000 search-assisted dialogs. One human conversant chose a topic and started the conversation, while another, if necessary, queried a search engine and formulated replies. The authors tracked which statements led to a search, and which statements and searches led to which responses.
- The authors trained one BART to take a dialogue-in-progress as input and generate the associated search query. The search engine returned five documents per query.
- The authors trained the other BART to generate representations of each document and the dialog in progress, concatenate the representations, and generate the response.
Results: Human volunteers chatted with both the authors’ system and a BART model without internet access, and scored the two according to various metrics. They rated the authors’ chatbot more consistent (76.1 percent versus 66.5 percent), engaging (81.4 percent versus 69.9 percent), knowledgeable (46.5 percent versus 38.6 percent), and factually correct (94.7 percent versus 92.9 percent).
Why it matters: This work enables chatbots to extend and update their knowledge on the fly. It may pave the way to more conversational internet search as well as a convergence of conversational agents and intelligent assistants like Siri, Google Assistant, and Alexa, which already rely on internet search.
We're thinking: When it comes to chatbots, things are looking up!