On Learning Neural End-to-End Goal-Oriented Dialog Systems

We identify and address some of the challenges in end-to-end learning of neural goal-oriented dialog systems: 1) The challenge posed by the presence of a large number of named entities in goal-oriented dialog tasks. We propose a method to build neural embeddings for named entities on the fly and store them with embeddings as keys and the actual named entities as values. The proposed method allows for comparison and retrieval, using embeddings as well as actual named entities. 2) The challenge of performing supervised learning of goal-oriented dialog systems with multiple valid next utterances. We propose a method to learn to use different parts of the neural network to encode different predictions of the next utterances without learning of one interfering with that of others. 3) The challenge of handling new user behaviors during deployment. We propose a method that learns to anticipate failures and transfers to human agents to maintain high user task success. 4) The challenge of requiring large amounts of training data for each new dialog task of interest. We show that by selectively learning from an already available related task’s data, we can improve the performance on a new task that only has a limited amount of training data.