HTML Elements
A major challenge that hinders the development of task- oriented dialog systems is the design of dialog schema, typ- ically composed of a set of intents with corresponding slots, to capture and handle the domain-specific dialog states. Pre- vious work on schema-guided dialog (Rastogi et al. 2020a; 2020b; Lee et al. 2021) focused on data-efficient joint di- alog state modeling across domains and zero-shot general- ization to new APIs. However for new emerging domains and novel services, the identification of key intents of such schema typically requires domain expertise and/or laborious analysis of a large volume of conversation transcripts. With increasing demand for and adoption of virtual assistants, re- cent work has investigated ways to accelerate schema design through the automatic induction of intents (Hakkani-Tur et ̈ al. 2015; Haponchyk et al. 2018; Perkins and Yang 2019; Chatterjee and Sengupta 2020) or the induction of slots and dialogue states (Hudecek, Du ˇ sek, and Yu 2021; Min et al. ˇ 2020). However, a lack of realistic shared benchmarks with public datasets, metrics, and task definitions has made it dif- ficult to track progress in this area. This track aims to evaluate methods for the automatic in- duction of customer intents in a realistic setting of customer service interactions between human agents and customers. As complete conversations will be provided, participants can make use of information in both agent and customer turns. To further encourage new research directions in the automatic discovery of intents, we also propose an open in- tent induction task that goes beyond the clustering paradigm commonly used for evaluation of intent discovery.