Enhancing Dialogue Systems with Adaptive Decision Boundaries and Multi-task Learning for Open Intent Recognition

  • Unique Paper ID: 168142
  • Volume: 11
  • Issue: 4
  • PageNo: 1323-1330
  • Abstract:
  • Modern dialogue systems have made significant strides in understanding and responding to user queries. Despite these advancements, they still encounter challenges when dealing with diverse user intents, especially those not covered in their training data. This limitation often leads to inaccurate responses and misinterpretations, impeding the potential for meaningful interaction. Addressing this issue, our project introduces an innovative approach by integrating two sophisticated techniques: Adaptive Decision Boundary (ADB) for open intent detection and Multi-task Pre-training and Contrastive Learning with Nearest Neighbors (MTP-CLNN) for open intent discovery. ADB's key feature is its ability to dynamically adjust decision boundaries surrounding known intent clusters, thereby effectively identifying open intents that fall outside these clusters. To further enhance the accuracy of the model, we have upgraded the ADB model to Adaptive Decision Boundary Learning via Expanding and Shrinking (ADBES). This enhancement includes an update to the model's loss function, by introducing the concept of shrinking boundaries. This modification allows for a more precise encapsulation of known intents and a better differentiation from emerging or unknown ones. Through the combination of ADBES and MTP-CLNN, our pipeline not only accurately identifies known intents but also uncovers new intent categories, facilitating a more robust and adaptable dialogue system capable of evolving with user needs.

Related Articles