=======================================================
The Ninth Dialog System Technology Challenge (DSTC9)
Call for Participation
Website:https://sites.google.com/dstc.community/dstc9/home
=======================================================
Background
-----------------
The DSTC shared tasks have provided common testbeds for the dialog research community since 2013. From its sixth edition, it has been rebranded as "Dialog System Technology Challenge" to cover a wider variety of dialog related problems.
For this year's challenge, we opened the call for track proposals and selected the following four parallel tracks by peer-reviews:
- Beyond Domain APIs: Task-oriented Conversational Modeling with Unstructured Knowledge Access (Amazon Alexa AI): This track aims to allow users to have requests that are out of the scope of APIs/DB but potentially available in external knowledge sources. Track participants will develop task-oriented dialogue systems to understand relevant domain knowledge, and generate system responses with the relevant selected knowledge. In addition, the track includes evaluation on generalization over unseen domains and modalities (i.e. moving from written to spoken conversations).
- Multi-domain Task-oriented Dialog Challenge II (Microsoft Research AI & Tsinghua University): This track follows its success in DSTC-8 continuing with the effort of building dialog systems under a multi-domain setting. This time extending the task by incorporating new datasets, creating new sub-tasks, and providing a new development platform. The new task specifically focuses on two aspects of dialog systems: language portability and end-to-end system complexity.
- Interactive Evaluation of Dialog (CMU & USC): This track targets the creation of systems that can be effectively used in interactive settings by real users. The task is intended to move research beyond datasets, and evaluate models in interactive environments with real users allowing several valuable properties of dialog to be measured: consistency, adaptiveness and user-centric development. DialPort, a platform for interactive assessment with real users will be used for evaluation.
- SIMMC: Situated Interactive Multi-Modal Conversational AI (Facebook Assistant & Facebook AI): This track aims to tackle grounding dialog in an evolving multi-modal contextual input. Unlike previous multimodal track challenges, where the context from the non-textual modalities (video and audio) remains unchanged as the dialog progresses, this track encompasses a rich, situated multi-modal user context in the form of a shared image or VR environment that evolves fluidly based on the dialog flow.
Participation is welcomed from any research team (academic, corporate, non-profit, government).
Important Dates
------------------------
- Jun 15, 2020: Training data is released
- Sep 21, 2020: Test data is released
- Oct 5, 2020: Entry submission deadline
- Nov 2020: Paper submission deadline
- Spring 2021: DSTC9 workshop (venue: TBD)
DSTC9 Organizing Committee
--------------------------------------------
- Chulaka Gunasekara - IBM Research AI, USA
- Abhinav Rastogi - Google Research, USA
- Yun-Nung (Vivian) Chen - National Taiwan University, Taiwan
- Luis Fernando D'Haro - Universidad Politécnica de Madrid, Spain
- Seokhwan Kim - Amazon Alexa AI, USA
DSTC9 Track Organizers
-------------------------------------
Beyond Domain APIs: Task-oriented Conversational Modeling with Unstructured Knowledge Access
- Seokhwan Kim, Mihail Eric, Behnam Hedayatnia, Karthik Gopalakrishnan, Yang Liu, Dilek Hakkani-tur (Amazon Alexa AI)
Multi-domain Task-oriented Dialog Challenge II
- Baolin Peng, Jianfeng Gao, Jinchao Li, Lars Liden, Minlie Huang, Qi Zhu, Runze Liang, Ryuichi Takanobu, Shahin Shayandeh, Swadheen Shukla, Zheng Zhang (Microsoft Research AI & Tsinghua University)
Interactive Evaluation of Dialog
- Shikib Mehri, Carla Gordon, David Traum, Maxine Eskenazi (CMU & USC)
SIMMC: Situated Interactive Multi-Modal Conversational AI
- Ahmad Beirami, Eunjoon (EJ) Cho, Paul A. Crook, Ankita De, Alborz Geramifard, Satwik Kottur, Seungwhan Moon, Shivani Poddar, Rajen Subba (Facebook Assistant & Facebook AI)
DSTC Steering Committee
---------------------------------------
- Koichiro Yoshino - Nara Institute of Science and Technology (NAIST), Japan
- Chiori Hori - Mitsubishi Electric Research Laboratories (MERL), USA
- Rafael E. Banchs - Nanyang Technological University (NTU), Singapore
- Michel Galley - Microsoft Research AI, USA
Contact Information
---------------------------------------
Join the DSTC mailing list to get the latest updates about DSTC9:
- To join the mailing list: visit https://groups.google.com/a/dstc.community/forum/#!forum/list/join
- To post a message: send your message to list@dstc.community
- To leave the mailing list: visit https://groups.google.com/a/dstc.community/forum/#!forum/list/unsubscribe
For specific enquiries about DSTC9:
Please feel free to contact dstc9-organizing-committee@dstc.community.