Department of
Computer Science
695 Park Ave.
NY, NY 10021

 

Susan L. Epstein

The CUNY Graduate School, Department of Computer Science and

 Hunter College, Department of Computer Science

 

 

 

 

     Home 
     Publications 
     Collaborators 
     Courses 
     Media Appearances 
     Contact

 

 

 

 

 

Spoken Dialogue System

Automated telephone dialogue systems rely disproportionately on accurate transcription of the speech signal into readable text. As the performance of automatic speech recognition (ASR) decreases, dialogue system performance often falls off sharply. Our project seeks better dialogue strategies that are less dependent on accurate ASR, and that degrade gracefully. Our novel methodology, wizard ablation, collects simulated human-system dialogues that vary in controlled ways.

CheckItOut

Our testbed application, the CheckItOut dialogue system, is modeled on a corpus of telephone transactions between patrons and librarians that we collected at New York City's Andrew Heiskell Braille & Talking Book Library. This application has appropriately limited complexity, and potentially broad social benefit. We based CheckItOut on the Olympus spoken dialogue system architecture and RavenClaw dialogue manager developed at Carnegie Mellon University. For further details, see the Loqui project.

Key references

Epstein, S.L., R.J. Passonneau, T. Ligorio and J. Gorden (In press). Data Mining to Support Human-Machine Dialogue for Autonomous Agents In Proceedings of Agents and Data Mining Interaction (ADMI2011), Springer-Verlag.

Passonneau, R., Epstein, S.L., Ligorio, T., & Gordon, J. (2011) Embedded Wizardry In Proceeding of SIGDIAL 2011

Passonneau, R. J., Epstein, S. L., Ligorio, T., Gordon, J., & Bhutada, P. Learning about Voice Search for Spoken Dialogue Systems. speech recognition. In 11th Annual Conference of the North American Chapter of the Association for Computational Linguistics (NAACL HLT 2010), Los Angeles, 2010 (pp. 840-848)

Ligorio, Tiziana; Epstein, Susan; Passonneau, Rebecca J. 2010. Wizards' dialogue strategies to handle noisy speech recognition. IEEE workshop on Spoken Language Technology (IEEE-SLT 2010).Berkeley, CA, December 12-15, 2010.

Ligorio, Tiziana; Epstein, Susan; Passonneau, Rebecca J.; Gordon, Joshua. 2010. What You Did and Didn't Mean: Noise, Context, and Human Skill. Annual Meeting of the Cognitive Science Society (COGSCI 2010) Portland, Oregon. August 11-August 14, 2010. .

Passonneau, Rebecca J.; Epstein, Susan; Ligorio, Tiziana; Gordon, Joshua; Bhutada, Pravin. 2010. Learning about voice search for spoken dialogue systems. In 11th Annual Conference of the North American Chapter of the Association for Computational Linguistics (NAACL HLT 2010). Los Angeles. June 1-6, 2010. Best Paper Nominee.

Epstein, S. L., R. J. Passonneau, J. Gordon and T. Ligorio 2010. Toward Spoken Dialogue as Mutual Agreement. In Proceedings of AAAI Workshop on Metacognition for Robust Social Systems.

Passonneau, R. J., Epstein, S. L., Gordon, J. B., & Ligorio, T. (2009). Seeing What You Said: How Wizards Use Voice Search Results. In Proceedings of the IJCAI-09 Workshop on Knowledge and Reasoning in Practical Dialogue Systems.

Passonneau, R., & Epstein, S. L. (2009). Help Me Understand You: Addressing the Speech Recognition Bottleneck. In Proceedings of AAAI Spring Symposium on Agents that Learn from Human Teachers.

FORRSooth

FORRSooth is an architecture for task-oriented human-computer dialogue. It is intended to interact effectively despite inaccurate speech recognition. FORRSooth exploits the high redundancy in human language to make effective communication possible without perfect comprehension of the audio signal. FORRSooth uses six FORR-based services that operate simultaneously: Interaction, Interpretation, Satisfaction, Grounding, Generation, and Discourse.

Key references

Epstein, S. L., R. J. Passonneau, T. Ligorio and J. Gordon 2012. Toward Habitable Assistance from Spoken Dialogue Systems. Innovative Applications of Artificial Intelligence (IAAI-2012).

Epstein, S.L., Passonneau, R.J., Gordon, J. Ligorio, T. The Role of Knowledge and Certainty in Understanding for Dialogue. Advances in Cognitive Systems. 1, 93-108 (2012)

Epstein, S. L., R. J. Passonneau, J. Gordon and T. Ligorio (2011). The Role of Knowledge and Certainty in Understanding for Dialogue. In Proceedings of the AAAI Fall Symposium on Advances in Cognitive systems. Washington DC.

Gordon, J., Epstein, S. L., & Passonneau, R. J. (2011). Learning to Balance Grounding Rationales for Dialogue Systems In Proceeding of SIGDIAL-11.

Gordon, J. B., R. J. Passonneau and S. L. Epstein (2011). Helping Agents Help Their Users Despite Imperfect Speech Recognition. In Proceeding of AAAI Symposium Help Me Help You: Bridging the Gaps in Human-Agent Collaboration. Palo Alto.

Epstein, S. L., R. J. Passonneau, T. Ligorio and J. Gordon (In press). Data Mining to Support Human-Machine Dialogue for Autonomous Agents. In Proceedings of Agents and Data Mining Interaction (ADMI2011), Springer-Verlag.

Epstein, S. L., R. J. Passonneau, J. Gordon and T. Ligorio (2011). The Role of Knowledge and Certainty in Understanding for Dialogue. In Proceedings of the AAAI Fall Symposium on Advances in Cognitive Systems. Washington, DC.

Passonneau, R., Epstein, S. L., Ligorio, T., & Gordon, J. (2011). Embedded Wizardry. In Proceeding of SIGDIAL 2011.

Gordon, J., Epstein, S. L., & Passonneau, R. J. (2011). Learning to Balance Grounding Rationales for Dialogue Systems. In Proceeding of SIGDIAL-11.

Gordon, J. B., R. J. Passonneau and S. L. Epstein (2011). Helping Agents Help Their Users Despite Imperfect Speech Recognition. In Proceeding of AAAI Symposium Help Me Help You: Bridging the Gaps in Human-Agent Collaboration. Palo Alto.

Ligorio, T., S. L. Epstein and R. J. Passonneau 2010. Wizards' Dialogue Strategies to Handle Noisy Speech Recognition. In Proceedings of IEEE-SLT workshop, Berkeley CA.

This material is based upon work supported by the National Science Foundation under Grant No. ISS-0744904.

Any opinions, findings, and conclusions or recommendations expressed in this material are those of the author(s) and do not necessarily reflect the views of the National Science Foundation.

[top of the page]

Home

Publications

Collaborators

Courses

Contact