The DARPA program CAST funded the development of four two-way speech-to-speech translation systems for four different language pairs across two domains.
Evaluation of Speech-to-Speech Translation Systems
Download Resources
PDF Accessibility
One or more of the PDF files on this page fall under E202.2 Legacy Exceptions and may not be completely accessible. You may request an accessible version of a PDF using the form on the Contact Us page.
The DARPA program CAST funded the development of four two-way speech-to-speech translation systems for four different language pairs across two domains. We describe the evaluation methodology, metrics and lessons learned from a noncomparative evaluation of the systems in one of these domains. Because the goal of the systems was to provide translation support for the performance goal-oriented tasks, the emphasis for assessment was on "live" real-time interactions between English subject matter experts and foreign language speakers engaged in realistic, unscripted scenario-based interactions. We measured performance in terms of how well these systems supported dialogue interaction between the participants. We discuss these metrics as well as some ideas for improvements in future evaluations.