Building a Tricorder: DARPA/NIST’s Universal Translator Smartphone

At dusk, a car stops at a checkpoint in Afghanistan. It is a tense moment for all. Because an interpreter is not available, U.S. Marines use hand gestures to ask the driver to step out of the car and open the trunk and hood for inspection. There’s a lot of room for error.

This scene was re-enacted recently during an evaluation at the National Institute of Standards and Technology (NIST)–but, this time, the Marine had a new smart phone-based device that translates his English into the driver’s native Pashto and the Pashto back into English.

For the past four years, scientists at NIST have been conducting detailed performance evaluations of speech translation systems for the Defense Advanced Research Projects Agency (DARPA). Previous systems used microphones and portable computers. In the most recent tests, the NIST team evaluated three two-way, real-time, voice-translation devices designed to improve communications between the U.S. military and non-English speakers in foreign countries.
Traditionally, the military has relied on human translators for communicating with non-English speakers in foreign countries, but the job is dangerous and skilled translators often are in short supply. And, sometimes, translators may have ulterior motives, according to NIST’s Brian Weiss. The DARPA project, called TRANSTAC (spoken language communication and TRANSlation system for TACtical use), aims to provide a technology-based solution. Currently, the focus is on Pashto, a native Afghani tongue, but NIST has also assessed machine translation systems for Dari–also spoken in Afghanistan–and Iraqi Arabic.

All new TRANSTAC systems all work much the same way, says project manager Craig Schlenoff. An English speaker talks into the phone. Automatic speech recognition distinguishes what is said and generates a text file that software translates to the target language. Text-to-speech technology converts the resulting text file into an oral response in the foreign language. This process is reversed for the foreign language speaker.

NIST researchers held focus groups with U.S. military personnel who have served overseas to determine critical communication interactions to simulate and evaluate in tests. The research team then devised 25 scenarios for evaluating the performance of translation devices. These included vehicle checkpoints; communication of key information, such as how long electricity will be available each day; facility inspections; medical assessments; and Afghani-U.S. military training exercises. Marines experienced in these tasks and native Kandahari-dialect Pashto speakers acted out the scenarios without a script. Each scenario was performed using the three industry-developed translation devices. For each test, on-site judges observed the scenarios, and the participating Marines and Pashto speakers were surveyed about the ease of interaction with the systems. Later, a separate panel of judges fluent in English and Pashto viewed videos of the exercise and evaluated each of the three systems in terms how accurately concepts were communicated in both languages, Schlenoff says. “We are writing a detailed assessment of the evaluation for DARPA so they can make an informed decision to determine where to direct funds and efforts in the TRANSTAC project,” says Schlenoff.