Breaking Down Technology Barriers for the deaf/Deaf and Hard of Hearing

Calling a friend. Joining a work conference call. Refilling a prescription by phone. Most of us use mobile devices and apps to communicate daily without a second thought. 

However, if you’re a member of the deaf/Deaf or hard of hearing communities in the U.S., you need a way to “see” the conversation. That could mean signing with an American Sign Language (ASL)-to-English interpreter via video relay or having a discussion facilitated by captions.

But how is communication delivered accurately, quickly, and reliably between people who are hearing and people who are deaf/Deaf or experience hearing loss? Is communication across multiple devices and platforms practical? And is the technology compatible? 

During the summer of 2018, nine MITRE interns—six of them members of the deaf/Deaf or hard of hearing communities—put several technologies to the test. The goal: Create an optimal user experience. They ran and re-ran baseline tests in our Federal Communications Commission (FCC)-sponsored National Test Lab (NTL) in McLean, Virginia, or in our NTL-Rochester, New York, lab. The testing and validation were part of an ongoing, growing effort as more interns from the community lend their time, talent, and consumer experience to dismantle barriers to communication.

The FCC funds the Telecommunications Relay Service, or TRS, to ensure that people who are deaf/Deaf, hard of hearing, deaf/blind, or speech-disabled have full access to advanced communications services and equipment free of charge. The service was launched in the 1970s. Since the passage of the Americans With Disabilities Act in 1990, fees from telecommunications and Voice over Internet Protocol providers have funded the TRS. 

Despite the clear need for the TRS, the FCC has just 1,688 employees and few engineers. So, four years ago, the agency requested MITRE’s systems engineering expertise to help modernize and improve TRS's options. The CMS Alliance to Modernize Healthcare took on the challenge.

Making the FCC’s services more reliable and accessible isn’t just the right thing to do, it also translates into making the world safer—MITRE’s mission from the start. Think of conveying a doctor’s advice or making a 911 call. In both cases, accuracy can be a matter of life or death. And for members of the deaf/Deaf or hard of hearing communities—everyday communication should be quick and easy. 

A Day in the Life in the NTL

Against a backdrop of 75-plus desktop phones, monitors, PCs, smartphones, spreadsheets, and cameras, interns, in the NTL, test video relay service (VRS) providers’ software and hardware performance against certain measures. (There are five VRS providers nationally.) The interns also test Internet Protocol Captioned Telephone Services (IP CTS), which allow individuals with some residual hearing to speak directly to the other individual over the internet, using both voice and captions.

For VRS testing, the interns—under the mentorship and guidance of Dr. Chern Liou—looked at time delays, pixilation, and video mail (vs. voicemail) delivery. For IP CTS testing, the interns measured text accuracy and delay in captions appearing on the screen. Through repeated, rigorous testing, they created objective “snapshots” of how the technology works on various devices. This testing is aimed at improving users’ routine communications.

“I never knew there was more than one provider—I only used one,” says Jaric Sloan, a fourth-year student at Rochester Institute of Technology (RIT)-National Technical Institute for the Deaf studying computer engineering technology. “I used an Excel matrix to validate and report different results. We color coded results. Yellow meant video mail failed. Red meant it failed to connect, or the quality wasn't good. Green meant it all worked great.”

In addition to gaining technical experience, the interns also benefitted from the opportunity to immerse themselves in MITRE’s culture. Mustafa Hussein, a senior studying information technology at Gallaudet University in Washington, D.C., says, “I’ve worked for a big commercial corporation. It was all about profits, a ‘time is money’ mentality. The expectations are very high at MITRE, but it’s more focused on collaboration. I learned lessons from this internship about leadership, teamwork, business culture, and public ethics.”

Pop-up Conversations and Proper Text Captioning

Spontaneous conversations came up in the lab setting, and Abbie Castillo was always ready to take part. Castillo, a Gallaudet student pursuing a bachelor’s degree in interpreting, was MITRE’s first ASL interpreter intern. 

“I stayed on the lookout for pop-up conversations between a Deaf client and a hearing person,” says Castillo, who is hearing. “When you’re interpreting, you’re constantly analyzing, tuning into the client’s preferred signing style, and making sure what you translate is conceptually accurate. It’s a lot. For long meetings, I’d interpret for 20 minutes, then my mentor—the project’s full-time, in-house interpreter—took the next 20 minutes. We went back and forth.”

Minnie Buenaventura, a systems engineering intern (on her second summer at MITRE) and a Gallaudet alumna, tested the overall production quality of IP CTS on a desktop phone, Android, and other platforms. She looked for time lapses before words appear on the screen and for missing or incorrect words. It’s like individualized captioning of live television programming, which is never perfect (unlike prerecorded programs' high level of accuracy).

“I really enjoyed testing all of the different providers,” says Buenaventura, who is Deaf. “I see errors, like the name 'Jim' might come across as 'Jen.' Or 'Reston' [a town in Virginia] might come across as 'Russia.'”

Buenaventura’s mentor, Evan Saltz, a former intern and now a MITRE subcontractor on the FCC project, assisted her with the IP CTS testing process he helped develop. Saltz’s mentor, Jim Malloy, taught him why text captioning must be accurate. 

“Consider the consequences if a caller speaks with a doctor and says, ‘I am not allergic to penicillin,’ and the doctor repeats the caller’s words, ‘You are not allergic to penicillin,’ but the captions show, ‘you are allergic to penicillin,’" Saltz says. “The patient might not get what they need.”

"It’s important to empower the deaf/Deaf and hard of hearing communities and educate vendors and others about the issues unique to both communities.”  

A Winning ACE for ASL-Fluent Customers

Through the FCC work, MITRE brings together deaf/Deaf and hard of hearing community stakeholders—the TRS users, government, vendors, and academia—to improve and modernize services and realize cost savings of up to $17 million annually, according to the FCC Contracting Officer Representative. 

To make direct video calls more efficient, for example, MITRE created a now-operational, open-source prototype called Accessible Communications for Everyone Direct, or ACE Direct. The technology supports the notion of an ASL-fluent customer service representative in a call center supporting Deaf ASL-using customers. 

On the vendor side, collaboration helps all parties. “The service providers have come to appreciate MITRE’s feedback and findings,” says Yus Johnson, MITRE systems engineer. “They’re getting a lot of testing at no cost, and we report agnostically to help make the service better overall.”

“This work is really disruptive—in a positive way,” says Jeff Rogers, who leads MITRE’s FCC work with Reeta Singh and Dwight Handon. “We’re enhancing the quality of service delivery to the Deaf and hard of hearing communities, which are so often discriminated against. And with the energy and enthusiasm of these students, we’re accelerating our ability to deliver impact at four times the speed.”

—by Karina Wright