Share this article on:

Breaking News: Sign of the Times: ASL Comes to Mobile Phones

Shaw, Gina

doi: 10.1097/01.HJ.0000421126.19676.34

If you have ever tried to Skype with your spouse while on a business trip or FaceTime with your grandchildren, you may have discovered that “mobile” communication is not entirely mobile. FaceTime demands a WiFi connection, so you cannot make calls on the go or in an area without a WiFi hotspot (yes, such places still exist). And it only works on the iPhone. Skype, on the other hand, is primarily a computer-based video chat, and many Smartphones do not have the processing speed or bandwidth needed to run Skype mobile. Mobile phones have been strictly text messaging devices for most deaf people, but now a new cell phone program, MobileASL, changes all that. (See FastLinks.)

Back to Top | Article Outline

No WiFi Needed

Developed by engineers at the University of Washington in Seattle, MobileASL works on any 3G phone and can be integrated into any mobile device that has a camera on the screen side. It uses approximately one-tenth of the bandwidth that FaceTime uses, so WiFi is not needed. The secret of MobileASL is its focus on what matters most when you are signing, which is called region of interest coding. (See Fastlinks.)

“We use an off-the-shelf algorithm to detect a person's face and hands and devote more bits to those areas while giving less resolution to the background,” explained Eve Riskin, PhD, a professor of electrical engineering at UW.

Dr. Riskin designed the program in partnership with Richard Ladner, PhD, a Boeing professor in computer science and engineering at UW, who is also a hearing child of deaf parents. They worked closely with a large team of undergraduate and graduate students to fine-tune the program. “We also save power by detecting which person isn't signing and lowering the video quality in that person's signal.”

MobileASL is able to broadcast the video signal at a remarkably frugal 30 kilobytes per second. That is less than 10 percent of the bit rate for the average YouTube video. It is not a perfect solution, of course. Signers cannot converse as quickly and nimbly as they do in person. “But they're having real conversations on it,” said Dr. Ladner.

Back to Top | Article Outline

Positive Test Feedback

MobileASL was initially field-tested in 2010 with a group of students at the UW Summer Academy for Advancing Deaf and Hard of Hearing in Computing. The students helped Drs. Ladner and Riskin adjust the software so it could hone in on the best areas of the screen for high resolution.

“It is good for fast communication,” Tong Song, a Chinese national studying at Gallaudet University in Washington, DC, told UW's news service at the time. “Texting sometimes is very slow because you send the message, and you're not sure that the person is going to get it right away. If you're using this kind of phone, then you're either able to get in touch with the person right away or not, and you can save a lot of time.”

“The conversations I had on it were quite good and clear,” said Robert Roth, the coordinator of the summer academy. “There were a few connection dropoffs that were annoying. And if your cell phone plan has a data limit like mine does, you may end up using up your monthly allotment of bytes. But being able to have conversations anywhere I want, with any cell phone, would be a great advantage.”

The program version in the field test was designed for use on older operating systems, and with the rapid advances in mobile phone technology in the past two years, Dr. Ladner and Dr. Riskin have had to move fast to adapt it.

“We've been working on porting it to the Android operating system for over a year now,” Dr. Riskin said. They had what they believed to be a good working version of the software for the Android by the end of the summer, but they said the real test would come in the fall when students returned to campus and could do more field testing.

Back to Top | Article Outline

Future Availability

Drs. Ladner and Riskin have been receiving emails from potential users eager to get the program after MobileASL first garnered publicity with the 2010 field test. Their goal is to offer the program online for free. That has not happened yet, but they say that they are hopeful that MobileASL will be available as open-source software on the web by 2013.

“The plan is to put it out and hopefully someone else will support it,” said Dr. Ladner. “We're a university, and we can't really provide ongoing software support. We don't have the capacity to do that. But the hope is that maybe some of the technology that we developed will be picked up by Apple or Skype or another video phone company, and they could potentially incorporate it directly into their phones.”

Mr. Roth said MobileASL will have significant appeal if its latest incarnation lives up to its promise and can be used with most mobile phones.

Back to Top | Article Outline


* Visit UW's MobileASL Web page at

* Read more about region of interest priority coding for sign language videoconferencing at

* Visit HJ's Student Blog at

* Check out HJ's R&D Blog at

* Click and Connect! Access the links in The Hearing Journal by reading this issue on our website or in our new iPad app, both available at

* Follow us on Twitter at and like us on Facebook at

© 2012 Lippincott Williams & Wilkins, Inc.