Archives: American Sign Language gets a boost from 'Paula'

American Sign Language gets a boost from 'Paula'
August 9, 2002 10:00
By Jeff Meredith

CHICAGO - Walking through an airport security checkpoint in these days of heightened security can be a trying experience. Between being asked to remove your laptop from its case, having to run your nail scissors back to checked baggage, and wondering what is setting off the metal detector, there's nothing but tension.

Now imagine that you're deaf and that you're trying to understand a security guard's commands; suddenly, the aforementioned complaints with the process seem trivial.

Combining computer technology and linguistics research to span the communication gap between the deaf and hearing worlds, DePaul University researchers may have a solution to the problem.

"Paula," a computer-generated synthetic interpreter developed by a team of faculty and students in DePaul's School of Computer Science, Telecommunications and Information Systems (CTI), could be the perfect translator for the deaf.

Using speech recognition and animation to translate spoken English into American Sign Language (ASL), researchers have Paula delivering ASL through hand gestures and facial expressions on a computer screen. A hearing person can speak through a headset connected to the computer, and Paula then translates spoken English into ASL.

Since human ASL translators are an expensive commodity and often difficult to come by, Paula could find a home in numerous settings, from grocery stores to service desks to airport security.

The idea for Paula was born in March 1998, when Karen Alkoby, a deaf DePaul CTI graduate student, mapped out the concept of an automated ASL synthesizer for her thesis. Faculty soon took to the idea and began work on a prototype in the fall of 1998; between 15-20 students and faculty have been involved with the project at any one point in its history.

Rosalee Wolfe, a professor of computer science at DePaul and one of the research team leaders, is encouraged by progress made with Paula, but notes that the system isn't perfect yet. The DePaul team is working on "proper placement of the hand when it comes into contact with different parts of the body, facial expressions...it's an ongoing process," said Wolfe.

"We're now producing American Sign Language that's recognizable, but it's still not perfect. I'm encouraged by the fact that many deaf people now look at it and say 'Well, I would sign that a little bit differently' - as opposed to just looking at it blankly and saying 'What is it'?"

The system may need refinement, but it could present a significant upgrade over the only other automated system on the market - a rather expensive product which is available for Signed English, which is not a language used by all the deaf of North America; it is merely an alternative. ASL on the other hand is the fourth most used language in the United States. Contrary to popular belief, ASL is different from English or any other spoken language.

"As a hearing person, I thought ASL was simply a signed form of English, but it's not. It has its own grammar, that was a big revelation," said Wolfe. "This is why simply writing things on paper does not work. Writing it out on paper, it's still in a foreign language."

Facial expressions are of particular importance in presenting ASL, and this is an area where the research team is trying to make strides. So far, Paula's expressions have been prone to "very dramatic or very exaggerated expressions," said Wolfe, and the team is looking to achieve greater subtlety.

"Deaf people and teachers of ASL tell you that the most important aspect of the language is what's happening on the person's face," said Wolfe. "Sometimes it's part of the vocabulary, sometimes it's part of the grammar. You can't ask a question in ASL without using an expression, it's intrinsic to the language."

Wolfe has spent 15 years at DePaul and has always had an interest in graphics and human-computer interaction. Looking forward, she envisions a Paula that will move beyond being simply intelligible to being widely understood by the deaf and implemented in a variety of settings. "Our main goal is to get this placed in as many situations as can be helpful to the deaf," said Wolfe.

For more information, visit http://asl.cs.depaul.edu.