The innovative system to automatically translate sign language | CDE Almería - Centro de Documentación Europea

Deaf people and the hard-of-hearing often struggle in today’s society, despite being many in number. The SignAll system directly translates sign language, which will not only help deaf people but teach others how to sign.

Around 5 % of the global population, or 466 million people, have disabling hearing loss. Around 800 000 people in the European Union use some form of sign language to communicate. Despite the widespread prevalence of deaf people in societies around the world, sign language is poorly understood by those who can hear. Suitable technologies able to bridge this semantic gap in human connectivity are lacking. As a result, many deaf people struggle to integrate fully – both in work, and in the use of public and private services.

SignAll is an innovative translation system based on artificial intelligence and designed to translate sign language into other languages and start improving the quality of life for the hard-of-hearing. The Horizon 2020 grant allowed for localising the SIGNALL project for the EU market.

The SignAll system allows for direct, seamless and automated translation between deaf and hearing people, without the need for a human translator. It is the first software-hardware solution developed in the world and is already heading to market.

The first version of SignAll was developed for use in the United States, piloting a translation system for American Sign Language (ASL). The SignAll team now plans to adapt the technology for European deaf communities and aims to bring it to market across the EU.

Beyond words

The SignAll system uses hundreds of tailor-made machine learning algorithms to analyse information gathered by a system of cameras and sensors, including everything from large skeletal movements to tiny micro-expressions in the face. Hand gestures are picked up on specially-developed gloves, while the state-of-the-art computer vision system monitors facial and body expressions – vital yet often ignored aspects of sign languages.

“One of the distinct solutions of the SignAll system is recognition of facial expressions, like ‘knowing’ if someone raises his or her eyebrows,” says Zsolt Robotka, CEO of SignAll Technologies and project coordinator. Aside from the difficulties faced with multiple sign languages, there are regional dialects, as well as context-specific information which can change the meaning of signs, but a group of linguists is in charge of eliminating this barrier.

SignAll is also developing and refining a complementary system – an interactive, customisable software tool to teach ASL. The tool gamifies the learning process, teaching and testing users on sign language. “Our Learn ASL tool is the most recent development. It motivates more people to learn and therefore results in more ASL speakers. Ultimately, this leads to better opportunities for social inclusion for the deaf in daily life,” says Robotka.

The EU promoter of development

The SignAll team are extremely grateful for the grant they received from the EU, which gave them the time to develop the technology while also learning European legislation and country-specific sign languages.

Más información

Full news

R+I Section

Beyond words

The EU promoter of development

Más información

Publicaciones relacionadas:

Reader Interactions

Leave a Reply Cancel reply