Promoting digital culture with innovative technologies: animated virtual humans teach sign language in the Institute of Mechanical Intelligence project developed for the[...]

Promoting digital culture with innovative technologies: animated virtual humans teach sign language in the Institute of Mechanical Intelligence project developed for the Biblioteca Comunale Guglielmo Marconi (Viareggio)

To create an interactive portal with applications in the Italian Sign Language (LIS) and supports for people with hearing and visual disabilities, accessible to all, inclusive, with the aim of illustrating the mission, the cultural and educational offers of the Biblioteca Comunale Guglielmo Marconi, Viareggio: this is the objective of the Project 'Communicating the library digital and virtual infrastructures for projects laboratories experiences of Viareggio City that Reads', developed by the Institute of Mechanical Intelligence of the Scuola Superiore Sant'Anna of Pisa, under the scientific supervision of Chiara Evangelista, and curated by Gaia Sasso, research fellow at the Institute.

In its first months of activity, the project has developed accessible communication strategies through the use of advanced technologies, such as the creation of animated Virtual Humans, human beings generated using specific software, to become users' artificial companions in simulations generated through the application.
The design of the avatars was adapted to the audience and the library's needs to offer, through customisation, tailor-made figures that can facilitate the use of content for visitors with hearing disabilities. These virtual assistants not only translate the educational content into Sign Language, but are also designed to educate and approach people interested in this form of communication. In fact, the implementation of virtual humans is not limited to serving only people with hearing disabilities, it is also designed for those who wish to learn and deepen their understanding of Sign Language, thus promoting a culture of inclusiveness and accessibility within the Viareggio community.

Thanks also to the collaboration with Polisportiva Silenziosi Versiliesi, the project carried out the 'Motion Capture' recording activities of the Sign Language, which conveys its meanings through a coded system of hand signs, facial expressions, lip movements and body movements.
Motion Capture is the technology that records movement by capturing it technologically by means of sensors, placed, in this case, on a Lycra suit worn by the subject, which return complete data on position, inclination and height. In this experiment, an Xsens suit was used, complemented by Manus gloves, which enabled high-precision tracking of the fingers, essential for successful recording.
The technological process not only captured body movement, but also facial expressions, lip movements and other more refined movements. The data was transferred in real time to software that enables the animation of Virtual Humans.

The next phase of the project involves the creation of videos, which will be posted on the library's portal, where the avatars will be the protagonists of the presentation of some informative and didactic content through the LIS language.
The project is part of the activities for the promotion and enhancement of digital culture through innovative technologies, as part of the Local Pact for Reading City of Viareggio, both in the technological and perceptual fields, for the enhancement of culture in general.

Gallery