Abstract

Deep learning has become a central part of everyday life, and has given us powerful digital tools such as machine translators, voice assistants, and large language models. The Deaf communities, however, benefit very

little from this progress. Sign languages are full natural languages, but they have no widely used written form, so the large text corpora and pretrained models that support these tools for spoken languages have no direct equivalent. As a result, sign language processing is held back by two related problems: a persistent shortage of annotated data, and a lack of tools to analyze the languages themselves.

This thesis focuses on one component of the sign language processing pipeline that has received comparatively little attention: sign language segmentation, the temporal partitioning of a continuous signing video into individual sign units. Segmentation sits between raw video and almost every symbolic task built on top of it, yet it remains poorly understood. The goal of this work is twofold: to advance the segmentation task itself, and to turn sign language processing research into tools that are usable by the Deaf community, with French Belgian Sign Language (LSFB) as the main setting.

The central contribution of this thesis is a set of segmentation models. We first study approaches based on recurrent neural networks, and show that their main difficulty lies in the modeling of the transitions between signs, the brief ambiguous movements that separate one sign from the next and that are easily confused with the signs themselves. This insight motivates the Hydra framework, a new approach that detects each sign as a whole unit rather than deciding, frame by frame, whether a sign is being produced. Concretely, instead of classifying every frame, the model predicts the distance from each frame to the nearest sign boundaries, which makes it less sensitive to the ambiguous regions and improves consistently over prior methods. Around this core, we present further contributions: a sign language-to-text dictionary that recognizes signs from an ordinary webcam, supporting work on isolated sign recognition, and a collaborative platform that gathers new sign data as a side effect of everyday use. Together, these contributions narrow the gap between sign language processing research and the people whose language it concerns.

Jury

  • Prof. Benoit Frénay - University of Namur, Belgium
  • Prof. Anthony Cleve - University of Namur, Belgium
  • Prof. Katrien Beuls - University of Namur, Belgium
  • Prof. Barbara Hammer - Bielefeld University, Germany
  • Prof. Michèle Gouiffès - Paris-Saclay University, France
  • Dr. Mathieu De Coster - Ghent University, Belgium