We offer a fully funded PhD Scholarship on Sign Language Recognition with Deep Learning.

Research topic description

Sign languages are the primary form of communication for many deaf or hard of hearing people. They are also entirely separate languages, which are regional and have local variations, and generally have very little relation to any spoken languages in the same region. For example, Flemish Sign Language (used in Flanders) is quite different from Dutch Sign Language (used in the Netherlands), but very similar to the French Belgian Sign Language used in Wallonia (the French speaking part of Belgium). Without any connection between written languages and the natural language structure of their native sign language, and without the intermediate step of the (loose) connection between sounds and letter combinations, learning to read and write is much more challenging for people who are born deaf, than it is for hearing people to learn a different spoken/written language.

However, only very few hearing people can use or understand a sign language, even at a basic level. This is why deaf people often rely on human sign language interpreters to communicate with hearing people and vice versa. Because an interpreter needs to be reserved and paid for, they are only available at specific times, for planned interaction with hearing people (e.g., a doctor’s visit, an administrative appointment, a job meeting, …). For unplanned or spontaneous interactions, the resulting communication barrier often restricts interactions to the absolute minimum, e.g. by using pointing and other gestures, facial expression or writing when possible.

One way to help mitigate the communication barrier between hearing and deaf people would be the use of machine learning techniques. In UGent’s AIRO team, we focus on automatically extracting information from sign language videos into a robust representation (embedding). This can then be used in two ways: either as input for transformer-based multilingual translation models, or to develop tools that give automatic learning support to sign language learners.

Either way, the main challenge is the fact that, compared to spoken languages, very little sign language data is available and only a fraction of that has been labeled. This means that any technique we develop needs to be data-efficient, and very robust to overfitting. It should also maximally use transfer learning from related data and/or tasks, as well as unsupervised (or self-supervised) learning to benefit from unlabeled data.

This research fits into the European project SignON, in which a large consortium of partners from academia, stakeholders and industry are developing a first prototype of an app that can translate in any direction between a number of European sign languages and a number of European spoken languages. Each step in this project is taken in close interaction with the user community, represented by the European Union of the Deaf as well as several regional representatives (e.g., in Flanders, VGTC – Centrum voor Vlaamse Gebarentaal). Their input is used for the identification of use cases and for the evaluation of our results.

Imec-IDLab-UGent

IDLab is a research group of UGent, as well as a core research group of imec. IDLab performs fundamental and applied research on data science and internet technology, and counts over 300 researchers. Our major research areas are machine learning and data mining; semantic intelligence; multimedia processing; distributed intelligence for IoT; cloud and big data infrastructures; wireless and fixed networking; electromagnetics, RF and high-speed circuits and systems. The AIRO (“AI and robotics”) team of IDLab has been studying various kinds of neural networks for more than 20 years. Our focus is on applications in robotics and in human-AI interaction. AIRO has been at the forefront of deep learning research ever since it became popular a decade ago. Illustrative of this success are an excellent track record at Kaggle competitions. thanks to a consistent focus on innovative ideas that are somewhat outside the mainstream “AI-hype”, many of our former PhD students are now working at top commercial research labs, such as Deepmind or Google Brain, or hold strategic senior R&D positions in companies.

Requirements

Degree and background

  • You have the degree of Master of Science, preferably in Computer Science (engineering), ICT or Informatics, or natural language processing
  • Your degree must be equivalent to 4 or 5 years of studies (bachelor + master) in the European Union, it must include a master’s thesis, and you must have excellent grades, at least for courses related to this position
  • You did not receive a Flemish PhD scholarship before
  • You must have a theoretical background in and experience with machine learning and state-of-the-art deep learning models (from master-level academic course grades and hands-on work)
  • We particularly encourage candidates who are deaf or hard of hearing to apply. For other candidates, a personal motivation (e.g. a close connection with deaf people) is an asset.

Language skills

  • If you are a hearing person: you are capable of spontaneous (unprepared) conversation about scientific and non-scientific topics in written and spoken English, you are willing to learn at least the basics of Flemish sign language
  • If you are a deaf person: you can communicate in Flemish sign language or international sign and are sufficiently proficient in written English to be able to follow the state-of-the-art in the relevant domains and publish your results in high impact journals and at conferences

Other skills

  • You have an applied and pragmatic mindset and are motivated to perform user-driven research in close interaction with the deaf community
  • You are interested in and motivated by the research topic, as well as in obtaining a PhD degree.
  • You are creative, have excellent analytical and problem solving skills, and can work independently as well as in team.
  • You have good communication skills, you have an open mind and a multi-disciplinary attitude.

Our offer

We offer a fully funded PhD scholarship for a maximal period of 4 years. Your initial contract will be for one year. In the case of a positive progress evaluation, you will then receive a second contract for the remaining three years.

The PhD research has fundamental and innovative aspects, but also a clear application target. You will research and develop data-efficient to extract the information from sign language video that is relevant for understanding and translation the sign language and for identifying possible weaknesses in the execution quality. You will use state-of-the art techniques from machine learning, deep learning and language technology. You will be encouraged to publish and present your work at project meetings and international conferences, or to attend useful summer schools.

You will join AIRO, a young and enthusiastic team of around 30 researchers, post-docs and professors. You will also collaborate with our many Flemish and European project partners: experts in various aspects of language technology, signing avatar generation, linguists who specialize in sign language, and the Flemish and European deaf communities.

Interested?

Apply with the following documents (Incomplete applications will not be considered!)

  • Motivation letter: explain why you are interested in and the right candidate for this particular project! Standard motivation letters that have no relation to the research topic of this vacancy will not be considered!
  • Detailed academic results (including course list, grades and percentile)
  • Pdf of Master’s thesis and relevant publications (if any), possibly a description of project work that is relevant to the vacancy
  • English proficiency scores for candidates with degrees outside the EU
  • Two reference contacts

For any questions, contact prof. dr. ir. Joni Dambre (Joni.Dambre@UGent.be).

After the first screening, good candidates will be invited to perform a skills test (a deep learning assignment).

A selection of candidates with good results on this will be invited for an interview (in-person or online), in which they will be asked to present themselves and a scientific topic of our choice and their skills will be assessed based on open and unprepared discussion..

Timeline and closing date

This PhD position is available as of September 1st, 2022.

You can apply until Sunday, July 20th by sending an email with all required documents attached to:

Prof. Joni Dambre – joni.dambre@ugent.be

Incomplete applications or generic applications that have no clear relation to this specific research position will be ignored.

Very early applicants who are top candidates (that meet the requirements above) may receive a reaction before the end of June. There will be no reactions between July 1st and July 21st.

After processing all applications, you will receive either a negative answer, or an invitation to the rest of the procedure, which will consider of an interview, in which you will present a research-related topic of our choice and your communication skills and technical background will be assessed during a spontaneous discussion. We may also invite you to participate in a skills assessment in the form of a coding assignment related to the field.