Speech translation for low-resource languages

Testing whether related, higher-resource languages can make speech translation better for languages with little data.

What I’m working on

I’m an undergraduate researcher in WAVLab (Watanabe’s Audio and Voice Lab) at Carnegie Mellon’s Language Technologies Institute, advised by Shinji Watanabe. The lab works on speech recognition, speech translation, spoken language understanding and machine learning for speech.

Most of the world’s languages have very little recorded, transcribed speech, so speech translation for them is poor. My question: can a closely related language with lots of data measurably improve translation quality for a low-resource one?

How

• Running controlled cross-lingual transfer experiments on the Common Voice and FLEURS speech datasets.
• Reproducing multilingual speech-translation baselines in ESPnet and PyTorch, with pretrained multilingual speech models as the point of comparison.