visual-speech-recognition

Visual Speech Recognition for Multiple Languages

Auto-AVSR: Lip-Reading Sentences Project

A PyTorch implementation of the Deep Audio-Visual Speech Recognition paper.

ReVoice-Lip｜唇声再生：Prosody-aware lip-to-speech and voice reconstruction for laryngectomy communication, built on USR 2.0.

"LipNet: End-to-End Sentence-level Lipreading" in PyTorch

Official Pytorch implementation of "Large Language Models are Strong Audio-Visual Speech Recognition Learners" [ICASSP 2025] and "Mitigating Attention Sinks and Massive Activations in Audio-Visual Speech Recognition with LLMs" [ICASSP 2026].

Python toolkit for Visual Speech Recognition

WildVSR

Visual speech recognition with face inputs: code and models for F&G 2020 paper "Can We Read Speech Beyond the Lips? Rethinking RoI Selection for Deep Visual Speech Recognition"

Deep Visual Speech Recognition in arabic words

Official source code for the paper "Tailored Design of Audio-Visual Speech Recognition Models using Branchformers"

MouthMind is a multi-view, sequence-to-sequence framework for visual speech recognition. It directly translates video frames into character sequences, without the need for autoregressive teacher forcing, resulting in robust, emergent learning of spoken words.

Lip2Vec

PyTorch implementation of SwinLip: an efficient Swin Transformer–based visual speech encoder for lip reading

Visual Speech Recognition using deep learing methods

EMOLIPS: TWO-LEVEL APPROACH FOR LIP-READING EMOTIONAL SPEECH

Implementation of "Combining Residual Networks with LSTMs for Lipreading" in Keras and Tensorflow2.0

Speaker-Independent Speech Recognition using Visual Features

Visual Speech Recognition for Spanish

Kaldi recipes that feed frozen self-supervised encoder features (HuBERT, WavLM, MMS, mHuBERT, AV-HuBERT, etc.) to GMM-HMM, TDNN-F chain and flat-start e2e chain models. Multilingual ASR recipes, 3.88% WER on Mini LibriSpeech, and GRID lipreading at 5.85% WER on unseen speakers.