I started to make translation feature

[SilasK]

I share results when it is working..

[Royalphax]

Hi @SilasK,

This would definitely be a killer feature. Transcribing is already great, but having live transcription and translation at the same time would be absolutely incredible!

I’ve been experimenting with a small prototype to integrate live translation using Meta’s NLLB model, which is currently one of the best open-source translation models available. You can find the snippet here – it’s a very minimal implementation that displays the translated text progressively in a Tkinter window (I know, not the most elegant solution, but it works for now).

My long-term vision (proposal) for WhisperLiveKit would be to integrate real-time translation directly into the web server UI, alongside the live transcription. Seeing both side by side in real time would be a game-changer!

I'd really love to see this feature officially implemented. I'm not entirely comfortable modifying the frontend myself, so having it natively supported in the project would be amazing (@QuentinFuxa – btw thanks for your hard work!).

[QuentinFuxa]

Hi @Royalphax
FYI it's already possible to get the translation of the transcription directly via whisper : large-v2 is good at it.

However that's not possible to get both native transcription and translation at the same time, and that would be indeed less GPU costly to run transcription + translation rather than 2 concurrent transcriptions, one in native and the other in translation mode.

I may implement that into WLK, however it seems that NLLB is not the best translation model right now : https://www.reddit.com/r/LocalLLaMA/comments/18kf5pb/translation_model_leaderboard/
Do you have an opinion about it ? I would prefer to build on what is currently the best small translation model

[valdecyralves]

I'm very interested in this live transcription and live translation, even if we get a very low latency, running everything locally. I have a use case to use it in my church and help the interpreters. in this case the preacher speaks in portuguese (primarily), and the translated text would be in english. in very few occasions, we would need to do the opposite. @SilasK have you had any success? thanks in advance

[SilasK]

Hey @valdecyralves could you contact me per mail. (See on my personal page https://silask.github.io/ )

[SilasK]

Hey @valdecyralves could you contact me per mail. (See on my personal page https://silask.github.io/ )

[QuentinFuxa]

Live transcription and translation are now implemented. Use --target-language with the default in-process NLLB backend, or --translation-backend alignatt for the streaming LLM sidecar. The original transcript and translated text are carried together through the server and web UI.

Closing this early proposal as delivered. New backend-specific translation bugs should use the issue form with the source and target languages.