I setup Bazarr using Dr Frankensteins Arrs guide found here. I followed this guide to the letter.

Everything works well so I thought I'd try setting up Whisper as a provider.

I set up a docker project on my Synology and used the YAML config from the Bazarr wiki found here.

Whisper installs successfully and I can access the web service by going to the port manually, but I'm struggling to get Bazarr to actually talk to it. Bazarr keeps giving me connection errors.

Can anybody advise on where I might have gone wrong?

I've got whisper working in unraid using the whisper-asr-webservice container, but I've noticed there is another whisper model 'faster-whisper', that's supposedly 4x faster than the original.. but, I can't seem to get bazarr to talk to it.

Anyone had any luck using bazarr with linuxserver/faster-whisper?

I've noticed Bazarr now includes Whisper to generate AI subtitles. I haven't tested it yet but it seems promising. They say the subs are vastly superiour to e.g. Youtube subs.

Did anyone test it?

Bazarr Ultimate Subtitle Generation Guide

This guide outlines how to set up Bazarr for automatic, perfectly synced subtitle generation using Whisper-ASR and Aeneas. This solution is ideal for users who want precise subtitle synchronization without manually searching for synced subtitles. Note that this process works best for English source videos and subtitles.

Overview

This guide is perfect for users who:

• Want perfectly synced subtitles for their media.

• Don’t mind minor literal translation inaccuracies.

• Prefer an automated pipeline for subtitle generation.

The setup involves leveraging Whisper-ASR for subtitle creation and Aeneas for fine-tuned subtitle synchronization.

Prerequisites

Before you begin, ensure you have the following:

1. Bazarr installed and configured with ARR tools like Sonarr and Radarr.

2. Basic familiarity with ARR workflows.

3. A system capable of running Whisper-ASR (preferably with GPU support for faster processing).

Steps to Set Up

1. Enable Custom Post-Processing

In Bazarr, add the following custom post-processing command in your configuration:

/config/postproces.sh "{{episode}}" "{{subtitles}}" "{{provider}}"

2. Set Language Profile

• The source language for this setup is English.

• Non-English source languages are not currently supported.

3. Integrate Whisper Provider with Bazarr

Modify the postproces.sh script in your Bazarr config directory. The script should handle the following:

• Identify if the subtitle provider is Whisper.

• Run post-processing if provider is not embedded subtitles; otherwise, exit without making changes.

How It Works

Process Flow

1. The Flask app interacts with the video file and subtitle file.

2. It converts the video to .mp3 format and syncs subtitles using Aeneas.

3. After processing:The original subtitle is replaced with the synced version.The intermediate .mp3 file is deleted.

4. The resulting English subtitle file is perfectly synced with the video.

You can then use Bazarr’s translate option to convert these synced subtitles into other languages.

Implementation Details

• The Flask app communicates with Aeneas running on aeneas:5000.

• The script sends the following parameters:series_path: Path to the video file.subtitle_path: Path to the subtitle file.provider: The subtitle provider.

Docker and GPU Support

• If using Whisper-ASR, GPU support is highly recommended for efficiency.

• Modify your Docker Compose file to specify the desired Whisper model or version.

Personal Pipeline Example

1. Input: English video source.

2. Processing:Generate English subtitles using Whisper-ASR.Sync subtitles perfectly with the video using Aeneas.

3. Output:Use Bazarr’s mass-translate feature to generate subtitles in other languages.

Notes & Tips

• This solution is tailored for English source to English subtitle workflows (for now).

• GPU support for Whisper is crucial for faster processing.

• Tutorials for configuring Whisper with Docker Compose can be found on the Bazarr Wiki.

Why Use This Solution?

This project was born out of the frustration of finding high-quality, perfectly synced subtitles. With this setup, you can ensure a seamless experience for all your media.

Happy subtitle syncing!

https://github.com/nik-dev-ops/bazarr-ultimate-subs

UPDATE:
Hey FYI i've been working on custom provider for Bazarr. Downside is it needs to be built from source but container will handle that by it self, this is a sneak peak. ( imgur link below )
https://imgur.com/a/c02ltaT

I would like to share my Bazarr Whisper AI setup on Synology. Hope it helps you.

Make sure Bazarr setup is correct

Before we begin, one of the reason you want AI subtitles is because you are not getting subtitles from your providers such as opensubtitles.com. Bazarr works in funny ways and may be buggy at times, but what we can do is make sure we are configuring correctly.

From Bazarr logs, I am only getting subtitles from opensubtitlescom and Gestdown, so I would recommend these two. I only use English ones so if you use other languages you would need to check your logs.

Opensubtitles.com

To use opensubtitles.com in Bazarr you would need VIP. It's mentioned in numerous forums. If you say it works without VIP or login, that's fine. I am not going to argue. It's $20/year I am ok to pay to support them. Just remember to check your Bazarr logs.

For opensubtitle provider configuration, make sure you use your username not email, your password not your token, do not use hash and enable ai subtitles.

For your language settings keep it simple, I only have English, you can have other languages. Deep analyze media, enable default settings for series and movies.

For Subtitle settings use Embedded subtitles, ffprobe, important: enable Upgrading subtitles and set 30 days to go back in history to upgrade and enable upgrade manually downloaded or translated subtitles. Most common mistake is setting days too low and Bazarr gives up before good subtitles are available. Do not enable Adaptive Searching.

For Sonarr and Radarr keep the minimum Score to 0. sometimes opensubtitles may return 0 even when the true score is 90+.

For Scheduler, Upgrade Previously Downloaded Subtitles to every 6 hours. Same for missing series and movies. Sometimes opensubtitles timeout. keeping it 6 hours will retry and also picking up latest subtitles faster.

Lastly, go to Wanted and search all, to download any missing subtitles from OpenSubtitles.

Now we have all the possible subtitles from opensubtitles. the rest we need Whisper AI.

subgen

subgen is Whisper AI but many generations ahead. First of all, it's using faster-whisper, not just whisper, and on top it uses stable-ts, third it support GPU acceleration, and fourth, but not least, it just works with Bazarr. So far this is the best Whisper AI I found.

I recommend to use Nvidia card on Synology to make use of Nvidia AI. with my T400 4GB I get 24-27sec/s transcribe performance. If you are interested check out my post https://www.reddit.com/r/synology/comments/16vl38e/guide_how_to_add_a_gpu_to_synology_ds1820/

If you want to use your NVidia GPU then you need to run the container from command line, here is my run.sh.

#!/bin/bash
docker run --runtime=nvidia --gpus all -e NVIDIA_DRIVER_CAPABILITIES=all -e TRANSCRIBE_DEVICE=gpu -e WHISPER_MODEL="base" -e UPDATE=True -e DEBUG=False -d --name=subgen -p 9000:9000 -v /volume1/nas/Media:/media --restart unless-stopped mccloud/subgen

After running, open your plex address and port 9000 to see the GUI, don't change anything, because Bazarr will send queries to it, the settings in GUI is only for if you want to run something standalone. If you want to know all the options, check out https://github.com/McCloudS/subgen

Whisper AI can only translate to English, it has many models: tiny, base, small, medium and large. From my experience, base is good enough. Also you can choose transcribe only (base.en) or translate and transcribe (base). I choose base because I also watch Anime and Korean shows. For more information check out https://github.com/openai/whisper

To monitor subgen, run the docker logs in terminal

docker logs -f subgen

Go back to Bazarr, add the Whisper AI provider, use subgen endpoint, for me it's http://192.168.2.56:9000 connection timout 3600, transctiption timeout 3600, logging level DEBUG, click Test Connection, you should see subgen version number, click save.

Now go to Wanted and click on any, it should trigger subgen. You can check from the docker log if it's running. Once confirmed, you may just search all and go to bed, with T400 you are looking at 2-3 mins per episode. Eventually all wanted will be cleared. If good you can press ctrl-c in terminal to stop seeing the docker logs. (or you can keep staring and admiring the speed :) ).

Is it any good?

I have been using it for a little over 48 hours and it generated 1150 subtitles in the meantime.

Having tried Spanish, English, and French shows. I can say that they are about 90-95% accurate, which beats no subs at all for me that has hearing issues.

Complete info here!

An example of the delay between generations:

Been trying whisper a bit. Running whisper asr docker on my unraid server as well as bazarr docker. Using base model.

I just setup whisper provider and it seems to work.

I however have some questions. Are there supposed to be some way to see the progress of whisper?

Now there is a spinning download notification. Then I get some weird error message -<h1>504 Gateway Time-out</h1></center> <hr><center>openresty</center> </body> </html> <!-- a padding to disable MSIE and Chrome friendly error page -->

However it keeps on working in the background. I have 100% cpu usage for two minutes or so. When it settles I actually have a new subtitle.

My cpu is quite slow i3-8100 but it seems to work.

Are there no way to track status of the whisper process or is it something weird with my setup?

Automate Subtitle Generation with Your Custom Script

Automate Subtitle Generation with Your Custom Script

If Bazarr doesn't find the subtitles you need, our custom solution will take care of it for you. This guide requires you to have an Nvidia GPU to leverage AI-driven transcription via Whisper AI. Here's how you can set it up:

Step 1: Clone the Repository

Start by cloning the repository from GitHub to get the necessary files.

git clone https://github.com/anubissbe/submaker.git

Step 2: Install Dependencies

Change to the project directory and install the required Python packages.

cd submaker
pip3 install -r requirements.txt

Step 3: Set Up Whisper Server

Ensure the Whisper AI server is running:

python3 whisper_server.py

Step 4: Run the Subtitle Script

Once everything is set up, you can run the script to start processing your video files:

python3 addsubs.py

Customizing File Locations and Languages

If you need to customize the locations of the files or change the languages for the subtitles:

1. File Location: Modify the base_directory variable in the addsubs.py script to the path where your video files are stored.

2. Languages: To change the languages for subtitles, edit the languages_needed list in the addsubs.py script. Add or remove language codes as per your requirements. The default setup includes English ('en'), Arabic ('ar'), and Dutch ('nl').

Conclusion

This solution extends Bazarr's functionality, ensuring you always have subtitles in the languages you need. By leveraging advanced AI models and custom scripts, managing your media library's subtitles becomes effortless.

I have a lot of files with no subs that I'm not getting via any of my configured providers. So whisper sounds like it will take care of everything until something better does show up. But the only way I can get transcribed subs from whisper is to search manually and select whisper. Is there something that I need to enable to make this work automatically?

Hi all, Bazarr is an AMAZING piece of software.

That being said, I've gotten lots of media where it seems silences are TOO long, leading to Whisper hallucinating text where there is silence.

I read a VAD might take care of this, but I'm not sure how to set up a VAD with Bazarr (running as Windows service)/whisper-asr-webservice (running on port 9020 in docker using medium).

Has anyone done this; if so, can you maybe help break down the process?

Note: I am using Docker on Windows 11 Pro. I will NOT switch this thing to Linux under any circumstances, due to the other software running on this machine.

Is it worth $20/year? Most of my subtitles are found, but it is really hard to find some in Ukrainian Russian, and opensubtitles.org mostly have them, opensubtitles.com is always on 6 hour cooldown. Here are my providers, what other you would recommend?

Hi All,

I thought i'd chuck this up there in case it helps anyone.

I setup and was using the whisper-asr-webservice docker on Unraid. Everything connected fine, it'd process a couple files but then inevitably it would throw a connection error in Bazarr and want to wait another 24 hours.

After scanning the log i noticed it was trying to process multiple files at a time. It would request one, start processing and then request another. I am unsure if it is capabale of doing multiple at once (I assume it is), but perhaps it was just trying too many for my system (i5 13500).

The solution that has worked thus far was changing the "connection/response timeout in seconds" under WhisperAI in providers from the default 5 seconds up to 120 seconds (2 minutes).

So far it has processed the most in a row that it has without any issues, and the logs show it (finally) starting and completing before going for another file with no timeout errors or connections aborted etc.

EDIT: Ran into an issue after a few hours and it seems to be due to a movie being transcribed which took ~10 minutes, so it tried a couple of times and failed so threw the error again and stopped. I've now upped to 12 minutes as i believe that should prevent the issue again, but it highlights so far that the frequency needs to be changed based on the largest media you have. I hope this helps someone having those connection issues.

When it finishes it seems to move straight on to the next file, so the time between connection attempts looks to be fine to be set for a long time.

Reposted from a few hours ago as i made a typo in the subject and couldn't figure out how to change it.

Hello, I’m configuring Whisperai as a provider on bazarr and I cannot get it to connect. I can type in the local ip address with port 9000 and I can see the interface. How can I fix the error “ConnectionError” on bazarr?

Hey, I've set up Bazarr and Whisper, they actually work pretty well and I'm loving it, the problem is that my mother tonge isn't english, and my family doesn's speak it. That's why i made a profile with english and spanish subs, I thought that Whisper would generate spanish subs even for shows that aren't in english.

In the logs I see the TASK transcribe, but never translate.

I also tried to manually request the spanish subs, but it just said "no es subtitles found".

I have also noticed that when the movie/show has more than one audio stream, bazarr logs state that only english stream will be used.

Is there a way to make whisper generate spanish subs from non-spanish movies/shows?

Thanks in advance!!