OpenAI's Whisper (off-line, MIT license, Python 3.9, CLI) yields some highly accurate transcription. To use (tested on Ubuntu 20.04 x64 LTS):
conda create -y --name whisperpy39 python==3.9
conda activate whisperpy39
pip install git+https://github.com/openai/whisper.git
sudo apt update && sudo apt install ffmpeg
whisper recording.wav
whisper recording.wav --model large
If using an Nvidia 3090 GPU, add the following after conda activate whisperpy39
pip install -f https://download.pytorch.org/whl/torch_stable.html
conda install pytorch==1.10.1 torchvision torchaudio cudatoolkit=11.0 -c pytorch
it can be used as a Python lib, e.g.:
import whisper
model = whisper.load_model("base")
# load audio and pad/trim it to fit 30 seconds
audio = whisper.load_audio("audio.mp3")
audio = whisper.pad_or_trim(audio)
# make log-Mel spectrogram and move to the same device as the model
mel = whisper.log_mel_spectrogram(audio).to(model.device)
# detect the spoken language
_, probs = model.detect_language(mel)
print(f"Detected language: {max(probs, key=probs.get)}")
# decode the audio
options = whisper.DecodingOptions()
result = whisper.decode(model, mel, options)
# print the recognized text
print(result.text)
Performance info below.
Model inference time:
Size |
Parameters |
English-only model |
Multilingual model |
Required VRAM |
Relative speed |
tiny |
39 M |
tiny.en |
tiny |
~1 GB |
~32x |
base |
74 M |
base.en |
base |
~1 GB |
~16x |
small |
244 M |
small.en |
small |
~2 GB |
~6x |
medium |
769 M |
medium.en |
medium |
~5 GB |
~2x |
large |
1550 M |
N/A |
large |
~10 GB |
1x |
WER on several corpus from https://cdn.openai.com/papers/whisper.pdf:

WER on several languages from https://github.com/openai/whisper/blob/main/language-breakdown.svg:
