I'm looking for a library in python that has fairly accurate speech recognition. It would preferably return a string indicating what was said, so that I can work with the string to do other things. Thanks!!!

I looked at this related question, but I don't think we're asking the same thing.


2 Answers 2


You can use CMU Sphinx:

  • free and open source
  • recognizer library written in C but provides Python bindings
  • often mentioned as one of the best open source speech recognition engines

When looking for a speech recognition software for Linux a while ago, I was told that CMU Sphinx' accuracy is significantly lower than Dragon (I'd be curious if anyone here has a benchmark Sphinx vs Dragon). However, if your voice recordings are in a well restricted domain, you might be able to train CMU Sphinx well enough.


OpenAI's Whisper (off-line, MIT license, Python 3.9, CLI) yields some highly accurate transcription. To use (tested on Ubuntu 20.04 x64 LTS):

conda create -y --name whisperpy39 python==3.9
conda activate whisperpy39
pip install git+https://github.com/openai/whisper.git 
sudo apt update && sudo apt install ffmpeg
whisper recording.wav
whisper recording.wav --model large

If using an Nvidia 3090 GPU, add the following after conda activate whisperpy39

pip install -f https://download.pytorch.org/whl/torch_stable.html
conda install pytorch==1.10.1 torchvision torchaudio cudatoolkit=11.0 -c pytorch

it can be used as a Python lib, e.g.:

import whisper

model = whisper.load_model("base")

# load audio and pad/trim it to fit 30 seconds
audio = whisper.load_audio("audio.mp3")
audio = whisper.pad_or_trim(audio)

# make log-Mel spectrogram and move to the same device as the model
mel = whisper.log_mel_spectrogram(audio).to(model.device)

# detect the spoken language
_, probs = model.detect_language(mel)
print(f"Detected language: {max(probs, key=probs.get)}")

# decode the audio
options = whisper.DecodingOptions()
result = whisper.decode(model, mel, options)

# print the recognized text

Performance info below.

Model inference time:

Size Parameters English-only model Multilingual model Required VRAM Relative speed
tiny 39 M tiny.en tiny ~1 GB ~32x
base 74 M base.en base ~1 GB ~16x
small 244 M small.en small ~2 GB ~6x
medium 769 M medium.en medium ~5 GB ~2x
large 1550 M N/A large ~10 GB 1x

WER on several corpus from https://cdn.openai.com/papers/whisper.pdf:

enter image description here

WER on several languages from https://github.com/openai/whisper/blob/main/language-breakdown.svg:

enter image description here

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service and acknowledge you have read our privacy policy.

Not the answer you're looking for? Browse other questions tagged or ask your own question.