Speech recognition SDK that distinguishes two speakers

Question

I want to transcribe phone conversation with an sdk.
I am developing in java php nodejs and c#.ned (windows or linux) Basically I dont want to use cloud based service, on premise is preferred, but not must.
So need to a library to have the following abilities:

1. must to have: advanced level of speech recognition (good quality)
2. must to have: to recognize each speaker (person1 one person2)
3. nice to have: male female recognition for each person
4. nice to have: punctuation marks.

Thanks

Answer 1

You can try Speechmatics, they have a good service.

For telephone quality conversations it is hard to get a good accuracy without model adaptation, so you probably won't get this software out of box. You'll have to build it. It is not an easy task, you will have to collect some data and combine several software components in order to archive the goal.

First of all you need to figure the language. If you are interested in US English, you are lucky, otherwise you will have to train a model for your language and there are not so much data available and you will have to go through very long acoustic model training process.

For US English you can use Kaldi Fisher models with Kaldi ASR.

You can use kaldi-offline-transcriber to run the whole process, it automates transcription process from beginning to end.

You’ll have to modify kaldi offline transcriber to transcribe callcenter speech.

Transcriber integrates LIUM speaker diarization to separate agent and customer on the call, but it is not very reliable. So you have to reimplement it or tune it.

Since call center calls are quite specific on topic and often have very bad quality you need to build a specific language model to make it easier for transcriber. You’ll have to transcribe at least 10 hours of recordings manually and use this manual data to adapt the language model for Kaldi. Then you’ll have to rebuild the graph.

You also need to run an evaluation process to ensure your accuracy is ok. You need to prepare a standalone testset which you can use to estimate accuracy of transcription. That will help you to evaluate decoder performance properties and improvements you made above.

Next, if you want to run large scale processing you’ll have to setup a job manager and cloud infrastructure for transcription, it’s not an easy task as well.

So as you see all components are available, but it’s probably not that easy to integrate them, you’ll have to learn a lot in the process.