A friend is transcribing a lot of handwritten music scores into Lilypond and I went looking for a better way. It would need 90-95% accuracy so the full process with verification and manual corrections would take less time than manual transcription. Here are some notes from the scan with a flat dotted eighth, a sixteenth, and a quarter:

handwritten sheet music

The originals are handwritten and have idiosyncrasies, such as the tail of the eighth running backwards. He has already transcribed a significant amount of music sheets, I would like to use that to tweak a pre-trained neural network that recognizes music scores and adapt it to this handwriting.

This thread, suggests that Audiveris is the state-of-the-art in open-source Optical Music Recognition. But the results are poor even with an image generated from digital music sheets. Furthermore, Audiveris seems antiquated, as it uses Java SDK 8 when we are now at 13, and uses a set of rules, so I suspect that the user would have to manually correct every eighth instead of the software learning that eighths have the tail going backward.

Is Audiveris the state-of-the-art in Optical Music Recognition, or do we now have solutions with neural networks?

Details on Audiveris

I installed OpenJDK 8 (not Oracle's because of a change in license) with:

brew tap adoptopenjdk/openjdk
brew cask install adoptopenjdk8

and then I downloaded and installed Audiveris:

git clone https://github.com/Audiveris/audiveris.git
cd audiveris
./gradlew build

The binarization step with a handwritten sheet of music was poor but I could pre-process images with edge detection in OpenCV. So I tried with an image from a digital music score at 300 ppi and ran the remaining steps in the software.

The result was OK and the accuracy is over 90%. But for such a clear black-and-white image, I expected 99% accuracy. In the image below, the eighth is mistaken for a quarter and the flat was skipped (I added it manually).

result of Optical Music Recognition on Audiveris

Manual corrections seem to take a long time: I cannot see in the Shape palette how to turn the quarter into an eighth because the eighth is not listed in the top hits of the "basic classifier" and no other menu item lists a shape palette.

This suggests that the results will be much below the threshold with handwritten music sheets and that it would need too many manual corrections.

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service, privacy policy and cookie policy

Browse other questions tagged or ask your own question.