I'm looking for a web application (either the third-party hosted or hosted by myself) that can:

  • load and display a text
  • allow the user to mark a word or a group of words as belonging to a predefined class
  • record for each such mark which user did it
  • free

If possible:

  • open-source
  • RDBMS back-end for storing the data

Here is my use case to clarify the need: I have 1000 news articles (plain text, written in English), I need to mark among those articles all words or a group of words that express a positive or negative sentiment (i.e. I have 2 classes). I'm going to use Amazon Turk/oDesk/etc. to crowdsource the task, hence the choice for a web interface. I will ask each worker to annotate a subset of those 1000 news articles. Each news articles will be annotated by 3 workers to ensure the annotations' quality.


I'm not sure how easy it would be to embed into Amazon Turk (it's a standalone webapp), but you might want to try brat (brat rapid annotation tool).

  • open source (MIT License)
  • Among other types of annotations, it supports annotating a span of text with a specific (predefined) class which sounds like it might fit your needs
  • supports multiple users (but again, some work might be necessary to tie this into Turk)
  • online demo, examples, GitHub repo

The development of brat is somehow stalled.

As at the beginning of 2019, I would highly recommend INCEpTION https://inception-project.github.io/

INCEpTION has builtin active learning support and a lot more.

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service, privacy policy and cookie policy

Not the answer you're looking for? Browse other questions tagged or ask your own question.