How – or with what specific software – do they create these massive labeled image databases, such as the ImageNet?

I need a very simple web application that works on top of some (mainly unlabeled) image collection, and lets users assign the images to a certain fixed (small) number of classes.

LabelMe is pretty close to what I want, but I really need something much simpler: the users should classify each full image to some category, rather than accurately annotate multiple regions of each image (yes there may be ambiguities, but that's another story). So this related question is not really a duplicate.

It should be as easy as possible for the users to go through and label hundreds or thousands of images (with a minimum number of clicks and such).

The software I'm looking for should:

  • Offer images to the user one by one and let the user select the correct class for each image (out of something like 3 choices);
  • Be open source and free to use and extend (GPL is fine);
  • Be available to install on my own server;
  • Be really, really simple to use (especially for the end user, but hopefully also for the admin, i.e. myself);
  • Not impose any specific pattern recognition methods (= it should focus on the manual labeling work only).

Also, it would be really nice if it did:

  • Work well on a mobile device (so that you could continue your categorization task while sitting in a bus, for instance);
  • Work on top of something like MongoDB (but flat files are fine too – I can put my data in almost any format, really);
  • Have at least some sort of user/login management;
  • Have the possibility to suggest some class to the users (based on a trained classifier), so that the user can either accept the suggestion or reject it (in which case the user could be asked to select the correct classification);
  • Offer the user some simple filtering capabilities (e.g., based on automatically assigned tags / date / even geolocation);
  • Readily have means to crowd-source the job, e.g. via Amazon Mechanical Turk.

Does something like this already exist (even partially), or do I need to roll my own? I'm not a great web app developer, so I'd rather not – but if there's no other way, then so be it. Anyone interested in participating in the effort (in, say, Node.js)?

  • I would suggest that it would be really useful to group visually similar images and either suggest the same classification as applied to others in the group or possibly better once one was classified offered batches of 10? to be given the same classification with individual and all tick boxes, ticked by default. – Steve Barnes Jul 14 '15 at 12:43
  • Regarding "Does something like this already exist (even partially), or do I need to roll my own?" - at best on this site you will get a comment about likelihood of it existing already. But often you will simply get no answers. Probably you will want to recruit to any project of your own elsewhere, and not on this site. Of course once it is made, if you decide to release it, that software or library could be an answer here . . . – Neil Slater Jul 17 '15 at 10:04
  • I have implemented my own, but at least for now it ended up being too application-specific. But maybe it could be generalized later... – Reunanen Aug 2 '15 at 11:48
  • You can try webbased imglab. It'll let you label images using rectangle for machine learning. You can also point landmark points or use external services to reduce your effort. – Amit Kumar Gupta Jan 23 '18 at 2:04

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service, privacy policy and cookie policy

Browse other questions tagged or ask your own question.