1

I have used, hacked and written a few neural network utilities before, and even performed OK in Kaggle competitions using them. My involvement is however just as a part-time hobbyist, and I struggle to keep up with developments in the field.

I would like to experiment with some specific "fun" AI projects using neural networks in the next year or few, and am trying to pick a Python library that will see me through. I am capable of coding low-level constructs - e.g. a Multi-Layer Perceptron in C - but that has ceased to be fun, and I am looking for a library that implements the low-level training support routines already.

This is my wish-list for an ideal NN library . . .

Must haves:

  • Python API. OS support for Mac OSX and Linux. Same Python scripts should work in OSX and Linux with little or no change.

  • CPU and GPU support. It is OK for GPU support to be restricted (e.g. Linux-only), but not CPU support - anything the GPU version can do should be possible using CPU version, albeit slower.

  • At least one RNN implementation (LSTM seems a current favourite and would be fine)

  • Suitable for Kaggle competitions. Older NN libraries, such as FANN, are perfectly good AI workhorses, but do not have enough optimisation features to compete.

  • Suitable for Reinforcement Learning projects - I know of no reason why a library would not be, but perhaps inability to use online learning mode would be a problem.

  • Suitable for Computational Creativity projects. The library needs to support structures that can "reverse sample" e.g. a classifier where you state the class and can generate features from an imagined example that would be in that class. I would like to experiment with generating images, music, architecture/maps for games.

  • Up-to-date algorithms. The field is moving quickly. If the library does not implement dropout, a couple of choices for regularisation, and a couple of decent gradient descent accelerators (e.g. Nesterov, RMSProp, AdaGrad) then it is too far behind the curve.

All the above should be available in current version of library, not promised for future releases (e.g. TensorFlow does not currently have GPU support in Python)

Nice to haves:

  • High-level API. The ideal scenario is that using the neural network is closer to configuration from a menu of options, leaving me free to implement higher-level logic that I am interested in.

  • Cutting-edge algorithms. The more up-to-date the better, but likely the more cutting edge it is, the more specialised it will be and more need for digging beneath the API, so this is a compromise.

  • Ideally there is an example of library's use in a Reinforcement Learning context. E.g. learning a board game from running simulations of play.

  • Ideally there is an example of library's use in a Computational Creativity context. E.g. generating new examples of images from selected class. This should go beyond RNN "hallucination" of structured content.

I have short-listed Keras, Lasagne, and Torch so far, and have been trying to learn a little about them. I like the high-level interface for Keras, and I am reasonably confident that I could use if for Kaggle competitions (and classifier/regression tasks in general), but am worried that it may not be suitable for the other kinds of project I have in mind.

In fact this is true for several promising-looking Python libraries. Also, because I value flexibility and want to use this for multiple purposes, I wonder if I should be learning a lower-level library e.g. Theano or TensorFlow instead (although I note that Python TensorFlow is CPU-only). Simply researching and trying all these could take me weeks of my precious hobby time. I am hoping someone else has recently taken a similar path and can comment on the library they chose and how it meets with some or all of the above expectations.

  • You could try pylearn2 it seems to support fairly new algorithms such as dropout. It also has gpu support for CUDA as far as i know. I do not know if it will work for reinforcement learning but someone may have made something for pylearn2 that can do it but I do not know. – Aiden Grossman Jan 23 '16 at 3:48
  • @AidenGrossman: Thank you for reminding me of that library. Pylearn2 started as a very promising project, but main problem with it is that it has been abandoned: If you read github.com/lisa-lab/pylearn2 there is a warning that it is no longer under development. That is a risk with any library that it loses support after learning it, but I think that puts pylearn2 out of the running for now. – Neil Slater Jan 23 '16 at 9:10
  • Also does remind me, there was a recommendation question here: datascience.stackexchange.com/questions/694/… - but that suffers from asking for unqualified "best" library, the sort of question in fact that softwarerecs is intended to do much better at. – Neil Slater Jan 23 '16 at 9:13
  • On a browse today, this review was an excellent find: github.com/zer0n/deepframeworks/blob/master/README.md – Neil Slater Jan 23 '16 at 9:52

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service, privacy policy and cookie policy

Browse other questions tagged or ask your own question.