Build tool for Python projects

Question

I come from Java/Scala background, and I'm looking for an adequate replacement for Maven/SBT that I can use for Python projects.

I am looking for a tool that helps me to

• Run unit and integration tests
• Compute test coverage
• Create "executable artifacts" (whatever this means in Python: there are no JAR's?)
• Since Python isn't really compiled and statically type checked, I will probably sooner or later want to run additional linters (any recommendations here?)
• Generate Reports
• Generate Documentation
• Distribute reusable packages in a way that they can be easily used by pip
• Run the built application
• Work with Python 3.6.x
• Be available for all major linux distributions

So far, I've tried using PyBuilder, because it comes up as first search hit on Google. It worked with the simple examples described in the tutorial, but even basic things such as running the built application turned out surprisingly counter-intuitive and complicated. Moreover, so far I couldn't make test coverage work for projects that are split up into multiple interdependent packages. I would like to hear whether there are any other recommendations before I invest more effort into making it work with PyBuilder.

Maybe I misunderstood something, and I'm wishing for the wrong things? Does it work somehow completely differently in Python? I'm currently struggling even with a tiny project that consists of maybe a dozen of modules and is <2kSLOC: without support by a proper build tool, everything starts to fall apart very quickly.

Answer 1

Have you looked into setuptools yet? That's the de-facto standard method for doing most of what you want:

• Run unit and integration tests

  Handled with minimal effort, and can be integrated into the build process directly.

• Compute test coverage

  Not sure about this one, but if it isn't handled by setuptools, it can probably be done easily by a plugin or as part of a custom step.

• Create "executable artifacts" (whatever this means in Python: there are no JAR's?)

  Python doesn't really have a concept of executables. There are a couple of options out tere that will bundle your application into an executable, but they're mostly for Windows, and are generally overly complicated in most cases. Python does have a concept of 'compiled' code, but it's only really used to speed up loading and execution of modules (if the interpreter finds a valid .pyc file alongside the .py file for the module, it will load that instead and avoid having to translate from the source to the internal bytecode representation), and setuptools does support pre-compiling modules.

• Since Python isn't really compiled and statically type checked, I will probably sooner or later want to run additional linters (any recommendations here?).

  Use a third party tool, and probably integrate it into your source control and not your build. Pylint is the most widely used option I know of for this.

• Generate Reports

  Not covered by setuptools, but probably not hard to do as a plugin or custom step. Some of the stuff offered by the above mentioned pylint may be of interest here (for example, it can generate UML diagrams from Python code).

• Generate Documentation

  May or may not be covered by setuptools inherently, but easy to add as part of a custom step. If you just care about the embedded documentation that is the standard for most Python modules, it's handled automatically and you don't need to do anything. If you want it to be accessed through means other than pydoc, you will have to look at using an external tool, though there isn't really any one standard for that.

• Distribute reusable packages in a way that they can be easily used by pip

  Setuptools together with the actual pip application covers everything you need to publish stuff to PyPI or bundle things as your own module (in fact, if you use setuptools, you can just distribute the source directory directly).

• Run the built application

  Easy to do with Python without needing any special framework, just make sure your sanely coding your imports.

• Work with Python 3.6.x

  Setuptools works with all Python versions since at least 2.7.

• Be available for all major linux distributions

  Because it's so widely used, setuptools is packaged in pretty much every major distribution natively, but it's also trivial to just install using pip if your distro doesn't have it.