Detect changes in a Python library's API

Question

I am a Python software developer relying on multiple third-party libraries with independent release schedules. Although any release requires testing before deployment, it would be helpful to know immediately that an update entails a guaranteed development cost.

In other words, has the API changed? Some libraries follow the semver versioning convention, which signals such a change explicitly (including deeper changes). But others don't. For those, I'd like to inspect the API (i.e. all method signatures) and generate a report on any changes.

I'm looking for:

• a free software Python library
• that can generate a report of class/method headers for an arbitrary module (folder)
• (ideally) perform a diff between two modules
• (ideally) alert on semver-level differences (i.e. breaking changes to the API)

An online equivalent that covered libraries available on PyPI would suffice, but is not ideal because it couldn't handle local/private libraries.

An IDE plugin would suffice, in which case a plugin for PyCharm Community Edition would be ideal, a plugin for any other free software IDE that has Windows support would be a fallback.

Answer 1

You could generate the interface signature for each class/function, which is what WingIDE does with its generate_pi.py file, and compare the signatures but I suspect that you will get way too many false positives from that, e.g.: if a positional parameter is renamed, (one that you never invoke by name), you will still be told that there is a problem.

The smart way to go is to have a decent automatic test suite for your code that exercises as near to 100% of the code functionality as possible - this is always a valuable investment in itself and to use pythons virtualenv, realy good writeup here, to setup a virtual environment for each supported python and to run your tests. You could periodically check the versions of the libraries that you are currently using and in each virtual environment run pip install -U library then run the tests and see if anything is broken. See the python testing tools taxonomy wiki for a list of possible tools and the Hitchhikers Guide for some great advice.

Personally I would say that the time spent writing functional tests is rarely wasted - in the process of writing the tests you often discover issues that have yet to be spotted by others and you ensure that your changes do not break anything or in this case that external changes have broken something.

Plus you could possibly use travis-ci or jenkins to automate this process.