Automatic python packaging: package code with a python distribution?

Question

This question is not about freezing / compilation tools like cx_Freeze, py2exe, pyinstaller or Nuitka.

Question

I want to distribute a python app myapp as python package(s), packaged together with a python distribution and an installation script. I have used following workflow:

1. Write code and a setup.py
2. Package the code and all the dependencies with pip wheel . into wheels.
3. Create zip file containing the wheels and a portable CPython distribution (WinPython)
4. Create a INSTALL.bat
5. Deliver to end user INSTALL.bat + package.zip.
6. Running INSTALL.bat will unzip the package, create virtual environment, install all the wheels, create UPDATE.bat and a desktop shortcut.

The steps 2-4 have been automated with a build.py. All good so long until the project grew and I had to separate mylib code from myapp code and now I have two interlinked packages: mylib and myapp. The build.py does not work after this refactoring. There is also a risk I have to refactor again into mylib1 and mylib2, for example.

Before I'm going to fix the broken build process, I would like to ask for the python community: Is there already a tool that would do automatic packaging of python package(s) + a python distribution? Am I reinventing the wheel?

Requirements

• Package code with all dependencies (python packages & assets) with self-contained, python distribution (or alternatively: can install a specific python version form internet)
• Installation should be easy for end user. Need to manually modify Path or create / activate venvs is absolute no-go.
• Installation does not mess up any existing python installations on end users' computer.
• Updating should be easy for end user.
• Works on Windows
• Optional: Creates shortcut for end user

Target as pictures

Q&A

1) Why not cx_Freeze, py2exe, pyinstaller, Nuitka, etc..?

These tools have their own users and use cases. My project is huge with lot's of dependencies and external files to include, and I used numerous hours to get any of them working without any signs of light at the end of the tunnel. Also, updating the app should be easy. Moreover, I want to stick to just running the app with python.exe since everything just works™ out of the box. It would be also easier for me to debug the app if/when anything goes wrong (even though, there is quite heavy logging included).

2) Why not just distribute wheels?

I can not distribute only .whl files, since the end users are not necessarily IT-experts. The app might only work on the CPython distributable I am using (Python 3.7.7 32-bit), and using virtual envs etc. is really too much asked for the end users. Clicking a shortcut is the only thing I can assume they can do for launching the program.

Answer 1

A few tips that might help:

• You can use the "embedded" python distribution to provide a local, non-system, python for your code. (See the downloads section for each python release for the embedded distributions download). You will need to provide one or more compiled stub executables that start the embedded interpreter and load/run your python code - the embedded distribution includes a python.exe & pythonw.exe. This may work better than using Portable Python which was having maintenance issues last time that I looked.
• If you modify your sys.path value to point to the zip file then most libraries can be run from inside of the zip file without unpacking it.
• As an alternative if you do your development in a venv that you have created with the --copies option and not the --system-site-packages then zipping that venv should get everything that you need.