Portable Python scripts with Docker and Make
October 21, 2022
Sometimes you need to share a script you quickly whipped up for some
one-off task, when your coworker has a very similar task where they
could use what you have with minor modifications. Getting the script
working on their machine is not always trivial: You may have different
Python versions and you most definitely have different
globally-installed dependencies. The first problem can be solved with
pyenv, the latter with virtual environments, but getting up
and running can still be a hassle that feels like an unnecessary detour.
And it is. It's the 2020s, we may not have flying cars but we have
Here I show how to create a setup that is as easy to get up and running as (1) cloning the repository, (2) running the script. That's it, assuming you have Docker and usual Unix/Linux command-line tools at your fingertips.
You can check out the full example at Github.
All this may seem like a bit too much overhead for a one-off script.
But if you make a
cookiecutter template out of it and use
that every time you start a new one-off script, that overhead is
Step 1: Create the Dockerfile
The main idea here is to create a Docker image with the dependencies of the script, but exclude the actual script. We'll mount that as a volume when running the container. The advantage of this approach is that we can iteratively improve the script without the overhead of building the Docker image unless the dependencies change.
For the package management we could use plain
conda, but I've chosen
pipenv in this example, purely because that's the tool I've
been mostly working with lately.
There is no
ENTRYPOINT in this
Dockerfile. We'll specify the exact Python command in the
docker run command. That will make it easy to pass along
ENV PYTHONDONTWRITEBYTECODE 1
ENV PYTHONUNBUFFERED 1
RUN pip install pipenv && apt-get update && apt-get install -y --no-install-recommends gcc
COPY Pipfile Pipfile.lock .
RUN PIPENV_VENV_IN_PROJECT=1 pipenv install
Step 2: Create the Makefile
How can we avoid rebuilding the image unless necessary? There's a
classic tool for that,
Makefile, which describes a dependency graph of
files that depend on the other files, plus instructions on how to
produce those other files.
There is one problem, however: Docker images are not created as files
in the working directory, but as images in the local Docker repository,
make cannot by default know if the image is older than
the files it depends on. The trick to solve this is to have a file with
a last-modified time of last Docker build. I've chosen to call that file
.build here. We should also put that file in the
# Build with `make image_name=<image-name> .build
.build: Dockerfile Pipfile Pipfile.lock
@echo 'Building the Docker image...'
docker build . -f Dockerfile -t @touch .build
Step 3: Create the run script
Now we can bring it all together with a tiny shell script (which I
make -s image_name=$DOCKER_IMAGE_NAME .build
docker run --rm -i -v "$(pwd)/src":/src -t $DOCKER_IMAGE_NAME python main.py "$@"
The script first runs
make in the silent mode, and
make builds the image if either the
file is missing or has a last-modified timestamp older than any of its
Then the script runs the Python script in the Docker container using
that image. We mount the
src/ directory, where I'm placing
the actual Python script as
main.py, and pass all the
command-line arguments onwards to it.
Step 4: The rest
Now that we've created
run script, we only have to add
a couple of necessary files before we can try this setup.
We need the
src/ directory with a
script -- That will be the script we wanted to make portable.
Because we chose to use
pipenv for dependencies, we
should create a
can be done easily by just running
pipenv install in the
project directory, assuming you have it installed system-wide.
In the end, the overall file structure should look like this:
| +- main.py