Portable Python scripts with Docker and Make

October 21, 2022

Sometimes you need to share a script you quickly whipped up for some one-off task, when your coworker has a very similar task where they could use what you have with minor modifications. Getting the script working on their machine is not always trivial: You may have different Python versions and you most definitely have different globally-installed dependencies. The first problem can be solved with pyenv, the latter with virtual environments, but getting up and running can still be a hassle that feels like an unnecessary detour. And it is. It's the 2020s, we may not have flying cars but we have Docker.

Here I show how to create a setup that is as easy to get up and running as (1) cloning the repository, (2) running the script. That's it, assuming you have Docker and usual Unix/Linux command-line tools at your fingertips.

You can check out the full example at Github.

All this may seem like a bit too much overhead for a one-off script. But if you make a cookiecutter template out of it and use that every time you start a new one-off script, that overhead is negligible.

Step 1: Create the Dockerfile

The main idea here is to create a Docker image with the dependencies of the script, but exclude the actual script. We'll mount that as a volume when running the container. The advantage of this approach is that we can iteratively improve the script without the overhead of building the Docker image unless the dependencies change.

For the package management we could use plain pip, or poetry, or conda, but I've chosen pipenv in this example, purely because that's the tool I've been mostly working with lately.

There is no CMD or ENTRYPOINT in this Dockerfile. We'll specify the exact Python command in the docker run command. That will make it easy to pass along command-line parameters.

FROM python:3.10-slim

ENV PYTHONDONTWRITEBYTECODE 1
ENV PYTHONUNBUFFERED 1

RUN pip install pipenv && apt-get update && apt-get install -y --no-install-recommends gcc
COPY Pipfile Pipfile.lock .
RUN PIPENV_VENV_IN_PROJECT=1 pipenv install

WORKDIR /src

Step 2: Create the Makefile

How can we avoid rebuilding the image unless necessary? There's a classic tool for that, make. The make tool needs a Makefile, which describes a dependency graph of files that depend on the other files, plus instructions on how to produce those other files.

There is one problem, however: Docker images are not created as files in the working directory, but as images in the local Docker repository, so make cannot by default know if the image is older than the files it depends on. The trick to solve this is to have a file with a last-modified time of last Docker build. I've chosen to call that file .build here. We should also put that file in the .gitignore.

# Build with `make image_name=<image-name> .build
.build: Dockerfile Pipfile Pipfile.lock
    @echo 'Building the Docker image...'
    docker build . -f Dockerfile -t $(image_name)
    @touch .build

Step 3: Create the run script

Now we can bring it all together with a tiny shell script (which I named simply run):

#!/bin/bash
set -e

DOCKER_IMAGE_NAME=dockerized-python-example

make -s image_name=$DOCKER_IMAGE_NAME .build
docker run --rm -i -v "$(pwd)/src":/src -t $DOCKER_IMAGE_NAME python main.py "$@"

The script first runs make in the silent mode, and make builds the image if either the .build file is missing or has a last-modified timestamp older than any of its three dependencies.

Then the script runs the Python script in the Docker container using that image. We mount the src/ directory, where I'm placing the actual Python script as main.py, and pass all the command-line arguments onwards to it.

Step 4: The rest

Now that we've created Dockerfile, Makefile, and run script, we only have to add a couple of necessary files before we can try this setup.

We need the src/ directory with a main.py script -- That will be the script we wanted to make portable.

Because we chose to use pipenv for dependencies, we should create a Pipfile and Pipfile.lock. This can be done easily by just running pipenv install in the project directory, assuming you have it installed system-wide.

In the end, the overall file structure should look like this:

project
 +- src
 |   +- main.py
 +- Dockerfile
 +- Makefile
 +- Pipfile
 +- Pipfile.lock
 +- run