Processing Files with Docker Volumes
Docker can be a great tool to process workloads, like optimising images, compiling source code or to do anything that has some very specific requirements. In this post we'll go over some examples on how to automate some workflows in containers.
I've written about an a bit more niche case in the past about updating php apps with breaking version changes, where I made use of a similar method, but you can do it purely with docker and mounting volumes as well.
Containers are great for having reproducable runtimes, but it's also a great place to stick everything you really don't want to deal with on your local development setup. You just need to automate once and can pass it on or dig it up when you need it, instead of taking great care of which python, node or whatever version is installed on your system, or even worse, to make sure you have that installed on all of your systems.
Creating your Docker Image
For this example we'll resize and convert some pictures, because it comes with some dependencies that we'll need in our image and we'll need to mount a volume to supply the running container with input.
Let's say we want to transform some images from
webp for this case, we'd create a
Dockerfile like this:
FROM alpine:3.16 RUN apk add --update \ libwebp-tools \ bash # create "in container" directories for source and destination images RUN mkdir -p /var/app/input RUN mkdir -p /var/app/output WORKDIR /var/app RUN ls RUN pwd RUN ls /var/app/input COPY run.sh . CMD /var/app/run.sh
The script we're going to run is the following:
for picture in input/*.jpg; do echo "processing $picture" cwebp -q 90 -resize 1600 0 $picture -o output/$(basename $picture .jpg).webp done ls -lash input/* ls -lash output/*
To build the container, we'll run this and save it to
mkdir input mkdir output docker build -t image-optimise .
Now we just need some pictures to drop into the input directory, let's take some full size street photography from the other day:
These images are HUGE, between 12 and 24 MB each, so resizing them sounds like a good plan, otherwise I'll get angry comments again that I'm burning through people's cell phone plans.
Mounting Volumes into your Docker Container
In oder to make the files on our host file system available inside the docker container, we can run it with attached volumes. You might already know volumes from using docker-compose with volumes for your database server or similar, they're forwarding a port, but instead we make a little part of our file system available.
Volumes follow the
host:container pattern so we'll create a little bash script to run our container as well,
"$PWD/input":/var/app/input \ -v "$PWD/output":/var/app/output \ image-optimisedocker run \ -v
Outside of the container the path is:
/home/jonathan/projects/docker/optimise-images/input, inside the container the path to the "linked" folder is:
We're going to mount two volumes, one for the input, one for the output and lastly we'll supply the name of the image to run.
Running your Docker Image
I tend to juggle a few different docker containers, so I like to put my build and run commands in shell scripts, so that I don't need to remember the oddities and names of each one every time, so to run both build and run script we need to remember to assign the
chmod +x *.sh
and then we can run:
./docker-build.sh && ./docker-run.sh
and we'll see output of first building the image:
Sending build context to Docker daemon 65.53MB Step 1/10 : FROM alpine:3.16 ---> 9c6f07244728 Step 2/10 : RUN apk add --update libwebp-tools bash ---> Using cache ---> a2ea3667de16 Step 3/10 : RUN mkdir -p /var/app/input ---> Using cache ---> c2543f93fc67 [...] ---> Using cache ---> a19ceb836203 Step 10/10 : CMD /var/app/run.sh ---> Using cache ---> f6c34207c17e Successfully built f6c34207c17e Successfully tagged image-optimise:latest
and then the run, which will contain some output from
cwebp actually resizing the images:
Output: 863108 bytes Y-U-V-All-PSNR 43.88 49.80 49.49 45.10 dB (1.80 bpp) block count: intra4: 13857 (91.77%) intra16: 1243 (8.23%) skipped: 1540 (10.20%) bytes used: header: 512 (0.1%) mode-partition: 74321 (8.6%) Residuals bytes |segment 1|segment 2|segment 3|segment 4| total macroblocks: | 5%| 18%| 32%| 45%| 15100 quantizer: | 12 | 12 | 9 | 6 | filter level: | 4 | 3 | 2 | 3 | 23.1M Oct 30 00:01 input/20221028_20-47-48-5L7A7970.jpg 496.4K Nov 3 21:08 output/20221028_20-47-48-5L7A7970.webp
As we can see we reduced the image size from 23.1 MB to under 0.5 MB, which is awesome.
Image optimisation is just one example of what you can process in docker and spit back out to your local hard drive through mounted volumes.
I'm going to remember stuffing things in a container whenever some dependency is just TOO ANNOYING to be maintained in a healthy state on a local system.