Docker compose on your server

My first article last year was about deploying a Django application on a VPS using a "quick and dirty" approach and for a while it was fine but, I soon noticed the need for a better strategy to manage my personal projects.

This article does, in contrast to the one linked, not aim to be a step-by-step guide but more of a dialogue to describe how my current setup works.

Pain points

The need didn't come from day to night but, after a few months of using the application I started to feel some friction in updating the application, a bit here and there and then the urge to update the application deployment and management came straight to my hands.

The first solution was one Docker Compose with all services e.g. reverse proxy and DB for each web app, but this requires too many resources, we need to be smarter.

The final solution allows me to have a Docker Compose file with shared services for the VPS and many Docker Compose applications pivoting on the shared Docker Compose without duplicating resources and wasting my small VPS's potential (with chip prices these days especially).

Build of the application

Probably one of the most annoying things was the building procedure in order to deploy it remotely, this was really a pain in the ass and I needed to find a better solution ASAP. I was aiming to build and deploy using Docker but, because of my recent transition to GitHub I don't have an easy way of managing Docker images in a remote registry nor did I want to deal with self-hosting one myself, I had to approach the topic in a different manner: build locally and save the build as a tar file, then manually place the tar file on my VPS and load it as a Docker image. Honestly, it is much better than before, when I had to execute several manual operations. In order to reduce friction even more I decided to introduce a tool I have been using at work: Just, see Makefile? Simplify it and you have Just, it's a command launcher where I've added two commands, one to build and the other one to push the build to my remote instance and immediately after load it into Docker.

Now the required operations to update the applications are to build, push and restart the Docker container, everything is already being taken care of by Docker.

Database

Personally I like SQLite, it's a good compromise in many cases but maybe, sometimes, having more than just integer, real, text and blob data types can help, or simply being able to access the remote database without having to download the file each time to view it with a DBMS. With this purpose in mind the first idea was to just throw in my repository's Docker Compose file a PostgreSQL instance, which is a great idea to saturate the server, then why not use a single database instance on the server? So let's build a new Docker Compose deploying shared resources to be accessed by other services.

HTTPS

Now the fun part, instead of handling HTTPS manually I wanted to adopt an out-of-the-box ready solution that also manages certificate renewal automatically and I ended up choosing Traefik. There are many different ones out there and Traefik simply inspired me, it provides a simple dashboard to visualize the service status and the configuration is fairly simple. The proxy is to be deployed within the shared resources Docker Compose stack to make it accessible for multiple web services to handle TLS certificates in a single spot.

New problems

At this point a few knots came up and it's time to untangle them, the issues are due to containerisation and are the following.

Database migrations

To handle this topic two were the possible solutions I was thinking of, such as executing migrations at container start via the entrypoint but, it doesn't feel clean and it's quite unnecessary to execute the migration command every time the container is rebuilt, in addition to not feeling safe, I want to be able to handle it manually. Alternatively I ended up adding a Docker Compose service running side by side with the web app with a container that terminates when it's done and is invoked only when a new migration is required. The image is the same as the web app written in Django, uses the manage.py script, and it can then be used to handle multiple operations such as migrating and deleting the sessions from the database on a schedule, the last topic for this article.

Scheduled tasks

As written in the previous paragraph there might be the need for an application to run scheduled tasks such as the manage.py clearsessions command, to delete old and expired sessions from the database. To do so, not having systemd or cron to rely on, I ended up using Ofelia, I don't really have much to say other than it works for now and does what's expected of it without additional overhead or configuring systemd services and schedules.

Conclusion

Maybe some of you are asking why not Docker Swarm, and simple enough, it solves problems that I am not currently facing while adding initial overhead that I didn't want to deal with. ¯\_(ツ)_/¯