How I setup my Kubernetes Cluster

I have several side projects that I continually work on and improve. Some of them are already launched, others are still in the building phase, and many more are in the planning stages 😅. You can see the full list under Projects.

Status Quo

Until now, I’ve been using VPS servers, Cloudflare, and/or Vercel to host my projects. My usual process involves buying a new VPS, applying my Ansible base role—which installs everything including Nginx, Node Exporter, and a WireGuard VPN tunnel for management and monitoring—and then creating a specific role for whatever I want to deploy.

This approach works great! I love Ansible and have invested a lot of time over the past 5-8 years optimizing my setup. I have roles for everything I need, and setting up a new server (whether on-prem or a cloud VPS) takes about 5 minutes. However, my Ansible setup is quite monolithic and is centered around a single repository where I manage my inventory, roles, and playbook (by the way, I only use one playbook and control execution with tags and limits, which I believe is the best approach). This makes it challenging to use in a per-project (repository) based CI/CD workflow.

Finding Alternatives

So, I started thinking about alternatives that could fulfill the following needs:

• Easy HTTP(S) ingress with Nginx
• Multiple services per project
• Cron jobs
• Deployment from multiple projects/repositories to the same cluster/instance

It became pretty obvious that Kubernetes was the right fit for me! But I still wanted to take advantage of the excellent VPS offers from Netcup.de and continue using my Ansible base setup (monitoring, WireGuard tunnel, etc.). Therefore, I decided to use k3s, a super lightweight Kubernetes distribution that I’m running on a Netcup.de VPS located in their US data center in Manassas.

Currently, it’s just a single-node cluster, but I can expand it if needed.

Setup

The setup was straightforward. After purchasing the VPS, I uploaded my Debian base image and ran my Ansible base role against it.

I then added these two k3s configuration files:

These files do the following:

• Disable Traefik—I’m using the Nginx ingress controller instead.
• Disable ServiceLB—I’m using a NodePort service with hostNetwork: true for the Nginx ingress controller.
• Explicitly set the SQLite DB path.
• Add my private Docker registry so I can pull images from there.
• Set the node token to a randomly generated string.
• Enable the Flannel WireGuard network, which encrypts cluster traffic with WireGuard between k3s nodes. If I need to expand, I can just add a second VPS and have the traffic encrypted.

After that, I simply ran curl -sfL https://get.k3s.io | INSTALL_K3S_EXEC="server" K3S_CLUSTER_INIT=true sh -s - and the cluster was installed!

Common Services

All of my side projects require some common services, such as:

• Let’s Encrypt certificates
• HTTPS ingress
• PostgreSQL database
• CI/CD deployment
• Umami web analytics
• Backup

These services need to be deployed to my k3s cluster, and I decided to use Helm together with Helmfiles for this.

What is Helm?

Helm is a package manager for Kubernetes, the popular container orchestration platform. It simplifies the deployment, management, and scaling of Kubernetes applications by using "charts," which are pre-configured templates. These charts contain all the necessary information to deploy a Kubernetes application, including YAML files for resources, configuration values, and dependencies.

This setup gives me the flexibility to use existing charts or create custom ones as needed.

What is Helmfile?

Helmfile is a declarative specification for deploying Helm charts. While Helm alone is powerful, managing multiple Helm charts across different environments or clusters can become cumbersome. Helmfile addresses this by enabling the management of multiple Helm releases using a single file.

I created the following Helmfiles:

• backup: Backs up the k3s cluster to my Nextcloud instance using a custom chart with a Kubernetes CronJob.
• cert-manager: Creates a ClusterIssuer with the DNS-01 challenge using Cloudflare DNS.
• github-deployment: Manages service accounts for all CI/CD-related deployments. This uses a custom chart that creates the service account, role, and role bindings for a namespace.
• nginx-ingress: Deploys the Nginx ingress controller to my cluster and makes it available through a NodePort (with hostNetwork: true) on ports 80/443 on my VPS.
• umami: Uses two custom charts—one for deploying the Umami Next.js application and another for the PostgreSQL database. I chose to create my own PostgreSQL chart for simplicity and to add backups to my Nextcloud instance.

What’s Next?

I am super happy with the setup and will start migrating all my side projects to my new cluster. This will enable me to have a thorough CI/CD setup with GitHub Actions.

I plan to have preview deployments for pull requests as well as production deployments on the main branch. Stay tuned!