I've been wanting to build and host my personal projects on my own hardware for a while now, but I didn't want to spend a fortune on hardware and energy bills. I needed something that didn't require a proper server rack, taking space in my apartment. I also wanted to improve my knowledge of kubernetes, and I wanted to learn how to use it to automatically deploy my own personal projects to my own cluster.

My Discovery

Initially, I bought a Raspberry Pi Zero 2W with 512MB of RAM, a USB C cable (to power it), a MicroSD usb adapter and a MicroSD card. It cost me about £60 and this allowed me to test out my ideas without spending too much money.

Thanks the imaging software from Raspberry PI, I was able to easily install PI OS on to the MicroSD.

After hooking up the power adapter, the mini HDMI to HDMI adapter, the MicroSD card, a keyboard and mouse (via a usb hub), my monitor via HDMI and a spare network cable. I had a tiny computer, surrounded by a rats nest of cables. The PI started up, and I worked great. I was surprised just how fast it was, and it used only 2 watts of energy.

However, I quickly noticed that the MicroSD card I had, just wasn't fast enough, and it was taking a long time to run commands and access files. This would become a problem, if I wanted to host my websites and web applications. I also didn't like the idea of having even more cables, when I go to buy the other PIs for my cluster.

Luckily, the imaging software has a feature, where you can configure the networking for the PI and set up a ssh server on the PI. That way, I could at least drop some of the cables I was using. I was still left with the power adapter and the network cable, but I wondered if there was a way to also get rid of the power cable.

What about kubernetes? how was I going to set that up? This is where k3s comes to the rescue. k3s is a lightweight Kubernetes distribution that runs well on limited hardware (such as ARM processors), so it's perfect for building small clusters. It also happens to be easy to install and use (see my post on configuring k3s for more details).

Choosing the right hardware

I spent a lot of time, researching the Raspberry Pi 5, trying to decide which one to buy. Most people recommend installing Ubuntu Server on the Pi, before setting up a k3s cluster. However, in order to run Ubuntu on a Raspberry Pi, you'll need a Raspberry Pi with a minimum of 4GB of RAM, a microSD card (4GB minimum, 8GB recommended), and MicroSD usb adapter. So you can't just buy an older Raspberry Pi, because of the RAM limitations.

Picking the wrong PI

In the end, I bought 4 Raspberry PI 5s with 8GB of RAM for just £80 each.

Specifications:

• Broadcom BCM2712 2.4GHz quad-core 64-bit Arm Cortex-A76 CPU, with cryptography extensions, 512KB per-core L2 caches and a 2MB shared L3 cache
• Dual-band 802.11ac Wi-Fi Bluetooth 5.0 / Bluetooth Low Energy (BLE)
• 2 × USB 3.0 ports, supporting simultaneous 5Gbps operation / 2 × USB 2.0 ports
• Gigabit Ethernet, with PoE+ support (requires separate PoE+ HAT)

I later realised, after setting them up, that they barely used any RAM or CPU, and that I could have gone with the 4GB model. This would have costed me half of what I had spent.

Wearing the right hat

A cool thing about the Raspberry Pi 5 is that you can extend its features by wearing a hat. A hat is a small board that sits on top of the Raspberry Pi, and it can be used to add additional features to the Pi. They typically connect to the PI, using the GPIO pins on the board.

Speeding up my web server

As I mentioned earlier, the access times when running a web server, were rather slow, due to using a MicroSD card. In order to speed up the storage, I needed to get something that supports a faster low latency interface, such as a NVMe or SATA hat. There are a number of options:

The idea choice is to go with the GeeekPi M.2 NVMe M-Key & POE+ HAT for Raspberry Pi 5, with Official Raspberry Pi 5 Active Cooler for £32 (source: Amazon), paired with a KingSpec 256GB M.2 2242 NVMe SSD for £30 (source: Amazon). However, with a cluster of 4 PI's, this would have racked up a £248 bill, just for storage alone.

So I looked around for a cheaper solution. I discovered that the Lexar NQ100 2.5” SATA III (6Gb/s) 240GB SSD, Up to 550MB/s Read for £14 (source: Amazon) was the right option for me. It was a bit slower than the KingSpec, but it was still way better than the MicroSD card. I could pair that with the JSAUX USB 3.0 to SATA Adapter for just £8 (source: Amazon).

I ended up spending just £88 instead of £248 for the cluster storage.

Power Over Ethernet (PoE)

I still wanted to get rid of the power adapter. Then I discovered Power Over Ethernet (PoE), a technology that allows devices to be connected to a network without the need for a separate power supply. This meant that I could power my Raspberry PIs and have it connected to my network, with just a single cable.

I found the Waveshare PoE HAT (G), Compatible with Raspberry Pi 5, 5V 5A on amazon for just £23.

It sits on top of the Raspberry Pi, and it powers the Raspberry PI using the PoE pins on the board. It has a very low profile, and it doesn't take up much space.

The Quest for the Best PoE Networking Switch

The Raspberry Pi 5 requires a power supply that is rated for 5V 5A (27 watts) in order to reach peak performance. I needed to get a network switch that supported Power over Ethernet (Poe). Most 4-port power over ethernet switches are rated for up to 30 watts per port but can only supply up to 40-65 watts across all ports. This poses a challenge, if we want to run 4 Raspberry Pi 5 computers at the same time (4 x 27 watts = 102 watts), each with a SSD (4 x 5 wats - 20 watts).

I did some digging, and I discovered that you can run a PI 5 on about 15 watts of power. Most SSDs actually run between 1-4 watts of power. Which meant that I could run my cluster on 80 watts of energy. It still wasn't the 65 watts, I was looking for, but we were getting closer. And then I came across the NETGEAR 5 Port Gigabit Ethernet PoE Switch (GS305PPV2) for just £80 on amazon. It was able to support 30 watts per port with a total budget of 83 watts for all ports.

BINGO!

My cluster now, would only need just 1 cable per node - the network cable. Since I knew how to make my own network cables, I could keep the rats nest under control by making really short cables.

Housing the cluster

The last piece of the puzzle, was how to keep them cool while protecting from dust and electrical static. I found the UCTRONICS Upgraded Complete Enclosure for Raspberry Pi Cluster for just £85 (source: Amazon). It was a nice, compact enclosure, that came with two fans, and a space for the network switch.

I didn't have any trouble installing the Raspberry PIs in the enclosure, and I was able to get them all up and running in about an hour.

Oh, but the noise from those fans was so loud, that it sounded like an F15 fighter jet! So I replaced them with 2 Noctua NF-A8 PWM for about £20 (source: Amazon) and a USB fan controller for £20 (source: Amazon). Now I could hear myself think!

Conclusion

After several hours of research, I was able to build a cluster of 4 Raspberry PI 5 nodes. I had spent about £600 on my credit card, and it was ready for me to configure k3s and start using it.