Month: March 2025

Proxmox VE Cluster – Made Simple

I love Proxmox VE and I love clustering Proxmox VE. What lead me down this journey was that I built a quick and dirty cluster, but because it was built quick and dirty, it lacks proper connections between each node. Each node only has a 1 Gbps copper link, and, well, it’s not great. It accomplishes the goal of a cluster, but things like HA, moving VMs for host maintenance, etc. are too much of a pain. Granted, local storage will always move slower than shared, and this is also without Ceph.

Building a Better Cluster – For Cheap

Normally, a cluster will be like a star topology – every node links to a switch at the center. This is great, but it requires having a switch. While I would like to have a switch, space and power are at a premium. So what is the solution? A fully redundant mesh network!

This is what we are going to be building:

As you can see, Node 1 is linked to Node 2 and Node 3, Node 2 is linked to Node 3. If any link is cut (or just unplugged), there is a redundant path. I’m using some 2x 10 gigabit SFP+ cards in each of the servers, some 10 gigabit SFP+ SR fiber transceivers from FS.com, and 3x OM4 50/150 Multimode 1.5 meter fiber cable.

Proxmox VE Setup

In Proxmox VE, we need to create a Linux Bond interface on each of the servers. The slave ports will be the interfaces of your SFP+ card, which for me is enp129s0f0 and enp129s0f1. You will need to set the mode to broadcast and then set an IPv4 and CIDR for the network. Do not set a gateway.

Once you do this, you can then create your cluster and use your bond interface IP for the cluster. Don’t forget to go to Datacenter > Options and set Migration Settings to use your cluster network.

And there you have it! A cluster without a switch!

Keep in mind that this does not scale well. Also remember that local storage will take awhile to migrate! You can use Ceph in this setup which will work better. Good luck!

Welcome (back) to “The IT Cave”

Welcome to The IT Cave, a tech site for IT professionals or for anyone who wants to learn a bit more about technology.

Brief History

I initially started “The IT Cave” in May 2012 as a way to document what I was working on and fixes for issues I came across in my job – basically to break out my tech posts from my personal posts on my personal blog. I was basically a year into my first job as a real systems administrator for a mortgage company in Orlando, Florida.

At the time, I had a private office without windows and I liked to work in the dark which lead the people in the office to start referring to my office as “the cave”. The name stuck, and so, welcome to The IT Cave!

This site was regularly updated from 2012 to 2018. In 2018, I forgot to renew the domain, and it got snatched up by some Chinese spam. For the last 7 years, I have been trying to get it back. I saw that the domain expired back in December of 2024 and was moving its way though the motions of being deleted (and I noticed it was in redemption in early March 2025, set to be deleted soon). I finally snagged my domain back – on auto-renew this time!

What I’m up to Now

Well, a lot has changed since I last blogged in 2018. I’ve left Florida and Ohio. I’m still working in IT, although I’ve changed jobs a few times. So much has happened, I wish I had this blog. I hope to restore useful content.

Last year, I wrote a book about cPanel (recently updated) and another book about WHM. Oh, I also wrote a book about customer service. Feel free to check them out and even buy a copy!

What’s in The Cave?

A knowledge base of information! I will share any tips, tricks, and wisdom. If you have questions, please ask. If there is a something you want to know, please ask. I’ll cover topics from Linux to Windows, virtualization, and everything in between. Enjoy!