Hi everyone. I have k3s with kube-vip for my control plane VIP via BGP. I also have MetalLB via ARP for the services. Before I decide to switch MetalLB to BGP, should I:

A) convert MetalLB to BGP for services

B) ditch MetalLB and enable kube-vip services

C) ditch both for something else?

Router is a Unifi UDM-SE and already have kube-vip BGP configured so should be easy to add more stuff.

Much appreciated!

Update: switched to Kube-vip and MetalLB over BGP. So far all is good, thanks for the help!

Hello community,

I'm finally deciding to upgrade my talos cluster from 1 controlplane node to 3 to enjoy the benefits of HA and minimal downtime. Even tho it's a lab environment, I'm wanting it to run properly.

So I configured the VIP on my eth0 interface following the official guide. Here is an extract :

machine:
    network:
      interfaces:
        - interface: eth0
          vip:
            ip: 192.168.200.139

The IP config is given by the proxmox cloud init network configuration, and this part works well.

Where I'm having some troubles undesrtanding what's happening is here :

• Since I upgraded to 3 CP nodes instead of one, I have weird messages regarding etcd that cannot do a propre healthcheck but sometimes manages to do it by miracle. This issue is "problematic" because it apparently triggers a new etcd election, which makes the VIP change node, and this process takes somewhere between 5 and 55s. Here is an extract of the logs :

user: warning: [2025-06-09T21:50:54.711636346Z]: [talos] service[etcd](Running): Health check failed: context deadline exceeded
user: warning: [2025-06-09T21:52:53.186020346Z]: [talos] controller failed {"component": "controller-runtime", "controller": "k8s.NodeApplyController", "error": "1 error(s) occurred:
 \n\ttimeout"}


user: warning: [2025-06-09T21:55:39.933493319Z]: [talos] service[etcd](Running): Health check successful
 user: warning: [2025-06-09T21:55:40.055643319Z]: [talos] enabled shared IP {"component": "controller-runtime", "controller": "network.OperatorSpecController", "operator": "vip", "link":
 "eth0", "ip": "192.168.200.139"}
 user: warning: [2025-06-09T21:55:40.059968319Z]: [talos] assigned address {"component": "controller-runtime", "controller": "network.AddressSpecController", "address":
 "192.168.200.139/32", "link": "eth0"}
 user: warning: [2025-06-09T21:55:40.078215319Z]: [talos] sent gratuitous ARP {"component": "controller-runtime", "controller": "network.AddressSpecController", "address":
 "192.168.200.139", "link": "eth0"}
 user: warning: [2025-06-09T21:56:22.786616319Z]: [talos] error releasing mutex {"component": "controller-runtime", "controller": "k8s.ManifestApplyController", "key":
 "talos:v1:manifestApplyMutex", "error": "etcdserver: request timed out"}
 user: warning: [2025-06-09T21:56:34.406547319Z]: [talos] service[etcd](Running): Health check failed: context deadline exceeded
 user: warning: [2025-06-09T21:57:04.072865319Z]: [talos] etcd session closed {"component": "controller-runtime", "controller": "network.OperatorSpecController", "operator": "vip"}
 user: warning: [2025-06-09T21:57:04.075063319Z]: [talos] removing shared IP {"component": "controller-runtime", "controller": "network.OperatorSpecController", "operator": "vip",
 "link": "eth0", "ip": "192.168.200.139"}
 user: warning: [2025-06-09T21:57:04.077945319Z]: [talos] removed address 192.168.200.139/32 from "eth0" {"component": "controller-runtime", "controller": "network.AddressSpecController"}
 user: warning: [2025-06-09T21:57:22.788209319Z]: [talos] controller failed {"component": "controller-runtime", "controller": "k8s.ManifestApplyController", "error": "error checking
 resource existence: etcdserver: request timed out"}

When it happens every 10-15mn, it's "okay"-ish but it happens every minute or so, it's very frustrating to have some delay in the kubectl commands or simply errors or failing tasks du to that. Some of the errors I'm encountering :

Unable to connect to the server: dial tcp 192.168.200.139:6443: connect: no route to host

or

Error from server: etcdserver: request timed out

It can also trigger instability in some of my pods that were stable with 1 cp node and that are now sometimes crashloopbackoff for no apparent reason.

Have any of you managed to make this run smoothly ? Or maybe it's possible to use another mechanism for the VIP that runs better ?

I also saw it can come from IO delay on the drives, but the 6-machines cluster runs on a full-SSD volume. I tried to allocate more resources (4 CPU cores instead of two and going from 4 to 8GB of memory), but it doesn't improve the behaviour.

Eager to read your thoughts on this (very annoying) issue !