Troubleshooting

I run a local Kubernetes cluster on macOS using Talos Linux with QEMU. The cluster uses MetalLB in L2 mode to assign external IPs to LoadBalancer services — the same way it works in production on bare metal. Everything inside the cluster was fine. But from my Mac, I couldn’t reach the LoadBalancer IP at all. $ curl -k --connect-timeout 3 https://10.5.0.10 curl: (28) Failed to connect to 10.5.0.10 port 443 after 3005 ms: Timeout was reached This took me a while to figure out, so here’s the full story. ...

The Problem I was using k9s to check logs on my K3s homelab cluster when I hit this frustrating error: to create fsnotify watcher: too many open files stream closed: EOF for development/drawdb-ddf8d9569-grfff (drawdb) The log stream would just close immediately, and I couldn’t see anything. Annoying. What’s Actually Happening Here’s the thing - this isn’t a problem with k9s on your laptop. It’s happening on the K3s node itself. When applications watch files (like development tools, file syncing services, or even k9s monitoring your cluster), Linux uses something called inotify watchers. There’s a limit to how many of these you can have. ...