Skip to content

Setting up Debian (Host OS)

Increasing default open file and watch limit

Why this is nessecary?

Kubernetes environments frequently encounter file handle exhaustion because multiple processes running under the same user need to open numerous files simultaneously and use filesystem watchers (fswatch/inotify) to monitor configuration changes, logs, and resources.

The default Debian limits (~1024 file handles, low inotify watchers) are insufficient for Kubernetes clusters where container runtimes, API components, and applications can easily exceed these quotas.

These increased limits prevent "too many open files" errors and ensure proper filesystem monitoring for Kubernetes controllers and logging systems.

Add the following content to the config files

/etc/security/limits.conf
*       soft    nofile          1048576
*       hard    nofile          1048576
root    soft    nofile          1048576
root    hard    nofile          1048576
/etc/sysctl.conf
fs.inotify.max_user_instances = 1280
fs.inotify.max_user_watches = 10028400

Not sure if I need to modify the following:

/etc/systemd/system.conf
# Below [manager] block!
DefaultLimitNOFILE=1048576
/etc/systemd/user.conf
# Below [manager] block!
DefaultLimitNOFILE=1048576

Setting up Kubernetes

Install the cluster

k0sctl apply
k0sctl kubeconfig > ~/.kube/config

Restore Sealed Secret key

(Restore sealed-secret.yaml from Backup)

k apply -f sealed-secret.yaml

Install ArgoCD

cd ops/argocd
k create ns ops
k kustomize --enable-helm | k apply -f -

From now on everything else will be installed / setup by ArgoCD