How do you manage the distribution of internal TLS network certificates? I’m using cert-manager to generate them, but the root self-signed certificate expires monthly which makes distribution to devices outside of K8s a challenge. It’s a PITA to keep doing this for the tablet, laptop and phones. I can bump the root cert to a year, but I’m concerned that the date will sneak up on me. Are there any automated solutions?
You must log in or register to comment.
Bump root cert to 10 years and use intermediate with shorter lifetime. root cert should be stored and processed off net.
Use a secret manager?
Cert is a secret, add a small agent to your containers that pings your secret manager and gets back the current cert. Then saves / imports it (or whatever is appropriate).
Not sure if it’d fit your use case 100%, but this has been a nice middle ground solution for LE certs in my lab: https://www.certwarden.com/
Some of my internal stuff goes out to Let’s Encrypt, so I don’t worry about it at all. My internal AD stuff is set for like three years. If anyone has compromised the CA, they’re already past where issuing malicious certs would be useful.
I would up your root cert expiration. You can keep the root CA offline if you’re concerned about compromise.
There are also ways to run LE-style automatic renewals internally, but I’ve never bothered because what I’ve described above means I don’t need it.
I looked into this recently.
There’s a DNS challenge designed for this exact scenario called (from memory) DNS01, but it’s more of a faff than I’m willing to get involved with.
Basically you push proof that you own the domain into a DNS record instead of to a file on a web server. It requires a DNS provider that has an API and a client that speaks that API.
It also leaks private DNS stuff into the public domain.
I’d love it if someone devised an easier way. Maybe there is an easier way?
I do the dns challenge with letsencrypt too but to not leak local dns names into the public I just run a pihole locally that can resolve those domains
The DNS server is only one thing you tell the domain, the other is the certificate authority. And those publish all issued certificates as part of certificate transparency. https://en.m.wikipedia.org/wiki/Certificate_Transparency
To mitigate the amount of published information, you can request wildcard certs to keep the subdomains private.
You can also use a wildcard cname entry to capture all subdomains and leave out the pihole faff, given that you use a reverse proxy that forwards to respective services by subdomain.
Yep I also run wildcard domains for simplicity
This is the way.
MONTHLY?? That’s a bit much, don’t you think?
If you’re regenerating certa that fast, I can’t think of anything that’s going to secure AND easy enough to satisfy automating this.
Whatever tool you want to use to secure the contents of the cert from its initial creation, to distribution, is fine enough. If you want super easy, use an SSH/SCP script. If you want something more elegant, think Hashicorp Vault or etcd.
Ansible is probably more effort than it’s worth (plus securing the secrets of the cert), and any other config mgmt tool won’t deal with the distribution portion simply, so I’d skip all of that.
Yes, monthly is too fast. I’m using a K8s operator for cert-manager which defaults to a month. I think I can patch the CSV with an annotation that will bump that out, but when the operator updates the CSV then I need to repatch it.
I was polling the community to see if there’s something that is easy to use but I was not able to find in my searches. It seems like a common problem.
Part of my problem is that I chose to use a K8s operator for cert-manager which isn’t easy to configure. Had I used a helm chart, i’d have bumped the root cert to 10 years and forgotten about it.
If the operator doesn’t allow it for some reason, uninstall it and try with the helm chart instead?
Or is there a reason to use the operator?
Yeah, either secure copy (SCP) or rsync are the ways to do this securely.
Synching maybe?
