Basically finish my vault-agents blog post

This commit is contained in:
Lol3rrr
2026-09-27 02:43:33 +02:00
parent 41bb1677ca
commit d4541950a9

View File

@@ -35,18 +35,27 @@ Deploying the vault agents to my servers now comes down to:
3. deploying the general config for the vault agents 3. deploying the general config for the vault agents
# Deploying templated Configs # Deploying templated Configs
TODO: Explain how I am using an ansible role to make this easily available Now that vault-agents are running on my servers, it is time to actually use them for something useful.
This means I need to make it easy to integrate the deployment of templated configurations for other services, which I decided to do by introducing a simple ansible-role.
This role takes care of creating the correct configuration for the vault-agent to template out a file and placing it in a shared space, where the vault-agent can pick it up.
It will then also trigger the vault-agent on the server to restart / reload its configuration to pick up the newly created config and start outputting the templated configuration.
A couple of examples how I am going to use the vault agents templating features So all I need to do for using templated configurations for a service that I want to deploy, is give the vault agent user the permissions to write my config to where my service expects it, give it the template to use and providing a command to execute whenever the written out file changes, in case the service does not automatically detect changes to it.
# Example Uses # Example Uses
## Nomad mTLS ## Nomad - mTLS
We can use vault agents to template the TLS certificates for nomads mTLS configuration. My nomad cluster uses mTLS to encrypt its communication over the network, this means every node needs the CA certificate and its own certificate to communicate with other nodes.
1. Get all currently valid CA certificates and store them in the trust bundle Previously this meant managing the certificates and their livecycles semi-manual, I automated all the steps, but needed to trigger the updates manually.
2. Get a leaf certificate for the current node
## Consul ACLs However now with the vault-agents, I can configure them to automatically fetch the shared CA certificate(s) and generate the certificate for the current node dynamically, including renewing it and notifying nomad accordingly.
To configure the ACL tokens for consul This workflow is a lot easier to maintain, as there is no human involvement to rotate the leaf-certificates, and allows for enforcing best practices, like short lifetimes for the leaf-certificates.
## Nomad - Consul ACLs
My nomad cluster is very tightly integrated into consul, to make use of its service-mesh capabilties and some other smaller features.
However because I have ACLs enabled on the consul cluster, every nomad client requires an ACL token to connect to consul.
Managing this manually would either mean a lot of maintenance for me or using tokens with very long lifetimes, which are both not good options.
Instead I can use the consul secret-engine in vault and the vault-agents, to dynamically obtain an ACL token for each server, which contains just enough permissions for nomad and has a short lifetime to follow best-practices as much as possible (for my context).
# Some Issues # Some Issues
I want to cover some of the issues that I encountered / still have, just so I am clear on the downsides of this setup. I want to cover some of the issues that I encountered / still have, just so I am clear on the downsides of this setup.
@@ -56,6 +65,10 @@ I have one server that likes to randomly freeze on the OS level, which is mostly
However because I use a vault token, which expires after at most one day with my current config, it frequently happens that the token for the server expires before I get around to restarting it, meaning I need to run my vault-agent deployment again to generate a new token for the server. However because I use a vault token, which expires after at most one day with my current config, it frequently happens that the token for the server expires before I get around to restarting it, meaning I need to run my vault-agent deployment again to generate a new token for the server.
This is not a huge issue for me in this case, but mostly annoying as I need to keep that in mind during maintenance and adds another kind of weird thing. This is not a huge issue for me in this case, but mostly annoying as I need to keep that in mind during maintenance and adds another kind of weird thing.
## One bad config infects the entire agent
Currently if I accidentally deploy one bad vault-agent config, for example by writing a custom config for templating a config, then the entire vault-agent on the server is not working properly and thus affects other services as well.
I have no good approach to prevent this, besides enforcing the use of the ansible-role, which could be updated to make sure that after every change the agent is still running without issues, but that is not yet implemented.
# References # References
[^vault_pki_api]: [Vault PKI API](https://developer.hashicorp.com/vault/api-docs/secret/pki) [^vault_pki_api]: [Vault PKI API](https://developer.hashicorp.com/vault/api-docs/secret/pki)
[^consul_template_secrets]: [Consul Template Secrets](https://github.com/hashicorp/consul-template/blob/main/docs/templating-language.md#secret) [^consul_template_secrets]: [Consul Template Secrets](https://github.com/hashicorp/consul-template/blob/main/docs/templating-language.md#secret)