3.5 KiB
+++ title = "Vault Agents" date = 2026-09-27 description = "Setting up vault-agents for my servers" draft = true
[taxonomies] categories = ["Homelab"] tags = ["Homelab", "Vault"]
[extra] toc = true +++
My journey to setting up vault-agents on my servers to handle dynamic secrets in configurations.
Configuring the Vault Server
We need to configure the vault server, such that we can generate the tokens for our agents and assign them the appropriate policies.
For this purpose we first create a kind of account for each of the servers, that a vault agent will be installed on. In vault this translates to an identity and an alias for each of the servers. All of these identities are then grouped into a vault-agent group, which will actually have the shared policies assigned to it, such that there is only one place to manage all of the shared access, while also retaining the option to assign policies to individual servers, for example if one of them required extra credentials as it fulfills a unique job not shared by other servers.
To then allow the vault agents to actually authenticate with vault, I decided to simply use tokens directly instead of generating credentials using something like AppRole, but I still do not love this approach yet. For this I created a backend role for the token auth mechanism in vault, which only allows the previously configured aliases to authenticate against it (as well as allowing orphaned tokens and renewing).
This means I can now generate a unique vault token for each of my servers and they can use it to authenticate as that server against vault to load any secrets they might need.
Deploying Vault Agents
Deploying the vault agents to my servers now comes down to:
- installing the vault agent software on the server
- getting a token for the server from vault
- deploying the general config for the vault agents
Deploying templated Configs
TODO: Explain how I am using an ansible role to make this easily available
A couple of examples how I am going to use the vault agents templating features
Example Uses
Nomad mTLS
We can use vault agents to template the TLS certificates for nomads mTLS configuration.
- Get all currently valid CA certificates and store them in the trust bundle
- Get a leaf certificate for the current node
Consul ACLs
To configure the ACL tokens for consul
Some Issues
I want to cover some of the issues that I encountered / still have, just so I am clear on the downsides of this setup.
Vault Token expiring while vault-agent is not running
I have one server that likes to randomly freeze on the OS level, which is mostly fine in my setup as all the normal workloads can just migrate and once I get to restarting the server everything just comes back. However because I use a vault token, which expires after at most one day with my current config, it frequently happens that the token for the server expires before I get around to restarting it, meaning I need to run my vault-agent deployment again to generate a new token for the server. This is not a huge issue for me in this case, but mostly annoying as I need to keep that in mind during maintenance and adds another kind of weird thing.