Compare commits

...

4 Commits

Author SHA1 Message Date
Lol3rrr
ae078f3a49 Add docker build workflow
Some checks failed
Build Docker Image / docker (push) Failing after 29s
2026-09-27 02:47:34 +02:00
Lol3rrr
d4541950a9 Basically finish my vault-agents blog post 2026-09-27 02:43:33 +02:00
Lol3rrr
41bb1677ca Overwrite some styles from the template 2026-09-27 02:20:10 +02:00
Lol3rrr
ab8ec68e77 Continue writing on the vault-agents post 2026-09-27 02:16:43 +02:00
5 changed files with 122 additions and 13 deletions

34
.github/workflows/docker.yaml vendored Normal file
View File

@@ -0,0 +1,34 @@
name: Build Docker Image
on:
push:
branches:
- main
permissions:
packages: write
jobs:
docker:
runs-on: ubuntu-latest
steps:
- name: Checkout
uses: actions/checkout@v4
- name: Log in to Gitea Container Registry
uses: docker/login-action@v3
with:
registry: gitea.lol3r.com
username: leon
password: ${{ secrets.DOCKER_PAT }}
- name: Build and push
uses: docker/build-push-action@v6
with:
context: .
file: ./Dockerfile
push: true
tags: |
gitea.lol3r.com/leon/blog:latest
gitea.lol3r.com/leon/blog:${{ gitea.sha }}

View File

@@ -1,6 +1,6 @@
+++ +++
title = "Vault Agents" title = "Vault Agents"
date = 2026-06-02 date = 2026-09-27
description = "Setting up vault-agents for my servers" description = "Setting up vault-agents for my servers"
draft = true draft = true
@@ -16,23 +16,58 @@ My journey to setting up vault-agents on my servers to handle dynamic secrets in
<!-- more --> <!-- more -->
# Configuring Vault Itself # Configuring the Vault Server
We first need to setup vault such that the vault agents can authenticate to vault itself. We need to configure the vault server, such that we can generate the tokens for our agents and assign them the appropriate policies.
For this purpose we first create a kind of account for each of the servers, that a vault agent will be installed on.
In vault this translates to an identity and an alias for each of the servers.
All of these identities are then grouped into a vault-agent group, which will actually have the shared policies assigned to it, such that there is only one place to manage all of the shared access, while also retaining the option to assign policies to individual servers, for example if one of them required extra credentials as it fulfills a unique job not shared by other servers.
To then allow the vault agents to actually authenticate with vault, I decided to simply use tokens directly instead of generating credentials using something like AppRole, but I still do not love this approach yet.
For this I created a backend role for the token auth mechanism in vault, which only allows the previously configured aliases to authenticate against it (as well as allowing orphaned tokens and renewing).
This means I can now generate a unique vault token for each of my servers and they can use it to authenticate as that server against vault to load any secrets they might need.
# Deploying Vault Agents # Deploying Vault Agents
TODO Deploying the vault agents to my servers now comes down to:
Testing 1. installing the vault agent software on the server
2. getting a token for the server from vault
3. deploying the general config for the vault agents
# Using them to template Configs # Deploying templated Configs
A couple of examples how I am going to use the vault agents templating features Now that vault-agents are running on my servers, it is time to actually use them for something useful.
This means I need to make it easy to integrate the deployment of templated configurations for other services, which I decided to do by introducing a simple ansible-role.
This role takes care of creating the correct configuration for the vault-agent to template out a file and placing it in a shared space, where the vault-agent can pick it up.
It will then also trigger the vault-agent on the server to restart / reload its configuration to pick up the newly created config and start outputting the templated configuration.
## Nomad mTLS So all I need to do for using templated configurations for a service that I want to deploy, is give the vault agent user the permissions to write my config to where my service expects it, give it the template to use and providing a command to execute whenever the written out file changes, in case the service does not automatically detect changes to it.
We can use vault agents to template the TLS certificates for nomads mTLS configuration.
1. Get all currently valid CA certificates and store them in the trust bundle
2. Get a leaf certificate for the current node
## Consul ACLs # Example Uses
To configure the ACL tokens for consul ## Nomad - mTLS
My nomad cluster uses mTLS to encrypt its communication over the network, this means every node needs the CA certificate and its own certificate to communicate with other nodes.
Previously this meant managing the certificates and their livecycles semi-manual, I automated all the steps, but needed to trigger the updates manually.
However now with the vault-agents, I can configure them to automatically fetch the shared CA certificate(s) and generate the certificate for the current node dynamically, including renewing it and notifying nomad accordingly.
This workflow is a lot easier to maintain, as there is no human involvement to rotate the leaf-certificates, and allows for enforcing best practices, like short lifetimes for the leaf-certificates.
## Nomad - Consul ACLs
My nomad cluster is very tightly integrated into consul, to make use of its service-mesh capabilties and some other smaller features.
However because I have ACLs enabled on the consul cluster, every nomad client requires an ACL token to connect to consul.
Managing this manually would either mean a lot of maintenance for me or using tokens with very long lifetimes, which are both not good options.
Instead I can use the consul secret-engine in vault and the vault-agents, to dynamically obtain an ACL token for each server, which contains just enough permissions for nomad and has a short lifetime to follow best-practices as much as possible (for my context).
# Some Issues
I want to cover some of the issues that I encountered / still have, just so I am clear on the downsides of this setup.
## Vault Token expiring while vault-agent is not running
I have one server that likes to randomly freeze on the OS level, which is mostly fine in my setup as all the normal workloads can just migrate and once I get to restarting the server everything just comes back.
However because I use a vault token, which expires after at most one day with my current config, it frequently happens that the token for the server expires before I get around to restarting it, meaning I need to run my vault-agent deployment again to generate a new token for the server.
This is not a huge issue for me in this case, but mostly annoying as I need to keep that in mind during maintenance and adds another kind of weird thing.
## One bad config infects the entire agent
Currently if I accidentally deploy one bad vault-agent config, for example by writing a custom config for templating a config, then the entire vault-agent on the server is not working properly and thus affects other services as well.
I have no good approach to prevent this, besides enforcing the use of the ansible-role, which could be updated to make sure that after every change the agent is still running without issues, but that is not yet implemented.
# References # References
[^vault_pki_api]: [Vault PKI API](https://developer.hashicorp.com/vault/api-docs/secret/pki) [^vault_pki_api]: [Vault PKI API](https://developer.hashicorp.com/vault/api-docs/secret/pki)

28
sass/styles.scss Normal file
View File

@@ -0,0 +1,28 @@
article[itemprop="articleBody"] {
h1 {
font-size: 1.2rem !important;
}
}
article[itemprop="articleBody"] {
h2,
h3,
h4,
h5,
h6 {
margin-top: 1rem !important;
font-size: 1.1rem !important;
}
}
.hack h2,
.hack h3,
.hack h4,
.hack h5,
.hack h6 {
position: relative;
margin-bottom: 1.2rem;
margin-top: 1.75rem
}

View File

@@ -21,3 +21,13 @@
<script src="/js/chart.min.js"></script> <script src="/js/chart.min.js"></script>
{% endblock %} {% endblock %}
{% block styles %}
{{ super() }}
<link
rel="stylesheet"
href="{{ get_url(path='styles.css', trailing_slash=false) | safe }}"
/>
{% endblock %}

View File

@@ -24,6 +24,8 @@ index_format = "elasticlunr_json"
[markdown] [markdown]
external_links_target_blank = true external_links_target_blank = true
insert_anchor_links = "left" insert_anchor_links = "left"
definition_list = true
github_alerts = true
[markdown.highlighting] [markdown.highlighting]
theme = "catppuccin-mocha" theme = "catppuccin-mocha"