diff --git a/content/homelab-ipv4-expose/index.md b/content/homelab-ipv4-expose/index.md index 23a1d02..44058f7 100644 --- a/content/homelab-ipv4-expose/index.md +++ b/content/homelab-ipv4-expose/index.md @@ -1,8 +1,7 @@ +++ title = "Homelab - IPv4 Expose" -date = 2026-04-04 +date = 2026-03-20 description = "How I expose my IPv6 only Homelab for IPv4 clients" -draft = true [taxonomies] categories = ["Homelab"] @@ -12,7 +11,7 @@ tags = ["Homelab", "IPv4", "IPv6"] toc = true +++ -How I am currently exposing the services running in my homelab to IPv4 users despite not having a publicly routable IPv4 address/subnet. +How I plan to expose the services running in my homelab to IPv4 users despite not having a publicly routable IPv4 address/subnet. @@ -26,9 +25,14 @@ So I natively expose my services to the internet using IPv6 and then only have t # Old Setup Currently I rent a VPS from DigitalOcean, which then naively forwards any TCP connections to my IPv6 services. -For this it periodically loads all services I have registered in Consul and looks for specific tags, which I use to mark services that should be exposed. + +For this I have created [ipv-proxy](https://github.com/Lol3rrr/ipv-proxy), which loads registered services from Consul and looks for specific tags in their configuration. +These allow me to easily configure a service to be exposed and how that should be done, like what external port etc. For each such service, it then starts a TCP-listener on it's public IPv4 address and for all incoming requests connects to the IPv6 service being exposed, forwarding all data in both directions. +However because this is centered around having a connection, connectionless things like UDP do not work with this setup. +And obviously it is not ideal to essentially split the one "logical" tcp connection from the client to my service, into two underlying tcp connections from the client to the vps and then the vps to the service. + # Idea 1 - SIIT-EAM ## The Idea SIIT-EAM (Stateless IP/ICMP Translation) allows one to have a "translator" in the network, which receives the incoming IPv4 packets and translates them into their corresponding IPv6 destinations. @@ -38,14 +42,11 @@ One can then configure how the public IPv4 addresses should map to IPv6 addresse For more details I would recommend going through the jool[^jool] documentation. ## The Plan -Services that need to be exposed get their own virtual IPv6 address using keepalived. The active/master node will be forced to the node on which the service is currently running using priorities. - -On my external server setup Jool with SIIT-DC and iptable rules. -1. Everything coming in at the given port for the service (for example HTTP, Teamspeak, etc.) is redirected to a different internal IPv4 address using iptables -2. Jool listens on the internal IPv4 address and performs SIIT-DC or SIIT-EAM forwarding to the correct virtual IPv6 address +I can rent a relatively cheap VPS on some cloud provider, with at least 1 public IPv4 and then it should ideally have a routable `/64` IPv6 subnet as well. +Then I configure SIIT-EAM in jool to forward the IPv4 traffic to one of my servers IPv6 addresses. ## Setup -{% details(summary="Detailed instructions on how to setup everything") %} +{% details(summary="Detailed instructions on how to setup the VPS") %} 1. Get a server that supports Dual-Stack networking and in the best case a /64 ipv6 subnet (I choose Scaleway as a European cloud provider, with cheap servers) [Scaleway IPv6 Docs](https://www.scaleway.com/en/docs/instances/how-to/use-flexips/#flexible-ipv6) [Scaleway Check neighbor discovery](https://www.scaleway.com/en/docs/dedibox-ipv6/how-to/debug-ipv6/#check-the-neighbor-discovery-protocol-ndp) @@ -75,37 +76,58 @@ On my external server setup Jool with SIIT-DC and iptable rules. ``` {% end %} -## Troubles -Jool missing pool6, because I first wanted to try with only the EAM entry +## Troubles encountered +This is a small list of issues that I have encountered, while setting all of this up and you might also run into if you are doing this yourself. -Lots of debugging with tcpdump on external and local server +Jool always requires a pool6 to be configured for SIIT, even if you are only configuring/using EAM entries. -Neighbor Solicitation not working -`13:31:35.919212 IP6 _gateway > ff02::1:ff52:2f24: ICMP6, neighbor solicitation, who has 2001:bc8:1640:6554::4a52:2f24, length 32` -fixed using ndppd. +There will be a lot of debugging of the network traffic going on. +I recommend getting comfortable with tools like tcpdump or wireshark on both the "target" machine, the client and the VPS, to hopefully get a full picture of how packets are flowing. -Fixed forwarding for everything received on the v4 ip, no way to forward based on service +Depending on your provider and specific configuration, Neighbor Solicitation might not work out of the box for you. +I first saw logs like `13:31:35.919212 IP6 _gateway > ff02::1:ff52:2f24: ICMP6, neighbor solicitation, who has 2001:bc8:1640:6554::4a52:2f24, length 32` in my tcpdumps on the VPS, which the server should have replied to, as it has the specific subnet assigned. +To fix this I added the last step in the setup, to configure ndppd, as that fixed my issue. + +The biggest drawback of this approach is that everything ariving on the IPv4 address gets forwarded to the specific IPv6 address, with no fine grained control. # Idea 2 - NAT64 with static BIB ## The Idea -The idea with this is to basically perform some static NAT64, to map ports on the ipv4 side to specific addresses and ports on the ipv6 side. -This would allow me to have one entry for every port that I want to expose, regardless of the IPv6 or port of the service. +The idea with this is to basically perform some static NAT64, to map ports on the IPv4 side to specific addresses and ports on the IPv6 side. +This would allow me to have one entry for every port that I want to expose, regardless of the IPv6 address or port of the service. ## Compared to Idea 1 -More fine grained stuff +The main difference between this and Idea 1 (SIIT-EAM), is that NAT64 allows me to map single ports and basically use a single IPv4 address to host a variety of services, +that might be running on different hosts. ## Setup -{% details(summary="Detailed instructions on how to setup everything") %} -1. Same -2. Same -3. Same -4. Same +{% details(summary="Detailed instructions on how to setup the VPS") %} +1. Get a server that supports Dual-Stack networking and in the best case a /64 ipv6 subnet (I choose Scaleway as a European cloud provider, with cheap servers) + [Scaleway IPv6 Docs](https://www.scaleway.com/en/docs/instances/how-to/use-flexips/#flexible-ipv6) + [Scaleway Check neighbor discovery](https://www.scaleway.com/en/docs/dedibox-ipv6/how-to/debug-ipv6/#check-the-neighbor-discovery-protocol-ndp) +2. apt-get update and apt-get upgrade +3. Install Jool + 1. Based on the [jool documentation](https://www.jool.mx/en/ubuntu.html) + 2. `sudo apt install jool-dkms jool-tools` + 3. Enable IP forwarding + - `/sbin/sysctl -w net.ipv4.conf.all.forwarding=1` + - `/sbin/sysctl -w net.ipv6.conf.all.forwarding=1` +4. Install NDP Proxy Daemon [ndppd](https://manpages.ubuntu.com/manpages/focal/man1/ndppd.1.html) + 1. `sudo apt-get install ndppd` + 2. `/sbin/sysctl -w net.ipv6.conf.all.proxy_ndp=1` 5. Configure [Jool](https://www.jool.mx/en/index.html) 1. `/sbin/modprobe jool` 2. `jool instance add "example" --netfilter --pool6 2001:0bc8:1640:6554:0:0:0:0/96`[^nat64_setup] 3. `jool -i "example" pool4 add --udp 51.158.177.228 1-65535`[^pool4_setup] 4. `jool -i "example" pool4 add --tcp 51.158.177.228 1-65535`[^pool4_setup] -6. Same +6. ndppd for neighbor discovery + 1. In `/etc/ndppd.conf` + ``` + proxy ens2 { + rule 2001:0bc8:1640:6554:0:0:0:0/96 { + static + } + } + ``` 7. Example Setup of bib[^bib_explained] entries (for teamspeak3 in this case) 1. `jool -i "example" bib add 2001:4dd5:b276:1:f652:14ff:fe94:dc00#9987 51.158.177.228#9987 --udp`[^bib_add_command] 2. `jool -i "example" bib add 2001:4dd5:b276:1:f652:14ff:fe94:dc00#30033 51.158.177.228#30033 --tcp`[^bib_add_command] @@ -113,10 +135,14 @@ More fine grained stuff # Future Work -- Automate the jool setup (ansible playbook or maybe even using cloud-init) -- Automate the configuration of the corresponding entries (likely a custom integration with consul) +I need to automate the setup of jool and the VPS. +My first thought would be to use ansible, as with most of my other services, but I might actually think that using something like terraform + cloud-init could be a better fit. +Using terraform together with cloud-init, I could avoid having to log into the VPS at all and the configuration will be done automatically upon creation/startup. -## References +I also want to integrate the configuration of jool, like the NAT64 entries, directly into ipv-proxy. +This way I can easily migrate from the old setup the new one (related [MR](https://github.com/Lol3rrr/ipv-proxy/pull/4)) and do not have to change anything else. + +# References [^jool]: [Jool Homepage](https://www.jool.mx/en/index.html) [^nat64_setup]: [basic NAT64 tutorial](https://www.jool.mx/en/run-nat64.html) [^pool4_setup]: We need to configure the [pool4](https://www.jool.mx/en/pool4.html) used by jool using the given [pool4 commands](https://nicmx.github.io/Jool/en/usr-flags-pool4.html)