Like many people on the internet, I recently saw this great
talk by Xe
Iaso about things you can do on your Tailscale
network with the
tsnet package. It got me wondering what more I
could do with tailscale myself. Then, three days later, I noticed that
my Kobo e-reader had stopped syncing with calibre-web and I knew
something about how I accessed my homelab-hosted services would have
I run about 10 services that are important to my personal life: home-assistant, calibre-web are examples of that; all of them live on machines that I run, and are accessible only on my private network. That used to be a weird OpenVPN-based monstrosity, but these days it’s all done via tailscale, a wireguard-based service that gets your computers (and only your computers) talking to each other really quickly & easily.
Those services each had a hostname under a domain, say
home-assistant.ts.example.com, and I ran an nginx instance that
serves as a reverse proxy.
I’d had most of that configured with a 100kB nginx.conf file that nix
mercifully generates for
and that nginx server was exposed to my tailnet by having nginx listen
on the tailscale-internal
100. IP address (and that address alone).
So there were a few problems:
- I want to share some of these services with my partner, but not all of these services. There are no access controls outside some manual jank (see).
- Some services really don’t expect to be run behind a reverse proxy
that buffers requests&responses, like nginx does in a
serverblock. You can use
streamblocks for some, but not all features I need for all the services I run are supported there.
- All this was serviced by a letsencrypt certificate for
ts.example.com, with tons of SubjectAltNames, which caused letsencrypt to email me about every name combination that ever got one issued, every time a certificate expires. That’s kinda a very tiny problem, but ugh.
- Configuring services to listen on the tailnet-internal address is great until you reboot the machine: The tailscale interface often would take an arbitrary amount of time to come up, and there was no good way to wait until tailscale was “up”… so nginx would croak on startup and had to be manually kicked (occasionally retries got it working automatically, and those were good days).
- And most importantly, it was ~impossible to teach the Kobo reader and calibre-web to connect to the port I had to listening on the internet for its external syncing service (8443, which for some reason the Kobo URL parser drops? From the URL?!).
Ok, so most things kinda worked, and that would be good enough for many! But knowing what I know now felt itchy: that you can easily build a reverse proxy that exposes services on your own tailscale network, and fix most, if not all, of these issues.
- Each service being proxied gets its own tailnet “machine” entry, which (in combination with ACL tags) allows very fine-grained control over who gets access to what services.
- Go’s ReverseProxy streams the request and response bodies, no more buffers that need their size adjusted or spilled to disk if I upload or download large files, e.g. in docspell.
- Tailscale manages the hostname and its TLS certificate automatically, which also means I don’t get email about potentially-expiring certs.
- There is literally no inter-service dependency. My local services listen on 127.0.0.1 (or a Unix domain socket!), and tsnsrv brings up its own tailnet-internal IP address, one per service.
- And best of all, it’s possible to run a tailscale funnel endpoint on the standard HTTPS port (443) which means my Kobo reader can sync with calibre-web again, without gross hacks. Joy.
tsnsrv is a little go program; each of its processes uses the
tsnet library to register itself as a node on my tailscale
network. Each of those nodes gets its own wireguard key, a hostname
provisioned under my
ts.net subdomain, and each of those hostnames
gets a TLS cert provisioned, too. Some services (notably calibre-web
sync) need exposure to the public internet via the tailscale funnel,
so it also supports registering with as a funnel service.
But mostly, I use this to run services that only authorized people should reach (namely me and not all the 5.07 billion people on the internet)!
And thanks to the ability to define ACLs for accessing each of these “tailnet machines”, even the other people on my tailnet (my partner, my parents, etc) can reach only exactly the services that they need… And not, say, my internal alertmanager, where they could accidentally silence an alarm.
This setup differs slightly from other systems like Caddy and Traefik, in that it leans heavily on the “machine = one service” idea. With Traefik, there’s only really one “service”, traefik, and it proxies to all your other services, likely under a subpath of your URL. That’s great until you try to use a service that doesn’t love being served under a subpath. Same for Caddy - the two proxies seem to use tailscale mostly for automatic TLS cert provisioning, and for providing a hostname (that of the server running the proxy).
So, with these services it’s no ACLs for you (other than “all or nothing” based on the server running the services), no convenient hostnames and occasionally, no service at all because whatever you’re running expects to own the entire URL path and not just a subdirectory.
tsnsrv instead, in the easiest case, is is a single commandline,
tsnsrv -name happy-computer http://127.0.0.1:8000 and then you point
your browser at