TheKrugers.com

Distilling Traefik lessons learned

Traefik is, put very simply, a reverse proxy that exposes services (such as web sites) on internal infrastructure to an external world (be that the Internet or a LAN).

The most commonly used setup for Traefik is to run it as a Docker container to proxy requests from outside to other containers and services on the Docker server. It can also be used to proxy requests to other stuff, like dedicated web servers sitting on the internal network.

Some extra stuff you get

The main reason I started looking at Traefik for my home lab is the easy integration with Let’s Encrypt, specifically using DNS Authentication for Let’s Encrypt certificate deployment.

Set up correctly, Traefic will manage certificates for all the services it proxies and deploy/renew Let’s Encrypt certificates for them as needed. For a configuration where the services that certificates are needed for are not on the Internet, Traefic can integrate with a DNS provider like Cloudflare to authenticate the domain, allowing for DNS-01 auth instead of HTTP-01 auth.

In my home lab, this means that I can have a whole lot of HTTPS services sitting internally in my lab without having to keep telling the browser that I’m happy with the fact that the certificate on the service is not properly signed.

You also get the ability to “magically” turn an unencrypted (http) service into an encrypted service (https) and automatically redirect from http to https.

Some basic concepts I had to learn

Traefik comes with a couple of object types at different levels of the configuration that may take some getting used to, so below I give a brief explanation of each as I understand it right now.

Note that I use TOML for my Traefik configuration, YAML is also an option. Using TOML in my config is purely based on the fact that most of the examples I discovered over time have been in TOML and not YAML.

Core config level

entryPoints

Entry points are basically the network ports that Traefik listens on for requests from things like web browsers. In the most basic of cases, you would have an entry point called web that listens on port 80, but it can be expanded some more. In my lab, I have two entry points called web (port 80) and websecure (port 443), with automatic redirect from web to websecure.

[entryPoints]
  [entryPoints.web]
    address = ":80"
    [entryPoints.web.http.redirections.entryPoint]
      to = "websecure"
      scheme = "https"

  [entryPoints.websecure]
    address = ":443"

serversTransport

This is just some extra parameters for the connection from Traefik to the services at the back end. I use this to tell Traefik not to care if my actual services don’t have properly signed certificates.

[serversTransport]
  insecureSkipVerify = true

providers

A provider is basically something that tells Traefik how to find out about services to proxy. In my home lab I make use of two providers. providers.docker integrates with the Docker API on the host that runs Traefik and the containers I want to serve, while providers.file reads a TOML configuration file for stuff outside of Docker.

[providers.docker]
  watch = true

[providers.file]
  filename = "traefik_dynamic.toml"

When I’m feeling brave, I may see what happens if I add watch = true to the file provider…

certificatesResolvers

Certificate resolvers tell Traefik how to get certificates for TLS services. This is where I set up my integration with Let’s Encrypt. 

[certificatesResolvers]
  [certificatesResolvers.lets-encrypt]
    [certificatesResolvers.lets-encrypt.acme]
      email = "A VALID EMAIL ADDRESS HERE"
      storage = "acme.json"
      [certificatesResolvers.lets-encrypt.acme.dnsChallenge]
        provider = "cloudflare"
        delayBeforeCheck = 0

The acme.json file must exist and must have permissions 0600 before you start the container for the first time, otherwise Bad Things Happen.

Note that the certificatesResolvers.lets-encrypt.acme.dnsChallenge bit also needs a couple of environment variables with secret info in them. These are CLOUDFLARE_EMAIL (the email address of your Cloudflare account) and CLOUDFLARE_API_KEY (the master API key of your Cloudflare account). This really should also work with a Cloudflare Access Token, but I haven’t really dug too deeply into it yet.

Middlewares

Middlewares are basically things that happen to a request as they are being processed, they sit “in the middle” between the service on one end and the endpoint on the other end. In my lab I make use of redirect-web-to-websecure@internal, which is automatically added by the use of entryPoints.web.http.redirections.entryPoint above and simpleAuth@file that is added in the configuration file.

[http.middlewares.simpleAuth.basicAuth]
  users = [
    "generate what goes in here on the command line with: htpasswd -n username"
  ]

simpleAuth@file is basically just used to secure my Traefik dashboard with a username and password. In general, services should handle their own user authentication.

Dynamic config level

Configuration about specific services being proxied by Traefik is considered dynamic, this is where Traefik learns what to serve to clients. A basic overview of the request flow, using all the layers, would be a bit like this:

Entry point -> Router -> Middlewares -> Service

The Middlewares part is not present in all routes, it only applies if the router says that middleware needs to be involved. Entry points and Middlewares are core components and described above, so let’s have a look at Routers and Services, but from the bottom up.

Services

A service is basically something, like a web server, running in a container or on a server that a client wants to talk to. The service definition tells Traefik how to find that service and talk to it so it can proxy requests to it. If the service is in a docker container on the same host as Traefik, this can be done by declaring labels in the Docker container that Traefik discovers automatically via providers.docker. Here’s an example from a container in my lab that has two different services sitting in the same container. This example uses docker compose file version 3.7

# Stuff removed
    labels:
# Stuff removed
# Web server 1 on port 8080
      traefik.http.services.web1.loadbalancer.server.port: 8080
# Stuff removed
# Web server 2 on port 8081
      traefik.http.services.web2.loadbalancer.server.port: 8081
# Stuff removed

Traefik will see this and create services called web1@docker that talks to port 8080 on the IP address of the container and another called web2@docker that talks to port 8081 on the IP address of the container. Note, it’s important that the Traefik container is connected to the Docker network that the container with the services is on.

The example below is for a web server running on a different server, so this is defined in my traefik_dynamic.toml file.

[http.services]
  [http.services.web3.loadBalancer]
    [[http.services.web3.loadBalancer.servers]]
      url = "https://10.1.2.3:9443/"

In this case, Traefik will have a service called web3@file that talks to port 9443 on 10.1.2.3. Again, Traefik naturally needs to be able to see 10.1.2.3 for this to work.

Routers

This is where the magic happens, this is where you tell Traefik what requests to expect from clients and what to do with those requests.

Firstly, there is one automatic router that gets set up by entryPoints.web.http.redirections.entryPoint above (the same thing that creates the redirect-web-to-websecure@internal middleware). This router is called web-to-websecure@internal. It uses the rule HostRegexp(`{host:.+}`) and entryPoint web to catch all requests coming in via http and redirect them to https.

The other kind of router is the type you specify by way of either labels in Docker containers, or by way of stuff in traefik_dynamic.toml. Let’s have a look at the Docker way first. Below, you see some other bits of the docker compose file for the service example above.

# Stuff removed
# Traefik magic
    labels:
      traefik.enable: "true"

# Web server 1 on port 8080
      traefik.http.services.web1.loadbalancer.server.port: 8080
      traefik.http.routers.web1.rule: "Host(`web1.mydomain`)"
      traefik.http.routers.web1.tls: true
      traefik.http.routers.web1.entrypoints: web,websecure
      traefik.http.routers.web1.tls.certResolver: lets-encrypt
# Important label for multi-service containers
      traefik.http.routers.web1.service: "web1"

# Web server 2 on port 8081
      traefik.http.services.web2.loadbalancer.server.port: 8082
      traefik.http.routers.web2.rule: "Host(`web2.mydomain`)"
      traefik.http.routers.web2.tls: true
      traefik.http.routers.web2.entrypoints: web,websecure
      traefik.http.routers.web2.tls.certResolver: lets-encrypt
# Important label for multi-service containers
      traefik.http.routers.web2.service: "web2"

# Another router that talks to web2@docker
      traefik.http.routers.cdn.rule: "Host(`cdn.mydomain`)"
      traefik.http.routers.cdn.tls: true
      traefik.http.routers.cdn.entrypoints: web,websecure
      traefik.http.routers.cdn.tls.certResolver: lets-encrypt
# Important label for multi-service containers
      traefik.http.routers.cdn.service: "web2"
  • Each router uses a rule to figure out what request this router should handle. Here I just use a simple check of the Host header in the HTTP request (so SNI). I could serve web2 and cdn in one router by making the rule "Host(`web2.mydomain`) || Host(`cdn.mydomain`)" on one of the routers and dumping the other router completely, it’s basically just a matter of personal taste. There are a good number of rules checks available, have a look at https://doc.traefik.io/traefik/routing/routers/ for a list.
  • The tls label tells Traefik that it must use TLS between itself and the client for this router.
  • The entrypoints label tells Traefik which entry points (read: ports) to expect requests on.
  • The tls.certResolver label tells Traefik how to find (and possibly issue) the certificate for this route.
  • The service label tells Traefik which service to talk to for this router.

The other way I configure routers is in my traefik_dynamic.toml file, like this.

[http.routers.web3]
  rule = "Host(`web3.mydomain`)"
  entrypoints = ["web","websecure"]
  service = "web3@file"
  [http.routers.web3.tls]
    certResolver = "lets-encrypt"

You’ll immediately notice that this is very similar to the Docker labels method. It’s basically the same thing but in TOML instead of docker compose labels.

Posted

in

by

johan

Tags:

, ,