Traefik 2.5 as Kubernetes Ingress controller

There is a list of quite a few third party ingress controllers supported by Kubernetes on their official website Kubernetes Ingress Controllers. Our choice fell on Traefik because this proxy, among other nice features, supports Let's Encrypt certificates. Let's Encrypt is a Certificate Authority which provides free SSL certificates, so we can esatblish secure connections to our applications. For demonstration, we are going to deploy a test application using this feature in this article.

We are going to use Kubernetes custom resource definitions (CRDs) and IngressRoute objects instead of Kubernetes native Ingress. The difference between the latter two is that IngressRoute is not a native Kubernetes object. Its functionality is more extendable, as it is defined by Traefik, in this case, and not by Kubernetes.

Most parts from the code below can be copied just as they are, we will separately point out what can/should be altered to use in your environment.

Traefik CRD, ClusterRole and ClusterRoleBinding

First, we need to create a manifest with CustomResourceDefinitions, ClusterRole and ClusterRoleBinding. With these objects we tell Kubernetes what functionality we want to use from Traefik. We took the manifests with the CustomResourceDefinitions (CRD), ClusterRoleBindings (CRB) and ClusterRole (CR) from the official Traefik documentation.

Through the link above, you can also find descriptions to every configuration parameter. We have created a single yaml-file with all the above-mentioned objects and named it traefik-api.yaml.

In order to apply it to your kubernetes cluster, use the following command:

kubectl apply -f traefik-api.yaml

PersistentVolume & PersistentVolumeClaim

In one of the next steps we are going to create an IngressRoute for secure connections - which means that Traefik needs some place where it can store certificates and keys and where they will stay even if the Traefik pod will be recreated.

For this purpose we will create a PersistentVolume and PersistentVolumeClaim that will be used by the Traefik pod:

apiVersion: v1
kind: PersistentVolume
metadata:
  name: taefik-pv
spec:
  capacity:
    storage: 32Mi
  accessModes:
  - ReadWriteOnce
  hostPath:
    path: /data/traefik
---
apiVersion: v1
kind: PersistentVolumeClaim
metadata:
  name: traefik-pvc
spec:
  accessModes:
  - ReadWriteOnce
  resources:
    requests:
      storage: 32Mi

In the example above (see lines 10-11) certificate data will be stored under /data/traefik on the host. Make sure you remember how you named your PersistentVolume (line 4) and PersistentVolumeClaim (line 17), you will need these names when deploying Traefiks pod.

You can adjust these lines if required and then create the resources:

kubectl apply -f traefik-pvc.yaml 

Traefik Deployment

Now we can deploy a Traefik pod. In the following manifest we include three types of objects: ServiceAccount, Deployment and Service.

• With the ServiceAccount we create an account to authenticate the Traefik pod, that speaks for itself.
• We are using a Deployment and not a DaemonSet because for a single-node cluster it is completely sufficient: the Deployment will create one pod on the node. If you have provisioned a multi-node cluster you might consider using a DaemonSet, so that on every node a copy of a Traefik pod is created. If you add a node later, a pod will be created on it automatically.
• Service will allow the pods to be accessible through the DNS-name of the Service.

There are few things you need to note and might need to edit in the yaml below:

• Make sure you specify the name you gave your ServiceAccount in the Traefik deployment (line 24).
• Also, in the Deployment just under ServiceAccount you should enter the name for PersistentVolume and PersistentVolumeClaim.
• With args in te Deployment we specify, among other things, the name of our certificate resolver. In our case it is certResolver. We will use this name in the IngressRoute of our test application to reference the certificate resolver.
• Make sure to specify a real e-mail address for certificates in the Deployment (line 46). Let's Encrypt will use this address to reach out to you (e.g. send warnings when your certificates are about to expire).
• Under ports in the Deployment we have three endpoints: web for normal HTTP requests, websecure for encrypted connections using TLS (here the Let's Encrypt certificates are used), and admin for the Traefk dashboard.
• The ports we have in the Service section have the same names and matching port numbers - this is how Traefik will forward incoming traffic to the specific ports.

apiVersion: v1
kind: ServiceAccount
metadata:
  name: traefik-ingress-controller
  namespace: default
---
kind: Deployment
apiVersion: apps/v1
metadata:
  name: traefik-deployment
  namespace: default
  labels:
    app: traefik-deployment
spec:
  replicas: 1
  selector:
    matchLabels:
      app: traefik-deployment
  template:
    metadata:
      labels:
        app: traefik-deployment
    spec:
      serviceAccountName: traefik-ingress-controller
      volumes:
        - name: traefik-pv
          persistentVolumeClaim:
            claimName: traefik-pvc
      containers:
        - name: traefik-application
          image: traefik:v2.5
          volumeMounts:
          - name: traefik-pv
            mountPath: "/acmeStorage"
          args:
            - --api
            - --api.insecure
            - --api.dashboard=true
            - --accesslog
            - --entrypoints.web.Address=:80
            - --entrypoints.websecure.Address=:443
            - --providers.kubernetescrd
            - --certificatesresolvers.certResolver.acme.storage=/acmeStorage/acme.json
            - --certificatesresolvers.certResolver.acme.httpchallenge
            - --certificatesresolvers.certResolver.acme.httpchallenge.entrypoint=web
            - --certificatesresolvers.certResolver.acme.email=mail@example.com
            - --providers.kubernetescrd=true
            - --entrypoints.web.http.redirections.entryPoint.to=websecure
            - --entrypoints.websecure.http.tls.certResolver=default
          ports:
            - name: web
              containerPort: 80
            - name: websecure
              containerPort: 443
            - name: admin
              containerPort: 8080
---
apiVersion: v1
kind: Service
metadata:
  name: traefik-service
  namespace: default

spec:
  ports:
    - protocol: TCP
      name: web
      port: 80
    - protocol: TCP
      name: admin
      port: 8080
    - protocol: TCP
      name: websecure
      port: 443
  selector:
    app: traefik-deployment

Now Traefik is installed and configured, and we can deploy a test application.

Application Deployment

We will create a simple Nginx "Hello World" deployment. Both the deployment object and the application service are included in the same manifest.

apiVersion: apps/v1
kind: Deployment
metadata:
  name: helloworld-deployment
  namespace: default
  labels:
    app: helloworld-deployment

spec:
  replicas: 1
  selector:
    matchLabels:
      app: helloworld-deployment
  template:
    metadata:
      labels:
        app: helloworld-deployment
    spec:
      containers:
      - name: helloworld-application
        image: nginxdemos/hello
        ports:
          - name: web
            containerPort: 80
---
apiVersion: v1
kind: Service
metadata:
  name: helloworld-service
  namespace: default

spec:
  selector:
    app: helloworld-deployment
  ports:
    - protocol: TCP
      name: web
      port: 80

Note the name of the service (line 29) - you will need to specify it later it in the IngressRoute manifest.

Port-Forwarding

Before we can create an IngressRoute for the newly deployed application, we need to make the Cluster available from the internet. For more details on port-forwarding, please see Traefik documentation on port-forwarding. The Traefik ACME provider will try to issue certificates as soon as it detects our IngressRoutes, and it will not work because if our pods cannot be reached from the outside.

kubectl port-forward --address 0.0.0.0 service/traefik 80:80 8080:8080 443:4443 -n default

Another way to do, which can be more practical for tests, is to save the command as bash-script and to run it in the background:

#!/bin/bash

nohup kubectl port-forward --address 0.0.0.0 service/traefik-service 80:80 8080:8080 443:4443 &

Keep in mind, that this way is not suitable for a production environments - if suddenly the script crashes the whole cluster will be cut from the Internet.

IngressRoute

In this final step we will create an IngressRoutes which uses TLS for a secure connection to the application. Here, we need to specify the name of the application service so that Traefik knows where to forward the incoming traffic to.

There are a couple of points you might want to change here:

• We use websecure under entryPoints in this case the IngressRoute will use the webSecure entry point we configured in our Traefik installation.
• Make sure to change match (line 10) to your domain name.
• Under services there should be the name you gave to the application service.
• Finally, in the last line we have to specifiy the name of the certResolver.

apiVersion: traefik.containo.us/v1alpha1
kind: IngressRoute
metadata:
  name: helloworld-ingress
  namespace: default
spec:
  entryPoints:
    - websecure
  routes:
  - match: Host(`your.domain.com`) 
    kind: Rule
    services:
    - name: helloworld-service
      port: 80
  tls:
    certResolver: certResolver

After a couple of seconds after applying the script to the cluster, it should be possible to access your application online.

If you want to have an insecure connection just change entryPoints to web and skip the tls part. If tls part is specified Traefik will ignore non TLS requests, so if you want to have both HTTP and HTTPS requests to be handled, you will need to specify two IngressRoutes.

Conclusion

We have deployed and configured Traefik 2.5 in Kubernetes and afterwords deployed and exposed a test application using a secured connection with the help of Let's Encrypt certificates.

Tags

• Kubernetes
• Ingress
• IngressRoute
• Traefik