Provision a Docker Swarm Host with Traefik (v2) on Hetzner Cloud using Terraform Modules - Part 2

Introduction

Part One covered the use of the docker-host Terraform module to provision a Docker host (in Swarm mode) on Hetzner Cloud.

We can now again make use of Terraform modules to deploy a Traefik container on this (or any) host to serve as a load balancer/reverse proxy for other containered applications running on the same host.

In this tutorial we’ll setup Traefik to route incoming requests for a Vault backend service.

Traefik Network Layout Network Layout - Traefik routing HTTP requests based on host name.

Two modules are used in a Terraform configuration to achieve this.

(traefik-v2) for provisioning a Traefik container on the Docker host.
(vault-dev) for provisioning a Vault container on the Docker host.

The docker-traefik-v2 module deploys a pre-configured Traefik (2.3.x) reverse proxy container. The same configuration deploys the HashiCorp Vault application using the docker-vault-dev module and connects it to the traefik overlay network so Traefik can route incoming requests to it.

Prerequisites

A Docker host (in Swarm mode)
Terraform ≥ v0.13
Knowledge of Docker & Terraform

Step 1 - Create the Terraform Configuration

Clone the example repository https://github.com/colinwilson/example-terraform-modules/tree/docker-traefik-and-vault

git clone -b docker-traefik-and-vault https://github.com/colinwilson/example-terraform-modules

example-terraform-modules
|-- .gitignore
|-- README.md
|-- main.tf
|-- outputs.tf
|-- terraform.tfvars
`-- variables.tf

Step 2 - Set Values for the Required Input Variables

Using the example below, update the input variables in the terraform.tfvars file to match your own.

# terraform.tfvars (example)
docker_host = "ssh://my-docker-host:22"

traefik_hostname = "traefik.example.com"

traefik_password = "my_secret_password"

vault_hostname = "vault.example.com"

acme_email = "myname@example.com"

The docker_host variable should be set to the connection string (required by the Docker provider).
Set a password for accessing the Traefik UI, traefik_password. (The Traefik module hashes your password using Bcrypt)
Set the hostnames via which both Traefik and Vault will be publicly accessible e.g.
traefik.example.com,
vault.example.com
(set DNS records for both hostnames pointing to the IP address of your Docker host). This allows Traefik to automatically configure an SSL certificate for both services via Let’s Encrypt using httpChallenge.
Note: The DNS records for both hostnames should be configured BEFORE deploying your configuration (running terraform apply)
Finally set the acme_email variable to an email address you wish to register with Let’s Encrypt for SSL certificate requests:

Step 3 - Deploy Traefik & Vault on the Docker Host

Switch to your example-terraform-modules directory and initialize your configuration by running terraform init.

terraform init

Terraform will proceed to download the required provider plugins for both modules.

EXAMPLE OUTPUT - (click to expand)

Initializing modules...
Downloading github.com/colinwilson/terraform-docker-traefik-v2 for docker-traefik...
- docker-traefik in .terraform\modules\docker-traefik
Downloading github.com/colinwilson/terraform-docker-vault-dev for docker-vault...
- docker-vault in .terraform\modules\docker-vault

Initializing the backend...

Initializing provider plugins...
- Finding latest version of kreuzwerker/docker...
- Finding latest version of hashicorp/template...
- Finding latest version of hashicorp/local...
- Using hashicorp/local v2.0.0 from the shared cache directory
- Using kreuzwerker/docker v2.8.0 from the shared cache directory
- Using hashicorp/template v2.2.0 from the shared cache directory

Terraform has created a lock file .terraform.lock.hcl to record the provider
selections it made above. Include this file in your version control repository
so that Terraform can guarantee to make the same selections by default when
you run "terraform init" in the future.

Terraform has been successfully initialized!

# ...

Now run terraform apply to create the Traefik and Vault containers.

terraform apply

EXAMPLE OUTPUT - (click to expand)

An execution plan has been generated and is shown below.
Resource actions are indicated with the following symbols:
  + create
 <= read (data resources)

Terraform will perform the following actions:

  # module.docker-traefik.docker_config.traefik-yaml will be created
  + resource "docker_config" "traefik-yaml" {
      + data = (sensitive value)
      + id   = (known after apply)
      + name = (known after apply)
    }

  # module.docker-traefik.docker_network.network will be created
  + resource "docker_network" "network" {
      + attachable      = true
      + check_duplicate = true
      + driver          = "overlay"
      + id              = (known after apply)
      + internal        = (known after apply)
      + ipam_driver     = "default"
      + name            = "traefik"
      + options         = (known after apply)
      + scope           = (known after apply)

      + ipam_config {
          + aux_address = (known after apply)
          + gateway     = (known after apply)
          + ip_range    = (known after apply)
          + subnet      = (known after apply)
        }
    }

  # module.docker-traefik.docker_service.traefik will be created
  + resource "docker_service" "traefik" {
      + id   = (known after apply)
      + name = "traefik"

      + endpoint_spec {
          + mode = (known after apply)

          + ports {
              + protocol       = "tcp"
              + publish_mode   = "host"
              + published_port = 80
              + target_port    = 80
            }
          + ports {
              + protocol       = "tcp"
              + publish_mode   = "host"
              + published_port = 443
              + target_port    = 443
            }
        }

      + labels {
          + label = (known after apply)
          + value = (known after apply)
        }

      + mode {
          + global = (known after apply)

          + replicated {
              + replicas = (known after apply)
            }
        }

      + task_spec {
          + force_update   = (known after apply)
          + networks       = [
              + "traefik",
            ]
          + restart_policy = (known after apply)
          + runtime        = (known after apply)

          + container_spec {
              + env               = {
                  + "CF_API_EMAIL"      = ""
                  + "CF_API_KEY"        = ""
                  + "CF_DNS_API_TOKEN"  = ""
                  + "CF_ZONE_API_TOKEN" = ""
                }
              + image             = "traefik:2.3.5"
              + isolation         = "default"
              + stop_grace_period = (known after apply)

              + configs {
                  + config_id   = (known after apply)
                  + config_name = (known after apply)
                  + file_gid    = "0"
                  + file_mode   = 292
                  + file_name   = "/traefik.yaml"
                  + file_uid    = "0"
                }

              + dns_config {
                  + nameservers = (known after apply)
                  + options     = (known after apply)
                  + search      = (known after apply)
                }

              + healthcheck {
                  + interval     = (known after apply)
                  + retries      = (known after apply)
                  + start_period = (known after apply)
                  + test         = (known after apply)
                  + timeout      = (known after apply)
                }

              + labels {
                  + label = "traefik.enable"
                  + value = "true"
                }
              + labels {
                  + label = "traefik.http.middlewares.https_redirect.redirectscheme.permanent"
                  + value = "true"
                }
              + labels {
                  + label = "traefik.http.middlewares.https_redirect.redirectscheme.scheme"
                  + value = "https"
                }
              + labels {
                  + label = "traefik.http.middlewares.traefik-basic-auth.basicauth.removeheader"
                  + value = "true"
                }
              + labels {
                  + label = "traefik.http.middlewares.traefik-basic-auth.basicauth.users"
                  + value = (known after apply)
                }
              + labels {
                  + label = "traefik.http.routers.http_catchall.entrypoints"
                  + value = "http"
                }
              + labels {
                  + label = "traefik.http.routers.http_catchall.middlewares"
                  + value = "https_redirect"
                }
              + labels {
                  + label = "traefik.http.routers.http_catchall.rule"
                  + value = "HostRegexp(`{any:.+}`)"
                }
              + labels {
                  + label = "traefik.http.routers.metrics.entrypoints"
                  + value = "http"
                }
              + labels {
                  + label = "traefik.http.routers.metrics.middlewares"
                  + value = "traefik-basic-auth"
                }
              + labels {
                  + label = "traefik.http.routers.metrics.rule"
                  + value = "Host(`traefik`) && PathPrefix(`/metrics`)"
                }
              + labels {
                  + label = "traefik.http.routers.metrics.service"
                  + value = "prometheus@internal"
                }
              + labels {
                  + label = "traefik.http.routers.traefik.entrypoints"
                  + value = "https"
                }
              + labels {
                  + label = "traefik.http.routers.traefik.middlewares"
                  + value = "traefik-basic-auth"
                }
              + labels {
                  + label = "traefik.http.routers.traefik.rule"
                  + value = "Host(`traefik-beta.example.com`)"
                }
              + labels {
                  + label = "traefik.http.routers.traefik.service"
                  + value = "api@internal"
                }
              + labels {
                  + label = "traefik.http.routers.traefik.tls.certresolver"
                  + value = "letsEncryptStaging"
                }

              + mounts {
                  + read_only = false
                  + source    = "traefik_acme"
                  + target    = "/etc/traefik/acme"
                  + type      = "volume"
                }
              + mounts {
                  + read_only = true
                  + source    = "/var/run/docker.sock"
                  + target    = "/var/run/docker.sock"
                  + type      = "bind"
                }
            }

          + placement {
              + constraints = (known after apply)
              + prefs       = (known after apply)

              + platforms {
                  + architecture = (known after apply)
                  + os           = (known after apply)
                }
            }

          + resources {
              + limits {
                  + memory_bytes = (known after apply)
                  + nano_cpus    = (known after apply)

                  + generic_resources {
                      + discrete_resources_spec = (known after apply)
                      + named_resources_spec    = (known after apply)
                    }
                }

              + reservation {
                  + memory_bytes = (known after apply)
                  + nano_cpus    = (known after apply)

                  + generic_resources {
                      + discrete_resources_spec = (known after apply)
                      + named_resources_spec    = (known after apply)
                    }
                }
            }
        }
    }

  # module.docker-traefik.docker_volume.traefik_acme will be created
  + resource "docker_volume" "traefik_acme" {
      + driver     = (known after apply)
      + id         = (known after apply)
      + mountpoint = (known after apply)
      + name       = "traefik_acme"
    }

  # module.docker-vault.data.local_file.vault_hcl will be read during apply
  # (config refers to values not yet known)
 <= data "local_file" "vault_hcl"  {
      + content        = (known after apply)
      + content_base64 = (known after apply)
      + filename       = ".terraform/modules/docker-vault/vault-config.hcl"
      + id             = (known after apply)
    }

  # module.docker-vault.data.template_file.vault_hcl will be read during apply
  # (config refers to values not yet known)
 <= data "template_file" "vault_hcl"  {
      + id       = (known after apply)
      + rendered = (known after apply)
      + template = <<-EOT
            storage "file" {
                path    = "/vault/file"
            }

            listener "tcp" {
                address     = "0.0.0.0:8200"
                tls_disable = true
            }

            ui = true
            disable_mlock = true
        EOT
    }

  # module.docker-vault.docker_config.vault_hcl will be created
  + resource "docker_config" "vault_hcl" {
      + data = (sensitive value)
      + id   = (known after apply)
      + name = (known after apply)
    }

  # module.docker-vault.docker_service.vault-dev will be created
  + resource "docker_service" "vault-dev" {
      + id   = (known after apply)
      + name = "vault-dev"

      + endpoint_spec {
          + mode = (known after apply)

          + ports {
              + name           = (known after apply)
              + protocol       = (known after apply)
              + publish_mode   = (known after apply)
              + published_port = (known after apply)
              + target_port    = (known after apply)
            }
        }

      + labels {
          + label = (known after apply)
          + value = (known after apply)
        }

      + mode {
          + global = (known after apply)

          + replicated {
              + replicas = (known after apply)
            }
        }

      + task_spec {
          + force_update   = (known after apply)
          + networks       = [
              + "traefik",
            ]
          + restart_policy = (known after apply)
          + runtime        = (known after apply)

          + container_spec {
              + args              = [
                  + "server",
                ]
              + env               = {
                  + "SKIP_SETCAP"    = "true"
                  + "VAULT_ADDR"     = "http://127.0.0.1:8200"
                  + "VAULT_API_ADDR" = "http://127.0.0.1:8200"
                }
              + image             = "vault:1.6.0"
              + isolation         = "default"
              + stop_grace_period = (known after apply)

              + configs {
                  + config_id   = (known after apply)
                  + config_name = (known after apply)
                  + file_gid    = "0"
                  + file_mode   = 292
                  + file_name   = "/vault/config/vault-config.hcl"
                  + file_uid    = "0"
                }

              + dns_config {
                  + nameservers = (known after apply)
                  + options     = (known after apply)
                  + search      = (known after apply)
                }

              + healthcheck {
                  + interval     = (known after apply)
                  + retries      = (known after apply)
                  + start_period = (known after apply)
                  + test         = (known after apply)
                  + timeout      = (known after apply)
                }

              + labels {
                  + label = "traefik.enable"
                  + value = "true"
                }
              + labels {
                  + label = "traefik.http.routers.vault-dev.entrypoints"
                  + value = "https"
                }
              + labels {
                  + label = "traefik.http.routers.vault-dev.rule"
                  + value = "Host(`vault-beta.example.com`)"
                }
              + labels {
                  + label = "traefik.http.routers.vault-dev.tls.certresolver"
                  + value = "letsEncryptStaging"
                }
              + labels {
                  + label = "traefik.http.services.vault-dev.loadbalancer.server.port"
                  + value = "8200"
                }

              + mounts {
                  + read_only = false
                  + source    = "vault_data"
                  + target    = "/vault/file"
                  + type      = "volume"
                }
              + mounts {
                  + read_only = false
                  + source    = "vault_logs"
                  + target    = "/vault/logs"
                  + type      = "volume"
                }
              + mounts {
                  + read_only = false
                  + source    = "vault_polices"
                  + target    = "/vault/policies"
                  + type      = "volume"
                }
            }

          + placement {
              + constraints = (known after apply)
              + prefs       = (known after apply)

              + platforms {
                  + architecture = (known after apply)
                  + os           = (known after apply)
                }
            }

          + resources {
              + limits {
                  + memory_bytes = (known after apply)
                  + nano_cpus    = (known after apply)

                  + generic_resources {
                      + discrete_resources_spec = (known after apply)
                      + named_resources_spec    = (known after apply)
                    }
                }

              + reservation {
                  + memory_bytes = (known after apply)
                  + nano_cpus    = (known after apply)

                  + generic_resources {
                      + discrete_resources_spec = (known after apply)
                      + named_resources_spec    = (known after apply)
                    }
                }
            }
        }
    }

  # module.docker-vault.docker_volume.vault_data will be created
  + resource "docker_volume" "vault_data" {
      + driver     = (known after apply)
      + id         = (known after apply)
      + mountpoint = (known after apply)
      + name       = "vault_data"
    }

  # module.docker-vault.docker_volume.vault_logs will be created
  + resource "docker_volume" "vault_logs" {
      + driver     = (known after apply)
      + id         = (known after apply)
      + mountpoint = (known after apply)
      + name       = "vault_logs"
    }

  # module.docker-vault.docker_volume.vault_policies will be created
  + resource "docker_volume" "vault_policies" {
      + driver     = (known after apply)
      + id         = (known after apply)
      + mountpoint = (known after apply)
      + name       = "vault_polices"
    }

Plan: 9 to add, 0 to change, 0 to destroy.

Changes to Outputs:
  + acme_mountpoint             = (known after apply)
  + traefik_network_name        = "traefik"
  + traefik_service_config_name = (known after apply)

Do you want to perform these actions?
  Terraform will perform the actions described above.
  Only 'yes' will be accepted to approve.

  Enter a value:

Respond to the prompt with yes to apply the changes and continue.

Terraform will now create the Traefik and Vault containers and you should see a message confirming the creation of resources (line 1 below). The output values generated by both modules should also be returned.


Apply complete! Resources: 9 added, 0 changed, 0 destroyed.

Outputs:

acme_mountpoint = "/mnt/docker_data_volume/volumes/traefik_acme/_data"
traefik_network_name = "traefik"
traefik_service_config_name = "traefik-yaml-2020-12-18T06.56.56Z"
vault_service_config_name = "vault_hcl-2020-12-18T06.57.07Z"

Step 4 - Confirm Traefik and Vault have been successfully deployed

After a few minutes Traefik should be proxying both its own UI and that of the Vault container. You should be able to visit the hostnames configured earlier to access each service.

Note: Both modules by default configure a staging SSL certificate, so expect to see browser warnings. Setting a live_cert variable in either module to true will configure a live cert for the corresponding service.

Visiting vault.example.com should present you with the default setup page to begin initialising the Vault install:

Vault UI

When visiting traefik.example.com, before being granted access to the Traefik dashboard your browser will prompt you to authenticate using the password configured in the terraform.tfvars file earlier:

Traefik Dashboard

Note: The Traefik module has the RedirectScheme middleware configured by default so all non-secure HTTP requests are automatically redirected to HTTPS

Step 5. Clean up

You can destroy both containers by running terraform destroy. After you respond to the prompt with yes, Terraform will destroy all the containers and Docker volumes created prior.

terraform destroy

EXAMPLE OUTPUT - (click to expand)

module.docker-vault.docker_service.vault-dev: Destroying... [id=ukpukp31n4mz55xoiqgy2k91l]
module.docker-vault.docker_service.vault-dev: Destruction complete after 2s
module.docker-vault.docker_volume.vault_data: Destroying... [id=vault_data]
module.docker-vault.docker_config.vault_hcl: Destroying... [id=p0hbxkitvpwf0t3dqvwyh6e0c]
module.docker-vault.docker_volume.vault_logs: Destroying... [id=vault_logs]
module.docker-vault.docker_volume.vault_policies: Destroying... [id=vault_polices]
module.docker-vault.docker_config.vault_hcl: Destruction complete after 0s
module.docker-vault.docker_volume.vault_data: Destruction complete after 3s
module.docker-vault.docker_volume.vault_logs: Destruction complete after 7s
module.docker-vault.docker_volume.vault_policies: Destruction complete after 7s
module.docker-traefik.docker_network.network: Destroying... [id=uufct6ond0qdzlnyzmna88lqe]
module.docker-traefik.docker_service.traefik: Destroying... [id=vs40s524qdkwj1bnenmqogeop]
module.docker-traefik.docker_service.traefik: Destruction complete after 2s
module.docker-traefik.docker_volume.traefik_acme: Destroying... [id=traefik_acme]
module.docker-traefik.docker_config.traefik-yaml: Destroying... [id=bbm9rnfaiqttfzdeur0lhtzgo]
module.docker-traefik.docker_config.traefik-yaml: Destruction complete after 1s
module.docker-traefik.docker_network.network: Destruction complete after 3s
module.docker-traefik.docker_volume.traefik_acme: Destruction complete after 3s

Destroy complete! Resources: 9 destroyed.

Conclusion

That’s it! You’ve now successfully deployed a Traefik load balancer/proxy container for routing incoming requests to backend services/containers. In this case to a Vault application service backend. The pre-built Terraform modules demonstrate how easy it is to automate the speedy deployment of infrastructure.

Provision a Docker Swarm Host with Traefik (v2) on Hetzner Cloud using Terraform Modules - Part 2

Introduction

Prerequisites

Step 1 - Create the Terraform Configuration

Step 2 - Set Values for the Required Input Variables

Step 3 - Deploy Traefik & Vault on the Docker Host

Step 4 - Confirm Traefik and Vault have been successfully deployed

Step 5. Clean up

Conclusion

Comments