Expected behavior:

• Deploying HA teleport cluster on AWS using Terraform with TLS routing enabled (TF_VAR_use_tls_routing = true) should work out of box.

Current behavior:

• After deployment completes, public proxy endpoint return 502 Bad Gateway.
• Target group of teleport-proxy-acm shows instances unhealthy.

Bug details:

• Teleport version: v14.1.1

• Recreation steps: Follow the official document

• Debug logs

Logs on proxy server

Nov 07 08:45:21 ip-172-31-2-101.eu-central-1.compute.internal teleport[3503]: 2023-11-07T08:45:21Z ERRO [PROC:1] Instance failed to establish connection to cluster: cluster pin does not match any provided certificate authority pin. This could have occurred if the Certificate Authority (CA) for the cluster was rotated, invalidating the old pin. This could also occur if a new HSM was added. Run "tctl status" to compare the pin used
Nov 07 08:45:21 ip-172-31-2-101.eu-central-1.compute.internal teleport[3503]: 2023-11-07T08:45:21Z ERRO [PROC:1] Check to see if the config has auth_server pointing to a Teleport Proxy. If it does, use proxy_server instead of auth_server. pid:3503.1 service/connect.go:110

Logs on auth server

Nov 07 09:22:43 ip-172-31-0-142.eu-central-1.compute.internal teleport[4381]: 2023-11-07T09:22:43Z WARN [MXTLS:1] Handshake failed. dst_addr:172.31.0.142:3025 error:[remote error: tls: bad certificate] src_addr:172.31.4.167:52380 multiplexer/tls.go:150

Resolution:

• Set proxy_protocol=off for auth and proxy and restart the service.