About

This creates an example kubernetes cluster hosted in the AWS Elastic Kubernetes Service (EKS) using a terraform program.

This will:

• Create an Elastic Kubernetes Service (EKS)-based Kubernetes cluster.
  • Enable the VPC CNI cluster addon.
• Create the Elastic Container Registry (ECR) repositories declared on the images local variable, and upload the corresponding container images.
• Demonstrate how to manually deploy a Kubernetes application.
  • Expose as a Kubernetes LoadBalancer Service. Note that this results in the creation of an EC2 Classic Load Balancer (CLB).

Usage (on a Ubuntu Desktop)

Install the dependencies:

• AWS CLI.
• Terraform.
• kubectl.
• Docker.
• Crane.

Set the AWS Account credentials using SSO:

# set the environment variables to use a specific profile.
# e.g. use the pattern <aws-sso-session-name>-<aws-account-name>-<aws-account-role>-<aws-account-id>
export AWS_PROFILE=example-dev-AdministratorAccess-123456
unset AWS_ACCESS_KEY_ID
unset AWS_SECRET_ACCESS_KEY
unset AWS_DEFAULT_REGION
# set the account credentials.
# see https://docs.aws.amazon.com/cli/latest/userguide/sso-configure-profile-token.html#sso-configure-profile-token-auto-sso
aws configure sso
# dump the configured profile and sso-session.
cat ~/.aws/config
# show the user, user amazon resource name (arn), and the account id, of the
# profile set in the AWS_PROFILE environment variable.
aws sts get-caller-identity

Or, set the AWS Account credentials using an Access Key:

# set the account credentials.
# NB get these from your aws account iam console.
#    see Managing access keys (console) at
#        https://docs.aws.amazon.com/IAM/latest/UserGuide/id_credentials_access-keys.html#Using_CreateAccessKey
export AWS_ACCESS_KEY_ID='TODO'
export AWS_SECRET_ACCESS_KEY='TODO'
unset AWS_PROFILE
# set the default region.
export AWS_DEFAULT_REGION='eu-west-1'
# show the user, user amazon resource name (arn), and the account id.
aws sts get-caller-identity

Review main.tf.

Initialize terraform:

make terraform-init

Launch the example:

make terraform-apply

Show the terraform state:

make terraform-state-list
make terraform-show

Show the OpenID Connect Discovery Document (aka OpenID Connect Configuration):

wget -qO- "$(terraform output -raw kubernetes_oidc_configuration_url)" \
  | jq

Access the EKS cluster:

export KUBECONFIG="$PWD/kubeconfig.yml"
kubectl cluster-info
kubectl get nodes -o wide

If for some reason the kubeconfig.yml is stale, you can re-created it, either with make terraform-apply, or:

rm "$KUBECONFIG"
aws eks update-kubeconfig \
  --region "$(terraform output -raw kubernetes_region)" \
  --name "$(terraform output -raw kubernetes_cluster_name)"

Log in the container registry:

NB You are logging in at the registry level. You are not logging in at the repository level.

aws ecr get-login-password \
  --region "$(terraform output -raw registry_region)" \
  | docker login \
      --username AWS \
      --password-stdin \
      "$(terraform output -raw registry_domain)"

NB This saves the credentials in the ~/.docker/config.json local file.

Show the example image manifest that was uploaded into the created container image repository:

image="$(terraform output --json images | jq -r .example)"
crane manifest "$image" | jq .

Log out the container registry:

docker logout \
  "$(terraform output -raw registry_domain)"

Launch the example application, using the image that was uploaded into the created image repository:

sed -E "s,ruilopes/example-docker-buildx-go:.+,$image,g" example-app.yml \
  | kubectl apply -f -
kubectl rollout status daemonset/example
kubectl get pods,services

Access the service from a kubectl port-forward local port:

kubectl port-forward service/example 6789:80 &
sleep 3 && printf '\n\n'
wget -qO- http://localhost:6789
kill %1 && sleep 3

Access the service from the Internet:

example_domain="$(kubectl get service/example -o jsonpath='{.status.loadBalancer.ingress[0].hostname}')"
example_url="http://$example_domain"
echo "example service url: $example_url"
# wait for the domain to resolve.
while [ -z "$(dig +short "$example_domain")" ]; do sleep 5; done && dig "$example_domain"
# finally, access the service.
wget -qO- "$example_url"

Destroy the example application:

sed -E "s,ruilopes/example-docker-buildx-go:.+,$image,g" example-app.yml \
  | kubectl delete -f -
kubectl get pods,services

Destroy the example:

make terraform-destroy

NB For some unknown reason, terraform shows the following Warning message. If you known how to fix it, please let me known!

╷
│ Warning: EC2 Default Network ACL (acl-004fd974909c20839) not deleted, removing from state
│
│
╵

Notes

• OpenID Connect Provider for EKS (aka Enable IAM Roles for Service Accounts (IRSA)) is enabled.
  • a aws_iam_openid_connect_provider resource is created.
• The EKS nodes virtual machines boot from a customizable Amazon Machine Image (AMI).
  • This example uses the Amazon Linux 2 AMI.
  • The official AMIs source code is available at the Amazon EKS AMI awslabs/amazon-eks-ami repository.

References

• Environment variables to configure the AWS CLI
• Token provider configuration with automatic authentication refresh for AWS IAM Identity Center (SSO)
• Managing access keys (console)
• AWS General Reference
  • Amazon Resource Names (ARNs)
• Amazon ECR private registry
  • Private registry authentication
• Network load balancing on Amazon EKS
• Amazon EKS add-ons
• Amazon EKS VPC-CNI
• EKS Workshop
  • Using Terraform
    • aws-samples/eks-workshop-v2 example repository
• Official Amazon EKS AMI awslabs/amazon-eks-ami repository