Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Adding kustomize templates for k8s onboarding #4754

Draft
wants to merge 27 commits into
base: main
Choose a base branch
from
Draft

Conversation

gizas
Copy link
Contributor

@gizas gizas commented May 15, 2024

  • Enhancement

What does this PR do?

WHAT: Introduces the onboarding templates of kustomize for k8s observability
WHY: #4657

Checklist

  • My code follows the style guidelines of this project
  • I have commented my code, particularly in hard-to-understand areas
  • [] I have made corresponding changes to the documentation
  • I have made corresponding change to the default configuration files
  • I have added tests that prove my fix is effective or that my feature works
  • I have added an entry in ./changelog/fragments using the changelog tool
  • I have added an integration test or an E2E test

How to test this PR locally

  1. Clone this repo
  2. Change to directory elastic/elastic-agent/deploy/kubernetes
  3. Use makefile to generate templates
GENERATEKUSTOMIZE=true make ci-create-kustomize
  1. test the templates with
cd elastic-agent-kustomize/onboarding/default
kubectl apply -k elastic-agent-managed
or 
kubectl apply -k elastic-agent-standalone

Examples of online references:

kubectl kustomize https://github.com/elastic/elastic-agent/deploy/kubernetes/elastic-agent-kustomize/default/elastic-agent-managed\?ref\=kustomizepoc

kubectl kustomize https://github.com/elastic/elastic-agent/deploy/kubernetes/elastic-agent-kustomize/default/elastic-agent-managed\?ref\=main

kubectl apply -k https://github.com/elastic/elastic-agent/deploy/kubernetes/elastic-agent-kustomize/default/elastic-agent-managed\?ref\=v8.13.1

Testing for onboarding procedure

Managed:

❯ kubectl kustomize elastic-agent-kustomize/default/elastic-agent-managed/ | sed -e "s/JUVOUk9MTE1FTlRfVE9LRU4l/base64_ENCODED_ENROLLMENT_TOKEN/g" -e "s/%FLEET_URL%/https:\/\/localhost:9200/g" | kubectl apply -f-

Standalone

❯ kubectl kustomize elastic-agent-kustomize/default/elastic-agent-standalone/ | sed -e "s/JUFQSV9LRVkl/<base64_encoded_APIKEY>/g" -e "s/%ES_HOST%/https:\/\/localhost:9200/g" -e "s/%CA_TRUSTED%/ca_trusted_fingerprint/g" -e "s/%ELASTIC_AGENT_ID%/12345/g" | kubectl apply -f-

Eg.

❯ kubectl kustomize https://github.com/elastic/elastic-agent/deploy/kubernetes/elastic-agent-kustomize/default/elastic-agent-managed\?ref\=kustomizepoc |  sed -e 's/JUFQSV9LRVkl/JUFQSV9LRVkl/g' -e "s/%ES_HOST%/https:\/\/localhost:9200/g" -e "s/%CA_TRUSTED%/ca_trusted_fingerprint/g" -e "s/%ELASTIC_AGENT_ID%/12345/g" |  kubectl apply -f-

Note: API_KEY should be given as bas64 encoded values from Kibana

Base 64 encoding decding:

echo '${enrollment_token}' | base64
JHtlbnJvbGxtZW50X3Rva2VufQo=
❯ echo '${api_key}' | base64
JHthcGlfa2V5fQo=
> echo JHtlbnJvbGxtZW50X3Rva2VufQ== | base64 -D
${enrollment_token}

Images

Successful enrollement in managed

Screenshot 2024-05-15 at 5 31 11 PM

Successful Installation of kube-state-metrics and

kgp -n kube-system
NAME                                         READY   STATUS    RESTARTS       AGE
coredns-5d78c9869d-b6np9                     1/1     Running   6 (7d8h ago)   22d
coredns-5d78c9869d-p62sz                     1/1     Running   6 (7d8h ago)   22d
elastic-agent-tvskp                          1/1     Running   0              11s
etcd-kind-control-plane                      1/1     Running   0              7d8h
kindnet-qpxrb                                1/1     Running   6 (7d8h ago)   22d
kube-apiserver-kind-control-plane            1/1     Running   0              7d8h
kube-controller-manager-kind-control-plane   1/1     Running   6 (7d8h ago)   22d
kube-proxy-96hfl                             1/1     Running   6 (7d8h ago)   22d
kube-scheduler-kind-control-plane            1/1     Running   6 (7d8h ago)   22d
kube-state-metrics-5f89fb6d84-8f2sm          1/1     Running   0              11s

Related issues

@gizas gizas requested a review from constanca-m May 15, 2024 11:27
@mergify mergify bot assigned gizas May 15, 2024
Copy link
Contributor

mergify bot commented May 15, 2024

This pull request does not have a backport label. Could you fix it @gizas? 🙏
To fixup this pull request, you need to add the backport labels for the needed
branches, such as:

  • backport-v./d./d./d is the label to automatically backport to the 8./d branch. /d is the digit

NOTE: backport-skip has been added to this pull request.

kind: DaemonSet
metadata:
name: elastic-agent
namespace: kube-system
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Wonder if all namespace fields should be removed from all files apart from the kustomization.yaml? The value gets correctly overwritten anyway, but it is not doing anything anymore.

@@ -28,19 +28,21 @@ spec:
# Uncomment if using hints feature
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Maybe we could have another base for hints, instead of leaving it in a comment here.

Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

As we discuss lets tackle this in another story

@constanca-m
Copy link
Contributor

constanca-m commented May 15, 2024

I am a bit confused. What is the different between default and onboarding directories?


Users can clone this repository to use the provided kustomize templates.

For *Managed Elastic Agent*, please update the following secrets inside main kustomization.yaml:
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Suggested change
For *Managed Elastic Agent*, please update the following secrets inside main kustomization.yaml:
For *Managed Elastic Agent*, please update the following secrets inside main [kustomization.yaml](./elastic-agent-managed/kustomization.yaml):

- api_key=${enrollment_token}
```

For *Standalone Elastic Agent*, please update the following secrets inside main [kustomization.yaml](./elastic-agent-managed/kustomization.yaml):
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Suggested change
For *Standalone Elastic Agent*, please update the following secrets inside main [kustomization.yaml](./elastic-agent-managed/kustomization.yaml):
For *Standalone Elastic Agent*, please update the following secrets inside main [kustomization.yaml](./elastic-agent-standalone/kustomization.yaml):


Managed Elastic Agent:

```bash
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Would this be the command our users would get from this Kibana page?

image

Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I think a more intuitive approach would be to change the kustomization.yaml file directly. So the user would see this file in Kibana:

apiVersion: kustomize.config.k8s.io/v1beta1
kind: Kustomization

namespace: kube-system

images:
  - name: docker.elastic.co/beats/elastic-agent
    newTag: ${IMAGE_TAG}

secretGenerator:
    - name: elastic-agent-creds
      literals:
        - host=${ES_HOST} 
        - enrollment_token=${ENROLLMENT_TOKEN} 

resources:
  - ./base  # <------------ This would need to point to our GitHub directory
  - https://github.com/kubernetes/kube-state-metrics

patches:
- path: environmental-variables-remove.yaml
- target:
    group: apps
    version: v1
    kind: DaemonSet
    name: elastic-agent
  path: fleet-enrollment-token-patch.yaml

And all ${...} variables should have literal values by that point.

I think the problem with that command is that it still requires us to evaluate base64 of the variables and we end up not making use of the literal of the secret map. And then the sed is not easy to read. What do you think?

Copy link

Quality Gate passed Quality Gate passed

Issues
0 New issues
0 Fixed issues
0 Accepted issues

Measures
0 Security Hotspots
No data about Coverage
No data about Duplication

See analysis details on SonarQube

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Projects
None yet
Development

Successfully merging this pull request may close these issues.

None yet

2 participants