Ability to add additional files from secret(s) to Velero pod without restart

[kaovilai]

Describe the problem/challenge you have

In a multi-user environment sharing the same velero instance, a user should be able to add their own BackupStorageLocation without restarting velero server pod and/or node-agent pods.

Currently there exists some BackupStorageLocation configurations that will require mounting secrets to velero deployment and/or nodeagents.

• AWS plugin config customerKeyEncryptionFile
• credentialsFile (normally populated by velero internally to /tmp/credentials when bsl.spec.credential is populated)

Mounting interrupts running pods which we would like to avoid.

Describe the solution you'd like

Reuse current namespacedFileStore code to write additional files from secret user want to add to velero pod.

This could be a new field in BSL, perhaps bsl.spec.additionalSecrets
or a new velero custom resource where the controller will write secret content to velero pod.

We can also just write more files to the current /tmp/credentials for the other keys in the secret bsl references.

Anything else you would like to add:

Not restarting velero pod/node-agent has previously been mentioned as a requirement for non object storage support (NFS/filesystem) enhancement.

Environment:

• Velero version (use velero version):
• Kubernetes version (use kubectl version):
• Kubernetes installer & version:
• Cloud provider or hardware configuration:
• OS (e.g. from /etc/os-release):

Vote on this issue!

This is an invitation to the Velero community to vote on issues, you can see the project's top voted issues listed here.
Use the "reaction smiley face" up to the right of this comment to vote.

• 👍 for "The project would be better with this feature added"
• 👎 for "This feature will not enhance the project in a meaningful way"

[sseago]

Having to augment BSL definition with a new secret list would be more work for end users and would require velero to treat plugin-specific fields differently. Instead, could we just modify the way we use /tmp?

Current behavior:
For the following BSL config:

  spec:
    credential:
      key: cloud
      name: my-credentials

Velero creates a file "/tmp/credentials/velero-ns/my-credentials-cloud".

Instead, why not create a dir "/tmp/credentials/velero-ns/my-credentials" with all keys created with keyname as filename and value as key value? That way no custom logic is needed to manage file creation. Then any BSL that uses named credentials can set:

    customerKeyEncryptionFile: "/tmp/credentials/velero-ns/secret-name/encryption-file-key-name"