Question about cluster- and namespace resource inclusion

[PieterVoet]

Hi all,

what I want to get done here is to create a backup that includes everything from a namespace, e.g. 'myns'.
In that namespace some operators are installed, which create corresponding clusterroles and clusterbindings,
which I also want to be included in the backup.
I found that the clusterroles/bindings are labelled : olm.owner.namespace=myns

I tried this Backup CR :

apiVersion: velero.io/v1
kind: Backup
metadata:
  name: myns-s3backup-try-2024-05-13
  namespace: openshift-adp
spec:
  csiSnapshotTimeout: 10m0s
  defaultVolumesToFsBackup: false
  hooks: {}
  includedNamespaces:
  - myns
  itemOperationTimeout: 4h0m0s
  snapshotMoveData: false
  storageLocation: backup-s3-1
  ttl: 720h0m0s
  orLabelSelectors: 
  - matchLabels:
      olm.owner.namespace: myns

This creates a Backup that backs up everything that matches the label, however I was expecting that this backup would back up :

• the 'myns' namespace recursively
• all clusterscoped resources with the label

Splitting this into two backups sort of gives my what I want : the namespace with it's resources, and the corresponding
clusterroles/bindings, but I prefer to have just one backup to manage.

( NOTE : I know that after restoring a namespace backup, deleting the operator CSV's will trigger a reinstall,
including the recreation of the required clusterroles/bindings, but the restore then will take a little longer )

OpenShifts OADP operator has Velero 1.12.4. Searching the web, I found #5838
but that was expected to be included in Velero 1.11.

Maybe I get it conceptually wrong, and I'm hoping to get some guidance here.
Help is appreciated.
Thanks and have a nice day.

[kaovilai]

There have been recent developments around this. See #7492

Please check if that satisfies your requirements.

[PieterVoet]

Hi kaovilai, thanks for responding.

After reading #5838 I understand that another way of backing up a namespace
is implemented, which uses only one label, added to the NameSpace resource. This would trigger all resources in that
namespace to be backed up. Like you said last week : 'Which is equivalent to --includedNamespaces right? '.

For now I can see no advantage to my issue. I do understand that this will enable making thatever is being backed up
is dynamic, if labels are changed runtime.

However, 'blackpiglet' states :

If the namespace I/E filters and the (Or)LabelSelectors selected namespaces are different. The tracker takes the union of them.

that would explain that my assumption on my Backup CR is wrong. So I think, unless the logic to assemble a union is not changed,
even #5838 is probably not solving my issue..

thanks !

[blackpiglet]

For your scenario, just creating a backup including the namespace should work.
The reason is that Velero can resolve some k8s resource dependency. That includes the ServiceAccount's relationship with ClusterRole, ClusterRoleBinding.

velero/pkg/backup/actions/service_account_action.go

Lines 92 to 118 in cdd5a4f

	for _, crb := range a.clusterRoleBindings {
	for _, s := range crb.ServiceAccountSubjects(namespace) {
	if s == name {
	a.log.Infof("Adding clusterrole %s and clusterrolebinding %s to additionalItems since serviceaccount %s/%s is a subject",
	crb.RoleRefName(), crb.Name(), namespace, name)
	bindings.Insert(crb.Name())
	roles.Insert(crb.RoleRefName())
	break
	}
	}
	}
	var additionalItems []velero.ResourceIdentifier
	for binding := range bindings {
	additionalItems = append(additionalItems, velero.ResourceIdentifier{
	GroupResource: kuberesource.ClusterRoleBindings,
	Name: binding,
	})
	}
	for role := range roles {
	additionalItems = append(additionalItems, velero.ResourceIdentifier{
	GroupResource: kuberesource.ClusterRoles,
	Name: role,
	})
	}

apiVersion: velero.io/v1
kind: Backup
metadata:
  name: myns-s3backup-try-2024-05-13
  namespace: openshift-adp
spec:
  csiSnapshotTimeout: 10m0s
  defaultVolumesToFsBackup: false
  hooks: {}
  includedNamespaces:
  - myns
  itemOperationTimeout: 4h0m0s
  snapshotMoveData: false
  storageLocation: backup-s3-1
  ttl: 720h0m0s

[PieterVoet]

Hi blackpiglet, thanks for your reply !

I started out using the Backup CR as you mentioned, and then after restoring it, all CSV's remained in the 'pending' state
because the clusterroles and clusterbindings did not exist. That's why I started to figure out if I have to explicitly include
those in the backup.

But since I use OpenShifts' OADP operator, it may not already be at the Velero version that incorporates this code.
As stated, the OADP operator has Velero version 1.12.4.
I'll go ahead check the operator logs for '"Adding clusterrole %s and clusterrolebinding %s to additionalItems since serviceaccount %s/%s is a subject"' lines..

thanks again for you help !

[PieterVoet]

And yes, you're right ! Logs read :
msg="Adding clusterrole cloud-native-postgresql.v1.18.10-b476b8498 and clusterrolebinding cloud-native-postgresql.v1.18.10-b476b8498 to additionalItems since serviceaccount cp4ba-baw/postgresql-operator-manager is a subject" backup=openshift-adp/cp4ba-baw-s3backup-try-2024-05-16 cmd=/velero logSource="/remote-source/velero/app/pkg/backup/service_account_action.go:95" pluginName=velero

this means that I have to investigate the Restore to check if the clusterroles/bindings are getting restored.

[PieterVoet]

And yes ! My issue is resolved. User error.

Because I created the Backup CR using

includedNamespaces:
  - myns

I automatically used that in the Restore CR too.. which caused the clusterscoped resources not to be restored.
Removing the 'includedNamespaces' directive from the Restore CR caused everything to restore fine !

Thanks Kaovilai and Blackpiglet for your help !

[kaovilai]

I totally missed the clusterrolebindings query.. whoops. Glad it's resolved!