Support NodePortLocal when CIS run in Multi-Cluster mode

[gzrt]

Title

Support NodePortLocal when CIS run in Multi-Cluster mode

Description

According to F5 CIS documents:
https://clouddocs.f5.com/containers/latest/userguide/config-options.html
There're 3 deployment options (pool-member-type) available to CIS: NodePort, ClusterIP, NodePortLocal.

However, when CIS running in Multi-Cluster mode:
https://clouddocs.f5.com/containers/latest/userguide/multicluster/#overview
"At present, nodePort mode is supported and Cluster mode is available with static route configuration on BIG-IP (No tunnels)."
https://clouddocs.f5.com/containers/latest/userguide/multicluster/#faq
"Currently, NodePort mode is supported. For cluster mode, static routing mode is supported to enable configuration of static routes on BIG-IP for pod network subnets for direct routing from BIG-IP to k8s pods"

The NodePortLocal support is missing. Hence we're looking for the support of NodePortLocal when CIS running in Multi-Cluster mode.

Actual Problem

We're buidling a K8s platform contains hundreds of node and services across clusters, based on the design, both F5 CIS and Antrea CNI's NodePortLocal feature are critical points to our success.

We've already tested F5 CIS with Antrea CNI's NodePortLocal (--pool_member_type=nodeportlocal), and it works well.

But when we tried to run F5 CIS in Multi-Cluster mode (--multi-cluster-mode=standalone) with Antrea CNI's NodePortLocal (--pool_member_type=nodeportlocal), only local cluster node IPs & ports been added, external cluster node IPs & ports are missing from pool members.

Solution Proposed

When CIS running in --multi-cluster-mode=standalone/primary/secondary, can support the paremeter --pool-member-type=nodeportlocal, so external cluster node IPs & ports can be added to/remove from pool members as expected.

Alternatives

No, neither NodePort or Static Route mode will work for our case.

Additional context

The tests we've done on K8s v1.28 and CIS version 2.16.0 with --custom-resource-mode=true

[trinaths]

@gzrt Please share CIS configuration and logs when external cluster node IPs & ports are missing from pool members to automation_toolchain_pm automation_toolchain_pm@f5.com

[gzrt]

@trinaths Thanks for your reply, I've sent the CIS helm chart values we're using and the logs to automation_toolchain_pm@f5.com, if any additional information required, please let me know, thank you!

[trinaths]

Created [CONTCNTR-4716] for internal tracking.