Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Feature Request - toggle for Prometheus scraping as ClusterRole #717

Open
dplusic opened this issue Nov 27, 2020 · 3 comments
Open

Feature Request - toggle for Prometheus scraping as ClusterRole #717

dplusic opened this issue Nov 27, 2020 · 3 comments

Comments

@dplusic
Copy link

dplusic commented Nov 27, 2020

Expected Behaviour

Helm Chart should not create ClusterRole when clusterRole=false.

Current Behaviour

It creates ClusterRole even if clusterRole=false.

Possible Solution

It should create Role instead of ClusterRole.

Steps to Reproduce (for bugs)

  1. Prepare a service account without permission creating ClusterRole
  2. Run helm install openfaas openfaas/openfaas --set clusterRole=false --set createCRDs=false
  3. Error message: Error: rendered manifests contain a resource that already exists. Unable to continue with install: could not get information about the resource: clusterroles.rbac.authorization.k8s.io "openfaas-prometheus" is forbidden: User "system:serviceaccount:default:default" cannot get resource "clusterroles" in API group "rbac.authorization.k8s.io" at the cluster scope

Context

I want namespaced installation of openfaas.

Your Environment

  • FaaS-CLI version ( Full output from: faas-cli version ): -

  • Docker version docker version (e.g. Docker 17.0.05 ): -

  • What version and distriubtion of Kubernetes are you using? kubectl version

Client Version: version.Info{Major:"1", Minor:"19", GitVersion:"v1.19.4", GitCommit:"d360454c9bcd1634cf4cc52d1867af5491dc9c5f", GitTreeState:"clean", BuildDate:"2020-11-11T13:17:17Z", GoVersion:"go1.15.2", Compiler:"gc", Platform:"linux/amd64"}
Server Version: version.Info{Major:"1", Minor:"18+", GitVersion:"v1.18.9-eks-d1db3c", GitCommit:"d1db3c46e55f95d6a7d3e5578689371318f95ff9", GitTreeState:"clean", BuildDate:"2020-10-20T22:18:07Z", GoVersion:"go1.13.15", Compiler:"gc", Platform:"linux/amd64"}
  • Operating System and version (e.g. Linux, Windows, MacOS): Ubuntu 18.04 on WSL

  • Link to your project or a code example to reproduce issue: -

  • What network driver are you using and what CIDR? i.e. Weave net / Flannel: -

@alexellis
Copy link
Member

alexellis commented Nov 28, 2020

This is expected, Prometheus needs to scrape in more than one namespace. The cluster role flag that you mention is only for the OpenFaaS Operator and does not configure the RBAC for Prometheus.

Happy for you to send a PR, otherwise you'll need to wait for this to be prioritized. Premium Subscription customers get priority.

@alexellis
Copy link
Member

/set title: Feature Request - toggle for Prometheus scraping as ClusterRole

@derek derek bot changed the title Helm Chart creates ClusterRole even if clusterRole=false Feature Request - toggle for Prometheus scraping as ClusterRole Nov 28, 2020
@alexellis
Copy link
Member

alexellis commented Nov 29, 2020

I took a look into this today to refresh my memory, you should read the notes on this PR:

71edafa#diff-48d9710e441cfca34edd16570dabeda39dc6ecfae662e42f4e9bab7813290aa7

And if converted to two Roles and RoleBindings to the Prometheus SA, the following error is given by Prometheus which fails to work:

level=error ts=2020-11-29T16:33:17.946Z caller=klog.go:94 component=k8s_client_runtime func=ErrorDepth msg="/app/discovery/kubernetes/kubernetes.go:283: Failed to list *v1.Pod: pods is forbidden: User \"system:serviceaccount:openfaas:openfaas-prometheus\" cannot list resource \"pods\" in API group \"\" in the namespace \"openfaas-fn\": RBAC: role.rbac.authorization.k8s.io \"openfaas-prometheus\" not found"

With:

alex@alex-nuc8:~/go/src/github.com/openfaas/faas-netes$ kubectl get sa -n openfaas
NAME                  SECRETS   AGE
default               1         4m55s
openfaas-prometheus   1         4m48s
openfaas-controller   1         4m48s
alex@alex-nuc8:~/go/src/github.com/openfaas/faas-netes$ kubectl get sa -n openfaas-fn
NAME      SECRETS   AGE
default   1         4m56s
alex@alex-nuc8:~/go/src/github.com/openfaas/faas-netes$ kubectl get role -n openfaas
NAME                  CREATED AT
openfaas-prometheus   2020-11-29T16:29:47Z
openfaas-profiles     2020-11-29T16:29:47Z
alex@alex-nuc8:~/go/src/github.com/openfaas/faas-netes$ kubectl get role -n openfaas-fn
NAME                  CREATED AT
openfaas-controller   2020-11-29T16:29:47Z
alex@alex-nuc8:~/go/src/github.com/openfaas/faas-netes$ kubectl get rolebinding -n openfaas
NAME                  ROLE                       AGE
openfaas-prometheus   Role/openfaas-prometheus   5m3s
openfaas-profiles     Role/openfaas-profiles     5m3s
alex@alex-nuc8:~/go/src/github.com/openfaas/faas-netes$ kubectl get rolebinding -n openfaas-fn
NAME                  ROLE                       AGE
openfaas-prometheus   Role/openfaas-prometheus   5m4s
openfaas-controller   Role/openfaas-controller   5m4s
alex@alex-nuc8:~/go/src/github.com/openfaas/faas-netes$ 

We would welcome your suggestions @dplusic

alexellis added a commit that referenced this issue Nov 29, 2020
A user was confused about the intent of the ClusterRole flag
in the faas-netes chart. This should resolve any future
ambiguity. Ref: #717

Signed-off-by: Alex Ellis (OpenFaaS Ltd) <alexellis2@gmail.com>
alexellis added a commit that referenced this issue Nov 29, 2020
Unless clusterRole is specified, the Prometheus role will be
restricted to scraping from only a single namespace.

This fixes issue: #717 where I user complained that they did
not want to create a ClusterRole in their cluster.

It has been tested with k3d and K8s 1.19 with and without the
--set clusterRole=true flag passed into the faas-netes helm
chart.

The second Role and RoleBinding needed a different name to the
ones in the primary namespace in order for the RBAC error to
go away in Prometheus.

Signed-off-by: Alex Ellis (OpenFaaS Ltd) <alexellis2@gmail.com>
alexellis added a commit that referenced this issue Dec 5, 2020
Unless clusterRole is specified, the Prometheus role will be
restricted to scraping from only a single namespace.

This fixes issue: #717 where I user complained that they did
not want to create a ClusterRole in their cluster.

It has been tested with k3d and K8s 1.19 with and without the
--set clusterRole=true flag passed into the faas-netes helm
chart.

The second Role and RoleBinding needed a different name to the
ones in the primary namespace in order for the RBAC error to
go away in Prometheus.

Signed-off-by: Alex Ellis (OpenFaaS Ltd) <alexellis2@gmail.com>
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

No branches or pull requests

2 participants