Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Jump to bottom

report an event to pod if kubelet does attach operation failed #124884

Open
wants to merge 1 commit into
base: master
Choose a base branch
from
Open

report an event to pod if kubelet does attach operation failed #124884

wants to merge 1 commit into from
+4 −3

Conversation

carlory
Copy link
Member

@carlory carlory commented May 15, 2024

What type of PR is this?

/kind feature

What this PR does / why we need it:

kubelet is running with --enable-controller-attach-detach=false.

kubelet log:

May 15 09:55:17 demo-control-plane kubelet[691]: E0515 09:55:17.993787     691 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/csi/hostpath.csi.k8s.io^fa71a846-129d-11ef-a424-4a1499c02f41 podName: nodeName:}" failed. No retries permitted until 2024-05-15 09:57:19.993627526 +0000 UTC m=+3002.459216475 (durationBeforeRetry 2m2s). Error: AttachVolume.Attach failed for volume "pvc-d8752815-33f2-4609-bf8f-9ca004058dd6" (UniqueName: "kubernetes.io/csi/hostpath.csi.k8s.io^fa71a846-129d-11ef-a424-4a1499c02f41") from node "demo-control-plane" : attaching volumes from the kubelet is not supported

no event:

(base) (⎈|kind-demo:N/A)➜  ~ kubectl get po
NAME                   READY   STATUS              RESTARTS   AGE
csi-hostpath-socat-0   1/1     Running             0          85m
csi-hostpathplugin-0   8/8     Running             0          72m
my-csi-app             0/1     ContainerCreating   0          47m
(base) (⎈|kind-demo:N/A)➜  ~ kubectl describe po my-csi-app
...
Events:
  Type    Reason     Age   From               Message
  ----    ------     ----  ----               -------
  Normal  Scheduled  47m   default-scheduler  Successfully assigned default/my-csi-app to demo-control-plane

Which issue(s) this PR fixes:

Fixes #

Special notes for your reviewer:

Does this PR introduce a user-facing change?

report an event to pod if kubelet does attach operation failed when kubelet is running with `--enable-controller-attach-detach=false`

Additional documentation e.g., KEPs (Kubernetes Enhancement Proposals), usage docs, etc.:

@carlory
report an event if kubelet does attach operation failed when kubelet …
02b1580 
…is running with --enable-controller-attach-detach=false
@k8s-ci-robot k8s-ci-robot added release-note Denotes a PR that will be considered when it comes time to generate release notes. kind/feature Categorizes issue or PR as related to a new feature. size/XS Denotes a PR that changes 0-9 lines, ignoring generated files. cncf-cla: yes Indicates the PR's author has signed the CNCF CLA. do-not-merge/needs-sig Indicates an issue or PR lacks a `sig/foo` label and requires one. needs-triage Indicates an issue or PR lacks a `triage/foo` label and requires one. labels May 15, 2024
@k8s-ci-robot
Copy link
Contributor

This issue is currently awaiting triage.

If a SIG or subproject determines this is a relevant issue, they will accept it by applying the triage/accepted label and provide further guidance.

The triage/accepted label can be added by org members by writing /triage accepted in a comment.

Instructions for interacting with me using PR comments are available here. If you have questions or suggestions related to my behavior, please file an issue against the kubernetes-sigs/prow repository.

@k8s-ci-robot k8s-ci-robot added the needs-priority Indicates a PR lacks a `priority/foo` label and requires one. label May 15, 2024
@k8s-ci-robot
Copy link
Contributor

[APPROVALNOTIFIER] This PR is NOT APPROVED

This pull-request has been approved by: carlory
Once this PR has been reviewed and has the lgtm label, please assign thockin for approval. For more information see the Kubernetes Code Review Process.

The full list of commands accepted by this bot can be found here.

Needs approval from an approver in each of these files:

Approvers can indicate their approval by writing /approve in a comment
Approvers can cancel approval by writing /approve cancel in a comment

@k8s-ci-robot k8s-ci-robot added area/kubelet sig/node Categorizes an issue or PR as relevant to SIG Node. and removed do-not-merge/needs-sig Indicates an issue or PR lacks a `sig/foo` label and requires one. labels May 15, 2024
carlory
carlory commented May 15, 2024
View reviewed changes
pkg/kubelet/volumemanager/reconciler/reconciler_common.go
@@ -250,9 +250,10 @@ func (rc *reconciler) waitForVolumeAttach(volumeToMount cache.VolumeToMount) {
// Volume is not attached to node, kubelet attach is enabled, volume implements an attacher,
Copy link
Member Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Is it legacy behavior? Should we deprecate and remove it in a future release?

@carlory
Copy link
Member Author

carlory commented May 15, 2024

/cc @jsafrane @xing-yang

@bart0sh bart0sh added this to Triage in SIG Node PR Triage May 15, 2024
@carlory
Copy link
Member Author

carlory commented May 15, 2024

/test pull-kubernetes-e2e-gce-storage-snapshot

3 similar comments
@carlory
Copy link
Member Author

carlory commented May 16, 2024

/test pull-kubernetes-e2e-gce-storage-snapshot

@carlory
Copy link
Member Author

carlory commented May 21, 2024

/test pull-kubernetes-e2e-gce-storage-snapshot

@carlory
Copy link
Member Author

carlory commented May 27, 2024

/test pull-kubernetes-e2e-gce-storage-snapshot

@haircommander
Copy link
Contributor

/sig storage
/remove-sig node

@k8s-ci-robot k8s-ci-robot added sig/storage Categorizes an issue or PR as relevant to SIG Storage. and removed sig/node Categorizes an issue or PR as relevant to SIG Node. labels Jun 5, 2024
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Reviewers

@chrishenzie chrishenzie Awaiting requested review from chrishenzie

@saikat-royc saikat-royc Awaiting requested review from saikat-royc

@jsafrane jsafrane Awaiting requested review from jsafrane

@xing-yang xing-yang Awaiting requested review from xing-yang

Assignees
No one assigned
Labels
area/kubelet cncf-cla: yes Indicates the PR's author has signed the CNCF CLA. kind/feature Categorizes issue or PR as related to a new feature. needs-priority Indicates a PR lacks a `priority/foo` label and requires one. needs-triage Indicates an issue or PR lacks a `triage/foo` label and requires one. release-note Denotes a PR that will be considered when it comes time to generate release notes. sig/storage Categorizes an issue or PR as relevant to SIG Storage. size/XS Denotes a PR that changes 0-9 lines, ignoring generated files.
Projects
SIG Node PR Triage
Status: not-only-sig-node
SIG Node PR Triage
Triage
Milestone
No milestone
Development

Successfully merging this pull request may close these issues.

None yet
3 participants
@carlory @k8s-ci-robot @haircommander