Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Jump to bottom

Restore gets stuck if the destination namespace is in Terminating phase #5697

Open
nilesh-akhade opened this issue Dec 15, 2022 · 2 comments · May be fixed by #7424
Open

Restore gets stuck if the destination namespace is in Terminating phase #5697

nilesh-akhade opened this issue Dec 15, 2022 · 2 comments · May be fixed by #7424
Assignees
@reasonerjt
Labels
Needs investigation Restore

Comments

@nilesh-akhade
Copy link
Contributor

What steps did you take and what happened

Ran a restore to a namespace where the destination namespace has a pod which was in the terminating phase. The restore did not complete or fail. The restore gets stuck in "InProgress" phase, with no errors or warnings in the logs.

What did you expect to happen:

Restore should fail due to timeout or with some descriptive error. Logs should contain the information, related to failed restores.

The following information will help us better understand what's going on:

bundle-2022-12-15-05-05-58.tar.gz

Environment:

  • Velero version (use velero version):

      $ velero version
  Client:
          Version: v1.10.0
          Git commit: 367f563072659f0bcd809bc33507fd75cd722344
  Server:
          Version: v1.10.0

  • Velero features (use velero client config get features):

  • Kubernetes version (use kubectl version):

      $ k version
  Client Version: version.Info{Major:"1", Minor:"24", GitVersion:"v1.24.1", GitCommit:"3ddd0f45aa91e2f30c70734b175631bec5b5825a", GitTreeState:"clean", BuildDate:"2022-05-24T12:26:19Z", GoVersion:"go1.18.2", Compiler:"gc", Platform:"linux/amd64"}
  Kustomize Version: v4.5.4
  Server Version: version.Info{Major:"1", Minor:"21", GitVersion:"v1.21.2", GitCommit:"092fbfbf53427de67cac1e9fa54aaa09a28371d7", GitTreeState:"clean", BuildDate:"2021-06-16T12:53:14Z", GoVersion:"go1.16.5", Compiler:"gc", Platform:"linux/amd64"}
  WARNING: version difference between client (1.24) and server (1.21) exceeds the supported minor version skew of +/-1

  • Kubernetes installer & version:

  • Cloud provider or hardware configuration:

  • OS (e.g. from /etc/os-release):

      $ cat /etc/os-release
  NAME="Ubuntu"
  VERSION="18.04.6 LTS (Bionic Beaver)"
  ID=ubuntu
  ID_LIKE=debian
  PRETTY_NAME="Ubuntu 18.04.6 LTS"
  VERSION_ID="18.04"

Vote on this issue!

This is an invitation to the Velero community to vote on issues, you can see the project's top voted issues listed here.
Use the "reaction smiley face" up to the right of this comment to vote.

  • 👍 for "I would like to see this bug fixed as soon as possible"
  • 👎 for "There are more important bugs to focus on right now"
@reasonerjt
Copy link
Contributor

@nilesh-akhade
Please clarify, by destination namespace, is it a namespace pre-existing before you ran restore? As for the pod in Terminating state, is it a pod in the backup with such state or is it a pod in the namespace? Or does it exist in both the destination ns and in the backup?

@reasonerjt reasonerjt self-assigned this Dec 19, 2022
@nilesh-akhade
Copy link
Contributor Author

nilesh-akhade commented Dec 20, 2022 •
edited

destination namespace, is it a namespace pre-existing before you ran restore?

Yes. Here's the scenario:

  • Create a namespace with the name "test"
  • Create a deployment called "nginx" in the "test" namespace
  • Create a backup of the namespace "test" and call it "b1"
  • Edit the "nginx" deployment and add the finalizer, which will prevent deletion of deployment
  • Delete the deployment "nginx", it should now be stuck in the terminating phase. The ns should also be shown as in the terminating phase.
  • Create a restore from backup "b1". To the destination namespace "test"(which is default)
  • Now wait for restore to be complete, it should never complete.

As for the pod in Terminating state, is it a pod in the backup with such state or is it a pod in the namespace?

It is a pod in the destination namespace. It was backed up before when it was in the Running state.

does it exist in both the destination ns and in the backup?

Yes. It exists in the both source and destination namespace. In the "running" state in the source namespace and in the "terminating" state in the destination namespace.

@kaovilai kaovilai linked a pull request Feb 13, 2024 that will close this issue
Open
3 tasks
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Assignees

@reasonerjt reasonerjt

Labels
Needs investigation Restore
Projects
None yet
Milestone
No milestone
Development

Successfully merging a pull request may close this issue.

Descriptive restore error on timeout due to terminating namespace kaovilai/velero
2 participants
@reasonerjt @nilesh-akhade