Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Maintenance job need have the timeout mechanism, abnormal detection, and grace shutdown #7748

Open
qiuming-best opened this issue Apr 28, 2024 · 1 comment
Assignees

Comments

@qiuming-best
Copy link
Contributor

Describe the problem/challenge you have

Describe the solution you'd like

Currently, the Maintenance job does not have a timeout mechanism, maybe the job runs a long time abnormally.

We need to detect the abnormal status of maintenance job, let it fail early by deleting the jobs

@qiuming-best qiuming-best self-assigned this Apr 28, 2024
@reasonerjt
Copy link
Contributor

If the job fails to start, the timeout mechanism probably won't help much, we just need to make sure the jobs won't pile up on the k8s side.

If the job started and took longer to finish, considering the job is running repository maintenance, it may be dangerous to kill the process b/c it may cause inconsistency in the repository.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Projects
None yet
Development

No branches or pull requests

3 participants