-
Notifications
You must be signed in to change notification settings - Fork 2.4k
New issue
Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.
By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.
Already on GitHub? Sign in to your account
[SUPPORT] Archive operation always release lock on the timeline when try lock failed #11104
Comments
archive itself holds an transaction lock, so we need to release it in any case, what is the wrong case you have ecountered? |
@danny0405 Sorry, I don't remember the details of the problem (I will confirm with my colleagues and provide the results later), but based on the logic here, wouldn't it be better to attempt to release the lock only when acquiring it, otherwise is there a chance of mistakenly releasing the lock of other operations and causing problems? |
That's reasonable, we should ensure the lock been acquired in the first place. |
@danny0405 @Ytimetravel The behavior of method |
@beyond1920 Yes, I am very willing to fix this issue. |
What is the general reason that the trasanction start of archival is failing? |
@danny0405 Failed to acquire lock. |
Okay, it would be great if you can fire a fix for it. |
Dear community,
I have discovered an issue when using Hudi.If multiple archive tasks run on COW table and set "hoodie.archive.automatic=false", it may cause data problems. If set hoodie.archive.automatic=true, then this issue will not occur.
And then I find that if the archive operation try lock failed, it will always release lock(if exist).
I suspect that this lock release operation may have affected other normal operations.
Perhaps the problem could be avoided by doing it this way?
Looking forward to your valuable suggestions.
Hudi version :0.14.0
The text was updated successfully, but these errors were encountered: