Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

fix zk session leak #678

Open
wants to merge 3 commits into
base: master
Choose a base branch
from

Conversation

sdomalap
Copy link
Contributor

This PR has a bug fix to a specific problem where the cluster could eventually go leaderless as leadership transitions occur. It handles the session re-establishment properly and avoid session leaks.

@@ -201,9 +207,15 @@ public void disconnect() {
* the actions that need to be taken with them, which are implemented in the Coordinator class
*/
public void connect() {
disconnect(); // Guard against leaking an existing zookeeper session
Copy link
Collaborator

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Removing this will not be sufficient. On Zookeeper session expiry, the live instance ephemeral node will be deleted. So, the leader will become follower and both needs to creates new live instance nodes. Once the role will change, all the listener cleanup will be required which is happening in disconnect.

Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Thanks for the review @vmaheshw.

It seems like you are correct. I might be leaking subscriptions. I may not have experienced any issues as the subscriptions are to the nodes that would never get updated.

I will add the clean up logic as necessary.

Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Why remove this line anyway? If we are connecting or reconnecting, then we should not have an existing session. The guard code defensively ensures that any existing connection is definitely closed.

public class ZkStateChangeListener implements IZkStateListener {
@Override
public void handleStateChanged(Watcher.Event.KeeperState state) {
}
Copy link
Collaborator

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

In the situation, when the session never gets reestablished, handleNewSession will not get invoked. So, handleStateChanged with state EXPIRY needs to be handled.

Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Noted. Will post a new commit. Thanks

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

Successfully merging this pull request may close these issues.

None yet

3 participants