Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Jump to bottom

Add support for memoization to resource group state endpoint #22764

Merged
merged 1 commit into from
May 22, 2024
Merged

Add support for memoization to resource group state endpoint #22764

merged 1 commit into from
May 22, 2024

Conversation

swapsmagic
Copy link
Contributor

@swapsmagic swapsmagic commented May 15, 2024 •
edited

Description

Adding memoization support to /v1/resourceGroupState endpoint.

Motivation and Context

This will help reduce load on the coordinator to
calculate the resource group state for concurrent/frequent requests in a short period of time.

Impact

Less contention on coordinator from /v1/resourceGroupState endpoint

Test Plan

unit test

Contributor checklist

  • Please make sure your submission complies with our development, formatting, commit message, and attribution guidelines.
  • PR description addresses the issue accurately and concisely. If the change is non-trivial, a GitHub Issue is referenced.
  • Documented new properties (with its default value), SQL syntax, functions, or other functionality.
  • If release notes are required, they follow the release notes guidelines.
  • Adequate tests were added if applicable.
  • CI passed.

Release Notes

Please follow release notes guidelines and fill in the release notes below.

== RELEASE NOTES ==

General Changes
* Add support for memoizing in resource group state info endpoint. This can be enabled by setting `cluster-resource-group-state-info-expiration-duration` to a non-zero duration. :pr:`22764`

@swapsmagic swapsmagic requested a review from a team as a code owner May 15, 2024 20:37
@swapsmagic swapsmagic changed the title [DRAFT] Improve Resource Group State Endpoint Improve Resource Group State Endpoint May 15, 2024
@swapsmagic swapsmagic changed the title Improve Resource Group State Endpoint Add support for memoization to resource group state endpoint May 15, 2024
@swapsmagic swapsmagic force-pushed the improve_rg_state_endpoint branch 2 times, most recently from c097be5 to 272fd92 Compare May 15, 2024 23:38
@steveburnett
Copy link
Contributor

Nit, request add PR number to the release note entry.

== RELEASE NOTES ==

General Changes
* Add support for memoizing in resource group state info endpoint. This can be enabled by setting `resource-group-state-info-expiration-duration` to a non-zero duration. :pr:`22764`

@facebook-github-bot
Copy link
Collaborator

@swapsmagic has imported this pull request. If you are a Meta employee, you can view this diff on Phabricator.

jainxrohit
jainxrohit previously approved these changes May 16, 2024
View reviewed changes
presto-main/src/main/java/com/facebook/presto/server/ServerConfig.java Outdated Show resolved Hide resolved
presto-main/src/main/java/com/facebook/presto/server/ResourceGroupStateInfoResource.java Outdated
@@ -76,6 +103,9 @@ public ResourceGroupStateInfoResource(
this.resourceGroupManager = requireNonNull(resourceGroupManager, "resourceGroupManager is null");
this.internalNodeManager = requireNonNull(internalNodeManager, "internalNodeManager is null");
this.proxyHelper = requireNonNull(proxyHelper, "proxyHelper is null");
this.resourceGroupStateInfoKeySupplierMap = new HashMap<>();
this.expirationDuration = requireNonNull(serverConfig, "serverConfig is null").getResourceGroupStateInfoExpirationDuration();
this.rootResourceGroupInfoSupplier = expirationDuration.getValue() > 0 ? memoizeWithExpiration(() -> resourceGroupManager.getRootResourceGroups(), expirationDuration.toMillis(), MILLISECONDS) : Suppliers.ofInstance(resourceGroupManager.getRootResourceGroups());
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Very long line. Lets break it with ? and :

presto-main/src/main/java/com/facebook/presto/server/ResourceGroupStateInfoResource.java Show resolved Hide resolved
presto-main/src/main/java/com/facebook/presto/server/ResourceGroupStateInfoResource.java Show resolved Hide resolved
@swapsmagic swapsmagic force-pushed the improve_rg_state_endpoint branch 2 times, most recently from 3ca9aab to 1f0f968 Compare May 21, 2024 17:45
@facebook-github-bot
Copy link
Collaborator

@swapsmagic has imported this pull request. If you are a Meta employee, you can view this diff on Phabricator.

@facebook-github-bot
Copy link
Collaborator

@swapsmagic has imported this pull request. If you are a Meta employee, you can view this diff on Phabricator.

NikhilCollooru
NikhilCollooru previously approved these changes May 21, 2024
View reviewed changes
Copy link
Contributor

@NikhilCollooru NikhilCollooru left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Except for the unit test comment. The changes look good to me.

@facebook-github-bot
Copy link
Collaborator

@swapsmagic has imported this pull request. If you are a Meta employee, you can view this diff on Phabricator.

@swapsmagic
Add support for memoization to resource group state endpoint
dbf6c37 
Adding memoization support to /v1/resourceGroupState endpoint. This will help reduce load on the coordinator to
calculate the resource group state for concurrent/frequent requests in a short period of time.
@facebook-github-bot
Copy link
Collaborator

@swapsmagic has imported this pull request. If you are a Meta employee, you can view this diff on Phabricator.

@swapsmagic swapsmagic merged commit 5f64ec8 into prestodb:master May 22, 2024
57 of 58 checks passed
@swapsmagic swapsmagic deleted the improve_rg_state_endpoint branch May 22, 2024 01:33
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Reviewers

@NikhilCollooru NikhilCollooru NikhilCollooru approved these changes

@jainxrohit jainxrohit jainxrohit left review comments

@presto-oss presto-oss Awaiting requested review from presto-oss presto-oss is a code owner automatically assigned from prestodb/committers

@jaystarshot jaystarshot Awaiting requested review from jaystarshot jaystarshot is a code owner

@feilong-liu feilong-liu Awaiting requested review from feilong-liu feilong-liu is a code owner

Assignees

@NikhilCollooru NikhilCollooru

@jainxrohit jainxrohit

Labels
None yet
Projects
None yet
Milestone
No milestone
Development

Successfully merging this pull request may close these issues.

None yet
5 participants
@swapsmagic @steveburnett @facebook-github-bot @NikhilCollooru @jainxrohit