Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

feat: client metrics #3988

Open
wants to merge 2 commits into
base: main
Choose a base branch
from
Open

Conversation

monosoul
Copy link

@monosoul monosoul commented Feb 27, 2024

Subsystem
Client

Motivation
Provide a plugin to instrument Ktor client similarly to how the server lib is instrumented with Micrometer.

Solution
This is basically just a PoC to discuss if the approach is okay or not. If it's okay, then I'll proceed with adding tests etc. If not, then we can discuss it here.

The idea here is to provide a client instrumentation similar to how it's done in OkHttpMetricsEventListener - introduce a timer for client requests.

The implementation is heavily inspired by the Logging plugin.

Since using an expanded URI as the value for the URI tag can lead to cardinality blow up, I had to introduce support for URI templates. This is done in 0e542bd. While those methods could potentially be added as extension functions in the plugin module, it feels like they belong to the core lib more.
( Probably a better option there would be to have a set of explicit builder functions like:

client.get(uriTemplate: String, vararg pathParameters: Pair<String, Any>, block: HttpRequestBuilder.() -> Unit = {})

Then it wouldn't be necessary to rebuild the URL. This is the path Spring followed with their client for example: https://github.com/spring-projects/spring-framework/blob/main/spring-web/src/main/java/org/springframework/web/client/RestTemplate.java#L400 )

The URI template is stored as a call attribute with the first call to HttpRequestBuilder.pathParameters and is later used in MicrometerMetrics plugin.

Here's how using the new methods would look like:

client.get("https://blog.jetbrains.com/kotlin/{year}/{month}/{name}/") {
    pathParameters(
         "year" to 2023,
         "month" to "11",
         "name" to "kotlin-1-9-20-released",
    )
}.bodyAsText()

The plugin is implemented in 2e9cab1 and provides configuration to make it flexible enough to cover most of the use cases.

The timer will be started when request has been sent and will be stopped either when an exception has occurred, or when the response has been received.

@monosoul monosoul force-pushed the feat/client-metrics branch 2 times, most recently from f3ea68a to 5b1c35c Compare February 28, 2024 11:36
@e5l e5l requested a review from marychatte March 18, 2024 13:05
@e5l e5l self-assigned this Mar 18, 2024
@e5l
Copy link
Member

e5l commented Mar 18, 2024

Hey @monosoul, thanks for the PR!

@marychatte, could you please check?

Copy link
Member

@marychatte marychatte left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Thanks a lot for the PR! And sorry for the big delay in the review
Good work, I like the idea of plugin. And I added some comments

* @see pathParameter
*/
public fun pathParameters(parameters: Iterable<Pair<String, Any>>) {
val currentUrl = url.build()
Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Can we remove url.build() here and make it call only once? Maybe we can use HttpRequestBuilder.attributes to save parameters and build later

public fun pathParameters(parameters: Iterable<Pair<String, Any>>) {
val currentUrl = url.build()
// save original URI pattern
attributes.computeIfAbsent(UriPatternAttributeKey) { "$currentUrl" }
Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Why does it need to be saved as a String? Later, it's used to parse, but maybe we can use functions URL.encodedQuery or URL.encodedPath

builder.host = host
builder.protocol = protocol
builder.port = port
builder.pathSegments =
Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

What about pathSegment = /{year}_{month}/ for example?

pathSegments.map { pathSegment ->
substitutionsMap[pathSegment] ?: pathSegment
}
parameters.forEach { parameterName, parameterValues ->
Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

We also replace query parameters here, so the names pathParameters(...) and pathParameter(...) can be confusing. And it's also better to add in example query parameter values for clarity

}
}
builder.user = user
builder.password = password
Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

We need to copy fragment and trailingQuery too

}

/**
* A client's plugin that provides the capability to meter HTTP calls with micrometer
Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Can you please add in Kdoc example of plugin usage?

Tag.of(
TAG_URI,
request.attributes.getOrNull(UriPatternAttributeKey)
.let { it ?: request.url.takeIf { useExpandedUrlWhenPatternUnavailable }?.toString() }
Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Can we extract it so it does not have copypaste?

try {
proceed()
} catch (cause: Throwable) {
timer.stop(
Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Can we add FailureHook so there is no need to write try-catch?

cause?.javaClass?.simpleName ?: javaClass.simpleName,
)

private fun String.removeHostPart(host: String) = replace("^.*$host[^/]*".toRegex(), "")
Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Can we extract regex here so as not to compile it every time?


private fun String.removeQueryPart() = replace(QUERY_PART_REGEX, "")

private object ResponseHook : ClientHook<suspend ResponseHook.Context.(response: HttpResponse) -> Unit> {
Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Can we, instead of creating new Hooks, use functions onRequest {... }, onResponse {... } etc?

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

Successfully merging this pull request may close these issues.

None yet

3 participants