Document the exact semantics of the maximum HTTP header size property

[StuAtGit]

Summary: max-http-header-size appears to cap requests based on the length of the request-line, which is not part of the header block.

Details:

Actual Behaviour
We were making GET requests, and received 400/Bad Request once the URL got past a certain length. Raising the max-http-header-size allowed the request to go through.

However, the URL is not part of the http header block, it is part of the http request line, so it took a bit of wild guessing and trial and error to identify this fix.

I can dig up the RFCs if needed, but a clear explanation of what constitutes a HTTP header block can be found in MDNs coverage of HTTP Basics:

https://developer.mozilla.org/en-US/docs/Web/HTTP/Messages

When debugging problems with URLs being rejected because of length, it would be easier to address issues if the config names weren't misleading.

To more precisely define "misleading"

-> Configs that used terms from the HTTP protocol (in the context of an http configuration) did not use them in the same way that they are defined in the protocol. Particularly for basic, elemental parts of the HTTP protocol.

Expected behaviour:

In a perfect world, it would be nice if:

max-http-[request]-header-size limited the the size of the headers the service accepted, with a clear http status message to that effect.

max-http-request-line-size limited the size of the request-line, with a clear http status message to that effect.

In a not-so-perfect-world, it would be nice if:
If spring decides to use basic, common, well-defined terms in ways that are in variance with the well-defined meaning, it would be good to, at least, define that variance explicitly.

I realize you folks may just be translating this into the underlying (Jetty/Undertow/Netty) configurations, so there may be limited flexibility here, but, at very least, documenting the "SURPRISE! WE mean something entirely different than what the word is defined to mean!" bits, so the poor users slogging through debugging and fixing these issues have some help, would be greatly appreciated.

[wilkinsona]

Thanks for reporting this, but I don't think there's any need for such a patronising tone. This is the first issue you're raised here and it's not a great first impression.

You haven't said which embedded web server you're using so I cannot talk about the specifics of your particular situation. What I can say, is that Boot's server.max-http-request-header-size is mapped onto the following APIs in each web server:

• org.apache.coyote.http11.AbstractHttp11Protocol.setMaxHttpRequestHeaderSize(int)

  Maximum size of the HTTP request message header.

  The maximum permitted size of the request line and headers associated with an HTTP request, specified in bytes. This is compared to the number of bytes received so includes line terminators and whitespace as well as the request line, header names and header values.

• org.eclipse.jetty.server.HttpConfiguration.setRequestHeaderSize(int)

  Larger headers will allow for more and/or larger cookies plus larger form content encoded in a URL. However, larger headers consume more memory and can make a server more vulnerable to denial of service attacks.

• reactor.netty.http.HttpDecoderSpec.maxHeaderSize(int)

  Configure the maximum header size that can be decoded for the HTTP request

• io.undertow.Undertow.Builder.setServerOption(UndertowOptions.MAX_HEADER_SIZE, T)

  The maximum size in bytes of a http request header

I think the naming of our property is consistent with the server-specific APIs onto which it's mapped and how they're documented. The maintainers of each of these servers also have far deeper HTTP expertise than we do so we prefer to defer to them on naming, aligning as closely as we can given that we have a single property name and four slightly different targets.

In the case of Reactor Netty, its setting maps down onto a Netty API:

• io.netty.handler.codec.http.HttpDecoderConfig.setMaxHeaderSize(int)

Set the maximum line length of header lines. This limits how much memory Netty will use when parsing HTTP header key-value pairs. You would typically set this to the same value as setMaxInitialLineLength(int)

From the descriptions of the properties, it's clear that Netty is making the distinction that you would like to see and that Jetty and Tomcat are not. Undertow isn't clear but looking at the code in it HttpReadListener, I believe it considers the header to be everything up until the request's body.

We can take a look at adding something to the documentation to make it clear that the exact meaning of the property varies depending on the underlying web server that's in use and to encourage readers to refer to the server's documentation for specifics.

[StuAtGit]

I made no attempt to be patronizing. I was making an effort to be diplomatic and not come across as aggressively complaining about http spec violations. Sometimes if I'm direct people say I'm aggressive, then I try to be excessively diplomatic, and .. patronizing?

I'll try to work on the tone, if you have any concrete feedback on that. I've raised issues on the old tracker (before git).

We're using undertow.

[mhalbritter]

We can take a look at adding something to the documentation to make it clear that the exact meaning of the property varies depending on the underlying web server that's in use and to encourage readers to refer to the server's documentation for specifics.

I'll turn this into a documentation issue.