Set Vary: Accept-Encoding header when compressing responses

[palant]

What is the problem your feature solves, or the need it fulfills?

When downstream compression is enabled in pingora-proxy, it won’t currently set Vary: Accept-Encoding header. This might result in downstream proxies caching the response incorrectly – e.g. caching a Zstandard response and serving it to a client without Zstandard support.

Describe the solution you'd like

ResponseCompressionCtx.response_header_filter() needs to add Vary: Accept-Encoding header. This could happen only for the responses that actually meet the criteria for being compressed (whether the compression is actually applied or not), or just for any responses going through that method if that’s easier to implement.

Describe alternatives you've considered

The header can be added by the code enabling the compression. It’s unexpected that this has to be done manually however.

Additional context

Vary header spec:

An origin server SHOULD generate a Vary header field on a cacheable response when it wishes that response to be selectively reused for subsequent requests. Generally, that is the case when the response content has been tailored to better fit the preferences expressed by those selecting header fields, such as when an origin server has selected the response's language based on the request's Accept-Language header field.