Hi community

I am starting a DISCUSS for "PIP-259: Make the config
httpMaxRequestHeaderSize of the pulsar web server configurable".

### Motivation

We have two ways to manage pulsar's resources:
- By client API (Can manage some resources, such as `create topic`, `create
subscriber`, and so on)
- By admin API (Can manage all the resources)

The `client API` has no limit on the request length. And the `admin API`
has a limit on the request length(such as HTTP request line and HTTP
request headers), this restriction is done by the built-in web container
Jetty.

Almost resources can be created by two APIs, but can only be modified and
deleted by `admin API`. This causes us to be unable to modify or delete
resources created by `client API` with too long a name because it exceeds
Jetty's default HTTP request URI length limit.

### Goal

#### 1. For web servers
Provide a way to modify Jetty's `httpMaxRequestHeaderSize` configuration
(involves two servers: the web server in pulsar and the web server in
pulsar-proxy)

#### 2.For the internal client in pulsar-proxy
Provide a way to modify Jetty-client's `httpClientRequestBufferSize`
configuration.

Since the pulsar-proxy handles HTTP requests like this: `pulsar-admin.sh`
-> `proxy web server` -> `(highlight) internal client in proxy` -> `pulsar
web server`.

When the internal client forwards a request, it forwards the request header
and the request body, and all the data passes through a buffer( we call it
Buf ), like this:
- Receive a request
- Put the request line and request headers input to the Buf.
- <strong>(highlight)</strong>Flush the Buf ( If the data in the request
line and request header exceeds the length of the buf, an error is reported
)
- Put the request body input to the Buf.
- Flush the Buf if it is full.

So we need a config to set the `buff size` of the Buf:
`pulsar-proxy.conf.httpClientRequestBufferSize` -> `buf size of the
internal client`.

### API Changes

#### ServiceConfiguration.java
```java
   @FieldContext(
            category = CATEGORY_HTTP,
            doc = """
                The maximum size in bytes of the request header.
                Larger headers will allow for more and/or larger cookies
plus larger form content encoded in a URL.
                However, larger headers consume more memory and can make a
server more vulnerable to denial of service
                attacks.
              """
    )
   private int httpMaxRequestHeaderSize = 8 * 1024;
```

#### ProxyConfiguration.java

```java
    @FieldContext(
        minValue = 1,
        category = CATEGORY_HTTP,
        doc = """
                The maximum size in bytes of the request header.
                Larger headers will allow for more and/or larger cookies
plus larger form content encoded in a URL.
                However, larger headers consume more memory and can make a
server more vulnerable to denial of service
                attacks.
              """
    )
    private int httpMaxRequestHeaderSize = 8 * 1024;

    @FieldContext(
        minValue = 1,
        category = CATEGORY_HTTP,
        doc = """
                 the size of the buffer used to write requests to Broker.
                 if "httpMaxRequestHeaderSize" is large than
"httpClientRequestBufferSize", will set
                 "httpClientRequestBufferSize" to the value of
"httpMaxRequestHeaderSize"
              """
    )
    private int httpClientRequestBufferSize = httpMaxRequestHeaderSize;
```

### Anything else?

This change should cherry-pick into the previous branches ( includes
`2.9~2.11` )

If the user uses the features `RETRY Topic` or `DLQ`, it is possible that
pulsar will automatically create some topics with names that are too long
and cannot be managed, the [scenario has been discussed in the email](
https://lists.apache.org/thread/q1m23ckyy10wvtzy65v8bwqwnh7r0gc8) before


Thanks
Yubiao Feng

Reply via email to