Thank you very much, that really helped! On Mon, Jan 18, 2021 at 6:48 PM Alex Rousskov < rouss...@measurement-factory.com> wrote:
> On 1/18/21 6:45 AM, Moti Berger wrote: > > > If the ICAP server sets 'Preview: 0' in the OPTIONS it means that when > > the ICAP client sends a request, it should not contain the body. > > The above summary may mislead many readers. I would describe the > protocol operation differently: > > * Preview in an OPTIONS response indicates that the server supports > Preview in general and specifies the maximum Preview size the client > should use (e.g., Preview:0 limits Preview to HTTP headers). > > * The Preview mode for a specific REQMOD or RESPMOD transaction is > signaled in the corresponding REQMOD or RESPMOD request (not a previous > OPTIONS response) by adding a Preview:N ICAP request header (Preview:0 > specifies a headers-only Preview for the current transaction). > > * The REQMOD or RESPMOD transaction with a Preview:0 request header is > split into two phases. During the first phase, the client must not send > the virgin body. During the second phase, if any, the client must send > the virgin body. Both phases comprise a single ICAP transaction, with a > single ICAP request and a single ICAP response. Thus, one cannot say > that this transaction (as a whole) "should not contain a body". > > > > This is the REQMOD request: > > > > F..n...DREQMOD icap://censor-req.proxy:14590/request ICAP/1.0 > > Host: censor-req.proxy:14590 > > Date: Mon, 18 Jan 2021 11:34:54 GMT > > Encapsulated: req-hdr=0, req-body=222 > > Preview: 0 > > Allow: 204, trailers > > X-custom-header: data > > > > POST http://www.dst-server.com:22222/v1/test HTTP/1.1 > > User-Agent: python-requests/2.25.1 > > Accept-Encoding: gzip, deflate > > Accept: */* > > Content-Length: 10 > > Content-Type: application/json > > Host: www.dst-server.com:22222 <http://www.dst-server.com:22222> > > > The ICAP 'Encapsulated' header has a req-body even though no 'body' > > should be in this request. > > Not exactly. The request may not be over at this point. Please see my > third bullet above for details. > > > > The ICAP server has no way to encapsulate the HTTP request body if it > > didn't get it. > > To get the request body in Preview:0 mode, the ICAP server must respond > with ICAP 100 (Continue). > > > > I want to avoid sending the body because the adaptation is body agnostic. > > Yes, I know, but you have to work within the ICAP protocol boundaries. > ICAP simply does not optimize your use case! After you have the basics > working well, you can invest in implementing a use-original-body ICAP > extension[1] that, in _some_ cases, can prevent the body exchange while > adapting HTTP headers. > > Alternatively, you can use an existing (extendible) ICAP server to do > the legwork for you [2]. Many individuals and companies have learned the > hard way that implementing an ICAP service correctly from scratch is > very difficult and often prohibitively expensive. > > [1] > > http://www.icap-forum.org/documents/specification/draft-icap-ext-partial-content-07.txt > > [2] https://wiki.squid-cache.org/Features/ICAP#ICAP_Servers > > > HTH, > > Alex. > > > > > On Sun, Jan 17, 2021 at 11:34 PM Alex Rousskov wrote: > > > > On 1/17/21 3:08 PM, Moti Berger wrote: > > > What should the ICAP response look like? > > > > The vast majority off ICAP responses containing an HTTP POST message > > will look like ICAP header + HTTP header + HTTP body. Please see RFC > > 3507 and its errata for examples of and discussion about those three > > components. It should help avoid guessing and developing by examples > > (which usually leads to bugs, especially where ICAP is involved). > > > > > > > What I do is to reply like this: > > > > > > (dI./M..ICAP/1.0 200 OK > > > ISTag: "SjIzlRA4te41axxcDOoiSl6rBRg4ZK" > > > Date: Sun, 17 Jan 2021 19:34:12 GMT > > > Server: BaseICAP/1.0 Python/3.6.12 > > > Encapsulated: req-hdr=0, req-body=360 > > > > > > POST http://www.dst-server.com:22222/v1/test HTTP/1.1 > > > x-new-header: {"key": "value"} > > > user-agent: python-requests/2.25.1 > > > accept-encoding: gzip, deflate > > > accept: */* > > > content-length: 16 > > > content-type: application/json > > > host: www.dst-server.com:22222 > > <http://www.dst-server.com:22222> <http://www.dst-server.com:22222> > > > > > > FYI: The above incomplete ICAP response promises an HTTP request > body, > > both on the ICAP level (req-body) and on the HTTP level > (content-length: > > 16). > > > > > > > As I said, I use 'Preview: 0' since I don't mind the body. The > > question > > > is whether declaring the body starts at X (req-body=X) is OK even > > though > > > I don't have a body to send? > > > > It is not OK not to send the body. Encapsulated:req-body does more > than > > declaring where the encapsulated headers end. It also promises an > > embedded HTTP body after those headers. You must encapsulate the > body if > > the HTTP message should have one. You cannot adapt the header of an > HTTP > > message with a body without also sending the HTTP body (virgin or > > adapted). > > > > Preview is pretty much irrelevant in this context -- the ICAP > protocol > > does not care how the ICAP service gets the HTTP body to include in > the > > ICAP response. > > > > There are unofficial ICAP extensions that make it possible to tell > the > > ICAP client to reuse the body it has buffered while adapting the > header, > > but you should get the baseline case working before bothering with > those > > extensions -- they are optimizations that are not applicable to some > > transactions. > > > > > > > I think having req-null=X is bad since it > > > probably tells squid that I decided the adapted request should > have no > > > body, but that's only a guess. > > > > If you meant to say "null-body", then you guessed correctly -- > null-body > > means the adapted HTTP message has no body. That is not what you > want to > > say when adapting most HTTP POST messages. > > > > > > HTH, > > > > Alex. > > > >
_______________________________________________ squid-users mailing list squid-users@lists.squid-cache.org http://lists.squid-cache.org/listinfo/squid-users
