Thanks for the detailed response Alex. In our case we don't need to modify the initial 10MB, just scan it for virus and if found, send a reset back to squid to not transmit the file.
Do you have any additional recommendation for such case? Thanks, On Mon, Sep 16, 2019 at 11:54 AM Alex Rousskov < rouss...@measurement-factory.com> wrote: > On 9/16/19 10:37 AM, Felipe Arturo Polanco wrote: > > > We would like to add some logic to our custom made ICAP server, one of > > these logics is to analyze up to 10MB of data of a given file and if the > > file is larger than that, squid should not keep sending it to icap, > > basically, a 204 message should be returned. > > > squid has a cap of 64K of preview size that limit us in the aspect. > > True. > > > > Is there a way to extend this limit? Modifying the source code perhaps? > > Yes, of course. > > > > If so, what are the disadvantages of doing so? > > Simply increasing BodyPipe::MaxCapacity might open you up to > difficult-to-find vulnerabilities where some old Squid code still > assumes that the buffers are limited by 64KB and crashes/asserts when > that assumption becomes false. FWIW, I have not audited all > BodyPipe-related code and do not know whether such bad code exists today. > > Increasing BodyPipe::MaxCapacity will also increase Squid RAM > requirements if your Squid receives requests (and/or ICAP > REQMOD-generated HTTP responses) that exceed 64KB body limit. I am not > sure about regular responses -- their accumulation is limited by > other/independent restrictions. > > > > Would it require more RAM per response > > [You should worry about "per message" accumulation -- requests will use > the same limit, even if you do not have REQMOD services.] > > Yes, but only for messages that exceed the limit. > > > > or is RAM dynamically allocated as the file is being received? > > Yes, the RAM in question is allocated dynamically on "as needed" basis. > > YMMV, but Squid may slow down a lot while dealing with the associated > reallocations because of the too-small 64KB increment used for each such > reallocation. > > > An alternative solution to consider here is extending ICAP features to > allow for splicing of the 10MB prefix (analyzed _and_ returned back by > the ICAP server) and the remaining virgin response suffix (paused > without excessive accumulation on the Squid side). This would be similar > to the existing ICAP 206 use-original-body="10MB" feature[1], but would > require additional negotiation and changes to disable storage of the > 10MB prefix on the Squid side. > > This alternative can be implemented without worrying about hidden 64KB > body buffer assumptions, but the implementation will not be trivial. > > [1] > > http://www.icap-forum.org/documents/specification/draft-icap-ext-partial-content-07.txt > > > HTH, > > Alex. >
_______________________________________________ squid-users mailing list squid-users@lists.squid-cache.org http://lists.squid-cache.org/listinfo/squid-users
