Hi Ruei-Bang, On Thu, Oct 12, 2023 at 09:44:22PM +0000, Ruei-Bang Chen wrote: > Hi team, > > I am writing to gather feedback on an idea before doing the implementation. > > > We have a use-case where we need all headers except for cookies. Currently, > the fetcher "req.hdrs" / "res.hdrs" returns all headers including cookie / > set-cookie headers. > > However, the cookie header itself can get pretty huge and cause unnecessary > overhead when we want to store / send / parse the string variable. > > In a high qps environment, this will cause big degradation to the latency of > the requests, so the proposal is to add a new fetcher for sending the headers > excluding cookies. > > Let me know what you think. If there are no concerns, I can send a patch.
I'm not necessarily against it but I fear that it opens a can of worms. Others will say that User-Agent is often too verbose and will want to skip it, then it will be Referer, then X-Pad, then all Connection and Cache-control headers etc. So if we go in that direction, we'd need to keep the exclusion list configurable. For example we could imagine that req.hdrs takes an optional list of arguments that are either a restricted list of headers to extract, or to exclude depending on the syntax. I think we can support up to about 10 arguments so maybe that could do the job. For example: req.hdrs() => all req.hdrs(host,referer,x-forwarded-for) => only these req.hdrs(!cookie,referer,connection) => all but these Alternately, have you tried to make your own sample-fetch function in Lua to deal with the headers you want ? I'm pretty sure that it was earlier used like this before becoming a native function. Thanks, Willy
