Makes sense. So it's about knowing if a plugin needs the plain text or not. I'm not arguing against gzip in the core, we should do that for performance reasons.
Not quite sure I see the argument for which plugin does the gzip though, that sounds somewhat of a configuration issue. Something w suck at :) The order of global plugins is well defined, but maybe txn hooks makes this trickier too. Cheers, -- Leif > On Mar 24, 2015, at 7:50 AM, Roland Zink <rola...@flashnetworks.com> wrote: > > I think they can see the updated headers but this doesn't help. A plugin > can't know if the next plugin in the chain needs the plain text or even if > there is a next plugin. So every plugin is doing the gzip at the end. The > next plugin in the chain then will ungzip again. > > Regards, > Roland > > -----Original Message----- > From: Leif Hedstrom [mailto:zw...@apache.org] > Sent: Tuesday, March 24, 2015 12:43 PM > To: dev@trafficserver.apache.org > Cc: Brian Geffon; Cynthia Gu > Subject: Re: Adding Gzip/Gunzip feature in ATS core > > I guess I don't understand why the chaining makes a difference. That's what I > was asking in the previous email. If two plugins are chained, one should be > able to detect gzip or not gzip the same way it does as if it was not > chained. What am I missing ? Does each chained plugin not see the updated > header? Do they all see the same unmodified headers? That seems unfortunate > to say the least if it's so ;/. > > -- Leif > > > >> On Mar 24, 2015, at 12:39 AM, Shu Kit Chan <chanshu...@gmail.com> wrote: >> >> ESI should handle these situations correctly. e.g. it won't gzip the >> response if it is already gzipped. >> So it works fine in a standalone way. >> >> The problems begin when we starts to chain a few of these standalone >> transformation plugins together. Each of them will try to do the right >> thing when they are used in a standalone way and perform gunzip/gzip >> correctly. So chaining them together means I will be unnecessarily >> doing multiple gunzip/gzip on the contents. >> >> The other way is make them plugins be aware of each other and work >> together and don't unnecessarily do gunzip/gzip. But then this makes >> the plugins no longer standalone >> >> +1 on Dzmitry's idea. >> >> Kit >> >>> On Mon, Mar 23, 2015 at 5:11 PM, Leif Hedstrom <zw...@apache.org> wrote: >>> >>> >>>>> On Mar 23, 2015, at 6:08 PM, Dzmitry Markovich >>>> <dmarkov...@linkedin.com.INVALID> wrote: >>>> >>>> Hello ATS experts, >>>> >>>> Today it is very common that http data that goes to the wire is >>> compressed. And we think it is a time to standardize this process in >>> ATS core, since it is very common operation. >>>> >>>> Today multiple plugins running on the same tier (that operates with >>> response body) do not have enough flexibility to handle gzip/gunzip - >>> and most of the times simply every plugin do gzip/gunzip. This lead >>> to the situation when we do gzip/gunzip multiple times while we >>> process the http response. This leads to the bad CPU utilization and >>> performance. >>>> >>>> Yes, there is gzip/gunzip logic in atscpp API - so plugins can >>>> simply >>> use those. But today's ATS architecture does not allow us to fully >>> control the order of hook callbacks for every plugin per request - it >>> means there is no non-hacky way to prevent multiple gzip/gunzip calls >>> while processing the request. And there is no way to do that with the >>> assumption that plugins dnot know about each other. >>>> >>>> >>>> Here is our high level proposal, just to start the conversation going: >>>> - Have gzip/gunzip logic landed on ATS core, so engineers can ask >>>> ATS >>> (vi config parameter) to take care about gzip/gunzip logic for them >>>> 1. Only ATS know when the body arrived to first plugin - so ATS at >>> this point ungzip the body; >>>> 2. Only ATS knows when body is processed by all plugins and should >>>> go >>> over the wire - so ATS at this point gzip the body if client supported it. >>> >>> >>> Moving gzip to the core seems a generally good idea. Probably in some >>> extensible way such that we can add future compression encoding (I >>> think Chrome has support for some better ones?). >>> >>> Now, I’m slightly confused, and/or concerned, that our plugins do not >>> handle this well. I would have imagined that something doing gzip >>> would not do so if the content comes back with Content-Encoding: >>> gzip. So shouldn’t e.g. ESI detect if the gzip plugin has already done so? >>> >>> Is there a reason why a plugin *can’t* detect this? If not, we should >>> fix the plugins regardless of this RFE, it seems like a broken >>> behavior if a plugin can gzip something that’s already CE: gzip. >>> >>> Cheers, >>> >>> — Leif >>> >>>
