That's only true for global hooks - in this case we can extrapolate the ordering, but even there it is tricky. Example, lets say we have three plugins A,B,C and all of them have registered: read request (lets call it req) headers and read respons headers (lets call it resp).
In plugin.conf they order is the following: A,B,C. In this case the callbacks ordering will look like: For request A - req B - req C - req For response: A - resp B - resp C - resp So, you can see that first plugin A will execute req and resp as a first one. Last plugin C will execute req last and resp last. But the request/response headers might change in between already., it means plugins are not stand alone. >From my point of view global callback ordering should look like: For request A - req B - req C - req For response: C - resp B - resp A - resp This should help with globals. But there are a lot of cases when based on some response headers new continuations are added, and those new continuations we work with response body. -Dzmitry ________________________________________ From: Leif Hedstrom [zw...@apache.org] Sent: Tuesday, March 24, 2015 10:36 AM To: Dzmitry Markovich Cc: dev@trafficserver.apache.org; Roland Zink; Brian Geffon; Cynthia Gu Subject: Re: Adding Gzip/Gunzip feature in ATS core Hmmm I'm not anywhere near a computer but continuations should get called in the order they were added. Is that not the case ? In the case of global plugins that's the order in the plugin.config. In remap plugins, it's the order in the remap rule. Where it gets iffy is when plugins adds new continuations as part of the handler of another callback. Maybe I'm still just confused :) -- Leif > On Mar 24, 2015, at 1:20 PM, Dzmitry Markovich <dmarkov...@linkedin.com> > wrote: > > Setting up unzip plugin at first and zip plugin at last will not help. We > tried this already. > There is no clear way to control that first response hook will go to unzip > plugin and last response hook will go to zip plugin. > > Also, there might be a case when based on some run-time logic plugins in the > chain will not touch the response body (it is already gzipped and it is > static content file - js/css) in this case we should not execute unzip and > zip plugin at all. But if it is just unzip and gzip plugin we cant > extrapolate this logic and we will execute gzip/gunzip where it is not > necessary. > > And at this point only ATS know about the "body/data" hooks registered for > transaction, so ATS will know whether it is necessary to unzip response, > because there are "body/data" hooks registered. And on the way out to the > client, based on Accept-Encoding header ATS will zip the body if necessary. > > -Dzmitry > > ________________________________________ > From: gang li [portl4t...@gmail.com] > Sent: Tuesday, March 24, 2015 6:38 AM > To: dev@trafficserver.apache.org > Cc: Roland Zink; Brian Geffon; Cynthia Gu > Subject: Re: Adding Gzip/Gunzip feature in ATS core > > I got this problem before, I think we can just set the unzip plugin at > first and zip plugin at last. > >> On Tue, Mar 24, 2015 at 8:39 PM, Leif Hedstrom <zw...@apache.org> wrote: >> >> Makes sense. So it's about knowing if a plugin needs the plain text or >> not. I'm not arguing against gzip in the core, we should do that for >> performance reasons. >> >> Not quite sure I see the argument for which plugin does the gzip though, >> that sounds somewhat of a configuration issue. Something w suck at :) The >> order of global plugins is well defined, but maybe txn hooks makes this >> trickier too. >> >> Cheers, >> >> -- Leif >> >> >> >>> On Mar 24, 2015, at 7:50 AM, Roland Zink <rola...@flashnetworks.com> >> wrote: >>> >>> I think they can see the updated headers but this doesn't help. A plugin >> can't know if the next plugin in the chain needs the plain text or even if >> there is a next plugin. So every plugin is doing the gzip at the end. The >> next plugin in the chain then will ungzip again. >>> >>> Regards, >>> Roland >>> >>> -----Original Message----- >>> From: Leif Hedstrom [mailto:zw...@apache.org] >>> Sent: Tuesday, March 24, 2015 12:43 PM >>> To: dev@trafficserver.apache.org >>> Cc: Brian Geffon; Cynthia Gu >>> Subject: Re: Adding Gzip/Gunzip feature in ATS core >>> >>> I guess I don't understand why the chaining makes a difference. That's >> what I was asking in the previous email. If two plugins are chained, one >> should be able to detect gzip or not gzip the same way it does as if it was >> not chained. What am I missing ? Does each chained plugin not see the >> updated header? Do they all see the same unmodified headers? That seems >> unfortunate to say the least if it's so ;/. >>> >>> -- Leif >>> >>> >>> >>>> On Mar 24, 2015, at 12:39 AM, Shu Kit Chan <chanshu...@gmail.com> >> wrote: >>>> >>>> ESI should handle these situations correctly. e.g. it won't gzip the >>>> response if it is already gzipped. >>>> So it works fine in a standalone way. >>>> >>>> The problems begin when we starts to chain a few of these standalone >>>> transformation plugins together. Each of them will try to do the right >>>> thing when they are used in a standalone way and perform gunzip/gzip >>>> correctly. So chaining them together means I will be unnecessarily >>>> doing multiple gunzip/gzip on the contents. >>>> >>>> The other way is make them plugins be aware of each other and work >>>> together and don't unnecessarily do gunzip/gzip. But then this makes >>>> the plugins no longer standalone >>>> >>>> +1 on Dzmitry's idea. >>>> >>>> Kit >>>> >>>>> On Mon, Mar 23, 2015 at 5:11 PM, Leif Hedstrom <zw...@apache.org> >> wrote: >>>>> >>>>> >>>>>>> On Mar 23, 2015, at 6:08 PM, Dzmitry Markovich >>>>>> <dmarkov...@linkedin.com.INVALID> wrote: >>>>>> >>>>>> Hello ATS experts, >>>>>> >>>>>> Today it is very common that http data that goes to the wire is >>>>> compressed. And we think it is a time to standardize this process in >>>>> ATS core, since it is very common operation. >>>>>> >>>>>> Today multiple plugins running on the same tier (that operates with >>>>> response body) do not have enough flexibility to handle gzip/gunzip - >>>>> and most of the times simply every plugin do gzip/gunzip. This lead >>>>> to the situation when we do gzip/gunzip multiple times while we >>>>> process the http response. This leads to the bad CPU utilization and >> performance. >>>>>> >>>>>> Yes, there is gzip/gunzip logic in atscpp API - so plugins can >>>>>> simply >>>>> use those. But today's ATS architecture does not allow us to fully >>>>> control the order of hook callbacks for every plugin per request - it >>>>> means there is no non-hacky way to prevent multiple gzip/gunzip calls >>>>> while processing the request. And there is no way to do that with the >>>>> assumption that plugins dnot know about each other. >>>>>> >>>>>> >>>>>> Here is our high level proposal, just to start the conversation going: >>>>>> - Have gzip/gunzip logic landed on ATS core, so engineers can ask >>>>>> ATS >>>>> (vi config parameter) to take care about gzip/gunzip logic for them >>>>>> 1. Only ATS know when the body arrived to first plugin - so ATS at >>>>> this point ungzip the body; >>>>>> 2. Only ATS knows when body is processed by all plugins and should >>>>>> go >>>>> over the wire - so ATS at this point gzip the body if client supported >> it. >>>>> >>>>> >>>>> Moving gzip to the core seems a generally good idea. Probably in some >>>>> extensible way such that we can add future compression encoding (I >>>>> think Chrome has support for some better ones?). >>>>> >>>>> Now, I’m slightly confused, and/or concerned, that our plugins do not >>>>> handle this well. I would have imagined that something doing gzip >>>>> would not do so if the content comes back with Content-Encoding: >>>>> gzip. So shouldn’t e.g. ESI detect if the gzip plugin has already done >> so? >>>>> >>>>> Is there a reason why a plugin *can’t* detect this? If not, we should >>>>> fix the plugins regardless of this RFE, it seems like a broken >>>>> behavior if a plugin can gzip something that’s already CE: gzip. >>>>> >>>>> Cheers, >>>>> >>>>> — Leif >>>>> >>>>> >>
