I think you might be misunderstanding certain continuation guarantees you'll have.
First, with transformations you'll never have a content length header because the content length cannot be committed to until the final transformation is complete so it will always happen with chunked transfer encoding. Then regarding transformations. You will NEVER be called into your consume() while it's currently running. That is, if you're currently consuming a chunk you need not worry about "pausing" as you're already paused in that sense, there is underlying mutual exclusion. The same is true with handleInputComplete(). Given that I still believe the method I'm describing will work for you, in your consume() method just keep buffering until you get a handleInputComplete() or you observe you've exceeded your limit. If you're in consume and your limit is exceeded you would read back in all of your buffered data and produce() it, set a flag that your no longer buffering and immediately produce all data that arrives in comsume() going forward. Finally if you get called in handleInputCompelte() before hitting the limit it means you were obviously below that limit and you would do your analysis and then produce the buffered content or modify it or discard it. Unless I'm missing something I believe what you want to do can be done today. Does that all make sense? If you'd like we can try to chat on IRC Brian On Wednesday, June 22, 2016, David Ben Zakai <david.benza...@gmail.com> wrote: > Hi guys, > Thanks for your feedback > > James, > I can expose another method - lets say *void resume(uint ms) *that will > schedule it for the user, sounds good? > > Brian, > I want to buffer to a certain limit and when the 'Content-Length' header is > missing we can't know the file size, so we'll start buffering to file while > keeping track on the aggregated size. If we figure out that the file size > is greater than the limit then we'd like to pass on the analysis and just > consume all the data. For that we first need to produce all the file data > (Read chunks of it and consume it * N times), that's why we need to pause > the transformation we don't want to start buffering the data that keeps on > coming while we consume the file, when we are done consuming the file we > can resume the transformation and consume the rest of the data. > > Thanks > > On Wed, Jun 22, 2016 at 3:59 AM Brian Geffon <briangef...@gmail.com > <javascript:;>> wrote: > > > The thing I'm missing about this, why can't you just keep buffering to > the > > same file while you're doing analysis? I don't see why you need to pause > > anything? > > > > Brian > > > > On Friday, June 17, 2016, David Ben Zakai <david.benza...@gmail.com > <javascript:;>> > > wrote: > > > > > Hi all, > > > > > > I'd like to suggest an API change in the CPP API Transformation > > interface. > > > > > > My own use case is that I'd like to be able to pause the > transformation, > > > handle what I can from the file and release the buffered content before > > > resuming and releasing the rest of the data. > > > Basically what I'm trying to achieve: > > > > > > 1. Write data to file (up to a certain amount) > > > 2. File analysis > > > 3. Produce file data (and any leftovers) downstream > > > > > > When the file size reaches a certain size limit I'd like to be able to > > > pause the transformation and produce all of its content downstream and > > then > > > resume the transformation. > > > > > > API Changes: > > > TransformationPlugin.h: > > > /** > > > * Call this method if you wish to pause the transformation. > > > * Schedule the return value continuation to resume the transforamtion. > > > * If the continuation is scheduled and called after the transform is > > > destroyed it will > > > * won't do anything beyond cleanups. > > > * Note: You must schedule the continuation or destroy it (using > > > TSContDestroy) yourself, > > > * otherwise it will leak. > > > */ > > > TSCont pause(); > > > > > > Internally, the continuation wraps the "resumeCallback" static > function: > > > static int resumeCallback(TSCont cont, TSEvent event, void *edata); /** > > > Resume callback*/ > > > > > > You can view my jira issue here > > > <https://issues.apache.org/jira/browse/TS-4523> > > > You can view my pull request here > > > <https://github.com/apache/trafficserver/pull/712>. > > > > > > Thank you! > > > > > >
