Hi Ahmed, Shally, > -----Original Message----- > From: Verma, Shally [mailto:[email protected]] > Sent: Thursday, February 15, 2018 5:53 AM > To: Ahmed Mansour <[email protected]>; Trahe, Fiona > <[email protected]>; > [email protected] > Cc: Athreya, Narayana Prasad <[email protected]>; Gupta, > Ashish > <[email protected]>; Sahu, Sunila <[email protected]>; De Lara > Guarch, Pablo > <[email protected]>; Challa, Mahipal > <[email protected]>; Jain, Deepak K > <[email protected]>; Hemant Agrawal <[email protected]>; Roy Pledge > <[email protected]>; Youri Querry <[email protected]> > Subject: RE: [RFC v2] doc compression API for DPDK > > > > >-----Original Message----- > >From: Ahmed Mansour [mailto:[email protected]] > >Sent: 14 February 2018 22:25 > >To: Verma, Shally <[email protected]>; Trahe, Fiona > ><[email protected]>; [email protected] > >Cc: Athreya, Narayana Prasad <[email protected]>; Gupta, > >Ashish > <[email protected]>; Sahu, Sunila > ><[email protected]>; De Lara Guarch, Pablo > ><[email protected]>; Challa, > Mahipal > ><[email protected]>; Jain, Deepak K <[email protected]>; > >Hemant Agrawal > <[email protected]>; Roy > >Pledge <[email protected]>; Youri Querry <[email protected]> > >Subject: Re: [RFC v2] doc compression API for DPDK > > > >On 2/14/2018 12:41 AM, Verma, Shally wrote: > >> Hi Ahmed > >> > >>> -----Original Message----- > >>> From: Ahmed Mansour [mailto:[email protected]] > >>> Sent: 02 February 2018 02:20 > >>> To: Trahe, Fiona <[email protected]>; Verma, Shally > >>> <[email protected]>; > [email protected] > >>> Cc: Athreya, Narayana Prasad <[email protected]>; Gupta, > >>> Ashish > <[email protected]>; Sahu, Sunila > >>> <[email protected]>; De Lara Guarch, Pablo > >>> <[email protected]>; Challa, > Mahipal > >>> <[email protected]>; Jain, Deepak K <[email protected]>; > >>> Hemant Agrawal > <[email protected]>; Roy > >>> Pledge <[email protected]>; Youri Querry <[email protected]> > >>> Subject: Re: [RFC v2] doc compression API for DPDK > >>> > >>>>>> [Fiona] I propose if BFINAL bit is detected before end of input > >>>>>> the decompression should stop. In this case consumed will be < > >>>>>> src.length. > >>>>>> produced will be < dst buffer size. Do we need an extra STATUS > >>>>>> response? > >>>>>> STATUS_BFINAL_DETECTED ? > >>>>> [Shally] @fiona, I assume you mean here decompressor stop after > >>>>> processing Final block right? > >>>> [Fiona] Yes. > >>>> > >>>> And if yes, > >>>>> and if it can process that final block successfully/unsuccessfully, > >>>>> then status could simply be > >>>>> SUCCESS/FAILED. > >>>>> I don't see need of specific return code for this use case. Just to > >>>>> share, in past, we have practically > run into > >>>>> such cases with boost lib, and decompressor has simply worked this way. > >>>> [Fiona] I'm ok with this. > >>>> > >>>>>> Only thing I don't like this is it can impact on performance, as > >>>>>> normally > >>>>>> we can just look for STATUS == SUCCESS. Anything else should be an > >>>>>> exception. > >>>>>> Now the application would have to check for SUCCESS || BFINAL_DETECTED > >>>>>> every time. > >>>>>> Do you have a suggestion on how we should handle this? > >>>>>> > >>> [Ahmed] This makes sense. So in all cases the PMD should assume that it > >>> should stop as soon as a BFINAL is observed. > >>> > >>> A question. What happens ins stateful vs stateless modes when > >>> decompressing an op that encompasses multiple BFINALs. I assume the > >>> caller in that case will use the consumed=x bytes to find out how far in > >>> to the input is the end of the first stream and start from the next > >>> byte. Is this correct? > >> [Shally] As per my understanding, each op can be tied up to only one > >> stream as we have only one > stream pointer per op and one > >stream can have only one BFINAL (as stream is one complete compressed data) > >but looks like you're > suggesting a case where one op > >can carry multiple independent streams? and thus multiple BFINAL?! , such > >as, below here is op > pointing to more than one streams > >> > >> -------------------------------------------- > >> op --> |stream1|stream2| |stream3| > >> -------------------------------------------- > >> > >> Could you confirm if I understand your question correct? > >[Ahmed] Correct. We found that in some storage applications the user > >does not know where exactly the BFINAL is. They rely on zlib software > >today. zlib.net software halts at the first BFINAL. Users put multiple > >streams in one op and rely on zlib to stop and inform them of the end > >location of the first stream. > > [Shally] Then this is practically case possible on decompressor and > decompressor doesn't regard flush > flag. So in that case, I expect PMD to internally reset themselves (say in > case of zlib going through cycle > of deflateEnd and deflateInit or deflateReset) and return with status = > SUCCESS with updated produced > and consumed. Now in such case, if previous stream also has some footer > followed by start of next > stream, then I am not sure how PMD / lib can support that case. Have you had > practically run of such > use-case on zlib? If yes, how then such application handle it in your > experience? > I can imagine for such input zlib would return with Z_FLUSH_END after 1st > BFINAL is processed to the > user. Then application doing deflateReset() or Init-End() cycle before > starting with next. But if it starts > with input that doesn't have valid zlib header, then likely it will throw an > error. > [Fiona] The consumed and produced tell the Application hw much data was processed up to the end of the first deflate block encountered with a bfinal set. If there is data, e.g. footer after the block with bfinal, then I think it must be the responsibility of the application to know this, the PMD can't have any responsibility for this. The next op sent to the PMD must start with a valid deflate block.
> >> > >> Thanks > >> Shally > >>
