Hi Ahmed, Shally, > -----Original Message----- > From: Verma, Shally [mailto:shally.ve...@cavium.com] > Sent: Thursday, February 15, 2018 5:53 AM > To: Ahmed Mansour <ahmed.mans...@nxp.com>; Trahe, Fiona > <fiona.tr...@intel.com>; > dev@dpdk.org > Cc: Athreya, Narayana Prasad <narayanaprasad.athr...@cavium.com>; Gupta, > Ashish > <ashish.gu...@cavium.com>; Sahu, Sunila <sunila.s...@cavium.com>; De Lara > Guarch, Pablo > <pablo.de.lara.gua...@intel.com>; Challa, Mahipal > <mahipal.cha...@cavium.com>; Jain, Deepak K > <deepak.k.j...@intel.com>; Hemant Agrawal <hemant.agra...@nxp.com>; Roy Pledge > <roy.ple...@nxp.com>; Youri Querry <youri.querr...@nxp.com> > Subject: RE: [RFC v2] doc compression API for DPDK > > > > >-----Original Message----- > >From: Ahmed Mansour [mailto:ahmed.mans...@nxp.com] > >Sent: 14 February 2018 22:25 > >To: Verma, Shally <shally.ve...@cavium.com>; Trahe, Fiona > ><fiona.tr...@intel.com>; dev@dpdk.org > >Cc: Athreya, Narayana Prasad <narayanaprasad.athr...@cavium.com>; Gupta, > >Ashish > <ashish.gu...@cavium.com>; Sahu, Sunila > ><sunila.s...@cavium.com>; De Lara Guarch, Pablo > ><pablo.de.lara.gua...@intel.com>; Challa, > Mahipal > ><mahipal.cha...@cavium.com>; Jain, Deepak K <deepak.k.j...@intel.com>; > >Hemant Agrawal > <hemant.agra...@nxp.com>; Roy > >Pledge <roy.ple...@nxp.com>; Youri Querry <youri.querr...@nxp.com> > >Subject: Re: [RFC v2] doc compression API for DPDK > > > >On 2/14/2018 12:41 AM, Verma, Shally wrote: > >> Hi Ahmed > >> > >>> -----Original Message----- > >>> From: Ahmed Mansour [mailto:ahmed.mans...@nxp.com] > >>> Sent: 02 February 2018 02:20 > >>> To: Trahe, Fiona <fiona.tr...@intel.com>; Verma, Shally > >>> <shally.ve...@cavium.com>; > dev@dpdk.org > >>> Cc: Athreya, Narayana Prasad <narayanaprasad.athr...@cavium.com>; Gupta, > >>> Ashish > <ashish.gu...@cavium.com>; Sahu, Sunila > >>> <sunila.s...@cavium.com>; De Lara Guarch, Pablo > >>> <pablo.de.lara.gua...@intel.com>; Challa, > Mahipal > >>> <mahipal.cha...@cavium.com>; Jain, Deepak K <deepak.k.j...@intel.com>; > >>> Hemant Agrawal > <hemant.agra...@nxp.com>; Roy > >>> Pledge <roy.ple...@nxp.com>; Youri Querry <youri.querr...@nxp.com> > >>> Subject: Re: [RFC v2] doc compression API for DPDK > >>> > >>>>>> [Fiona] I propose if BFINAL bit is detected before end of input > >>>>>> the decompression should stop. In this case consumed will be < > >>>>>> src.length. > >>>>>> produced will be < dst buffer size. Do we need an extra STATUS > >>>>>> response? > >>>>>> STATUS_BFINAL_DETECTED ? > >>>>> [Shally] @fiona, I assume you mean here decompressor stop after > >>>>> processing Final block right? > >>>> [Fiona] Yes. > >>>> > >>>> And if yes, > >>>>> and if it can process that final block successfully/unsuccessfully, > >>>>> then status could simply be > >>>>> SUCCESS/FAILED. > >>>>> I don't see need of specific return code for this use case. Just to > >>>>> share, in past, we have practically > run into > >>>>> such cases with boost lib, and decompressor has simply worked this way. > >>>> [Fiona] I'm ok with this. > >>>> > >>>>>> Only thing I don't like this is it can impact on performance, as > >>>>>> normally > >>>>>> we can just look for STATUS == SUCCESS. Anything else should be an > >>>>>> exception. > >>>>>> Now the application would have to check for SUCCESS || BFINAL_DETECTED > >>>>>> every time. > >>>>>> Do you have a suggestion on how we should handle this? > >>>>>> > >>> [Ahmed] This makes sense. So in all cases the PMD should assume that it > >>> should stop as soon as a BFINAL is observed. > >>> > >>> A question. What happens ins stateful vs stateless modes when > >>> decompressing an op that encompasses multiple BFINALs. I assume the > >>> caller in that case will use the consumed=x bytes to find out how far in > >>> to the input is the end of the first stream and start from the next > >>> byte. Is this correct? > >> [Shally] As per my understanding, each op can be tied up to only one > >> stream as we have only one > stream pointer per op and one > >stream can have only one BFINAL (as stream is one complete compressed data) > >but looks like you're > suggesting a case where one op > >can carry multiple independent streams? and thus multiple BFINAL?! , such > >as, below here is op > pointing to more than one streams > >> > >> -------------------------------------------- > >> op --> |stream1|stream2| |stream3| > >> -------------------------------------------- > >> > >> Could you confirm if I understand your question correct? > >[Ahmed] Correct. We found that in some storage applications the user > >does not know where exactly the BFINAL is. They rely on zlib software > >today. zlib.net software halts at the first BFINAL. Users put multiple > >streams in one op and rely on zlib to stop and inform them of the end > >location of the first stream. > > [Shally] Then this is practically case possible on decompressor and > decompressor doesn't regard flush > flag. So in that case, I expect PMD to internally reset themselves (say in > case of zlib going through cycle > of deflateEnd and deflateInit or deflateReset) and return with status = > SUCCESS with updated produced > and consumed. Now in such case, if previous stream also has some footer > followed by start of next > stream, then I am not sure how PMD / lib can support that case. Have you had > practically run of such > use-case on zlib? If yes, how then such application handle it in your > experience? > I can imagine for such input zlib would return with Z_FLUSH_END after 1st > BFINAL is processed to the > user. Then application doing deflateReset() or Init-End() cycle before > starting with next. But if it starts > with input that doesn't have valid zlib header, then likely it will throw an > error. > [Fiona] The consumed and produced tell the Application hw much data was processed up to the end of the first deflate block encountered with a bfinal set. If there is data, e.g. footer after the block with bfinal, then I think it must be the responsibility of the application to know this, the PMD can't have any responsibility for this. The next op sent to the PMD must start with a valid deflate block.
> >> > >> Thanks > >> Shally > >>
