Hi Shally, > -----Original Message----- > From: Verma, Shally [mailto:shally.ve...@cavium.com] > Sent: Tuesday, March 13, 2018 8:15 AM > To: Trahe, Fiona <fiona.tr...@intel.com>; Ahmed Mansour > <ahmed.mans...@nxp.com>; dev@dpdk.org > Cc: De Lara Guarch, Pablo <pablo.de.lara.gua...@intel.com>; Athreya, Narayana > Prasad > <narayanaprasad.athr...@cavium.com>; Gupta, Ashish <ashish.gu...@cavium.com>; > Sahu, Sunila > <sunila.s...@cavium.com>; Challa, Mahipal <mahipal.cha...@cavium.com>; Jain, > Deepak K > <deepak.k.j...@intel.com>; Hemant Agrawal <hemant.agra...@nxp.com>; Roy Pledge > <roy.ple...@nxp.com>; Youri Querry <youri.querr...@nxp.com>; > fiona.tr...@gmail.com; Daly, Lee > <lee.d...@intel.com>; Jozwiak, TomaszX <tomaszx.jozw...@intel.com> > Subject: RE: [dpdk-dev] [PATCH] compressdev: implement API - mbuf alternative > > HI Fiona > > So I understand we're moving away from mbufs because of its size limitation > (64k) and cacheline overhead > and their more suitability to n/w applications. Given that, I understand > benefit of having another structure > to input data but then what is proposal for ipcomp like application where > mbuf usage may be a better > option? Should we keep support for both (mbuf and this structure) so that > apps can use appropriate data > structure depending on their requirement. [Fiona] An application can use pass buffers from an mbuf or mbuf chain to compressdev by filling in the compressdev struct fields with the mbuf meta-data, using rte_pktmbuf_data_len(), rte_pktmbuf_mtod(), rte_pktmbuf_mtophys(), etc For simplicity I'd prefer to offer only 1 rather than 2 data formats on the API. We see storage applications rather than IPComp as the main use-case for compressdev, so would prefer to optimise for that. Do you think otherwise?
> > Further comments, on github. > > Thanks > Shally > > >-----Original Message----- > >From: Trahe, Fiona [mailto:fiona.tr...@intel.com] > >Sent: 12 March 2018 21:31 > >To: Ahmed Mansour <ahmed.mans...@nxp.com>; Verma, Shally > ><shally.ve...@cavium.com>; > dev@dpdk.org > >Cc: De Lara Guarch, Pablo <pablo.de.lara.gua...@intel.com>; Athreya, > >Narayana Prasad > <narayanaprasad.athr...@cavium.com>; > >Gupta, Ashish <ashish.gu...@cavium.com>; Sahu, Sunila > ><sunila.s...@cavium.com>; Challa, Mahipal > ><mahipal.cha...@cavium.com>; Jain, Deepak K <deepak.k.j...@intel.com>; > >Hemant Agrawal > <hemant.agra...@nxp.com>; Roy > >Pledge <roy.ple...@nxp.com>; Youri Querry <youri.querr...@nxp.com>; > >fiona.tr...@gmail.com; Daly, > Lee <lee.d...@intel.com>; > >Jozwiak, TomaszX <tomaszx.jozw...@intel.com> > >Subject: RE: [dpdk-dev] [PATCH] compressdev: implement API - mbuf alternative > > > >Hi Shally, Ahmed, and anyone else interested in compressdev, > > > >I mentioned last week that we've been exploring using something other than > >mbufs to pass src/dst > buffers to compressdev PMDs. > > > >Reasons: > > - mbuf data is limited to 64k-1 in each segment of a chained mbuf. Data for > > compression > > can be greater and it would add cycles to have to break up into smaller > > segments. > > - data may originate in mbufs, but is more likely, particularly for storage > > use-cases, to > > originate in other data structures. > > - There's a 2 cache-line overhead for every segment in a chain, most of > > this data > > is network-related, not needed by compressdev > >So moving to a custom structure would minimise memory overhead, remove > >restriction on 64k-1 size and > give more flexibility if > >compressdev ever needs any comp-specific meta-data. > > > >We've come up with a compressdev-specific structure using the struct iovec > >from sys/uio.h, which is > commonly used by storage > >applications. This would replace the src and dest mbufs in the op. > >I'll not include the code here - Pablo will push that to github shortly and > >we'd appreciate review > comments there. > >https://github.com/pablodelara/dpdk-draft-compressdev > >Just posting on the mailing list to give a heads-up and ensure this reaches > >a wider audience than may see > it on github. > > > >Note : We also considered having no data structures in the op, instead the > >application > >would supply a callback which the PMD would use to retrieve meta-data (virt > >address, iova, length) > >for each next segment as needed. While this is quite flexible and allow the > >application > >to keep its data in its native structures, it's likely to cost more cycles. > >So we're not proposing this at the moment, but hope to benchmark it later > >while the API is still > experimental. > > > >General feedback on direction is welcome here on the mailing list. > >For feedback on the details of implementation we would appreciate comments > >on github. > > > >Regards, > >Fiona.
