Hi Fiona, > Hi Shally, Ahmed, and anyone else interested in compressdev, > > I mentioned last week that we've been exploring using something other than > mbufs to pass src/dst buffers to compressdev PMDs. > > Reasons: > - mbuf data is limited to 64k-1 in each segment of a chained mbuf. Data for > compression > can be greater and it would add cycles to have to break up into smaller > segments. > - data may originate in mbufs, but is more likely, particularly for storage > use-cases, to > originate in other data structures. > - There's a 2 cache-line overhead for every segment in a chain, most of this > data > is network-related, not needed by compressdev > So moving to a custom structure would minimise memory overhead, remove > restriction on 64k-1 size and give more flexibility if > compressdev ever needs any comp-specific meta-data. > > We've come up with a compressdev-specific structure using the struct iovec > from sys/uio.h, which is commonly used by storage > applications. This would replace the src and dest mbufs in the op. > I'll not include the code here - Pablo will push that to github shortly and > we'd appreciate review comments there. > https://github.com/pablodelara/dpdk-draft-compressdev > Just posting on the mailing list to give a heads-up and ensure this reaches a > wider audience than may see it on github. > > Note : We also considered having no data structures in the op, instead the > application > would supply a callback which the PMD would use to retrieve meta-data (virt > address, iova, length) > for each next segment as needed. While this is quite flexible and allow the > application > to keep its data in its native structures, it's likely to cost more cycles.
As I said in different thread it will not only slowdown things, but will make it difficult (if possible at all) for compressdev PMDs to support DPDK MP model. Konstantin > So we're not proposing this at the moment, but hope to benchmark it later > while the API is still experimental. > > General feedback on direction is welcome here on the mailing list. > For feedback on the details of implementation we would appreciate comments on > github. > > Regards, > Fiona.
