> -----Original Message-----
> From: Thomas Monjalon [mailto:thomas.monja...@6wind.com]
> Sent: Wednesday, March 15, 2017 8:16 PM
> To: Dumitrescu, Cristian <cristian.dumitre...@intel.com>
> Cc: De Lara Guarch, Pablo <pablo.de.lara.gua...@intel.com>; Singh,
> Jasvinder <jasvinder.si...@intel.com>; dev@dpdk.org; Doherty, Declan
> <declan.dohe...@intel.com>
> Subject: Re: [dpdk-dev] [PATCH v2 1/2] librte_net: add crc init and compute
> APIs
>
> 2017-03-15 19:03, Dumitrescu, Cristian:
> > ... <snip>
> >
> > > > > > > I think it should be in librte_hash.
> > > > > > >
> > > > > > > Please check lib/librte_hash/rte_hash_crc.h
> > > > > >
> > > > > > Is it good to include payload crc calculation in hash library as I
> > > > > > see all
> > > hash
> > > > > related functionality there?
> > > > >
> > > > > I think yes. Pablo?
> > > >
> > > > I think this doesn't belong in the hash library. These new functions
> calculate
> > > CRC, but not as a hash function.
> > >
> > > Can't we say that a CRC is a hash? What is a hash?
> > > A function generating the same output bytes from given input bytes.
> > >
> > > I think you must separate hash functions and hash table management.
> > >
> >
> > The fact that CRC32 instruction is opportunistically used to compute a hash
> digest/signature for load balancing (affinity-based) or hash tables (flow
> tables, ARP cache, etc) does not mean that all the code that uses CRC32
> instruction should be placed in librte_hash.
> >
> > The purpose of the hash functions in librte_hash is to compute a
> digest/signature for a given array of bytes (the key) as fast as possible. Any
> hash function that produces a hash signature with good uniform distribution
> in a small amount of cycles belongs here, including those opportunistically
> using specialized CPU instructions such as CRC32 (or XOR, AESNI, etc).
> >
> > The code proposed in this patch is used to compute packet fields for
> various protocols that have a CRC field, such as FCS of Ethernet frames, etc.
> according to the relevant standard (IEEE 802, others). It is an utility to be
> used
> for implementing protocol-level functionality for various protocols from the
> network stack, similar to e.g. IP or UDP checksum. If we were to add an IP or
> UDCP checksum calculator, would you put it in librte_hash?
> >
> > The code from this patch is never going to be used to compute a fast
> signature/digest. Typically this CRC is computed over the entire frame/packet
> rather than just selected fields from the packet representing the application-
> specific flow key. Also note that the signature produced by CRC32 hash
> function from librte_hash is actually not the correct Cyclic Redundancy Check
> of that array of bytes (or, for math guys, of the associated polynomial), it
> is
> just a partial/intermediate value.
> >
> > Therefore, I suggest placing this code into: librte_ether (given that it
> > can be
> used to compute Ethernet FCS), or librte_net, or librte_crc. Definitely not in
> librte_hash.
>
> I agree with you Cristian that the protocol layer must be in librte_net.
> But I think most of this patch is not protocol level.
Nope, this is the true CRC computed over entire protocol header and/or payload.
Similar to to IPv4/UDP/TCP checksum. The only reason for computing it is
because the protocol specs require it for data integrity checks, nothing to do
with our signature for load balancing/hash tables.
More details on covered protocols from a reliable source :) [1]:
CRC-32 (polynomial = 0x04C11DB7): used for HDLC, ANSI X3.66, ITU-T
V.42, Ethernet, Serial ATA, MPEG-2, PKZIP, Gzip, Bzip2, PNG, many others
CRC-16-CCITT (polynomial = 0x1021): used for X.25, V.41, HDLC FCS,
XMODEM, Bluetooth, PACTOR, SD, DigRF, many others
[1] Wikipedia:
https://en.wikipedia.org/wiki/Cyclic_redundancy_check#Standards_and_common_use
> I think you agree with me that the code computing a
> "digest/signature for a given array of bytes" must go to librte_hash?
Yes, but this is the true protocol-level CRC, not a digest/signature.
Non-cryptographic hash digest/signature:
-computed over selected packet fields (flow key) for load balancing (affinity
scheme) or hash table; key size typical range: 8 .. 192 bytes
-required by application requirements (such as flow packet ordering
preservation), not by protocol standards
-has uniform distribution
-requires small amount of cycles to compute
-used as meta-data, not written in the packet
-can be opportunistically generated using specialized CPU instructions, such as
CRC32 (or XEOR, or AESNI); in this case, it is a partial/intermediate value,
not the correct CRC of the array of bytes
Protocol CRC:
-computed over entire packet header and/or payload
-protocol overhead (required by standards)
-computational cost is typically big and proportional with the packet length;
packet length typical range: 64 .. 1514 .. 9K
-written in the packet (by the application SW or by the HW)
