Hello Bruno and all, On Tue, May 08, 2018 at 01:10:52AM +0200, Bruno Haible wrote: > Let me try to summarize.
Thanks for writing this - informative and helpful. > * We need to consider each of the algorithms md5, sha1 .... sha256 separately, > because each algorithm has a different performance characteristic [1]. > This is due to the following factors: > - Some non-Intel hardware has crypto devices. [2] > - Intel hardware has special instructions for special crypto algorithms. > [3][4] A comment about the above: Talking about "crypto API" lumps together encryption (e.g. AES) and hashing. The link [3] talks about the "AES-NI" which is used for encryption (and from personal anecdotal evidence indeed provide nice speedups). These instructions do not help in sha1/256/512 hashing (which is what we're interested in). Intel also has "SHA Extensions" [7][8], but the only information I could find is that the instructions are only for sha1/sha256, and are only availble on the weaker models (e.g. Celeron) - because these models do not have more advanced SIMD/MIMD instructions. For CPUs with advanced instructions, Intel seems to promote AVX-512 instructions as the key to fast hashing [10]. [7] https://en.wikipedia.org/wiki/Intel_SHA_extensions [8] https://software.intel.com/en-us/isa-extensions/intel-sha [9] https://neosmart.net/blog/2017/will-amds-ryzen-finally-bring-sha-extensions-to-intels-cpus/ [10] https://software.intel.com/en-us/articles/intel-xeon-scalable-processor-cryptographic-performance > - The Linux kernel has specially optimized code for specific crypto > algorithms. [4] While the linux kernel code is indeed optimized with special CPU instructions, they don't actually require kernel mode privileges, and can be impemented in user mode [11] [12]. [11] https://software.intel.com/en-us/articles/improving-openssl-performance [12] https://github.com/minio/sha256-simd And so, I think it would be incorrect to say that most CPUs have dedicated SHA instructions that require kernel privileges, and it's very likely that coming versions of OpenSSL/libreSSL will already include support for these efficient implementations. Sorry to be a party-pooper, but just like gnulib/coreutils hasn't copied other (more efficient) hashing implementations, and decided to delegate it to OpenSSL, I think this case is very similar: If users/distributions want the most efficient implementations, let them link to a dedicated library that specializes in these kind of things (at least by default). > * The reasons for our disappointment are: > - The original presentation [2] was misleading because, as Assaf noticed > [5], > a large portion of the reported speedup (at least for Intel processors) > is due to a test case that > 1. is a corner case, > 2. exhibits a speedup that is due to sendfile(), not a different crypto > implementation. And also the fact that the performance improvements was measured against the (knowningly slow) C default implementation, and not compared to the faster OpenSSL implementation. OpenSSL since version 1.0.2 seems to support SIMD/AVX extensions for hashing. > > [1] https://lists.gnu.org/archive/html/bug-gnulib/2018-05/msg00043.html > [2] https://lists.gnu.org/archive/html/bug-gnulib/2018-04/msg00062.html > [3] https://en.wikipedia.org/wiki/AES_instruction_set > [4] https://lists.gnu.org/archive/html/bug-gnulib/2018-05/msg00038.html > [5] https://lists.gnu.org/archive/html/bug-gnulib/2018-04/msg00088.html > [6] https://lists.gnu.org/archive/html/bug-gnulib/2018-05/msg00044.html regards, - assaf
