Re: Adding slice-by-4 and slice-by-8 to CRC32

The coreutils code is GPL and the crc module in gnulib is LGPL. I'm using the gnulib crc module in some LGPL projects. Is it wortwhile to keep the optimization GPL? I would prefer a LGPL crc module that is performant, with #ifdef-varianted support for 1) no tables at all, 2) small table like tod

Re: Adding slice-by-4 and slice-by-8 to CRC32

The math nerd in me really likes pclmul but agree it adds a lot of complexity for effectively x64-only features. I'll focus on getting slice-by-8 ready in this case anyway, I am waiting for a reply from the GNU copyright team and then am planning to just get this one thing all the way to the end.

Re: Adding slice-by-4 and slice-by-8 to CRC32

On 15/10/2024 12:58, Sam Russell wrote: I'm happy with the slice-by-8 code I have > but the  cksum_pclmul implementation is quite detailed so it would be useful if we

Re: Adding slice-by-4 and slice-by-8 to CRC32

On Tue, Oct 15, 2024 at 10:59 AM Pádraig Brady wrote: > > On 15/10/2024 12:58, Sam Russell wrote: > > I'm happy with the slice-by-8 code I have > > > > but the > > c

Re: Adding slice-by-4 and slice-by-8 to CRC32

I'm happy with the slice-by-8 code I have < https://github.com/samrussell/gnulib/blob/slice_by_8/lib/crc.c> but the cksum_pclmul implementation is quite detailed so it would be useful if we could relicense that for gnulib. There's a minor optimisation around dealing with a non-round number of byte

Re: [PATCH] tests/test-crc.c: New tests for future optimised implementation.

Thanks this looks better. Let's keep it around for more feedback/review until your copyright papers arrive. /Simon Sam Russell writes: >> Or repeat "Gnulib crc test string" many times. > > done > >> This isn't correct until the new implementation is added > > done > >> I would make it test mu

Re: Adding slice-by-4 and slice-by-8 to CRC32

On 15/10/2024 08:07, Simon Josefsson wrote: The coreutils code is GPL and the crc module in gnulib is LGPL. I'm using the gnulib crc module in some LGPL projects. Is it wortwhile to keep the optimization GPL? I would prefer a LGPL crc module that is performant, with #ifdef-varianted support fo

Re: [PATCH] tests/test-crc.c: New tests for future optimised implementation.

> Or repeat "Gnulib crc test string" many times. done > This isn't correct until the new implementation is added done > I would make it test much larger > mis-alignment too, how about a ~700 byte string and test up to ~300 byte > mis-alignment? I'm not sure what problem this solves. I've used

Re: Adding slice-by-4 and slice-by-8 to CRC32

I suggest to focus on the immediate use-case and see if we can complete that, to not get overwhelmed with the variety of ideas. Just getting your bigger table speedup into a LGPL'd crc.c in gnulib, and to get gzip to use that module, seems like a good improvement. You should start on the copyrigh

Re: Adding slice-by-4 and slice-by-8 to CRC32

Sounds good, I've just sent a patch with a test to cover alignment issues we might run into in the optimised implementation, and I emailed ass...@fsf.org and asked to kick off the copyright assignment process. Anything else I should be starting now to make things run more smoothly? On Tue, 15 Oct

Re: [PATCH] tests/test-crc.c: New tests for future optimised implementation.

Sam Russell writes: > + char plaintext[] = "This file is free software: you can redistribu" > + "te it and/or modify it under the terms of the " > + "GNU Lesser General Public License as published" > + " by th"; Using license texts lik

Re: Adding slice-by-4 and slice-by-8 to CRC32

IANAL but it appears the source (paper, not code) for both implementations in coreutils is free, I'm happy to write from scratch based off the papers: Slice-by-8: "Novel Table Lookup-Based Algorithms for High-Performance CRC Generation" PCLMUL: "Fast CRC Computation for Numeric Polynomials Using P

[PATCH] tests/test-crc.c: New tests for future optimised implementation.

Patch file attached. As far as I can tell, the main concern with the new optimisations is that we're moving from processing byte-by-byte to processing in blocks of up to 128 bytes, and these will be read in either 8 or 16 byte chunks. This new test calls the inner CRC update function with data ali

Re: Adding slice-by-4 and slice-by-8 to CRC32

On 15/10/2024 16:50, Jeffrey Walton wrote: On Tue, Oct 15, 2024 at 10:59 AM Pádraig Brady wrote: On 15/10/2024 12:58, Sam Russell wrote: I'm happy with the slice-by-8 code I have

[PATCH] crc: New optimised slice-by-8 implementation

This is my implementation of the slice-by-8 algorithm for CRC32 generation. I've added a flag CRC_ENABLE_SLICE_BY_8, I'd appreciate if someone can give me a hint on how to set this up in the makefile config. I get the impression that we want this to be on by default or for the major architectures,

Re: Adding slice-by-4 and slice-by-8 to CRC32

Pádraig Brady writes: > In general gnulib focuses on portable routines, so for example leaves > platform specific crypto optimizations to libcrypto, only providing fallback > cross platform implementations where needed. OpenSSL libcrypto doesn't provide CRC, does it? I agree with what you say