> ### Description > > The CRC32 kernel used on machines without AVX-512 > (MacroAssembler::kernel_crc32) folds 512 bits of input per iteration, 128 > bits per each of four parallel streams, but it reduces the final 128-bit > state to 32 bits with the byte-wise table: two vpclmulqdq and a vpand fold > the state down to 64 bits using the K_M_64 constant, and the rest is folded > eight bits at a time with fold_8bit_crc32 - four folds on a 128-bit vector > register, then four more on a general purpose register. > > Each 8-bit fold computes crc = T[crc & 0xFF] ^ (crc >> 8) and depends on the > result of the previous fold, so the reduction is a serial chain of eight > table lookups whose addresses depend on the incoming data, plus five XMM->GPR > moves (four movdl to index the table, one to hand the value over to the GPR > folds). > > This change replaces the reduction with the folding and Barrett sequence that > kernel_crc32_avx512 already uses: > - vpclmulqdq folds the 128-bit state down to 96 and then to 64 bits using the > new K_128_64 constant; > - a Barrett reduction with the new K_64_32 constant cancels the upper 32 bits > with two carry-less multiplies; > - pextrd extracts the CRC directly from the vector register; pextrd is SSE4.1 > only, so it is guarded with VM_Version::supports_sse4_1() and falls back to > vpsrldq + movdl; > - the byte-wise table is still used by the prologue and for the trailing > bytes, but not by the reduction anymore. > > K_128_64 and K_64_32 are added to _crc_by128_masks, following the naming and > one-bit pre-shift conventions of the existing entries so that the table stays > ordered by fold distance. K_M_64 was only used by the removed reduction and > is dropped; the AOT address table is updated since the generated stub > references the new entries. Only the non-AVX-512 CRC32 path changes: > kernel_crc32_avx512 and CRC32C are untouched. > > ### TEST > - env: Intel Xeon 6980P (Granite Rapids), single socket, 128 cores / 256 > threads (SMT2), 3 NUMA nodes, 755 GiB RAM, Ubuntu 22.04.4 LTS, glibc 2.35 > - JMH: test/micro/org/openjdk/bench/java/util/TestCRC32.java > - jvmArgs: "-Xms4g -Xmx4g -XX:UseAVX=2" > - results (ops/ms): > | size | baseline | patched | delta | > | ---: | ---: | ---: | ---: | > | 16 | 23588.77 | 23589.00 | +0.0% | > | 32 | 14586.33 | 19589.51 | +34.3% | > | 64 | 13845.50 | 18218.98 | +31.6% | > | 128 | 12564.96 | 16064.29 | +27.8% | > | 256 | 12075.03 | 15205.08 | +25.9% | > | 512 | 11147.73 | 13791.16 | +23.7% | > | 1024 | 9623.20 | 11574.89 | +20.3% | > | 2048 | 7378.65 | 8544.03 | +...
Larry has updated the pull request incrementally with one additional commit since the last revision: 8394259: Remove unused CRC32 helper ------------- Changes: - all: https://git.openjdk.org/jdk/pull/33342/files - new: https://git.openjdk.org/jdk/pull/33342/files/35613760..1c9983f8 Webrevs: - full: https://webrevs.openjdk.org/?repo=jdk&pr=33342&range=03 - incr: https://webrevs.openjdk.org/?repo=jdk&pr=33342&range=02-03 Stats: 29 lines in 2 files changed: 0 ins; 29 del; 0 mod Patch: https://git.openjdk.org/jdk/pull/33342.diff Fetch: git fetch https://git.openjdk.org/jdk.git pull/33342/head:pull/33342 PR: https://git.openjdk.org/jdk/pull/33342
