gsmiller commented on PR #13658:
URL: https://github.com/apache/lucene/pull/13658#issuecomment-2296882889
I ran `wikimedium10m` and can (sort of) see the regression on AndHighMed
(but only that task). Results below, but I see `2.4% with a 0.058 p-value. This
is on an AWS m4.12xlarge x86-based ec2 instance (lscpu info also below).
```
TaskQPS baseline StdDevQPS
my_modified_version StdDev Pct diff p-value
PKLookup 137.15 (11.7%) 131.58
(9.7%) -4.1% ( -22% - 19%) 0.232
BrowseDayOfYearSSDVFacets 11.12 (16.3%) 10.74
(13.7%) -3.4% ( -28% - 31%) 0.475
LowSpanNear 200.12 (16.2%) 194.44
(15.0%) -2.8% ( -29% - 33%) 0.566
Prefix3 625.50 (12.8%) 608.03
(14.4%) -2.8% ( -26% - 28%) 0.517
AndHighHighDayTaxoFacets 16.48 (6.5%) 16.02
(7.3%) -2.8% ( -15% - 11%) 0.201
HighSloppyPhrase 56.05 (5.1%) 54.49
(5.9%) -2.8% ( -13% - 8%) 0.110
HighTermDayOfYearSort 348.17 (7.3%) 338.70
(7.4%) -2.7% ( -16% - 12%) 0.241
AndHighMed 399.50 (4.1%) 389.84
(3.9%) -2.4% ( -10% - 5%) 0.058
AndHighMedDayTaxoFacets 40.17 (5.7%) 39.21
(5.1%) -2.4% ( -12% - 8%) 0.167
MedSpanNear 37.99 (4.4%) 37.23
(4.7%) -2.0% ( -10% - 7%) 0.168
HighIntervalsOrdered 21.57 (6.0%) 21.16
(5.6%) -1.9% ( -12% - 10%) 0.314
TermDTSort 187.47 (9.5%) 184.23
(6.7%) -1.7% ( -16% - 15%) 0.504
HighTermTitleSort 102.21 (4.6%) 100.51
(4.7%) -1.7% ( -10% - 7%) 0.256
OrHighNotMed 475.86 (8.2%) 468.17
(6.5%) -1.6% ( -15% - 14%) 0.490
Fuzzy2 61.34 (8.7%) 60.41
(7.6%) -1.5% ( -16% - 16%) 0.557
MedTermDayTaxoFacets 39.23 (6.9%) 38.63
(6.3%) -1.5% ( -13% - 12%) 0.467
OrNotHighMed 552.58 (5.5%) 544.55
(4.1%) -1.5% ( -10% - 8%) 0.339
HighSpanNear 15.81 (6.8%) 15.59
(5.8%) -1.4% ( -13% - 12%) 0.484
OrHighNotLow 331.39 (9.2%) 327.30
(7.6%) -1.2% ( -16% - 17%) 0.645
AndHighLow 1591.14 (6.9%) 1573.26
(5.6%) -1.1% ( -12% - 12%) 0.571
MedPhrase 164.71 (13.0%) 162.95
(10.8%) -1.1% ( -22% - 26%) 0.778
BrowseRandomLabelSSDVFacets 7.71 (12.4%) 7.63
(8.9%) -1.1% ( -19% - 23%) 0.758
HighTermMonthSort 1163.43 (3.0%) 1152.28
(5.3%) -1.0% ( -8% - 7%) 0.482
OrNotHighLow 1282.20 (2.0%) 1270.00
(3.4%) -1.0% ( -6% - 4%) 0.279
LowPhrase 94.18 (4.3%) 93.37
(4.1%) -0.9% ( -8% - 7%) 0.514
OrHighMed 381.27 (3.5%) 378.01
(4.1%) -0.9% ( -8% - 7%) 0.482
LowIntervalsOrdered 88.52 (3.3%) 87.85
(2.7%) -0.8% ( -6% - 5%) 0.425
OrHighHigh 129.35 (8.1%) 128.45
(7.5%) -0.7% ( -15% - 16%) 0.775
Respell 34.66 (8.6%) 34.44
(7.5%) -0.6% ( -15% - 16%) 0.802
BrowseMonthSSDVFacets 11.15 (15.7%) 11.08
(14.4%) -0.6% ( -26% - 34%) 0.895
OrHighMedDayTaxoFacets 6.12 (5.8%) 6.09
(7.1%) -0.6% ( -12% - 13%) 0.776
LowSloppyPhrase 169.38 (5.0%) 168.65
(5.2%) -0.4% ( -10% - 10%) 0.788
OrHighNotHigh 457.60 (5.9%) 455.89
(6.9%) -0.4% ( -12% - 13%) 0.854
OrHighLow 646.02 (5.2%) 643.87
(3.6%) -0.3% ( -8% - 8%) 0.813
Wildcard 515.01 (4.8%) 513.75
(4.5%) -0.2% ( -9% - 9%) 0.869
AndHighHigh 179.57 (6.3%) 179.15
(6.0%) -0.2% ( -11% - 12%) 0.904
MedIntervalsOrdered 75.31 (9.1%) 75.25
(7.9%) -0.1% ( -15% - 18%) 0.977
HighPhrase 8.55 (7.4%) 8.57
(7.0%) 0.1% ( -13% - 15%) 0.954
OrNotHighHigh 628.86 (5.5%) 630.10
(4.5%) 0.2% ( -9% - 10%) 0.902
MedSloppyPhrase 102.99 (14.2%) 103.38
(14.2%) 0.4% ( -24% - 33%) 0.933
IntNRQ 683.08 (10.0%) 687.13
(9.6%) 0.6% ( -17% - 22%) 0.848
LowTerm 1076.27 (4.9%) 1083.49
(4.5%) 0.7% ( -8% - 10%) 0.652
HighTerm 507.46 (7.6%) 511.43
(6.9%) 0.8% ( -12% - 16%) 0.733
HighTermTitleBDVSort 67.41 (4.9%) 68.10
(5.2%) 1.0% ( -8% - 11%) 0.514
MedTerm 519.28 (6.2%) 526.10
(6.2%) 1.3% ( -10% - 14%) 0.504
Fuzzy1 73.78 (9.7%) 75.28
(10.5%) 2.0% ( -16% - 24%) 0.525
BrowseDateTaxoFacets 22.84 (26.5%) 23.67
(21.4%) 3.6% ( -35% - 70%) 0.633
BrowseDayOfYearTaxoFacets 23.25 (27.7%) 24.35
(22.4%) 4.7% ( -35% - 75%) 0.553
BrowseRandomLabelTaxoFacets 14.84 (23.7%) 15.76
(18.2%) 6.2% ( -28% - 62%) 0.353
BrowseDateSSDVFacets 2.03 (18.9%) 2.17
(19.1%) 6.7% ( -26% - 54%) 0.266
BrowseMonthTaxoFacets 22.77 (26.6%) 25.55
(21.9%) 12.2% ( -28% - 82%) 0.113
```
```
gmiller-m4.12xl% lscpu
[/workplace/gmiller/upstream/benchmarks/luceneutil
Architecture: x86_64
CPU op-mode(s): 32-bit, 64-bit
Byte Order: Little Endian
CPU(s): 48
On-line CPU(s) list: 0-47
Thread(s) per core: 2
Core(s) per socket: 24
Socket(s): 1
NUMA node(s): 1
Vendor ID: GenuineIntel
CPU family: 6
Model: 85
Model name: Intel(R) Xeon(R) Platinum 8175M CPU @ 2.50GHz
Stepping: 4
CPU MHz: 3100.189
BogoMIPS: 4999.99
Hypervisor vendor: KVM
Virtualization type: full
L1d cache: 32K
L1i cache: 32K
L2 cache: 1024K
L3 cache: 33792K
NUMA node0 CPU(s): 0-47
Flags: fpu vme de pse tsc msr pae mce cx8 apic sep mtrr pge
mca cmov pat pse36 clflush mmx fxsr sse sse2 ss ht syscall nx pdpe1gb rdtscp lm
constant_tsc arch_perfmon rep_good nopl xtopology nonstop_tsc cpuid aperfmperf
tsc_known_freq pni pclmulqdq monitor ssse3 fma cx16 pcid sse4_1 sse4_2 x2apic
movbe popcnt tsc_deadline_timer aes xsave avx f16c rdrand hypervisor lahf_lm
abm 3dnowprefetch invpcid_single pti fsgsbase tsc_adjust bmi1 avx2 smep bmi2
erms invpcid mpx avx512f avx512dq rdseed adx smap clflushopt clwb avx512cd
avx512bw avx512vl xsaveopt xsavec xgetbv1 xsaves ida arat pku ospke
```
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: [email protected]
For queries about this service, please contact Infrastructure at:
[email protected]
---------------------------------------------------------------------
To unsubscribe, e-mail: [email protected]
For additional commands, e-mail: [email protected]
