uschindler commented on PR #12873:
URL: https://github.com/apache/lucene/pull/12873#issuecomment-1837669165
Hi, I prepared a JMH benchmark and compared results. I made the "enforce MH
mode" configurable when compiling the JS expression:
```java
package org.apache.lucene.benchmark.jmh;
import java.io.IOException;
import java.lang.invoke.MethodHandle;
import java.lang.invoke.MethodHandles;
import java.lang.invoke.MethodHandles.Lookup;
import java.lang.invoke.MethodType;
import java.text.ParseException;
import java.util.HashMap;
import java.util.Map;
import java.util.concurrent.ThreadLocalRandom;
import java.util.concurrent.TimeUnit;
import org.apache.lucene.expressions.Expression;
import org.apache.lucene.expressions.js.JavascriptCompiler;
import org.apache.lucene.search.DoubleValues;
import org.openjdk.jmh.annotations.Benchmark;
import org.openjdk.jmh.annotations.BenchmarkMode;
import org.openjdk.jmh.annotations.Fork;
import org.openjdk.jmh.annotations.Level;
import org.openjdk.jmh.annotations.Measurement;
import org.openjdk.jmh.annotations.Mode;
import org.openjdk.jmh.annotations.OutputTimeUnit;
import org.openjdk.jmh.annotations.Param;
import org.openjdk.jmh.annotations.Scope;
import org.openjdk.jmh.annotations.Setup;
import org.openjdk.jmh.annotations.State;
import org.openjdk.jmh.annotations.Warmup;
@BenchmarkMode(Mode.Throughput)
@OutputTimeUnit(TimeUnit.MILLISECONDS)
@State(Scope.Benchmark)
@Warmup(iterations = 3, time = 5)
@Measurement(iterations = 5, time = 8)
@Fork(value = 1)
public class ExpressionsBenchmark {
private static final Map<String, MethodHandle> FUNCTIONS = getFunctions();
private static Map<String, MethodHandle> getFunctions() {
try {
var lookup = MethodHandles.lookup();
Map<String, MethodHandle> m = new
HashMap<>(JavascriptCompiler.DEFAULT_FUNCTIONS);
m.put("func_identity", lookup.findStatic(lookup.lookupClass(),
"ident", MethodType.methodType(double.class, double.class)));
m.put("mh_identity", MethodHandles.identity(double.class));
return m;
} catch (ReflectiveOperationException e) {
throw new AssertionError(e);
}
}
static double ident(double v) {
return v;
}
private double[] randomData;
private Expression expression;
@Param({"x", "func_identity(x)", "mh_identity", "func_identity(x) + x",
"cos(x)", "cos(x) + sin(x)"})
String js;
@Param({"true", "false"})
boolean useMH;
@Setup(Level.Iteration)
public void init() throws ParseException {
ThreadLocalRandom random = ThreadLocalRandom.current();
randomData = random.doubles().limit(1024).toArray();
expression = JavascriptCompiler.compile(js, FUNCTIONS, false, useMH);
}
@Benchmark
public double expression() throws IOException {
var it = new ValuesIterator(randomData);
var values = it.newDoubleValues();
double result = 0d;
while (it.next()) {
result += expression.evaluate(values);
}
return result;
}
static final class ValuesIterator {
final double[] data;
final DoubleValues[] dv;
int pos = -1;
ValuesIterator(double[] data) {
this.data = data;
var dv = new DoubleValues() {
@Override
public double doubleValue() throws IOException {
return data[pos];
}
@Override
public boolean advanceExact(int doc) throws IOException {
throw new UnsupportedOperationException();
}
};
this.dv = new DoubleValues[] { dv };
}
boolean next() {
pos++;
return (pos < data.length);
}
DoubleValues[] newDoubleValues() {
return dv;
}
}
}
```
Here are the results of the benchmark (my Intel box):
```
Benchmark (js) (useMH) Mode Cnt
Score Error Units
ExpressionsBenchmark.expression x true thrpt 5
204,253 ± 8,302 ops/ms
ExpressionsBenchmark.expression x false thrpt 5
203,082 ± 4,484 ops/ms
ExpressionsBenchmark.expression func_identity(x) true thrpt 5
201,631 ± 10,111 ops/ms
ExpressionsBenchmark.expression func_identity(x) false thrpt 5
202,196 ± 9,016 ops/ms
ExpressionsBenchmark.expression mh_identity true thrpt 5
200,443 ± 7,391 ops/ms
ExpressionsBenchmark.expression mh_identity false thrpt 5
200,415 ± 10,642 ops/ms
ExpressionsBenchmark.expression func_identity(x) + x true thrpt 5
198,585 ± 9,959 ops/ms
ExpressionsBenchmark.expression func_identity(x) + x false thrpt 5
195,611 ± 13,522 ops/ms
ExpressionsBenchmark.expression cos(x) true thrpt 5
59,574 ± 2,952 ops/ms
ExpressionsBenchmark.expression cos(x) false thrpt 5
60,598 ± 1,993 ops/ms
ExpressionsBenchmark.expression cos(x) + sin(x) true thrpt 5
35,756 ± 0,905 ops/ms
ExpressionsBenchmark.expression cos(x) + sin(x) false thrpt 5
35,890 ± 0,452 ops/ms
```
Some hints:
- The JS function `x` has no method calls, so the `useMH=true/false` should
be identical. There are some differences.
- `func_identity` is a function returning itsself, which is part of
benchmark class and referred to by a MethodHandle. As this is a direct
MethodHandle, the compiler translates this with `useMH=false` (current PR) to
an `invokestatic`. When MHs are used it uses the new dynamic constants and
executes `invokeExact` on the MH from the function table.
- `mh_identity` is just the `MethodHandles.identity(double.class)` handle.
As this is no direct MethodHandle, the results of `useMH=true/false` should be
identical.
- the rest are more expensive method calls to numeric functions like cosine,
sine.
When looking at those results, my observations:
- there's no difference between useMH enabled or not. So the dynamic
constant glued into the bytecode makes Hotspot completely remove the
MethodHandle and it is identical to `invokestatic`
- you also see some small differences for runs where the produced bytecode
is identical (as there's no direct MethodHandle that can be converted to
`invokestatic`. This shows that the error is larger. I may run another
benchmark with longer runtimes tomorrow (to lower error bars).
I will remove the old `invokestatic` code and cracking of MethodHandles.
With the new dynamic constants we can directly invoke the handles and it does
not slow down. Code will get much smaller and compilation might get faster, too
(as we no longer need to create those two function maps with cracked handles).
Uwe
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: [email protected]
For queries about this service, please contact Infrastructure at:
[email protected]
---------------------------------------------------------------------
To unsubscribe, e-mail: [email protected]
For additional commands, e-mail: [email protected]
