gortiz commented on issue #15343: URL: https://github.com/apache/pinot/issues/15343#issuecomment-3150206579
> We use a lot of Lucene for our Pinot use cases. Lucene recently gets a lot of performance boost by utilizing Java 21 vector API. So moving to Java 21 source code could help us to get the best out of Lucnene Luckily, we probably don't need to upgrade the _source_ Java version to get these improvements. I'm not aware of how Lucene implemented the change, but probably they did it using multi-release jars, in which case we should get the performance boost for free just by changing the Java runtime version. So using Java 21 at runtime should be good enough. > Can you elaborate how to do this? Plugin sounds a good option so that we do not need to fork. It is a trivial change. In the past, during https://github.com/apache/pinot/pull/10528, we had a PR where we changed the Pinot buffers we used to use Foreign Memory API (see https://github.com/gortiz/pinot/blob/dc227182efaebff29e12320f5ba833ad269f6a54/pinot-plugins/pinot-foreign-memory/pom.xml) instead of larray (the older and deprecated code) or unsafe (the one we use right now). We decided to close that PR because we didn't have time to upgrade the GitHub Actions pipelines and Maven to compile only a sub-module using Java 21 and the rest using Java 11. This is totally possible to do, but given our pipelines take so long to run, it is an exhausting process we didn't have time to invest on. -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: [email protected] For queries about this service, please contact Infrastructure at: [email protected] --------------------------------------------------------------------- To unsubscribe, e-mail: [email protected] For additional commands, e-mail: [email protected]
