gortiz opened a new pull request, #16755: URL: https://github.com/apache/pinot/pull/16755
This PR improves https://github.com/apache/pinot/issues/8837 The main idea is to break the _ascending only_ invariant on docid iterators and support both ascending and descending. Due to how the code is right now, in order to do that we need to modify several operators. Then in SelectionPlanNode, if the order is descending, we call `ProjectorOperator.withOrder(false)`, which either returns: - A new ProjectorOperator whose order is descending, in which case we use SelectionPartiallyOrderedByLinearOperator, which is similar to the old SelectionPartiallyOrderedByAscOperator. - Or fails with UnsupportedOperationException, in which case we call the same old SelectionPartiallyOrderedByDescOperator we used before, which has a higher cost. A prelimilary benchmark returns (on my M1) the following data, which is significantly better than the current state. I still need to verify that everything works as expected, so I'm opening this as a draft ``` Benchmark (_asc) (_limit) (_numRows) (_primaryRepetitions) (_scenario) Mode Cnt Score Error Units BenchmarkOrderByQueries.sortedPartially true 1500 1500000 1 EXP(0.5) avgt 1.521 ms/op BenchmarkOrderByQueries.sortedPartially:jfr true 1500 1500000 1 EXP(0.5) avgt NaN --- BenchmarkOrderByQueries.sortedPartially true 1500 1500000 1000 EXP(0.5) avgt 3.853 ms/op BenchmarkOrderByQueries.sortedPartially:jfr true 1500 1500000 1000 EXP(0.5) avgt NaN --- BenchmarkOrderByQueries.sortedPartially true 150000 1500000 1 EXP(0.5) avgt 35.481 ms/op BenchmarkOrderByQueries.sortedPartially:jfr true 150000 1500000 1 EXP(0.5) avgt NaN --- BenchmarkOrderByQueries.sortedPartially true 150000 1500000 1000 EXP(0.5) avgt 59.138 ms/op BenchmarkOrderByQueries.sortedPartially:jfr true 150000 1500000 1000 EXP(0.5) avgt NaN --- BenchmarkOrderByQueries.sortedPartially false 1500 1500000 1 EXP(0.5) avgt 1.900 ms/op BenchmarkOrderByQueries.sortedPartially:jfr false 1500 1500000 1 EXP(0.5) avgt NaN --- BenchmarkOrderByQueries.sortedPartially false 1500 1500000 1000 EXP(0.5) avgt 2.388 ms/op BenchmarkOrderByQueries.sortedPartially:jfr false 1500 1500000 1000 EXP(0.5) avgt NaN --- BenchmarkOrderByQueries.sortedPartially false 150000 1500000 1 EXP(0.5) avgt 37.283 ms/op BenchmarkOrderByQueries.sortedPartially:jfr false 150000 1500000 1 EXP(0.5) avgt NaN --- BenchmarkOrderByQueries.sortedPartially false 150000 1500000 1000 EXP(0.5) avgt 53.551 ms/op BenchmarkOrderByQueries.sortedPartially:jfr false 150000 1500000 1000 EXP(0.5) avgt NaN --- BenchmarkOrderByQueries.sortedTotally true 1500 1500000 1 EXP(0.5) avgt 0.694 ms/op BenchmarkOrderByQueries.sortedTotally:jfr true 1500 1500000 1 EXP(0.5) avgt NaN --- BenchmarkOrderByQueries.sortedTotally true 1500 1500000 1000 EXP(0.5) avgt 2.983 ms/op BenchmarkOrderByQueries.sortedTotally:jfr true 1500 1500000 1000 EXP(0.5) avgt NaN --- BenchmarkOrderByQueries.sortedTotally true 150000 1500000 1 EXP(0.5) avgt 14.721 ms/op BenchmarkOrderByQueries.sortedTotally:jfr true 150000 1500000 1 EXP(0.5) avgt NaN --- BenchmarkOrderByQueries.sortedTotally true 150000 1500000 1000 EXP(0.5) avgt 15.271 ms/op BenchmarkOrderByQueries.sortedTotally:jfr true 150000 1500000 1000 EXP(0.5) avgt NaN --- BenchmarkOrderByQueries.sortedTotally false 1500 1500000 1 EXP(0.5) avgt 0.629 ms/op BenchmarkOrderByQueries.sortedTotally:jfr false 1500 1500000 1 EXP(0.5) avgt NaN --- BenchmarkOrderByQueries.sortedTotally false 1500 1500000 1000 EXP(0.5) avgt 0.462 ms/op BenchmarkOrderByQueries.sortedTotally:jfr false 1500 1500000 1000 EXP(0.5) avgt NaN --- BenchmarkOrderByQueries.sortedTotally false 150000 1500000 1 EXP(0.5) avgt 28.011 ms/op BenchmarkOrderByQueries.sortedTotally:jfr false 150000 1500000 1 EXP(0.5) avgt NaN --- BenchmarkOrderByQueries.sortedTotally false 150000 1500000 1000 EXP(0.5) avgt 21.024 ms/op BenchmarkOrderByQueries.sortedTotally:jfr false 150000 1500000 1000 EXP(0.5) avgt NaN --- ``` -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: [email protected] For queries about this service, please contact Infrastructure at: [email protected] --------------------------------------------------------------------- To unsubscribe, e-mail: [email protected] For additional commands, e-mail: [email protected]
