Kontinuation commented on PR #24: URL: https://github.com/apache/sedona-db/pull/24#issuecomment-3375175278
I agree that we should benchmark individual ST functions using one thread (scenario 2), as it is a must for making the benchmark results comparable across multiple engines. We focus on the quality of function implementation rather than the overall performance of the system in this scenario, which helps us catch performance bugs. By the time I authored this PR, all the benchmarks we've written were for benchmarking individual ST functions, that is why I decided to force using single thread. I also agree that we need to use the default settings of the systems to benchmark for perceived performance. This setting should be used for benchmarking more complex scenarios such as joins, spatial bench, etc. The conclusion is that we'd better use different settings for different types of benchmarks. -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: [email protected] For queries about this service, please contact Infrastructure at: [email protected]
