Re: Polaris benchmarks proposal

2025-03-22 Thread Yufei Gu
I’m leaning toward placing it in a separate repository rather than in https://github.com/apache/polaris. The benchmark tool is largely self-contained and doesn’t have a strong dependency on the main codebase. IIUC, the only requirement is a running Polaris instance, which the tool can connect to u

Re: Polaris benchmarks proposal

2025-03-22 Thread Pierre Laporte
I don't mind contributing the benchmarks to `polaris-tools`. It seems that the consensus is clearly in that direction. I want to address some comments that were made in the PR but that are not really related to code review per se. > You can write gatling benchmarks in a language other than Scala

Re: Polaris benchmarks proposal

2025-03-22 Thread Russell Spitzer
I think we should start a new thread just to gauge consensus on whether Scala will be allowed in the tools repository or not. To go through my quick thoughts here. I like Scala but I have to be realistic in saying that it is a rather esoteric language choice and limits the number of community memb

Re: Polaris benchmarks proposal

2025-03-22 Thread Russell Spitzer
I think it makes sense for us to also build some capabilities into the tools repo to build Polaris at a specific commit for testing purposes. If the Spark Catalog and Benchmarking code goes there they could both share this code for testing, ditto for the migration code. On Fri, Mar 21, 2025 at 4:5