I’m guessing this is mostly LLM generated code? The question I think we need to answer as a community is if we are okay merging a large chunk of LLM generated code assuming we can put in place the right guardrails, get it to pass all tests, all upstream tests in datafusion, benchmarks look better and the high level design is accepted.
> On Nov 9, 2026, at 5:46 AM, Andy Grove <[email protected]> wrote: > > Hi all, > > Over the weekend, I started from a clean slate and implemented a new > version of the sqlparser-rs crate using the design that I had > originally wanted but did not know how to build (I was still quite new > to Rust at the time). > > The new version has one trait per dialect and all dialect-specific > parsing lives in that trait. There is no longer one huge "core" parser > containing conditional logic for different dialects. Related dialects > can also share logic. For example, the Hive and Databricks dialects > both call into the Spark dialect for common pieces. > > The new version also has ~10x fewer memory allocations (no more > copying Strings everywhere) and is ~10x faster. > > I am nearly finished porting all of the existing dialects over. The > actual parsing code is taken from the current implementation, so that > is not re-written. I have ported over all the tests as well, and am > working towards 100% parity with v1, although v2 contains quite a lot > of correctness fixes compared to v1. > > What is the appetite in the community for me pushing this to a new 2.0 > branch in the Apache repo? > > Another option would be for me to file a huge issue explaining the > design in detail, and let the community rebuild it. > > WDYT? > > I filed an issue for this discssion as well, so that is probably the > best place to continue the discussion. > > https://github.com/apache/datafusion-sqlparser-rs/issues/2893 > > Thanks, > > Andy. > > --------------------------------------------------------------------- > To unsubscribe, e-mail: [email protected] > For additional commands, e-mail: [email protected] > --------------------------------------------------------------------- To unsubscribe, e-mail: [email protected] For additional commands, e-mail: [email protected]
