Apologize for quick fix, This won't go to VOTE "unless" there are "valid" objections. The message was sent accidentally before I double checked.
On Sun, Mar 16, 2025 at 5:00 PM Jungtaek Lim <kabhwan.opensou...@gmail.com> wrote: > Just to clarify: This won't go to VOTE as long as there are "valid" > objections. Let's not waste more time on our end. > > On Sun, Mar 16, 2025 at 4:52 PM Jungtaek Lim <kabhwan.opensou...@gmail.com> > wrote: > >> Hi dev, >> >> I'm really tired of the discussion which does not move forward because >> the argument is not backed by strict ASF policy. We debate based on >> the interpretation of ASF policy by individuals, which I think makes zero >> sense. >> >> I really thought about this a lot how to resolve this, and now I'm open >> to have a hack on the migration logic to eliminate the main concern on the >> debate, which I can think like following: >> >> 1) Checks the config string via pattern, like the string contains >> ".optimizer.pruneFiltersCanPruneStreamingSubplan" at the end and the string >> is longer than "spark.sql.optimizer.pruneFiltersCanPruneStreamingSubplan" >> (or strictly 12 chars longer). >> >> 2) Encode the incorrect config name with any hashing algorithm, and use >> this to compare with. We do not document about the origin string, but >> probably leave the offending ticket to at least figure out when we forgot >> about what was the origin string by ourselves when we ever debug this. >> >> 3) etcetc (I'm open for better ideas, except just removing the migration >> logic.) >> >> In overall, indirect comparison. >> >> This definitely complicates the logic, but the logic is really just 4 >> lines to begin with, and maybe it will be just 10 lines or so, so it's >> still something manageable. >> >> It might be slightly tricky on testing, but this is a lot easier than >> debating in there. I'm now regretting not allowing myself to introduce a >> hack in earlier days - we should have saved 3 weeks. >> >> If we are OK to allow a bit of indirect checking on the logic to just >> remove out the debate, I'm happy to do that. It's obvious that we can just >> leave this migration logic much longer than what I was proposing, because >> we eliminate the main concern. >> >> I'm open to hear about support and objections. >> >> Thanks, >> Jungtaek Lim (HeartSaVioR) >> >
