Do I understand correctly that now we can merge https://github.com/apache/spark/pull/49984 and unblock Spark 4.0?
To make sure that everyone is on the same page as not all the people have read all the discussion threads: 1. This PR *DOES* *NOT* add back the misnamed configuration. People cannot set that configuration anymore with Spark 4.0. 2. This PR adds a migration path in case the misnamed configuration is persistent in the streaming checkpoint. This is necessary otherwise the streaming query needs to re-process all the inputs, or users need to modify the streaming checkpoint files manually. On Tue, Mar 18, 2025 at 1:45 PM Mark Hamstra <markhams...@gmail.com> wrote: > As you noted previously, this does allow the original vote to proceed > without a valid veto, but I will also note that this does not preclude > modifying the migration logic later to avoid explicitly including > “databricks” in the code if people think that is important and an > agreeable, technically sound alternative is proposed. > > > On Mon, Mar 17, 2025 at 3:37 PM Hyukjin Kwon <gurwls...@apache.org> wrote: > >> The vote passes with 5 +1s (4 binding +1s) and 3 -1s (3 binding -1s). >> >> (* = binding) >> +1: >> - Mark Hamstra * >> - Jungtaek Lim >> - Wenchen Fan * >> - Reynold Xin * >> - Yuanjian Li * >> >> -1: >> - Holden Karau * >> - Hyukjin Kwon * >> - Dongjoon Hyun * >> >> Thanks. >> >>
