I think 3.2.0 is seen as a minor release, not a maintenance release. Eg. 3.2.1 would be a maintenance release which typically includes bug fixes, minor usability improvements and security fixes. If we block new features from going into minor releases that would be a step back in my opinion. I am pretty sure we have released small features in minor releases before (e.g 2.2.0 release notes <https://issues.apache.org/jira/secure/ReleaseNote.jspa?version=12335837&styleName=Text&projectId=12310843>). I would request not to club new features and breaking features together in the same decision.
For breaking changes, I don't think we have published guidelines before so I am open to that. Especially for the HoS tests which are failing, some of the failures are related to dependency upgrades (done for security reasons) while some have no clear root-cause yet. For a feature which has been removed in master branch, I think it is okay to deprecate it and disable the tests given that we have spent considerable time on them already and none of the original HoS contributors are available to help. Thanks, Vihang On Mon, Feb 27, 2023 at 1:24 PM Stamatis Zampetakis <zabe...@gmail.com> wrote: > Some people raised a valid point that branch-3 is a maintenance branch. If > we really aim 3.2.0 to be a maintenance release then we should minimize > breaking changes and prohibit new features. In this case Spark cannot go > away and the only thing we can do is deprecate it. It also means that we > should fix the tests cause failures typically indicate breaking changes > which again are not tolerable for a maintenance release. > > On the other hand, > I got the impression that some people were interested for getting new > features in 3.2.0 ( some may be in already). Furthermore, some dependency > upgrades may also lead to breaking changes/different behavior so we should > definitely agree on what is acceptable and what is not for branch-3. > > Summing up the question boils down to the following. Do we allow breaking > changes and new features in branch-3 or not? > > Best, > Stamatis > > On Fri, Feb 24, 2023, 10:41 AM Aman Raj <raja...@microsoft.com.invalid> > wrote: > > > Hi Laszlo, > > > > I am perfectly fine with disabling the Hive on Spark tests. In fact, I > > prefer that. I agree with Vihang and you on this. I had proposed this > idea > > long back (of disabling the test cases) and then we had discussed on the > > community that either we fix the Hive on Spark test cases or remove Hive > on > > Spark. Therefore, I initiated this thread of removing Hive on Spark since > > we are not still able to resolve the test cases since the past couple of > > months. > > > > Thanks, > > Aman. > > > > ________________________________ > > From: László Bodor <bodorlaszlo0...@gmail.com> > > Sent: Friday, February 24, 2023 2:57 PM > > To: dev@hive.apache.org <dev@hive.apache.org> > > Subject: Re: [EXTERNAL] Re: Proposal to deprecate Hive on Spark from > > branch-3 > > > > +1 on Vihang's suggestion > > I remember that spark removal was a debated thing even on master, so > > completely removing it backwards from a "maintenance" branch-3 line is > not > > really acceptable (actually, I'm surprised it's not -1ed yet by hive on > > spark folks), but it depends on what *deprecation* really means: I mean > > disabling some spark tests to stabilize precommit is completely fine in > the > > absence of community aspiration to fix them properly > > > > regarding the motivation: "This would ensure that branch-3 is aligned > with > > the master as done in ..." <-- I don't think we're targeting this, we > are > > about to make 3.x releases as simply as possible > > > > I'm hoping/assuming that most of the +1s so far are in line with Vihang's > > suggestion > > > > vihang karajgaonkar <vihan...@apache.org> ezt írta (időpont: 2023. febr. > > 23., Cs, 16:37): > > > > > +1 to deprecate Hive on Spark. > > > > > > I feel directly removing it in a minor release is probably a bad idea. > > Most > > > users will upgrade to 3.2 first and go to 4.0 later. If we deprecate it > > in > > > 3.2 it transitions well into its removal as users upgrade to 4.0 > > > eventually. > > > > > > If the goal to stabilize the branch-3, we can disable the failing tests > > on > > > Hive on Spark. > > > > > > Thanks, > > > Vihang > > > > > > On Thu, Feb 23, 2023 at 12:32 AM Alessandro Solimando < > > > alessandro.solima...@gmail.com> wrote: > > > > > > > +1 from me too > > > > > > > > On Thu, 23 Feb 2023 at 06:09, Ayush Saxena <ayush...@gmail.com> > wrote: > > > > > > > > > +1 on removing Hive on Spark from branch-3 > > > > > > > > > > -Ayush > > > > > > > > > > > On 23-Feb-2023, at 6:40 AM, Wang, Yuming > <yumw...@ebay.com.invalid > > > > > > > > wrote: > > > > > > > > > > > > +1. > > > > > > > > > > > > From: Naresh P R <nares...@apache.org> > > > > > > Date: Thursday, February 23, 2023 at 02:49 > > > > > > To: dev@hive.apache.org <dev@hive.apache.org> > > > > > > Subject: Re: [EXTERNAL] Re: Proposal to deprecate Hive on Spark > > from > > > > > branch-3 > > > > > > External Email > > > > > > > > > > > > +1 to remove Hive on Spark in branch-3 > > > > > > --- > > > > > > Regards, > > > > > > Naresh P R > > > > > > > > > > > >> On Wed, Feb 22, 2023 at 5:37 AM Sankar Hariappan > > > > > >> <sankar.hariap...@microsoft.com.invalid> wrote: > > > > > >> > > > > > >> +1, to remove Hive on Spark in branch-3. > > > > > >> > > > > > >> Thanks, > > > > > >> Sankar > > > > > >> > > > > > >> -----Original Message----- > > > > > >> From: Rajesh Balamohan <rbalamo...@apache.org> > > > > > >> Sent: Wednesday, February 22, 2023 6:58 PM > > > > > >> To: dev@hive.apache.org > > > > > >> Subject: [EXTERNAL] Re: Proposal to deprecate Hive on Spark from > > > > > branch-3 > > > > > >> > > > > > >> +1 on removing Hive on Spark in branch-3. > > > > > >> > > > > > >> It was not done earlier since it was removing a feature in the > > > branch. > > > > > But > > > > > >> if there is enough consensus, we should consider removing it. > > > > > >> > > > > > >> ~Rajesh.B > > > > > >> > > > > > >> On Wed, Feb 22, 2023 at 12:48 PM Aman Raj > > > > <raja...@microsoft.com.invalid > > > > > > > > > > > >> wrote: > > > > > >> > > > > > >>> Hi team, > > > > > >>> > > > > > >>> We have been trying to fix Hive on Spark test failures for a > long > > > > > >>> time. As of now, branch-3 has less than 12 test failures (whose > > fix > > > > > >>> have not been identified). 8 of them are related to Hive on > > Spark. > > > I > > > > > >>> had mailed about the failures in my previous mail threads. > Thanks > > > to > > > > > >>> Vihang for working on them as well. But we have not been able > to > > > > > >> identify the root cause till now. > > > > > >>> These fixes can be tracked in the following tickets : > > [HIVE-27087] > > > > Fix > > > > > >>> TestMiniSparkOnYarnCliDriver test failures on branch-3 - ASF > > JIRA ( > > > > > >>> apache.org)< > > > > > > > > > > > > > > > https://nam10.safelinks.protection.outlook.com/?url=https%3A%2F%2Fnam06.safelinks.protection.outlook.com%2F%3Furl%3Dhttps%2525&data=05%7C01%7Cyumwang%40ebay.com%7C2bd54cc0c84a4e44a59e08db150574e5%7C46326bff992841a0baca17c16c94ea99%7C0%7C0%7C638126885411646147%7CUnknown%7CTWFpbGZsb3d8eyJWIjoiMC4wLjAwMDAiLCJQIjoiV2luMzIiLCJBTiI6Ik1haWwiLCJXVCI6Mn0%3D%7C3000%7C%7C%7C&sdata=KORvfBkmdpqIFOoWr6J4X%2BqAQO6jcykzjY3%2FU0pq0y4%3D&reserved=0 > > > > > >>> 3A%2F%2Fissues.apache.org > > > > %2Fjira%2Fbrowse%2FHIVE-27087&data=05%7C01%7C > > > > > >>> Sankar.Hariappan%40microsoft.com > > > > %7C687a6a4dbd41454568e008db14d8cc23%7C > > > > > >>> > > > > > 72f988bf86f141af91ab2d7cd011db47%7C1%7C0%7C638126693641861742%7CUnknow > > > > > >>> > > > > > n%7CTWFpbGZsb3d8eyJWIjoiMC4wLjAwMDAiLCJQIjoiV2luMzIiLCJBTiI6Ik1haWwiLC > > > > > >>> > > > > > JXVCI6Mn0%3D%7C3000%7C%7C%7C&sdata=RbAqrwK6fQFDStufXYfpusNc81EzjtpiaHm > > > > > >>> qv5CFiAs%3D&reserved=0> and [HIVE-26940] Backport of > HIVE-19882 : > > > Fix > > > > > >>> QTestUtil session lifecycle - ASF JIRA > > > > > >>> (apache.org)< > > > > > > > > > > > > > > > https://nam06.safelinks.protection.outlook.com/?url=http%3A%2F%2Fhttps%2F&data=05%7C01%7Crajaman%40microsoft.com%7C7342dd78ed7f424b36a608db16495672%7C72f988bf86f141af91ab2d7cd011db47%7C1%7C0%7C638128276496747003%7CUnknown%7CTWFpbGZsb3d8eyJWIjoiMC4wLjAwMDAiLCJQIjoiV2luMzIiLCJBTiI6Ik1haWwiLCJXVCI6Mn0%3D%7C3000%7C%7C%7C&sdata=rokHWpxWQ0xYFDr%2BC2p%2BFv4gjLtv%2FCXeZGxdcCSFcVM%3D&reserved=0 > > > > > >>> %3A%2F%2Fissues.apache.org > > > > %2Fjira%2Fbrowse%2FHIVE-26940&data=05%7C01%7 > > > > > >>> CSankar.Hariappan%40microsoft.com > > > > %7C687a6a4dbd41454568e008db14d8cc23%7 > > > > > >>> > > > > > C72f988bf86f141af91ab2d7cd011db47%7C1%7C0%7C638126693641861742%7CUnkno > > > > > >>> > > > > > wn%7CTWFpbGZsb3d8eyJWIjoiMC4wLjAwMDAiLCJQIjoiV2luMzIiLCJBTiI6Ik1haWwiL > > > > > >>> > > > > > CJXVCI6Mn0%3D%7C3000%7C%7C%7C&sdata=PaDtEZD569Sv0ER9sC4l6q1ZxyoBeER3zn > > > > > >>> Bsc51PWI8%3D&reserved=0> > > > > > >>> > > > > > >>> Until we have a green branch-3, we cannot go ahead to push new > > > > > >>> features for the Hive-3.2.0 release. This is kind of a blocker > > for > > > > this > > > > > >> release. > > > > > >>> Already bringing the test fixes to the current state took more > > > than 2 > > > > > >>> months. > > > > > >>> > > > > > >>> I wanted to bring up a proposal to deprecate Hive on Spark from > > > > > >>> branch-3 altogether. This would ensure that branch-3 is aligned > > > with > > > > > >>> the master as done in > > > > > >>> > > > > > > > > > > > > > > > https://nam06.safelinks.protection.outlook.com/?url=https%3A%2F%2Fissu%2F&data=05%7C01%7Crajaman%40microsoft.com%7C7342dd78ed7f424b36a608db16495672%7C72f988bf86f141af91ab2d7cd011db47%7C1%7C0%7C638128276496747003%7CUnknown%7CTWFpbGZsb3d8eyJWIjoiMC4wLjAwMDAiLCJQIjoiV2luMzIiLCJBTiI6Ik1haWwiLCJXVCI6Mn0%3D%7C3000%7C%7C%7C&sdata=Ajkc6Ev5ts7dQobFo1gwBDRvTt9ggRFTasOz%2B5k3Ocg%3D&reserved=0 > > > > > >>> es.apache.org > > > > > >> %2Fjira%2Fbrowse%2FHIVE-26134&data=05%7C01%7CSankar.Hariappan% > > > > > >> 40microsoft.com > > > > > > > > > > > > > > > %7C687a6a4dbd41454568e008db14d8cc23%7C72f988bf86f141af91ab2d7cd011db47%7C1%7C0%7C638126693641861742%7CUnknown%7CTWFpbGZsb3d8eyJWIjoiMC4wLjAwMDAiLCJQIjoiV2luMzIiLCJBTiI6Ik1haWwiLCJXVCI6Mn0%3D%7C3000%7C%7C%7C&sdata=j%2F%2BYTakSvfk1Mm%2Fq8kI84gKm6s%2Fg2iA1abIPzY7t%2Bsg%3D&reserved=0. > > > > > >> Just wanted to have a vote on this in parallel working on the > test > > > > > fixes. > > > > > >> If we have the approval from the community, we can deprecate it > > > > > altogether. > > > > > >>> > > > > > >>> Please feel free to suggest any concerns or suggestions you > have. > > > > > >>> Also, I welcome any possible fix suggestion for the test > > failures. > > > > > >>> > > > > > >>> Thanks, > > > > > >>> Aman. > > > > > >>> > > > > > >> > > > > > > > > > > > > > > >
