Hi Laszlo, I am perfectly fine with disabling the Hive on Spark tests. In fact, I prefer that. I agree with Vihang and you on this. I had proposed this idea long back (of disabling the test cases) and then we had discussed on the community that either we fix the Hive on Spark test cases or remove Hive on Spark. Therefore, I initiated this thread of removing Hive on Spark since we are not still able to resolve the test cases since the past couple of months.
Thanks, Aman. ________________________________ From: László Bodor <bodorlaszlo0...@gmail.com> Sent: Friday, February 24, 2023 2:57 PM To: dev@hive.apache.org <dev@hive.apache.org> Subject: Re: [EXTERNAL] Re: Proposal to deprecate Hive on Spark from branch-3 +1 on Vihang's suggestion I remember that spark removal was a debated thing even on master, so completely removing it backwards from a "maintenance" branch-3 line is not really acceptable (actually, I'm surprised it's not -1ed yet by hive on spark folks), but it depends on what *deprecation* really means: I mean disabling some spark tests to stabilize precommit is completely fine in the absence of community aspiration to fix them properly regarding the motivation: "This would ensure that branch-3 is aligned with the master as done in ..." <-- I don't think we're targeting this, we are about to make 3.x releases as simply as possible I'm hoping/assuming that most of the +1s so far are in line with Vihang's suggestion vihang karajgaonkar <vihan...@apache.org> ezt írta (időpont: 2023. febr. 23., Cs, 16:37): > +1 to deprecate Hive on Spark. > > I feel directly removing it in a minor release is probably a bad idea. Most > users will upgrade to 3.2 first and go to 4.0 later. If we deprecate it in > 3.2 it transitions well into its removal as users upgrade to 4.0 > eventually. > > If the goal to stabilize the branch-3, we can disable the failing tests on > Hive on Spark. > > Thanks, > Vihang > > On Thu, Feb 23, 2023 at 12:32 AM Alessandro Solimando < > alessandro.solima...@gmail.com> wrote: > > > +1 from me too > > > > On Thu, 23 Feb 2023 at 06:09, Ayush Saxena <ayush...@gmail.com> wrote: > > > > > +1 on removing Hive on Spark from branch-3 > > > > > > -Ayush > > > > > > > On 23-Feb-2023, at 6:40 AM, Wang, Yuming <yumw...@ebay.com.invalid> > > > wrote: > > > > > > > > +1. > > > > > > > > From: Naresh P R <nares...@apache.org> > > > > Date: Thursday, February 23, 2023 at 02:49 > > > > To: dev@hive.apache.org <dev@hive.apache.org> > > > > Subject: Re: [EXTERNAL] Re: Proposal to deprecate Hive on Spark from > > > branch-3 > > > > External Email > > > > > > > > +1 to remove Hive on Spark in branch-3 > > > > --- > > > > Regards, > > > > Naresh P R > > > > > > > >> On Wed, Feb 22, 2023 at 5:37 AM Sankar Hariappan > > > >> <sankar.hariap...@microsoft.com.invalid> wrote: > > > >> > > > >> +1, to remove Hive on Spark in branch-3. > > > >> > > > >> Thanks, > > > >> Sankar > > > >> > > > >> -----Original Message----- > > > >> From: Rajesh Balamohan <rbalamo...@apache.org> > > > >> Sent: Wednesday, February 22, 2023 6:58 PM > > > >> To: dev@hive.apache.org > > > >> Subject: [EXTERNAL] Re: Proposal to deprecate Hive on Spark from > > > branch-3 > > > >> > > > >> +1 on removing Hive on Spark in branch-3. > > > >> > > > >> It was not done earlier since it was removing a feature in the > branch. > > > But > > > >> if there is enough consensus, we should consider removing it. > > > >> > > > >> ~Rajesh.B > > > >> > > > >> On Wed, Feb 22, 2023 at 12:48 PM Aman Raj > > <raja...@microsoft.com.invalid > > > > > > > >> wrote: > > > >> > > > >>> Hi team, > > > >>> > > > >>> We have been trying to fix Hive on Spark test failures for a long > > > >>> time. As of now, branch-3 has less than 12 test failures (whose fix > > > >>> have not been identified). 8 of them are related to Hive on Spark. > I > > > >>> had mailed about the failures in my previous mail threads. Thanks > to > > > >>> Vihang for working on them as well. But we have not been able to > > > >> identify the root cause till now. > > > >>> These fixes can be tracked in the following tickets : [HIVE-27087] > > Fix > > > >>> TestMiniSparkOnYarnCliDriver test failures on branch-3 - ASF JIRA ( > > > >>> apache.org)< > > > > > > https://nam10.safelinks.protection.outlook.com/?url=https%3A%2F%2Fnam06.safelinks.protection.outlook.com%2F%3Furl%3Dhttps%2525&data=05%7C01%7Cyumwang%40ebay.com%7C2bd54cc0c84a4e44a59e08db150574e5%7C46326bff992841a0baca17c16c94ea99%7C0%7C0%7C638126885411646147%7CUnknown%7CTWFpbGZsb3d8eyJWIjoiMC4wLjAwMDAiLCJQIjoiV2luMzIiLCJBTiI6Ik1haWwiLCJXVCI6Mn0%3D%7C3000%7C%7C%7C&sdata=KORvfBkmdpqIFOoWr6J4X%2BqAQO6jcykzjY3%2FU0pq0y4%3D&reserved=0 > > > >>> 3A%2F%2Fissues.apache.org > > %2Fjira%2Fbrowse%2FHIVE-27087&data=05%7C01%7C > > > >>> Sankar.Hariappan%40microsoft.com > > %7C687a6a4dbd41454568e008db14d8cc23%7C > > > >>> > > 72f988bf86f141af91ab2d7cd011db47%7C1%7C0%7C638126693641861742%7CUnknow > > > >>> > > n%7CTWFpbGZsb3d8eyJWIjoiMC4wLjAwMDAiLCJQIjoiV2luMzIiLCJBTiI6Ik1haWwiLC > > > >>> > > JXVCI6Mn0%3D%7C3000%7C%7C%7C&sdata=RbAqrwK6fQFDStufXYfpusNc81EzjtpiaHm > > > >>> qv5CFiAs%3D&reserved=0> and [HIVE-26940] Backport of HIVE-19882 : > Fix > > > >>> QTestUtil session lifecycle - ASF JIRA > > > >>> (apache.org)< > > > > > > https://nam06.safelinks.protection.outlook.com/?url=http%3A%2F%2Fhttps%2F&data=05%7C01%7Crajaman%40microsoft.com%7C7342dd78ed7f424b36a608db16495672%7C72f988bf86f141af91ab2d7cd011db47%7C1%7C0%7C638128276496747003%7CUnknown%7CTWFpbGZsb3d8eyJWIjoiMC4wLjAwMDAiLCJQIjoiV2luMzIiLCJBTiI6Ik1haWwiLCJXVCI6Mn0%3D%7C3000%7C%7C%7C&sdata=rokHWpxWQ0xYFDr%2BC2p%2BFv4gjLtv%2FCXeZGxdcCSFcVM%3D&reserved=0 > > > >>> %3A%2F%2Fissues.apache.org > > %2Fjira%2Fbrowse%2FHIVE-26940&data=05%7C01%7 > > > >>> CSankar.Hariappan%40microsoft.com > > %7C687a6a4dbd41454568e008db14d8cc23%7 > > > >>> > > C72f988bf86f141af91ab2d7cd011db47%7C1%7C0%7C638126693641861742%7CUnkno > > > >>> > > wn%7CTWFpbGZsb3d8eyJWIjoiMC4wLjAwMDAiLCJQIjoiV2luMzIiLCJBTiI6Ik1haWwiL > > > >>> > > CJXVCI6Mn0%3D%7C3000%7C%7C%7C&sdata=PaDtEZD569Sv0ER9sC4l6q1ZxyoBeER3zn > > > >>> Bsc51PWI8%3D&reserved=0> > > > >>> > > > >>> Until we have a green branch-3, we cannot go ahead to push new > > > >>> features for the Hive-3.2.0 release. This is kind of a blocker for > > this > > > >> release. > > > >>> Already bringing the test fixes to the current state took more > than 2 > > > >>> months. > > > >>> > > > >>> I wanted to bring up a proposal to deprecate Hive on Spark from > > > >>> branch-3 altogether. This would ensure that branch-3 is aligned > with > > > >>> the master as done in > > > >>> > > > > > > https://nam06.safelinks.protection.outlook.com/?url=https%3A%2F%2Fissu%2F&data=05%7C01%7Crajaman%40microsoft.com%7C7342dd78ed7f424b36a608db16495672%7C72f988bf86f141af91ab2d7cd011db47%7C1%7C0%7C638128276496747003%7CUnknown%7CTWFpbGZsb3d8eyJWIjoiMC4wLjAwMDAiLCJQIjoiV2luMzIiLCJBTiI6Ik1haWwiLCJXVCI6Mn0%3D%7C3000%7C%7C%7C&sdata=Ajkc6Ev5ts7dQobFo1gwBDRvTt9ggRFTasOz%2B5k3Ocg%3D&reserved=0 > > > >>> es.apache.org > > > >> %2Fjira%2Fbrowse%2FHIVE-26134&data=05%7C01%7CSankar.Hariappan% > > > >> 40microsoft.com > > > > > > %7C687a6a4dbd41454568e008db14d8cc23%7C72f988bf86f141af91ab2d7cd011db47%7C1%7C0%7C638126693641861742%7CUnknown%7CTWFpbGZsb3d8eyJWIjoiMC4wLjAwMDAiLCJQIjoiV2luMzIiLCJBTiI6Ik1haWwiLCJXVCI6Mn0%3D%7C3000%7C%7C%7C&sdata=j%2F%2BYTakSvfk1Mm%2Fq8kI84gKm6s%2Fg2iA1abIPzY7t%2Bsg%3D&reserved=0. > > > >> Just wanted to have a vote on this in parallel working on the test > > > fixes. > > > >> If we have the approval from the community, we can deprecate it > > > altogether. > > > >>> > > > >>> Please feel free to suggest any concerns or suggestions you have. > > > >>> Also, I welcome any possible fix suggestion for the test failures. > > > >>> > > > >>> Thanks, > > > >>> Aman. > > > >>> > > > >> > > > > > >
