+1 on Vihang's suggestion I remember that spark removal was a debated thing even on master, so completely removing it backwards from a "maintenance" branch-3 line is not really acceptable (actually, I'm surprised it's not -1ed yet by hive on spark folks), but it depends on what *deprecation* really means: I mean disabling some spark tests to stabilize precommit is completely fine in the absence of community aspiration to fix them properly
regarding the motivation: "This would ensure that branch-3 is aligned with the master as done in ..." <-- I don't think we're targeting this, we are about to make 3.x releases as simply as possible I'm hoping/assuming that most of the +1s so far are in line with Vihang's suggestion vihang karajgaonkar <vihan...@apache.org> ezt írta (időpont: 2023. febr. 23., Cs, 16:37): > +1 to deprecate Hive on Spark. > > I feel directly removing it in a minor release is probably a bad idea. Most > users will upgrade to 3.2 first and go to 4.0 later. If we deprecate it in > 3.2 it transitions well into its removal as users upgrade to 4.0 > eventually. > > If the goal to stabilize the branch-3, we can disable the failing tests on > Hive on Spark. > > Thanks, > Vihang > > On Thu, Feb 23, 2023 at 12:32 AM Alessandro Solimando < > alessandro.solima...@gmail.com> wrote: > > > +1 from me too > > > > On Thu, 23 Feb 2023 at 06:09, Ayush Saxena <ayush...@gmail.com> wrote: > > > > > +1 on removing Hive on Spark from branch-3 > > > > > > -Ayush > > > > > > > On 23-Feb-2023, at 6:40 AM, Wang, Yuming <yumw...@ebay.com.invalid> > > > wrote: > > > > > > > > +1. > > > > > > > > From: Naresh P R <nares...@apache.org> > > > > Date: Thursday, February 23, 2023 at 02:49 > > > > To: dev@hive.apache.org <dev@hive.apache.org> > > > > Subject: Re: [EXTERNAL] Re: Proposal to deprecate Hive on Spark from > > > branch-3 > > > > External Email > > > > > > > > +1 to remove Hive on Spark in branch-3 > > > > --- > > > > Regards, > > > > Naresh P R > > > > > > > >> On Wed, Feb 22, 2023 at 5:37 AM Sankar Hariappan > > > >> <sankar.hariap...@microsoft.com.invalid> wrote: > > > >> > > > >> +1, to remove Hive on Spark in branch-3. > > > >> > > > >> Thanks, > > > >> Sankar > > > >> > > > >> -----Original Message----- > > > >> From: Rajesh Balamohan <rbalamo...@apache.org> > > > >> Sent: Wednesday, February 22, 2023 6:58 PM > > > >> To: dev@hive.apache.org > > > >> Subject: [EXTERNAL] Re: Proposal to deprecate Hive on Spark from > > > branch-3 > > > >> > > > >> +1 on removing Hive on Spark in branch-3. > > > >> > > > >> It was not done earlier since it was removing a feature in the > branch. > > > But > > > >> if there is enough consensus, we should consider removing it. > > > >> > > > >> ~Rajesh.B > > > >> > > > >> On Wed, Feb 22, 2023 at 12:48 PM Aman Raj > > <raja...@microsoft.com.invalid > > > > > > > >> wrote: > > > >> > > > >>> Hi team, > > > >>> > > > >>> We have been trying to fix Hive on Spark test failures for a long > > > >>> time. As of now, branch-3 has less than 12 test failures (whose fix > > > >>> have not been identified). 8 of them are related to Hive on Spark. > I > > > >>> had mailed about the failures in my previous mail threads. Thanks > to > > > >>> Vihang for working on them as well. But we have not been able to > > > >> identify the root cause till now. > > > >>> These fixes can be tracked in the following tickets : [HIVE-27087] > > Fix > > > >>> TestMiniSparkOnYarnCliDriver test failures on branch-3 - ASF JIRA ( > > > >>> apache.org)< > > > > > > https://nam10.safelinks.protection.outlook.com/?url=https%3A%2F%2Fnam06.safelinks.protection.outlook.com%2F%3Furl%3Dhttps%2525&data=05%7C01%7Cyumwang%40ebay.com%7C2bd54cc0c84a4e44a59e08db150574e5%7C46326bff992841a0baca17c16c94ea99%7C0%7C0%7C638126885411646147%7CUnknown%7CTWFpbGZsb3d8eyJWIjoiMC4wLjAwMDAiLCJQIjoiV2luMzIiLCJBTiI6Ik1haWwiLCJXVCI6Mn0%3D%7C3000%7C%7C%7C&sdata=KORvfBkmdpqIFOoWr6J4X%2BqAQO6jcykzjY3%2FU0pq0y4%3D&reserved=0 > > > >>> 3A%2F%2Fissues.apache.org > > %2Fjira%2Fbrowse%2FHIVE-27087&data=05%7C01%7C > > > >>> Sankar.Hariappan%40microsoft.com > > %7C687a6a4dbd41454568e008db14d8cc23%7C > > > >>> > > 72f988bf86f141af91ab2d7cd011db47%7C1%7C0%7C638126693641861742%7CUnknow > > > >>> > > n%7CTWFpbGZsb3d8eyJWIjoiMC4wLjAwMDAiLCJQIjoiV2luMzIiLCJBTiI6Ik1haWwiLC > > > >>> > > JXVCI6Mn0%3D%7C3000%7C%7C%7C&sdata=RbAqrwK6fQFDStufXYfpusNc81EzjtpiaHm > > > >>> qv5CFiAs%3D&reserved=0> and [HIVE-26940] Backport of HIVE-19882 : > Fix > > > >>> QTestUtil session lifecycle - ASF JIRA > > > >>> (apache.org)< > > > > > > https://nam10.safelinks.protection.outlook.com/?url=http%3A%2F%2Fhttps%2F&data=05%7C01%7Cyumwang%40ebay.com%7C2bd54cc0c84a4e44a59e08db150574e5%7C46326bff992841a0baca17c16c94ea99%7C0%7C0%7C638126885411646147%7CUnknown%7CTWFpbGZsb3d8eyJWIjoiMC4wLjAwMDAiLCJQIjoiV2luMzIiLCJBTiI6Ik1haWwiLCJXVCI6Mn0%3D%7C3000%7C%7C%7C&sdata=nmosJ%2FQ2JM1UBDuightSWLL9haNQFuc24Zkvo4RnUh4%3D&reserved=0 > > > >>> %3A%2F%2Fissues.apache.org > > %2Fjira%2Fbrowse%2FHIVE-26940&data=05%7C01%7 > > > >>> CSankar.Hariappan%40microsoft.com > > %7C687a6a4dbd41454568e008db14d8cc23%7 > > > >>> > > C72f988bf86f141af91ab2d7cd011db47%7C1%7C0%7C638126693641861742%7CUnkno > > > >>> > > wn%7CTWFpbGZsb3d8eyJWIjoiMC4wLjAwMDAiLCJQIjoiV2luMzIiLCJBTiI6Ik1haWwiL > > > >>> > > CJXVCI6Mn0%3D%7C3000%7C%7C%7C&sdata=PaDtEZD569Sv0ER9sC4l6q1ZxyoBeER3zn > > > >>> Bsc51PWI8%3D&reserved=0> > > > >>> > > > >>> Until we have a green branch-3, we cannot go ahead to push new > > > >>> features for the Hive-3.2.0 release. This is kind of a blocker for > > this > > > >> release. > > > >>> Already bringing the test fixes to the current state took more > than 2 > > > >>> months. > > > >>> > > > >>> I wanted to bring up a proposal to deprecate Hive on Spark from > > > >>> branch-3 altogether. This would ensure that branch-3 is aligned > with > > > >>> the master as done in > > > >>> > > > > > > https://nam10.safelinks.protection.outlook.com/?url=https%3A%2F%2Fissu%2F&data=05%7C01%7Cyumwang%40ebay.com%7C2bd54cc0c84a4e44a59e08db150574e5%7C46326bff992841a0baca17c16c94ea99%7C0%7C0%7C638126885411646147%7CUnknown%7CTWFpbGZsb3d8eyJWIjoiMC4wLjAwMDAiLCJQIjoiV2luMzIiLCJBTiI6Ik1haWwiLCJXVCI6Mn0%3D%7C3000%7C%7C%7C&sdata=n51cF5fFuwSFFhX%2B0S828W3jYN3G3YwRwJWne1AMGtg%3D&reserved=0 > > > >>> es.apache.org > > > >> %2Fjira%2Fbrowse%2FHIVE-26134&data=05%7C01%7CSankar.Hariappan% > > > >> 40microsoft.com > > > > > > %7C687a6a4dbd41454568e008db14d8cc23%7C72f988bf86f141af91ab2d7cd011db47%7C1%7C0%7C638126693641861742%7CUnknown%7CTWFpbGZsb3d8eyJWIjoiMC4wLjAwMDAiLCJQIjoiV2luMzIiLCJBTiI6Ik1haWwiLCJXVCI6Mn0%3D%7C3000%7C%7C%7C&sdata=j%2F%2BYTakSvfk1Mm%2Fq8kI84gKm6s%2Fg2iA1abIPzY7t%2Bsg%3D&reserved=0. > > > >> Just wanted to have a vote on this in parallel working on the test > > > fixes. > > > >> If we have the approval from the community, we can deprecate it > > > altogether. > > > >>> > > > >>> Please feel free to suggest any concerns or suggestions you have. > > > >>> Also, I welcome any possible fix suggestion for the test failures. > > > >>> > > > >>> Thanks, > > > >>> Aman. > > > >>> > > > >> > > > > > >
