Hi Chris,
I plan on going through this diff and making a comprehensive list of all the
major bug fixes that went into branch-3 and not in hive-313. This will be
included in the umbrella JIRA that I am creating.
In this email thread I have only mentioned CVEs and upgrades that will go on
top of these changes in branch-3.
Thanks,
Aman.
________________________________
From: Chris Nauroth <cnaur...@apache.org>
Sent: Friday, November 4, 2022 3:44 AM
To: dev@hive.apache.org <dev@hive.apache.org>
Subject: Re: [EXTERNAL] Re: Proposal : New Release 3.2.0 | Fixing CVE's and
Bugs on apache hive branch-3
I noticed that there is a pretty large delta (256 commits) between release
3.1.3 and the current branch-3:
> git log --oneline rel/release-3.1.3..upstream-branch-3 | wc
256 4208 33558
I just wanted to mention that a release from branch-3 would include far
more than what we are cataloging on this mail thread.
Chris Nauroth
On Thu, Nov 3, 2022 at 12:16 PM Pravin Sinha <mailpravi...@gmail.com> wrote:
> +1,
>
> Thanks for driving this, Aman. Apart from CVE fixes, do you have a list of
> JIRAs to be targeted?
>
> -Pravin
>
> On Thu, Nov 3, 2022 at 11:12 PM Chris Nauroth <cnaur...@apache.org> wrote:
>
> > Thank you for driving this!
> >
> > To kick things off, I have filed HIVE-26702 for a backport of HIVE-17315
> (a
> > total of 5 sub-tasks/patches) to 3.2.0. This adds support for more
> flexible
> > configuration of the metastore's database connection pooling. Dataproc's
> > distribution has been running this in production backported onto release
> > 3.1.3, so I can provide the patches.
> >
> > May I assume that our intent is to keep 3.2.x backward-compatible with
> > 3.1.x?
> >
> > Chris Nauroth
> >
> >
> > On Thu, Nov 3, 2022 at 3:53 AM Sankar Hariappan
> > <sankar.hariap...@microsoft.com.invalid> wrote:
> >
> > > +1, I'm excited to see the scope includes important upgrades and CVE
> > fixes.
> > > We should carefully port the relevant patches from master as code has
> > been
> > > heavily refactored. But, it make perfect sense to give another 3.x
> > release
> > > from Hive to keep the users delighted.
> > > Thanks Aman for the initiative!
> > >
> > > Thanks,
> > > Sankar
> > >
> > > -----Original Message-----
> > > From: 张铎(Duo Zhang) <palomino...@gmail.com>
> > > Sent: Thursday, November 3, 2022 2:53 PM
> > > To: dev@hive.apache.org
> > > Subject: [EXTERNAL] Re: Proposal : New Release 3.2.0 | Fixing CVE's and
> > > Bugs on apache hive branch-3
> > >
> > > [You don't often get email from palomino...@gmail.com. Learn why this
> is
> > > important at https://aka.ms/LearnAboutSenderIdentification ]
> > >
> > > +1, and please include HIVE-24694...
> > >
> > > Thanks.
> > >
> > > Aman Raj <raja...@microsoft.com.invalid> 于2022年11月3日周四 17:03写道：
> > > >
> > > > Hi team,
> > > >
> > > >
> > > > We know that Hive 4.0.0 release is ongoing but considering the number
> > of
> > > changes going into the release, it will take some iterations to come up
> > > with the stable version for the same. Meanwhile there are a lot of
> issues
> > > in Hive 3.1.3 which our customers have reported. In this scenario, it
> > makes
> > > sense to make a release from branch-3 which will have all the necessary
> > > upgrades, bug and CVE fixes which are causing issues to the existing
> > > customers. Also, Hive is still using Hadoop 3.1.0 whereas Spark 3.3 has
> > > already moved to Hadoop 3.3.1. Therefore, we need to do the same for
> > hive.
> > > >
> > > >
> > > >
> > > > I will be happy to take the ownership of this new release and will be
> > > creating JIRA's for all the fixes that will go on with this release.
> > > >
> > > >
> > > >
> > > > Therefore, I am proposing a new release cut out from branch-3. The
> > > release version would be hive-3.2.0.
> > > >
> > > >
> > > >
> > > > This version will include major upgrades as:
> > > >
> > > > 1. Hadoop version upgrade to 3.3.4
> > > > 2. Zookeeper version upgrade to 3.6.3
> > > > 3. Tez version upgrade to 0.10.2
> > > > 4. Calcite version upgrade to 1.25.0
> > > > 5. Orc version upgrade to 1.6.9
> > > >
> > > > This version will also include major CVE fixes as follows:
> > > >
> > > > 1. NVD - CVE-2020-13949 (nist.gov)<
> > >
> >
> https://nam06.safelinks.protection.outlook.com/?url=https%3A%2F%2Fnvd.nist.gov%2Fvuln%2Fdetail%2FCVE-2020-13949&data=05%7C01%7Crajaman%40microsoft.com%7C46f055069d244ce0128508dabde8cd84%7C72f988bf86f141af91ab2d7cd011db47%7C1%7C0%7C638031104888618917%7CUnknown%7CTWFpbGZsb3d8eyJWIjoiMC4wLjAwMDAiLCJQIjoiV2luMzIiLCJBTiI6Ik1haWwiLCJXVCI6Mn0%3D%7C3000%7C%7C%7C&sdata=DSR9uajrYVrUTUj7RK779CYM3KCI2Y%2FSXE%2FF9T%2FPqtE%3D&reserved=0
> > >
> > > - Libthrift Upgrade to 0.14.1 (OSS Jira :
> > >
> >
> https://nam06.safelinks.protection.outlook.com/?url=https%3A%2F%2Fissues.apache.org%2Fjira%2Fbrowse%2FHIVE-25098&data=05%7C01%7Crajaman%40microsoft.com%7C46f055069d244ce0128508dabde8cd84%7C72f988bf86f141af91ab2d7cd011db47%7C1%7C0%7C638031104888618917%7CUnknown%7CTWFpbGZsb3d8eyJWIjoiMC4wLjAwMDAiLCJQIjoiV2luMzIiLCJBTiI6Ik1haWwiLCJXVCI6Mn0%3D%7C3000%7C%7C%7C&sdata=4Fv1qyr2V8%2F82DKLOe3h%2BAQl3Xuhk89Hbsl5avb4r4M%3D&reserved=0
> > > <
> > >
> >
> https://nam06.safelinks.protection.outlook.com/?url=https%3A%2F%2Fissues.apache.org%2Fjira%2Fbrowse%2FHIVE-25098&data=05%7C01%7Crajaman%40microsoft.com%7C46f055069d244ce0128508dabde8cd84%7C72f988bf86f141af91ab2d7cd011db47%7C1%7C0%7C638031104888618917%7CUnknown%7CTWFpbGZsb3d8eyJWIjoiMC4wLjAwMDAiLCJQIjoiV2luMzIiLCJBTiI6Ik1haWwiLCJXVCI6Mn0%3D%7C3000%7C%7C%7C&sdata=4Fv1qyr2V8%2F82DKLOe3h%2BAQl3Xuhk89Hbsl5avb4r4M%3D&reserved=0
> > > >)
> > > >
> > > > 1. NVD - CVE-2015-1832 (nist.gov)<
> > >
> >
> https://nam06.safelinks.protection.outlook.com/?url=https%3A%2F%2Fnvd.nist.gov%2Fvuln%2Fdetail%2FCVE-2015-1832&data=05%7C01%7Crajaman%40microsoft.com%7C46f055069d244ce0128508dabde8cd84%7C72f988bf86f141af91ab2d7cd011db47%7C1%7C0%7C638031104888618917%7CUnknown%7CTWFpbGZsb3d8eyJWIjoiMC4wLjAwMDAiLCJQIjoiV2luMzIiLCJBTiI6Ik1haWwiLCJXVCI6Mn0%3D%7C3000%7C%7C%7C&sdata=AZtgQ9q27QYN66z6XRReY8hJyXMuHZnoKCPt4nQPeOU%3D&reserved=0
> > >
> > > - Derby upgrade to 10.14.2.0 (OSS Jira :
> > >
> >
> https://nam06.safelinks.protection.outlook.com/?url=https%3A%2F%2Fwww.mail-archive.com%2Fdev%2540hive.apache.org%2Fmsg142721.html&data=05%7C01%7Crajaman%40microsoft.com%7C46f055069d244ce0128508dabde8cd84%7C72f988bf86f141af91ab2d7cd011db47%7C1%7C0%7C638031104888618917%7CUnknown%7CTWFpbGZsb3d8eyJWIjoiMC4wLjAwMDAiLCJQIjoiV2luMzIiLCJBTiI6Ik1haWwiLCJXVCI6Mn0%3D%7C3000%7C%7C%7C&sdata=46Vpp%2BU1FVTT1u01fIhTrzKC%2BTQeHWsYMHpbJpbsBug%3D&reserved=0
> > > <
> > >
> >
> https://nam06.safelinks.protection.outlook.com/?url=https%3A%2F%2Fwww.mail-archive.com%2Fdev%2540hive.apache.org%2Fmsg142721.html&data=05%7C01%7Crajaman%40microsoft.com%7C46f055069d244ce0128508dabde8cd84%7C72f988bf86f141af91ab2d7cd011db47%7C1%7C0%7C638031104888618917%7CUnknown%7CTWFpbGZsb3d8eyJWIjoiMC4wLjAwMDAiLCJQIjoiV2luMzIiLCJBTiI6Ik1haWwiLCJXVCI6Mn0%3D%7C3000%7C%7C%7C&sdata=46Vpp%2BU1FVTT1u01fIhTrzKC%2BTQeHWsYMHpbJpbsBug%3D&reserved=0
> > > >)
> > > >
> > > > 1. NVD - CVE-2013-4002 (nist.gov)<
> > >
> >
> https://nam06.safelinks.protection.outlook.com/?url=https%3A%2F%2Fnvd.nist.gov%2Fvuln%2Fdetail%2FCVE-2013-4002&data=05%7C01%7Crajaman%40microsoft.com%7C46f055069d244ce0128508dabde8cd84%7C72f988bf86f141af91ab2d7cd011db47%7C1%7C0%7C638031104888618917%7CUnknown%7CTWFpbGZsb3d8eyJWIjoiMC4wLjAwMDAiLCJQIjoiV2luMzIiLCJBTiI6Ik1haWwiLCJXVCI6Mn0%3D%7C3000%7C%7C%7C&sdata=Nh7Osuvb8RV6EJJ2MU9PMhy5ENCXrQKs7JIrgqGrmhk%3D&reserved=0
> > >
> > > - Xerces Upgrade to 2.12.2 (OSS Jira :
> > >
> >
> https://nam06.safelinks.protection.outlook.com/?url=https%3A%2F%2Fissues.apache.org%2Fjira%2Fbrowse%2FHIVE-25920&data=05%7C01%7Crajaman%40microsoft.com%7C46f055069d244ce0128508dabde8cd84%7C72f988bf86f141af91ab2d7cd011db47%7C1%7C0%7C638031104888618917%7CUnknown%7CTWFpbGZsb3d8eyJWIjoiMC4wLjAwMDAiLCJQIjoiV2luMzIiLCJBTiI6Ik1haWwiLCJXVCI6Mn0%3D%7C3000%7C%7C%7C&sdata=%2BEypMVloduSBSKsa6dNPakG%2F%2BYppSNpobtuBM8S2Fe4%3D&reserved=0
> > > <
> > >
> >
> https://nam06.safelinks.protection.outlook.com/?url=https%3A%2F%2Fissues.apache.org%2Fjira%2Fbrowse%2FHIVE-25920&data=05%7C01%7Crajaman%40microsoft.com%7C46f055069d244ce0128508dabde8cd84%7C72f988bf86f141af91ab2d7cd011db47%7C1%7C0%7C638031104888618917%7CUnknown%7CTWFpbGZsb3d8eyJWIjoiMC4wLjAwMDAiLCJQIjoiV2luMzIiLCJBTiI6Ik1haWwiLCJXVCI6Mn0%3D%7C3000%7C%7C%7C&sdata=%2BEypMVloduSBSKsa6dNPakG%2F%2BYppSNpobtuBM8S2Fe4%3D&reserved=0
> > > >)
> > > >
> > > > 1. NVD - CVE-2020-36518 (nist.gov)<
> > >
> >
> https://nam06.safelinks.protection.outlook.com/?url=https%3A%2F%2Fnvd.nist.gov%2Fvuln%2Fdetail%2FCVE-2020-36518&data=05%7C01%7Crajaman%40microsoft.com%7C46f055069d244ce0128508dabde8cd84%7C72f988bf86f141af91ab2d7cd011db47%7C1%7C0%7C638031104888618917%7CUnknown%7CTWFpbGZsb3d8eyJWIjoiMC4wLjAwMDAiLCJQIjoiV2luMzIiLCJBTiI6Ik1haWwiLCJXVCI6Mn0%3D%7C3000%7C%7C%7C&sdata=yrcbP59wk0Wa9DsgY3Hs1vkDmGkLHvLHeHK40VQvEuI%3D&reserved=0
> > >
> > > - Jackson upgrade to 2.12.7 (OSS Jira :
> > >
> >
> https://nam06.safelinks.protection.outlook.com/?url=https%3A%2F%2Fwww.mail-archive.com%2Fdev%40hive.apache.org%2Fmsg142871.html&data=05%7C01%7Crajaman%40microsoft.com%7C46f055069d244ce0128508dabde8cd84%7C72f988bf86f141af91ab2d7cd011db47%7C1%7C0%7C638031104888618917%7CUnknown%7CTWFpbGZsb3d8eyJWIjoiMC4wLjAwMDAiLCJQIjoiV2luMzIiLCJBTiI6Ik1haWwiLCJXVCI6Mn0%3D%7C3000%7C%7C%7C&sdata=ON24JQ%2F7Hq6mObSRpQ2uvmBodv%2BwuaKwQIu6BIx4Y1w%3D&reserved=0
> > > <
> > >
> >
> https://nam06.safelinks.protection.outlook.com/?url=https%3A%2F%2Fwww.mail-archive.com%2Fdev%40hive.apache.org%2Fmsg142871.html&data=05%7C01%7Crajaman%40microsoft.com%7C46f055069d244ce0128508dabde8cd84%7C72f988bf86f141af91ab2d7cd011db47%7C1%7C0%7C638031104888618917%7CUnknown%7CTWFpbGZsb3d8eyJWIjoiMC4wLjAwMDAiLCJQIjoiV2luMzIiLCJBTiI6Ik1haWwiLCJXVCI6Mn0%3D%7C3000%7C%7C%7C&sdata=ON24JQ%2F7Hq6mObSRpQ2uvmBodv%2BwuaKwQIu6BIx4Y1w%3D&reserved=0
> > > >)
> > > >
> > > > 1. NVD - CVE-2022-23221 (nist.gov)<
> > >
> >
> https://nam06.safelinks.protection.outlook.com/?url=https%3A%2F%2Fnvd.nist.gov%2Fvuln%2Fdetail%2FCVE-2022-23221&data=05%7C01%7Crajaman%40microsoft.com%7C46f055069d244ce0128508dabde8cd84%7C72f988bf86f141af91ab2d7cd011db47%7C1%7C0%7C638031104888618917%7CUnknown%7CTWFpbGZsb3d8eyJWIjoiMC4wLjAwMDAiLCJQIjoiV2luMzIiLCJBTiI6Ik1haWwiLCJXVCI6Mn0%3D%7C3000%7C%7C%7C&sdata=JW7KrKsXgKgP18FG2SZJeZl0bhYUHHvc5amuAhuieE4%3D&reserved=0
> > >
> > > - Upgrade H2 database version to 2.1.210 (OSS Jira :
> > >
> >
> https://nam06.safelinks.protection.outlook.com/?url=https%3A%2F%2Fissues.apache.org%2Fjira%2Fbrowse%2FHIVE-25945&data=05%7C01%7Crajaman%40microsoft.com%7C46f055069d244ce0128508dabde8cd84%7C72f988bf86f141af91ab2d7cd011db47%7C1%7C0%7C638031104888618917%7CUnknown%7CTWFpbGZsb3d8eyJWIjoiMC4wLjAwMDAiLCJQIjoiV2luMzIiLCJBTiI6Ik1haWwiLCJXVCI6Mn0%3D%7C3000%7C%7C%7C&sdata=6ePiXEA1r%2F%2F94cNW54vU%2FRXay4IvHge%2FLfPhBE0jxGs%3D&reserved=0
> > > <
> > >
> >
> https://nam06.safelinks.protection.outlook.com/?url=https%3A%2F%2Fissues.apache.org%2Fjira%2Fbrowse%2FHIVE-25945&data=05%7C01%7Crajaman%40microsoft.com%7C46f055069d244ce0128508dabde8cd84%7C72f988bf86f141af91ab2d7cd011db47%7C1%7C0%7C638031104888618917%7CUnknown%7CTWFpbGZsb3d8eyJWIjoiMC4wLjAwMDAiLCJQIjoiV2luMzIiLCJBTiI6Ik1haWwiLCJXVCI6Mn0%3D%7C3000%7C%7C%7C&sdata=6ePiXEA1r%2F%2F94cNW54vU%2FRXay4IvHge%2FLfPhBE0jxGs%3D&reserved=0
> > > >)
> > > >
> > > > 1. WS-2021-0419 | Mend Vulnerability Database<
> > >
> >
> https://nam06.safelinks.protection.outlook.com/?url=https%3A%2F%2Fwww.mend.io%2Fvulnerability-database%2FWS-2021-0419&data=05%7C01%7Crajaman%40microsoft.com%7C46f055069d244ce0128508dabde8cd84%7C72f988bf86f141af91ab2d7cd011db47%7C1%7C0%7C638031104888618917%7CUnknown%7CTWFpbGZsb3d8eyJWIjoiMC4wLjAwMDAiLCJQIjoiV2luMzIiLCJBTiI6Ik1haWwiLCJXVCI6Mn0%3D%7C3000%7C%7C%7C&sdata=gKe%2BkrRB0o%2FIwdZo5Bk8JTBQZWXuV6vqXs9sv5SdLHw%3D&reserved=0
> > >
> > > - Upgrade gson to 2.8.9 (OSS Jira :
> > >
> >
> https://nam06.safelinks.protection.outlook.com/?url=https%3A%2F%2Fissues.apache.org%2Fjira%2Fbrowse%2FHIVE-26078&data=05%7C01%7Crajaman%40microsoft.com%7C46f055069d244ce0128508dabde8cd84%7C72f988bf86f141af91ab2d7cd011db47%7C1%7C0%7C638031104888618917%7CUnknown%7CTWFpbGZsb3d8eyJWIjoiMC4wLjAwMDAiLCJQIjoiV2luMzIiLCJBTiI6Ik1haWwiLCJXVCI6Mn0%3D%7C3000%7C%7C%7C&sdata=CTO8Muczt79f7k3FyD3f7OI%2FJt%2BixpcoasKunPeCX%2FQ%3D&reserved=0
> > > <
> > >
> >
> https://nam06.safelinks.protection.outlook.com/?url=https%3A%2F%2Fissues.apache.org%2Fjira%2Fbrowse%2FHIVE-26078&data=05%7C01%7Crajaman%40microsoft.com%7C46f055069d244ce0128508dabde8cd84%7C72f988bf86f141af91ab2d7cd011db47%7C1%7C0%7C638031104888618917%7CUnknown%7CTWFpbGZsb3d8eyJWIjoiMC4wLjAwMDAiLCJQIjoiV2luMzIiLCJBTiI6Ik1haWwiLCJXVCI6Mn0%3D%7C3000%7C%7C%7C&sdata=CTO8Muczt79f7k3FyD3f7OI%2FJt%2BixpcoasKunPeCX%2FQ%3D&reserved=0
> > > >)
> > > >
> > > > 1. NVD - CVE-2020-11979 (nist.gov)<
> > >
> >
> https://nam06.safelinks.protection.outlook.com/?url=https%3A%2F%2Fnvd.nist.gov%2Fvuln%2Fdetail%2FCVE-2020-11979&data=05%7C01%7Crajaman%40microsoft.com%7C46f055069d244ce0128508dabde8cd84%7C72f988bf86f141af91ab2d7cd011db47%7C1%7C0%7C638031104888618917%7CUnknown%7CTWFpbGZsb3d8eyJWIjoiMC4wLjAwMDAiLCJQIjoiV2luMzIiLCJBTiI6Ik1haWwiLCJXVCI6Mn0%3D%7C3000%7C%7C%7C&sdata=r8JxY%2FVWEaatiNixm%2FkoxNRd2dBSozs8LO5mQpBxagE%3D&reserved=0
> > >
> > > - Upgrade ant to 1.10.9 (OSS Jira : [HIVE-26081] Upgrade ant to 1.10.9
> -
> > > ASF JIRA (apache.org)<
> > >
> >
> https://nam06.safelinks.protection.outlook.com/?url=https%3A%2F%2Fissues.apache.org%2Fjira%2Fbrowse%2FHIVE-26081&data=05%7C01%7Crajaman%40microsoft.com%7C46f055069d244ce0128508dabde8cd84%7C72f988bf86f141af91ab2d7cd011db47%7C1%7C0%7C638031104888618917%7CUnknown%7CTWFpbGZsb3d8eyJWIjoiMC4wLjAwMDAiLCJQIjoiV2luMzIiLCJBTiI6Ik1haWwiLCJXVCI6Mn0%3D%7C3000%7C%7C%7C&sdata=mWnu5wbEP6cTQWny5aEFz3viB%2B%2B8xCIOEM5OwyRu%2Bbk%3D&reserved=0
> > > >)
> > > >
> > > > 1. NVD - CVE-2020-17533 (nist.gov)<
> > >
> >
> https://nam06.safelinks.protection.outlook.com/?url=https%3A%2F%2Fnvd.nist.gov%2Fvuln%2Fdetail%2FCVE-2020-17533&data=05%7C01%7Crajaman%40microsoft.com%7C46f055069d244ce0128508dabde8cd84%7C72f988bf86f141af91ab2d7cd011db47%7C1%7C0%7C638031104888618917%7CUnknown%7CTWFpbGZsb3d8eyJWIjoiMC4wLjAwMDAiLCJQIjoiV2luMzIiLCJBTiI6Ik1haWwiLCJXVCI6Mn0%3D%7C3000%7C%7C%7C&sdata=FnvQ4DOGd2xZyw%2B65OLPqxHtmz9LY7q3SkOnCkFGvco%3D&reserved=0
> > >
> > > - Upgrade accumulo-core to 1.10.1 (OSS Jira : [HIVE-26080] Upgrade
> > > accumulo-core to 1.10.1 - ASF JIRA (apache.org)<
> > >
> >
> https://nam06.safelinks.protection.outlook.com/?url=https%3A%2F%2Fissues.apache.org%2Fjira%2Fbrowse%2FHIVE-26080&data=05%7C01%7Crajaman%40microsoft.com%7C46f055069d244ce0128508dabde8cd84%7C72f988bf86f141af91ab2d7cd011db47%7C1%7C0%7C638031104888618917%7CUnknown%7CTWFpbGZsb3d8eyJWIjoiMC4wLjAwMDAiLCJQIjoiV2luMzIiLCJBTiI6Ik1haWwiLCJXVCI6Mn0%3D%7C3000%7C%7C%7C&sdata=79fLG5PR5rOAy2tf70RpcQjHSAsdMpgbgpk9HBIKDZ4%3D&reserved=0
> > > >)
> > > >
> > > >
> > > >
> > > > The version can also contain critical bug fixes that have been fixed
> in
> > > Open-Source master. Please suggest any other important backports that
> can
> > > be included in this section.
> > > >
> > > > I am thinking of the backport of transaction statistics related
> patches
> > > to enable better CBO for ACID tables and datanucleus changes to 5.x can
> > be
> > > some bug fixes that we can consume in this release. This is an Open
> forum
> > > and I welcome your suggestions on the same.
> > > >
> > > >
> > > >
> > > > We can take a month or two to make this release after validating the
> > > test scenarios and use cases. I will come up with the proper timelines
> > for
> > > this 3.2.0 release once we get the community approval for the same.
> > > >
> > > >
> > > >
> > > > Thanks,
> > > >
> > > > Aman.
> > > >
> > >
> >
>
