On Thu, Nov 21, 2019 at 12:53 AM Dongjoon Hyun <dongjoon.h...@gmail.com> wrote:
> Thank you for much thoughtful clarification. I agree with your all options. > > Especially, for Hive Metastore connection, `Hive isolated client loader` > is also important with Hive 2.3 because Hive 2.3 client cannot talk with > Hive 2.1 and lower. `Hive Isolated client loader` is one of the good design > in Apache Spark. > > One of the reason I started this thread focusing on the fork is that we *don't > use* that fork actually. > > https://mvnrepository.com/artifact/org.spark-project.hive/ > > Big companies (and vendors) maintains their own fork of that fork or > upgrade its hive dependency already. So, when we say it's battle-tested, it > does not mean it really. It's not tested. > > I'm not up to date with the cloudera fork. Last time I went near the then-hortonworks fork was for this : https://github.com/pwendell/hive/pull/2 ; think there were a couple of security patches too. I don't think anyone would have added new features to the branch, but bug fixes and security patches are inevitable. The above repository becomes something like a stranded phantom. We pointed > that repo as a legacy interface, and we don't use the code really in the > large production environments. Since there is no way to contribute back > to that repo, we also have a segmentation problem on the experience with > Hive 1.2.1. Someone may say it's good while the others still struggles without > any community support. > > Anyway, thank you so much for the conclusion. > I'll try to make a JIRA and PR for `hive-1.2` profile first as a > conclusion. > +1 >>>>>>>
