Hi Mich Hive also has non-validated primary key, foreign key etc constraints. Whilst I’m not too familiar with the modelling tools you mention, perhaps they’re able to use these for generating SQL etc?
ORC files have indexes (min, max, bloom filters) - not particularly relevant to the data modelling tools question, but mentioning it for completeness… Thanks Austin > On 11 Nov 2020, at 17:14, Mich Talebzadeh <mich.talebza...@gmail.com> wrote: > > Many thanks Peter. > > > > > LinkedIn > https://www.linkedin.com/profile/view?id=AAEAAAAWh2gBxianrbJd6zP6AcPCCdOABUrV8Pw > > <https://www.linkedin.com/profile/view?id=AAEAAAAWh2gBxianrbJd6zP6AcPCCdOABUrV8Pw> > > > > Disclaimer: Use it at your own risk. Any and all responsibility for any loss, > damage or destruction of data or any other property which may arise from > relying on this email's technical content is explicitly disclaimed. The > author will in no case be liable for any monetary damages arising from such > loss, damage or destruction. > > > > On Wed, 11 Nov 2020 at 16:58, Peter Vary <pv...@cloudera.com > <mailto:pv...@cloudera.com>> wrote: > Hi Mich, > > Index support was removed from hive: > https://issues.apache.org/jira/browse/HIVE-21968 > <https://issues.apache.org/jira/browse/HIVE-21968> > https://issues.apache.org/jira/browse/HIVE-18715 > <https://issues.apache.org/jira/browse/HIVE-18715> > > Thanks, > Peter > >> On Nov 11, 2020, at 17:25, Mich Talebzadeh <mich.talebza...@gmail.com >> <mailto:mich.talebza...@gmail.com>> wrote: >> >> Hi all, >> >> I wrote these notes earlier this year. >> >> I heard today that someone mentioned Hive 1 does not support indexes but >> hive 2 does. >> >> I still believe that Hive does not support indexing as per below. Has this >> been changed? >> >> Regards, >> >> Mich >> >> ---------- Forwarded message --------- >> From: Mich Talebzadeh <mich.talebza...@gmail.com >> <mailto:mich.talebza...@gmail.com>> >> Date: Thu, 2 Apr 2020 at 12:17 >> Subject: How useful are tools for Hive data modeling >> To: user <user@hive.apache.org <mailto:user@hive.apache.org>> >> >> >> Hi, >> >> Fundamentally Hive tables have structure and support provided by desc >> formatted <TABLE> and show partitions <TABLE>. >> >> Hive does not support indexes in real HQL operations (I stand corrected). So >> what we have are tables, partitions and clustering (AKA hash partitioning). >> >> Hive does not support indexes because Hadoop lacks blocks locality necessary >> for indexes. So If I use a tool like Collibra, Ab-intio etc what >> advantage(s) one is going to gain on top a simple sell scrip to get table >> and partition definitions? >> >> Thanks, >> >> >> Disclaimer: Use it at your own risk. Any and all responsibility for any >> loss, damage or destruction of data or any other property which may arise >> from relying on this email's technical content is explicitly disclaimed. The >> author will in no case be liable for any monetary damages arising from such >> loss, damage or destruction. >> >
