Re: Metastore: How is the unique ID of new databases and tables determined?

2023-10-24 Thread Venu Reddy
Hi Eugene, HMS depends on DataNucleus for the identity value generation for the HMS tables. It is generated by DataNucleus when an object is made persistent. DataNucleus value generator will generate values uniquely across different JVMs. As Zoltan said, DataNucleus tracks with the SEQUENCE_TABLE

Re: Metastore: How is the unique ID of new databases and tables determined?

2023-10-24 Thread Zoltán Rátkai
Hi Eugene, the TBL_ID in TBLS table is handled by Datanucleus, so AUTO_INCREMENT won't help, since the TBL_ID is not defined as AUTO_INCREMENT. Datanucleus uses SEQUENCE_TABLE to store the actual value for primary keys. In this table this two rows is what you need to modify: org.apache.hadoop.hi

Re: Announce: Hive-MR3 with Celeborn,

2023-10-24 Thread lisoda
Thanks. I will try. Replied Message | From | Sungwoo Park | | Date | 10/24/2023 20:08 | | To | user@hive.apache.org | | Cc | | | Subject | Announce: Hive-MR3 with Celeborn, | Hi Hive users, Before the impending release of MR3 1.8, we would like to announce the release of Hive-MR3 wi

Announce: Hive-MR3 with Celeborn,

2023-10-24 Thread Sungwoo Park
Hi Hive users, Before the impending release of MR3 1.8, we would like to announce the release of Hive-MR3 with Celeborn (Hive 3.1.3 on MR3 1.8 with Celeborn 0.3.1). Apache Celeborn [1] is remote shuffle service, similar to Magnet [2] and Apache Uniffle [3] (which was discussed in this Hive mailin

Re: Re: Hive's performance for querying the Iceberg table is very poor.

2023-10-24 Thread Ayush Saxena
HIVE-27734 is in progress, as I see we have a POC attached to the ticket, we should have it in 2-3 week I believe. > Also, after the release of 4.0.0, will we be able to do all TPCDS queries on ICEBERG except for normal HIVE tables? Yep, I believe most of the TPCDS queries would be supported even

Re:Re: Hive's performance for querying the Iceberg table is very poor.

2023-10-24 Thread lisoda
Thanks. I would like to know if hive currently supports push to ICEBERG table partition under JOIN condition. Because I see HIVE-27734 is not yet complete, what is its progress so far? Also, after the release of 4.0.0, will we be able to do all TPCDS queries on ICEBERG except for normal HIVE tabl