I have seen this kind of problem when the catalog was not configured for the 
table/session and we ended up using the default catalog instead of HiveCatalog

> On Mar 2, 2021, at 18:49, Edgar Rodriguez 
> <edgar.rodrig...@airbnb.com.INVALID> wrote:
> 
> Hi,
> 
> I'm trying to run a simple query in Hive 2.3.4 with a join of a Hive table 
> and an Iceberg table, each configured accordingly - Iceberg table has the 
> `storage_handler` defined and running with MR engine.
> 
> I'm using the `iceberg.mr.catalog.loader.class` class to load our internal 
> catalog. In the logs I can see Hive loading the Iceberg table, but then I can 
> see the Driver doing some traversal through the FS path under the table 
> location, getting statuses for all data within the directory - this is not 
> the behavior I see when querying an Iceberg table in Hive by itself, where I 
> can see the splits being computed correctly. 
> Due to this behavior, the query basically scans the full FS structure under 
> the path - which if large it looks like it's stuck, however I do see the wire 
> activity fetching the FS listings.
> 
> Question is, has anyone experienced this behavior on querying Hive tables 
> with joins on Iceberg tables? If so, what's the best way to approach this?
> 
> Best,
> -- 
> Edgar R

Reply via email to