I have seen this kind of problem when the catalog was not configured for the table/session and we ended up using the default catalog instead of HiveCatalog
> On Mar 2, 2021, at 18:49, Edgar Rodriguez > <edgar.rodrig...@airbnb.com.INVALID> wrote: > > Hi, > > I'm trying to run a simple query in Hive 2.3.4 with a join of a Hive table > and an Iceberg table, each configured accordingly - Iceberg table has the > `storage_handler` defined and running with MR engine. > > I'm using the `iceberg.mr.catalog.loader.class` class to load our internal > catalog. In the logs I can see Hive loading the Iceberg table, but then I can > see the Driver doing some traversal through the FS path under the table > location, getting statuses for all data within the directory - this is not > the behavior I see when querying an Iceberg table in Hive by itself, where I > can see the splits being computed correctly. > Due to this behavior, the query basically scans the full FS structure under > the path - which if large it looks like it's stuck, however I do see the wire > activity fetching the FS listings. > > Question is, has anyone experienced this behavior on querying Hive tables > with joins on Iceberg tables? If so, what's the best way to approach this? > > Best, > -- > Edgar R
