Hi Jark, You raised a good point: Creating the Hive temporary table. AFAIK, Hive temporary tables should be stored in metastore, Hive metastore will maintain their life cycle. Correct me if I am wrong.
So actually, if we want to support Hive temporary tables, we should finish one thing: - A temporary table should belong to Catalog. - Instead of current, a temporary table belongs to CatalogManager. It means, `createTemporaryTable` and `dropTemporaryTable` should be proxied into the Catalog. In this situation, actually, we don't need the "isTemporary" flag. (But we can have it too) Best, Jingsong On Tue, Aug 25, 2020 at 4:32 PM Jark Wu <imj...@gmail.com> wrote: > Hi, > > I'm wondering if we always fallback to using SPI for temporary tables, then > how does the create Hive temporary table using Hive dialect work? > > IMO, adding an "isTemporary" to the factory context sounds reasonable to > me, because the factory context should describe the full content of create > table DDL. > > Best, > Jark > > > On Tue, 25 Aug 2020 at 16:01, Jingsong Li <jingsongl...@gmail.com> wrote: > > > Hi Dawid, > > > > But the temporary table does not belong to Catalog, actually > > Catalog doesn't know the existence of the temporary table. Let the table > > factory of catalog to create source/sink sounds a little sudden. > > > > If we want to make temporary tables belong to Catalog, I think we need to > > involve catalog when creating temporary tables. > > > > Best, > > Jingsong > > > > On Tue, Aug 25, 2020 at 3:55 PM Dawid Wysakowicz <dwysakow...@apache.org > > > > wrote: > > > > > Hi Rui, > > > > > > My take is that temporary tables should use the factory of the catalog > > > they were registered with. > > > > > > What you are describing sounds very much like a limitation/bug in Hive > > > catalog only. I'd be in favor of passing the *isTemporary* flag. > > > > > > Best, > > > > > > Dawid > > > > > > On 25/08/2020 09:37, Rui Li wrote: > > > > Hi Dev, > > > > > > > > Currently temporary generic tables cannot work with hive catalog [1]. > > > When > > > > hive catalog is chosen as the current catalog, planner will use > > > > HiveTableFactory to create source/sink for the temporary > > > > table. HiveTableFactory cannot tell whether a table is temporary or > > not, > > > > and considers it as a Hive table, which leads to job failure. > > > > I've discussed with Jingsong offline and we believe one solution is > to > > > make > > > > planner avoid using catalog table factory for temporary tables. But > I'd > > > > also like to hear more opinions from others whether this is the right > > way > > > > to go. I think a possible alternative is to add an *isTemporary* > field > > > > to TableSourceFactory.Context & TableSinkFactory.Context, so that > > > > HiveTableFactory knows how to handle such tables. What do you think? > > > > > > > > [1] https://issues.apache.org/jira/browse/FLINK-18999 > > > > > > > > > > > > > > -- > > Best, Jingsong Lee > > > -- Best, Jingsong Lee
