Agree. : ) 2017-01-22 11:20 GMT-08:00 Reynold Xin <r...@databricks.com>:
> To be clear there are two separate "hive" we are talking about here. One > is the catalog, and the other is the Hive serde and UDF support. We want to > get to a point that the choice of catalog does not impact the functionality > in Spark other than where the catalog is stored. > > > On Sun, Jan 22, 2017 at 11:18 AM Xiao Li <gatorsm...@gmail.com> wrote: > >> We have a pending PR to block users to create the Hive serde table when >> using InMemroyCatalog. See: https://github.com/apache/spark/pull/16587 I >> believe it answers your question. >> >> BTW, we still can create the regular data source tables and insert the >> data into the tables. The major difference is whether the metadata is >> persistently stored or not. >> >> Thanks, >> >> Xiao Li >> >> 2017-01-22 11:14 GMT-08:00 Reynold Xin <r...@databricks.com>: >> >> I think this is something we are going to change to completely decouple >> the Hive support and catalog. >> >> >> On Sun, Jan 22, 2017 at 4:51 AM Shuai Lin <linshuai2...@gmail.com> wrote: >> >> Hi all, >> >> Currently when the in-memory catalog is used, e.g. through `--conf >> spark.sql.catalogImplementation=in-memory`, we can create a persistent >> table, but inserting into this table would fail with error message "Hive >> support is required to insert into the following tables..". >> >> sql("create table t1 (id int, name string, dept string)") // OK >> sql("insert into t1 values (1, 'name1', 'dept1')") // ERROR >> >> >> This doesn't make sense for me, because this table would always be empty >> if we can't insert into it, thus would be of no use. But I wonder if there >> are other good reasons for the current logic. If not, I would propose to >> raise an error when creating the table in the first place. >> >> Thanks! >> >> Regards, >> Shuai Lin (@lins05) >> >> >> >> >> >> >> >>
