Hi,
In Spark 1.6.2, we were able to very quickly -- nearly instantly -- search
through the list of (many) table names in our Hive metastore with
sqlContext.tableNames().filter(_.matches("some regex")).foreach { println }
In Spark 2.0.2, however, this takes forever. Similarly, queries with
Catalog that should return a Dataset like
spark.catalog.listTables("default")
take forever. Setting the log level to DEBUG in the spark-shell, I can see
the above command is scrolling through every table name in the metastore.
Does anyone have a better way to quickly search through the metastore for a
table names matching a regexp in Spark 2?
Thanks!
- Everett
