[SS] Writing a test for a possible bug in StateStoreSaveExec with Append output mode?

2017-09-03 Thread Jacek Laskowski
Hi, I may have found a bug in StateStoreSaveExec with Append output mode and would love proving myself I'm wrong or help squashing it by writing a test for the case. Is there a test for StateStoreSaveExec with Append output mode? If not, is there a streaming test template that could be very close

Re: Spark 2.2.0 - Odd Hive SQL Warnings

2017-09-03 Thread Liang-Chi Hsieh
Hi Don, There is a new SQL config `spark.sql.hive.caseSensitiveInferenceMode` which sets the action to take when a case-sensitive schema cannot be read from a Hive table's properties. The default setting of this config is `INFER_AND_SAVE` that goes toinfer the case-sensitive schema from the unde

No rows in Apache Tajo table reading(Spark SQL)

2017-09-03 Thread Cinyoung Hur
Hi, I want to read Apache Tajo table using spark sql. Tajo JDBC driver is added to spark-shell, but Tajo table doesn't show anything. The followings are Spark code and the result. $ spark-shell --jars tajo-jdbc-0.11.3.jar scala> val componentDF = spark.sqlContext.load("jdbc", Map( "url"-> "