Re: Specifying orc.stripe.size in Spark

2016-12-18 Thread Mich Talebzadeh
You can use straight sql command to create ORC table in Hive. Assuming you have registered a temp table val HiveContext = new org.apache.spark.sql.hive.HiveContext(sc) s.registerTempTable("tmp") sqltext = """ CREATE TABLE test.dummy2 ( ID INT , CLUSTERED INT , SCATTERED INT , RANDOM

Specifying orc.stripe.size in Spark

2016-12-18 Thread Daniel Haviv
Hi, When writing a dataframe using: df.write.orc("/path/to/orc") How can I specify orc parameters like orc.stripe.size ? Thank you, Daniel