Hi,

I am able to successfully create shark table with 3 columns  and 2 rows.


 val recList = List((" value A1", "value B1","value C1"),
                                 ("value A2", "value B2","value c2"));
   val dbFields =List ("Col A", "Col B","Col C")
    val rdd = sc.parallelize(recList)
    RDDTable(rdd).saveAsTable("table_1", dbFields)


I have a scenario where table will have 60 columns. How to achieve it using
RDDTable.

I tried creating a List[(Seq[String],Seq[String])] , but it throws below
exception.Any help /pointer will help.

Exception in thread "main" shark.api.DataTypes$UnknownDataTypeException:
scala.collection.Seq
    at shark.api.DataTypes.fromClassTag(DataTypes.java:133)
    at shark.util.HiveUtils$$anonfun$1.apply(HiveUtils.scala:106)
    at shark.util.HiveUtils$$anonfun$1.apply(HiveUtils.scala:105)
    at
scala.collection.TraversableLike$$anonfun$map$1.apply(TraversableLike.scala:244)
    at
scala.collection.TraversableLike$$anonfun$map$1.apply(TraversableLike.scala:244)
    at scala.collection.immutable.List.foreach(List.scala:318)
    at scala.collection.TraversableLike$class.map(TraversableLike.scala:244)
    at scala.collection.AbstractTraversable.map(Traversable.scala:105)
    at shark.util.HiveUtils$.createTableInHive(HiveUtils.scala:105)
    at shark.api.RDDTableFunctions.saveAsTable(RDDTableFunctions.scala:63)

Thanks
Subacini

Reply via email to