I am trying to use the posexplode function in the HiveContext to
auto-generate a sequence number. This feature is supposed to be available
Hive 0.13.0.
SELECT name, phone FROM contact LATERAL VIEW
posexplode(phoneList.phoneNumber) phoneTable AS pos, phone
My test program failed with the following
java.lang.ClassNotFoundException: posexplode
at java.net.URLClassLoader.findClass(URLClassLoader.java:665)
at java.lang.ClassLoader.loadClassHelper(ClassLoader.java:942)
at java.lang.ClassLoader.loadClass(ClassLoader.java:851)
at sun.misc.Launcher$AppClassLoader.loadClass(Launcher.java:335)
at java.lang.ClassLoader.loadClass(ClassLoader.java:827)
at
org.apache.spark.sql.hive.HiveFunctionWrapper.createFunction(Shim13.scala:147)
at
org.apache.spark.sql.hive.HiveGenericUdtf.function$lzycompute(hiveUdfs.scala:274)
at
org.apache.spark.sql.hive.HiveGenericUdtf.function(hiveUdfs.scala:274)
Does spark support this Hive function posexplode? If not, how to patch it
to support this? I am on Spark 1.3.1
Thanks,
Jeff Li