Ayan, The location of the logging class was moved from Spark 1.6 to Spark 2.0. Looks like you are trying to run 1.6 code on 2.0, I have ported some code like this before and if you have access to the code you can recompile it by changing reference to Logging class and directly use the slf4 Logger class, most of the code tends to be easily portable.
Following is the release note for Spark 2.0 Removals, Behavior Changes and Deprecations Removals The following features have been removed in Spark 2.0: * Bagel * Support for Hadoop 2.1 and earlier * The ability to configure closure serializer * HTTPBroadcast * TTL-based metadata cleaning * Semi-private class org.apache.spark.Logging. We suggest you use slf4j directly. * SparkContext.metricsSystem Thanks, Mahesh From: ayan guha [mailto:guha.a...@gmail.com] Sent: Monday, June 26, 2017 6:26 AM To: Weiqing Yang Cc: user Subject: Re: HDP 2.5 - Python - Spark-On-Hbase Hi I am using following: --packages com.hortonworks:shc:1.0.0-1.6-s_2.10 --repositories http://repo.hortonworks.com/content/groups/public/ Is it compatible with Spark 2.X? I would like to use it.... Best Ayan On Sat, Jun 24, 2017 at 2:09 AM, Weiqing Yang <yangweiqing...@gmail.com<mailto:yangweiqing...@gmail.com>> wrote: Yes. What SHC version you were using? If hitting any issues, you can post them in SHC github issues. There are some threads about this. On Fri, Jun 23, 2017 at 5:46 AM, ayan guha <guha.a...@gmail.com<mailto:guha.a...@gmail.com>> wrote: Hi Is it possible to use SHC from Hortonworks with pyspark? If so, any working code sample available? Also, I faced an issue while running the samples with Spark 2.0 "Caused by: java.lang.ClassNotFoundException: org.apache.spark.Logging" Any workaround? Thanks in advance.... -- Best Regards, Ayan Guha -- Best Regards, Ayan Guha DISCLAIMER ========== This e-mail may contain privileged and confidential information which is the property of Persistent Systems Ltd. It is intended only for the use of the individual or entity to which it is addressed. If you are not the intended recipient, you are not authorized to read, retain, copy, print, distribute or use this message. If you have received this communication in error, please notify the sender and delete all copies of this message. Persistent Systems Ltd. does not accept any liability for virus infected mails.
