Hi Rajesh,
I have a feeling that this is not directly related to spark but I might be
wrong. The reason why is that when you do:
Configuration configuration = HBaseConfiguration.create();
by default, it reads the configuration files hbase-site.xml in your
classpath and ... (I don't remember all the configuration files hbase has).
I noticed that you overwrote some configuration settings in the code but
I'm not if you have other configurations that might have conflicted with
those.
Could you try the following, remove anything that is spark specific leaving
only hbase related codes. uber jar it and run it just like any other simple
java program. If you still have connection issues, then at least you know
the problem is from the configurations.
HTH,
Jerry
On Tue, Jul 15, 2014 at 12:10 PM, Krishna Sankar <ksanka...@gmail.com>
wrote:
> One vector to check is the HBase libraries in the --jars as in :
> spark-submit --class <your class> --master <master url> --jars
> hbase-client-0.98.3-hadoop2.jar,commons-csv-1.0-SNAPSHOT.jar,hbase-common-0.98.3-hadoop2.jar,hbase-hadoop2-compat-0.98.3-hadoop2.jar,hbase-it-0.98.3-hadoop2.jar,hbase-protocol-0.98.3-hadoop2.jar,hbase-server-0.98.3-hadoop2.jar,htrace-core-2.04.jar,spark-assembly-1.0.0-hadoop2.2.0.jar
> badwclient.jar
> This worked for us.
> Cheers
> <k/>
>
>
> On Tue, Jul 15, 2014 at 6:47 AM, Madabhattula Rajesh Kumar <
> mrajaf...@gmail.com> wrote:
>
>> Hi Team,
>>
>> Could you please help me to resolve the issue.
>>
>> *Issue *: I'm not able to connect HBase from Spark-submit. Below is my
>> code. When i execute below program in standalone, i'm able to connect to
>> Hbase and doing the operation.
>>
>> When i execute below program using spark submit ( ./bin/spark-submit )
>> command, i'm not able to connect to hbase. Am i missing any thing?
>>
>>
>> import java.util.HashMap;
>> import java.util.List;
>> import java.util.Map;
>> import java.util.Properties;
>>
>> import org.apache.hadoop.conf.Configuration;
>> import org.apache.hadoop.hbase.HBaseConfiguration;
>> import org.apache.hadoop.hbase.client.Put;
>> import org.apache.log4j.Logger;
>> import org.apache.spark.SparkConf;
>> import org.apache.spark.api.java.JavaRDD;
>> import org.apache.spark.api.java.function.Function;
>> import org.apache.spark.streaming.Duration;
>> import org.apache.spark.streaming.api.java.JavaDStream;
>> import org.apache.spark.streaming.api.java.JavaStreamingContext;
>> import org.apache.hadoop.hbase.HTableDescriptor;
>> import org.apache.hadoop.hbase.client.HBaseAdmin;
>>
>> public class Test {
>>
>>
>> public static void main(String[] args) throws Exception {
>>
>> JavaStreamingContext ssc = new
>> JavaStreamingContext("local","Test", new Duration(40000), sparkHome, "");
>>
>> JavaDStream<String> lines_2 = ssc.textFileStream(hdfsfolderpath);
>>
>> Configuration configuration = HBaseConfiguration.create();
>> configuration.set("hbase.zookeeper.property.clientPort", "2181");
>> configuration.set("hbase.zookeeper.quorum", "localhost");
>> configuration.set("hbase.master", "localhost:600000");
>>
>> HBaseAdmin hBaseAdmin = new HBaseAdmin(configuration);
>>
>> if (hBaseAdmin.tableExists("HABSE_TABLE")) {
>> System.out.println(" ANA_DATA table exists ......");
>> }
>>
>> System.out.println(" HELLO HELLO HELLO ");
>>
>> ssc.start();
>> ssc.awaitTermination();
>>
>> }
>> }
>>
>> Thank you for your help and support.
>>
>> Regards,
>> Rajesh
>>
>
>
