Hi Nathan and Jerry,
Thank you for the details.
Jerry,
I've installed Spark, Hbase and Hadoop in same machine. Please let me know
do you need more information. I'm not able to identify the issue why it is
not connected to Hbase when i use spark-submit
Do you have a example program? if yes, please share with me
Thank you for your help and support.
Regards,
Rajesh
On Tue, Jul 15, 2014 at 8:00 PM, Jerry Lam <chiling...@gmail.com> wrote:
> Hi Rajesh,
>
> can you describe your spark cluster setup? I saw localhost:2181 for
> zookeeper.
>
> Best Regards,
>
> Jerry
>
>
> On Tue, Jul 15, 2014 at 9:47 AM, Madabhattula Rajesh Kumar <
> mrajaf...@gmail.com> wrote:
>
>> Hi Team,
>>
>> Could you please help me to resolve the issue.
>>
>> *Issue *: I'm not able to connect HBase from Spark-submit. Below is my
>> code. When i execute below program in standalone, i'm able to connect to
>> Hbase and doing the operation.
>>
>> When i execute below program using spark submit ( ./bin/spark-submit )
>> command, i'm not able to connect to hbase. Am i missing any thing?
>>
>>
>> import java.util.HashMap;
>> import java.util.List;
>> import java.util.Map;
>> import java.util.Properties;
>>
>> import org.apache.hadoop.conf.Configuration;
>> import org.apache.hadoop.hbase.HBaseConfiguration;
>> import org.apache.hadoop.hbase.client.Put;
>> import org.apache.log4j.Logger;
>> import org.apache.spark.SparkConf;
>> import org.apache.spark.api.java.JavaRDD;
>> import org.apache.spark.api.java.function.Function;
>> import org.apache.spark.streaming.Duration;
>> import org.apache.spark.streaming.api.java.JavaDStream;
>> import org.apache.spark.streaming.api.java.JavaStreamingContext;
>> import org.apache.hadoop.hbase.HTableDescriptor;
>> import org.apache.hadoop.hbase.client.HBaseAdmin;
>>
>> public class Test {
>>
>>
>> public static void main(String[] args) throws Exception {
>>
>> JavaStreamingContext ssc = new
>> JavaStreamingContext("local","Test", new Duration(40000), sparkHome, "");
>>
>> JavaDStream<String> lines_2 = ssc.textFileStream(hdfsfolderpath);
>>
>> Configuration configuration = HBaseConfiguration.create();
>> configuration.set("hbase.zookeeper.property.clientPort", "2181");
>> configuration.set("hbase.zookeeper.quorum", "localhost");
>> configuration.set("hbase.master", "localhost:600000");
>>
>> HBaseAdmin hBaseAdmin = new HBaseAdmin(configuration);
>>
>> if (hBaseAdmin.tableExists("HABSE_TABLE")) {
>> System.out.println(" ANA_DATA table exists ......");
>> }
>>
>> System.out.println(" HELLO HELLO HELLO ");
>>
>> ssc.start();
>> ssc.awaitTermination();
>>
>> }
>> }
>>
>> Thank you for your help and support.
>>
>> Regards,
>> Rajesh
>>
>
>
