Hi Team,

Could you please help me to resolve the issue.

*Issue *: I'm not able to connect HBase from Spark-submit. Below is my
code.  When i execute below program in standalone, i'm able to connect to
Hbase and doing the operation.

When i execute below program using spark submit ( ./bin/spark-submit )
command, i'm not able to connect to hbase. Am i missing any thing?


import java.util.HashMap;
import java.util.List;
import java.util.Map;
import java.util.Properties;

import org.apache.hadoop.conf.Configuration;
import org.apache.hadoop.hbase.HBaseConfiguration;
import org.apache.hadoop.hbase.client.Put;
import org.apache.log4j.Logger;
import org.apache.spark.SparkConf;
import org.apache.spark.api.java.JavaRDD;
import org.apache.spark.api.java.function.Function;
import org.apache.spark.streaming.Duration;
import org.apache.spark.streaming.api.java.JavaDStream;
import org.apache.spark.streaming.api.java.JavaStreamingContext;
import org.apache.hadoop.hbase.HTableDescriptor;
import org.apache.hadoop.hbase.client.HBaseAdmin;

public class Test {


    public static void main(String[] args) throws Exception {

        JavaStreamingContext ssc = new JavaStreamingContext("local","Test",
new Duration(40000), sparkHome, "");

        JavaDStream<String> lines_2 = ssc.textFileStream(hdfsfolderpath);

        Configuration configuration = HBaseConfiguration.create();
        configuration.set("hbase.zookeeper.property.clientPort", "2181");
        configuration.set("hbase.zookeeper.quorum", "localhost");
        configuration.set("hbase.master", "localhost:600000");

        HBaseAdmin hBaseAdmin = new HBaseAdmin(configuration);

            if (hBaseAdmin.tableExists("HABSE_TABLE")) {
                System.out.println(" ANA_DATA table exists ......");
            }

        System.out.println(" HELLO HELLO HELLO ");

        ssc.start();
        ssc.awaitTermination();

    }
}

Thank you for your help and support.

Regards,
Rajesh

Reply via email to