Good catch. I thought the largest port number is 65535.
Cheers
<k/>
On Tue, Jul 15, 2014 at 4:33 PM, Spark DevUser <spark.devu...@gmail.com>
wrote:
> Are you able to launch *hbase shell* and run some commands (list,
> describe, scan, etc)? Seems *configuration.set("hbase.**master",
> "localhost:600000")* is wrong.
>
>
> On Tue, Jul 15, 2014 at 3:00 PM, Tathagata Das <
> tathagata.das1...@gmail.com> wrote:
>
>> Also, it helps if you post us logs, stacktraces, exceptions, etc.
>>
>> TD
>>
>>
>> On Tue, Jul 15, 2014 at 10:07 AM, Jerry Lam <chiling...@gmail.com> wrote:
>>
>>> Hi Rajesh,
>>>
>>> I have a feeling that this is not directly related to spark but I might
>>> be wrong. The reason why is that when you do:
>>>
>>> Configuration configuration = HBaseConfiguration.create();
>>>
>>> by default, it reads the configuration files hbase-site.xml in your
>>> classpath and ... (I don't remember all the configuration files hbase has).
>>>
>>> I noticed that you overwrote some configuration settings in the code but
>>> I'm not if you have other configurations that might have conflicted with
>>> those.
>>>
>>> Could you try the following, remove anything that is spark specific
>>> leaving only hbase related codes. uber jar it and run it just like any
>>> other simple java program. If you still have connection issues, then at
>>> least you know the problem is from the configurations.
>>>
>>> HTH,
>>>
>>> Jerry
>>>
>>>
>>> On Tue, Jul 15, 2014 at 12:10 PM, Krishna Sankar <ksanka...@gmail.com>
>>> wrote:
>>>
>>>> One vector to check is the HBase libraries in the --jars as in :
>>>> spark-submit --class <your class> --master <master url> --jars
>>>> hbase-client-0.98.3-hadoop2.jar,commons-csv-1.0-SNAPSHOT.jar,hbase-common-0.98.3-hadoop2.jar,hbase-hadoop2-compat-0.98.3-hadoop2.jar,hbase-it-0.98.3-hadoop2.jar,hbase-protocol-0.98.3-hadoop2.jar,hbase-server-0.98.3-hadoop2.jar,htrace-core-2.04.jar,spark-assembly-1.0.0-hadoop2.2.0.jar
>>>> badwclient.jar
>>>> This worked for us.
>>>> Cheers
>>>> <k/>
>>>>
>>>>
>>>> On Tue, Jul 15, 2014 at 6:47 AM, Madabhattula Rajesh Kumar <
>>>> mrajaf...@gmail.com> wrote:
>>>>
>>>>> Hi Team,
>>>>>
>>>>> Could you please help me to resolve the issue.
>>>>>
>>>>> *Issue *: I'm not able to connect HBase from Spark-submit. Below is
>>>>> my code. When i execute below program in standalone, i'm able to connect
>>>>> to Hbase and doing the operation.
>>>>>
>>>>> When i execute below program using spark submit ( ./bin/spark-submit )
>>>>> command, i'm not able to connect to hbase. Am i missing any thing?
>>>>>
>>>>>
>>>>> import java.util.HashMap;
>>>>> import java.util.List;
>>>>> import java.util.Map;
>>>>> import java.util.Properties;
>>>>>
>>>>> import org.apache.hadoop.conf.Configuration;
>>>>> import org.apache.hadoop.hbase.HBaseConfiguration;
>>>>> import org.apache.hadoop.hbase.client.Put;
>>>>> import org.apache.log4j.Logger;
>>>>> import org.apache.spark.SparkConf;
>>>>> import org.apache.spark.api.java.JavaRDD;
>>>>> import org.apache.spark.api.java.function.Function;
>>>>> import org.apache.spark.streaming.Duration;
>>>>> import org.apache.spark.streaming.api.java.JavaDStream;
>>>>> import org.apache.spark.streaming.api.java.JavaStreamingContext;
>>>>> import org.apache.hadoop.hbase.HTableDescriptor;
>>>>> import org.apache.hadoop.hbase.client.HBaseAdmin;
>>>>>
>>>>> public class Test {
>>>>>
>>>>>
>>>>> public static void main(String[] args) throws Exception {
>>>>>
>>>>> JavaStreamingContext ssc = new
>>>>> JavaStreamingContext("local","Test", new Duration(40000), sparkHome, "");
>>>>>
>>>>> JavaDStream<String> lines_2 =
>>>>> ssc.textFileStream(hdfsfolderpath);
>>>>>
>>>>> Configuration configuration = HBaseConfiguration.create();
>>>>> configuration.set("hbase.zookeeper.property.clientPort",
>>>>> "2181");
>>>>> configuration.set("hbase.zookeeper.quorum", "localhost");
>>>>> configuration.set("hbase.master", "localhost:600000");
>>>>>
>>>>> HBaseAdmin hBaseAdmin = new HBaseAdmin(configuration);
>>>>>
>>>>> if (hBaseAdmin.tableExists("HABSE_TABLE")) {
>>>>> System.out.println(" ANA_DATA table exists ......");
>>>>> }
>>>>>
>>>>> System.out.println(" HELLO HELLO HELLO ");
>>>>>
>>>>> ssc.start();
>>>>> ssc.awaitTermination();
>>>>>
>>>>> }
>>>>> }
>>>>>
>>>>> Thank you for your help and support.
>>>>>
>>>>> Regards,
>>>>> Rajesh
>>>>>
>>>>
>>>>
>>>
>>
>
