And which version of HBase and Hadoop are you running? Did you try to put the hbase-site.xml in the jar? Moreover, I don't know how much reliable is at the moment the web client UI..my experience is that the command line client is much more reliable.
You just need to run from the flink dir something like: bin/flink run -c xxx.yyy.MyMainClass /path/to/shadedJar.jar On Wed, May 11, 2016 at 4:19 PM, Palle <pa...@sport.dk> wrote: > I run the job from the cluster. I run it through the web UI. > The jar file submitted does not contain the hbase-site.xml file. > > ----- Original meddelelse ----- > > *Fra:* Flavio Pompermaier <pomperma...@okkam.it> > *Til:* user <user@flink.apache.org> > *Dato:* Ons, 11. maj 2016 09:36 > > *Emne:* Re: HBase write problem > > Do you run the job from your IDE or from the cluster? > > On Wed, May 11, 2016 at 9:22 AM, Palle <pa...@sport.dk> wrote: > >> Thanks for the response, but I don't think the problem is the classpath - >> hbase-site.xml should be added. This is what it looks like (hbase conf is >> added at the end): >> >> 2016-05-11 09:16:45,831 INFO org.apache.zookeeper.ZooKeeper >> - Client >> environment:java.class.path=C:\systems\packages\flink-1.0.2\lib\flink-dist_2.11-1.0.2.jar;C:\systems\packages\flink-1.0.2\lib\flink-python_2.11-1.0.2.jar;C:\systems\packages\flink-1.0.2\lib\guava-11.0.2.jar;C:\systems\packages\flink-1.0.2\lib\hbase-annotations-1.2.1-tests.jar;C:\systems\packages\flink-1.0.2\lib\hbase-annotations-1.2.1.jar;C:\systems\packages\flink-1.0.2\lib\hbase-client-1.2.1.jar;C:\systems\packages\flink-1.0.2\lib\hbase-common-1.2.1-tests.jar;C:\systems\packages\flink-1.0.2\lib\hbase-common-1.2.1.jar;C:\systems\packages\flink-1.0.2\lib\hbase-examples-1.2.1.jar;C:\systems\packages\flink-1.0.2\lib\hbase-external-blockcache-1.2.1.jar;C:\systems\packages\flink-1.0.2\lib\hbase-hadoop-compat-1.2.1.jar;C:\systems\packages\flink-1.0.2\lib\hbase-hadoop2-compat-1.2.1.jar;C:\systems\packages\flink-1.0.2\lib\hbase-it-1.2.1-tests.jar;C:\systems\packages\flink-1.0.2\lib\hbase-it-1.2.1.jar;C:\systems\packages\flink-1.0.2\lib\hbase-prefix-tree-1.2.1.jar;C:\systems\packages\flink-1.0.2\lib\hbase-procedure-1.2.1.jar;C:\systems\packages\flink-1.0.2\lib\hbase-protocol-1.2.1.jar;C:\systems\packages\flink-1.0.2\lib\hbase-resource-bundle-1.2.1.jar;C:\systems\packages\flink-1.0.2\lib\hbase-rest-1.2.1.jar;C:\systems\packages\flink-1.0.2\lib\hbase-server-1.2.1-tests.jar;C:\systems\packages\flink-1.0.2\lib\hbase-server-1.2.1.jar;C:\systems\packages\flink-1.0.2\lib\hbase-shell-1.2.1.jar;C:\systems\packages\flink-1.0.2\lib\hbase-thrift-1.2.1.jar;C:\systems\packages\flink-1.0.2\lib\log4j-1.2.17.jar;C:\systems\packages\flink-1.0.2\lib\slf4j-log4j12-1.7.7.jar;C:\systems\master_flink\bin;C:\systems\packages\flink-1.0.2\lib;;C:\systems\packages\hbase-1.2.1\lib;C:\systems\hbase\conf;C:\systems\hbase\conf\hbase-site.xml; >> >> 2016-05-11 09:16:45,831 INFO org.apache.zookeeper.ZooKeeper >> - Client >> environment:java.library.path=C:\systems\packages\jre-1.8.0_74_x64\bin;C:\Windows\Sun\Java\bin;C:\Windows\system32;C:\Windows;C:\systems\master_flink\bin;C:\systems\packages\appsync-1.0.6\bin;C:\systems\packages\flink-1.0.2\bin;C:\systems\packages\jre-1.8.0_74_x64\bin;C:\Windows\system32;C:\Windows;C:\Windows\System32\Wbem;C:\Windows\System32\WindowsPowerShell\v1.0\;C:\Program >> Files\Microsoft SQL Server\120\DTS\Binn\;C:\Program Files\Microsoft SQL >> Server\Client SDK\ODBC\110\Tools\Binn\;C:\Program Files (x86)\Microsoft SQL >> Server\120\Tools\Binn\;C:\Program Files\Microsoft SQL >> Server\120\Tools\Binn\;C:\Program Files (x86)\Microsoft SQL >> Server\120\Tools\Binn\ManagementStudio\;C:\Program Files (x86)\Microsoft >> SQL >> Server\120\DTS\Binn\;C:\systems\packages\hadoop-2.7.2\bin;C:\systems\packages\jdk-1.8.0_74_x64\bin;C:\systems\packages\apache-maven-3.3.9\bin;C:\systems\packages\protoc-2.5.0-win32\;C:\systems\packages\cygwin64\bin\;C:\systems\packages\cmake-3.5.2-win32-x86\bin;C:\Program >> Files\Microsoft Windows Performance >> Toolkit\;C:\systems\packages\perl-5.6.0-win\bin;C:\systems\hbase\conf;. >> >> ----- Original meddelelse ----- >> >> *Fra:* Flavio Pompermaier <pomperma...@okkam.it> >> *Til:* user <user@flink.apache.org> >> *Dato:* Ons, 11. maj 2016 00:05 >> *Emne:* Re: HBase write problem >> >> >> Do you have the hbase-site.xml available in the classpath? >> On 10 May 2016 23:10, "Palle" <pa...@sport.dk> wrote: >> >>> HBase write problem >>> >>> Hi all. >>> >>> I have a problem writing to HBase. >>> >>> I am using a slightly modified example of this class to proof the >>> concept: >>> >>> https://github.com/apache/flink/blob/master/flink-batch-connectors/flink-hbase/src/test/java/org/apache/flink/addons/hbase/example/HBaseWriteExample.java >>> >>> However all the HBase-specific stuff is exactly the same as in the >>> HBaseWriteExample. >>> >>> The problem I see is that the job never completes (been running for more >>> than an hour now) and it is only 13 key/value pairs that is to be written >>> to HBase :-) >>> I have tested the map/reduce stuff works if I replace the HBase >>> connection stuff with just a write to a text file - works OK. I have also >>> tested that I can insert data in HBase from a similar Hadoop MapReduce job. >>> >>> Here is the part of the code where I guess the problem is: >>> >>> @Override >>> public Tuple2<Text, Mutation> map(Tuple2<String, Integer> t) >>> throws Exception { >>> LOG.info("Tuple2 map() called"); >>> reuse.f0 = new Text(t.f0); >>> Put put = new Put(t.f0.getBytes()); >>> put.add(MasterConstants.CF_SOME, MasterConstants.COUNT, >>> Bytes.toBytes(t.f1)); >>> reuse.f1 = put; >>> return reuse; >>> } >>> }).output(new HadoopOutputFormat<Text, Mutation>(new >>> TableOutputFormat<Text>(), job)); >>> >>> env.execute("Flink HBase Event Count Hello World Test"); >>> >>> This code matches the code in the HBaseWriteExample.java I should think. >>> >>> The "Tuple2" log line I see exactly the 13 times I expect, and the last >>> log line I see is this: >>> 2016-05-10 21:48:42,715 INFO >>> org.apache.hadoop.hbase.mapreduce.TableOutputFormat - Created >>> table instance for event_type_count >>> >>> Any suggestions to what the problem could be? >>> >>> Thanks, >>> Palle >> >> >> >> > >
