Re: significant scan performance difference between Thrift(c++) and Java: 4X slower

2015-03-12 Thread Demai Ni
Andrey, thanks. You are right that I am using Thrift v1. I was following example under : hbase-examples/src/main/cpp/DemoClient.cpp. It looks like pretty old, and actually its scan example: > scanner = client.scannerOpenWithStop(t, "00020", "00040", columnNames, > dummyAttributes); > doesn't work

Re: significant scan performance difference between Thrift(c++) and Java: 4X slower

2015-03-09 Thread Andrey Stepachev
Sorry Demai, I have no access to that code currently. But what you described seems that you use thrift v1. I'd recommend to use thrift2. Also it is a good idea to check thrift server configuration: 1. blocking/nonblocking/hsha, and framed or not 2. size of thread pool On Mon, Mar 9, 2015 at 9:

Re: significant scan performance difference between Thrift(c++) and Java: 4X slower

2015-03-09 Thread Demai Ni
Andrey and all, thanks for the input. Andrey, if possible, do you mind share your code segment so I can follow the setting on your side? I have exactly the same thought when face the result first time. I was expecting a little bit performance issue (10~20%) when using Thrift(C++), and not as much

Re: significant scan performance difference between Thrift(c++) and Java: 4X slower

2015-03-08 Thread Andrey Stepachev
Hi Demai. Thats seems odd for me, in my tests I got very similar performance. I'd like to suggest to check that scans have identical parameters (cache size in particular). That can bring very different performance in you case. Thanks. On Sun, Mar 8, 2015 at 6:50 PM, Mike Axiak wrote: > If you'

Re: significant scan performance difference between Thrift(c++) and Java: 4X slower

2015-03-08 Thread Mike Axiak
If you're going the JNI route, the best bet is to embed a VM in your C project. You use "java -s -p" to create the required header files and compile linking against the java library. This article talks about how to talk from C to Java: http://www.codeproject.com/Articles/22881/How-to-Call-Java-Fun

Re: significant scan performance difference between Thrift(c++) and Java: 4X slower

2015-03-08 Thread Michael Segel
JNI example? I don’t have one… my client’s own the code so I can’t take it with me and share. (The joys of being a consultant means you can’t take it with you and you need to make sure you don’t xfer IP accidentally. ) Maybe in one of the HBase books? Or just google for a JNI example on the

Re: significant scan performance difference between Thrift(c++) and Java: 4X slower

2015-03-07 Thread Demai Ni
Nick, thanks. I will give REST a try. However, if it use the same design, the result probably will be the same. Michael, I was thinking about the same thing through JNI. Is there an example I can follow? Mike (Axiak), I run the C++ client on the same linux machine as the hbase and thrift. The HBa

Re: significant scan performance difference between Thrift(c++) and Java: 4X slower

2015-03-07 Thread Mike Axiak
What if you install the thrift server locally on every C++ client machine? I'd imagine performance should be similar to native java performance at that point. -Mike On Sat, Mar 7, 2015 at 4:49 PM, Michael Segel wrote: > Or you could try a java connection wrapped by JNI so you can call it from >

Re: significant scan performance difference between Thrift(c++) and Java: 4X slower

2015-03-07 Thread Michael Segel
Or you could try a java connection wrapped by JNI so you can call it from your C++ app. > On Mar 7, 2015, at 1:00 PM, Nick Dimiduk wrote: > > You can try the REST gateway, though it has the same basic architecture as > the thrift gateway. May be the details work out in your favor over rest. >

Re: significant scan performance difference between Thrift(c++) and Java: 4X slower

2015-03-07 Thread Nick Dimiduk
You can try the REST gateway, though it has the same basic architecture as the thrift gateway. May be the details work out in your favor over rest. On Fri, Mar 6, 2015 at 11:31 PM, nidmgg wrote: > Stack, > > Thanks for the quick response. Well, the extra layer really kill the > Performance. The

Re: significant scan performance difference between Thrift(c++) and Java: 4X slower

2015-03-06 Thread nidmgg
Stack, Thanks for the quick response. Well, the extra layer really kill the Performance. The 'hop' is so expensive Is there another C/C++ api to try out? I saw there is a jira Hbase-1015, but was inactive for a while. Demai Stack wrote: >Is it because of the 'hop'?  Java goes against RS.

Re: significant scan performance difference between Thrift(c++) and Java: 4X slower

2015-03-06 Thread Stack
Is it because of the 'hop'? Java goes against RS. The thrift C++ goes to a thriftserver which hosts a java client and then it goes to the RS? St.Ack On Fri, Mar 6, 2015 at 4:46 PM, Demai Ni wrote: > hi, guys, > > I am trying to get a rough idea about the performance comparison between > c++ and

significant scan performance difference between Thrift(c++) and Java: 4X slower

2015-03-06 Thread Demai Ni
hi, guys, I am trying to get a rough idea about the performance comparison between c++ and java client when access HBase table, and is surprised to find out that Thrift (c++) is 4X slower The performance result is: C++: real*16m11.313s*; user5m3.642s; sys2m21.388s Java: real*4m6.