Re: HBase Bulk Lookup

2014-09-22 Thread Bin Wang
Hi Demai, I am talking about point-search, there is no clearly pattern in the key. I assume range-search is something like product id from 001 ~ 999 or something like that. Thanks, Bin On Mon, Sep 22, 2014 at 12:13 PM, Demai Ni wrote: > Bin, > > Are you talking about point-search or range-se

Re: HBase Bulk Lookup

2014-09-22 Thread Demai Ni
Bin, Are you talking about point-search or range-search? While I think HBase is very good at point-search(comparing to other big data store), I have similar questions about range-search. To get a good performance of range-search, I am afraid of the hot-spot issue during bulk load phase, and loo

Re: HBase Bulk Lookup

2014-09-22 Thread Jean-Marc Spaggiari
Hi Bin, 100M rows 1 byte is about 100MB. 100M rows 1KB is about 100GB. What is your record size and what is your SLA? Do you expected 100GB to be transfered in few seconds? How to you query your data? A single get? All of it? You might want to give way more details of your usecases if you want

Re: HBase Bulk Lookup

2014-09-22 Thread Bin Wang
Hi Ted, I have not dive into the programming part yet... I am still at the POC and pick up the right tool stage. Based on your experience, do you think the get(List gets) will return the result from 100M level in an interactive time? say a few seconds? If that is the case, I will start working on

Re: HBase Bulk Lookup

2014-09-22 Thread Ted Yu
bq. upload a list of product ids Have you looked at the following API in HTable ? public Result[] get(List gets) throws IOException { Cheers On Mon, Sep 22, 2014 at 8:14 AM, Bin Wang wrote: > Hi there, > > I have a use case that I need to do bulk look up in a table of size 100 > million key