Assuming you will be using hdfs as the file system: wouldn't saving those large objects in the fs and keeping a pointer to them in a hbase table serve the purpose.
[I haven't done it myself but I can't see it not working. In fact, I remember reading it somewhere in the list.] ~Himanshu On Mon, Sep 6, 2010 at 11:40 PM, William Kang <[email protected]>wrote: > Hi JG, > Thanks for your reply. As far as I have read in Hbase's documentation and > wiki, the cell size is not supposed to be larger than 10 MB. For the row, I > am not quite sure, but it looks like 256 MB is the upper limit. I am > considering store some binary data used to be stored in RDBM blob field. > The > size of those binary objects may vary from hundreds of KB to hundreds of > MB. > What would be a good way to use Hbase for it? We really want to use hbase > to > avoid that scaling problem. > Many thanks. > > > William > > On Mon, Sep 6, 2010 at 7:10 PM, Jonathan Gray <[email protected]> wrote: > > > I'm not sure what you mean by "optimized cell size" or whether you're > just > > asking about practical limits? > > > > HBase is generally used with cells in the range of tens of bytes to > > hundreds of kilobytes. However, I have used it with cells that are > several > > megabytes, up to about 50MB. Up at that level, I have seen some weird > > performance issues. > > > > The most important thing is to be sure to tweak all of your settings. If > > you have 20MB cells, you need to be sure to increase the flush size > beyond > > 64MB and the split size beyond 256MB. You also need enough memory to > > support all this large object allocation. > > > > And of course, test test test. That's the easiest way to see if what you > > want to do will work :) > > > > When you run into problems, e-mail the list. > > > > As far as row size is concerned, the only issue is that a row can never > > span multiple regions so a given row can only be in one region and thus > be > > hosted on one server at a time. > > > > JG > > > > > -----Original Message----- > > > From: William Kang [mailto:[email protected]] > > > Sent: Monday, September 06, 2010 1:57 PM > > > To: hbase-user > > > Subject: Limits on HBase > > > > > > Hi folks, > > > I know this question may have been asked many times, but I am wondering > > > if > > > there is any update on the optimized cell size (in megabytes) and row > > > size > > > (in megabytes)? Many thanks. > > > > > > > > > William > > >
