Re: hbase (coprocessors & cell tags) used in hadoop-yarn

2016-01-04 Thread ramkrishna vasudevan
I saw the patches some time back but got lost in other work. You are creating the new cells with Tags inside the Coprocessors? Do you see any need for introducing Tags to be added directly from the client side as part of Puts (for your usecase)? Currently HBase does not support Tags on the client s

Re: What is the max size of row value?

2016-01-04 Thread Ted Yu
See the following in hbase-default.xml : hbase.client.keyvalue.maxsize 10485760 Specifies the combined maximum allowed size of a KeyValue instance. FYI On Mon, Jan 4, 2016 at 4:34 PM, Keith Lim wrote: > I am using an c# API to write to HBASE, is there a max value for row val

What is the max size of row value?

2016-01-04 Thread Keith Lim
I am using an c# API to write to HBASE, is there a max value for row value? It seems to be able to insert a max blob size of 74 bytes and I am wondering if this is configurable somewhere? Is there a default size setting for the row value in HBASE? Thanks, Keith

Re: Hbase master not starting

2016-01-04 Thread Shane O'Donnell
Thanks for this, Dejan! Where did you have your tables to move them in? I know the names of all of my tables, but I'm not sure how I'd go about "copying my table to the new Hbase location". Can you explain? Shane O. Shane O'Donnell 4819 Em

Re: hbase (coprocessors & cell tags) used in hadoop-yarn

2016-01-04 Thread Vrushali Channapattan
I see, thanks Anoop. We wanted to use cell tags for indicating the context of information in cells in that cells for aggregation purpose. It is referred to only in the coprocessor. We also use in the flush/compaction processing to decide which cells to discard/what info to keep. I will be on the l

Re: Hbase master not starting

2016-01-04 Thread Shane O'Donnell
And the error was (no surprise) not finding 'namespace': 2016-01-04 16:28:45,323 INFO [PriorityRpcServer.handler=7,queue=1,port=60020] regionserver.RSRpcServices: Open hbase:namespace,,1451917103275.82432aca9ede964943b40753cb64e808. 2016-01-04 16:28:45,326 ERROR [RS_OPEN_REGION-ip-10-0-1-151:6002

Re: Hbase master not starting

2016-01-04 Thread Shane O'Donnell
I looked for region 82432aca9ede964943b40753cb64e808 on each of my region servers and none of them had it. They all had identical failures in the log of attempting to open it, but none had it (or at least were successful in opening it). My solution to "finding it" was grepping for "82432aca9ede96

Re: Hbase master not starting

2016-01-04 Thread Shane O'Donnell
It's not there. The directory listing was for the right directory, but the namespace directory is not there. Here it is one from directory level up: [user@hbase-prod2-master (ip-10-0-1-165) ~]$ sudo -u hdfs hdfs dfs -ls -R /hbase/data drwxr-xr-x - hdfs hadoop 0 2016-01-04 14:47 /hbas

Re: Hbase master not starting

2016-01-04 Thread Ted Yu
In your listing, meta table directory structure was shown. Please look under /hbase/data/hbase for namespace table. Cheers On Mon, Jan 4, 2016 at 8:42 AM, Shane O'Donnell wrote: > I attempted to restore a backed-up "meta" directory from a copy made by > 'hbase hbck', so my directory structure

Re: Hbase master not starting

2016-01-04 Thread Shane O'Donnell
I attempted to restore a backed-up "meta" directory from a copy made by 'hbase hbck', so my directory structure may be messed up: [user@hbase-prod2-master (ip-10-0-1-165) ~]$ sudo -u hdfs hdfs dfs -ls -R /hbase/data/hbase drwxr-xr-x - hbase hadoop 0 2016-01-04 14:48 /hbase/data/hbase/m

Re: issue in migration

2016-01-04 Thread Ted Yu
Please see http://hbase.apache.org/book.html#upgrade1.0.from.0.94 On Mon, Jan 4, 2016 at 6:10 AM, Parmeet Arora wrote: > Hii, > How to migrate data in hbase-0.94.8 to hbase-1.1.0? With hbase-0.94.8,I > was using Hadoop-1.x and with hbase-1.1.0 ,I was using hadoop2.x,so how I > should migrate dat

Re: Hbase master not starting

2016-01-04 Thread Dejan Menges
I had once exactly this situation, and was able (but that was some development database, so wasn't super production big deal) to get over it with: - Stop HBase - Copy your table to your new HBase location - Start your HBase - Run HBCK until it says all is good and you are sure you have your data -

Re: Hbase master not starting

2016-01-04 Thread Ted Yu
Can you log onto the server hosting region 82432aca9ede964943b40753cb64e808 and see what happened ? See if the namespace table can be found under rootdir. e.g. assuming /apps/hbase/data is the rootdir, you should see something similar to the following: hdfs dfs -ls /apps/hbase/data/data/hbase/nam

Re: Hbase master not starting

2016-01-04 Thread Shane O'Donnell
Some progress... /hbase did NOT have either the hbase.id or hbase.version files so I temporarily changed hbase.rootdir and started the master so the files would be recreated elsewhere and copied them in. Now it starts fine, but my hbase tables are gone. Specifically, I'm getting this error: hba

RE: issue in migration

2016-01-04 Thread Parmeet Arora
Hii, How to migrate data in hbase-0.94.8 to hbase-1.1.0? With hbase-0.94.8,I was using Hadoop-1.x and with hbase-1.1.0 ,I was using hadoop2.x,so how I should migrate data in hbase-0.94.8 to hbase-1.1.0? -Original Message- From: Ted Yu [mailto:yuzhih...@gmail.com] Sent: Monday, January 0

Re: issue in fuzzy filter

2016-01-04 Thread Ted Yu
Can you pastebin the complete error you encountered ? What dependencies have you added ? Thanks > On Jan 4, 2016, at 3:44 AM, Rajeshkumar J wrote: > > Hi, > > We can use fuzzy row filter when rowkey has fixed length.I decided to > design row key as -mm-dd| platform id | control id where pl

issue in fuzzy filter

2016-01-04 Thread Rajeshkumar J
Hi, We can use fuzzy row filter when rowkey has fixed length.I decided to design row key as -mm-dd| platform id | control id where platform id and control id are a two digit nos. we have combo box through which we can select differen values of date, platform and control for search. For this w

Re: Type of filter to use

2016-01-04 Thread Rajeshkumar J
Hi Tariq, I have tried rowfilter both with substring and regex filter it is very slow but i need data retrieval to be less than 3 seconds. Thanks On Mon, Jan 4, 2016 at 4:33 PM, Mohammad Tariq wrote: > Hi Rajesh, > > To begin with I would suggest you to rethink about your key design. If y

Re: Type of filter to use

2016-01-04 Thread Mohammad Tariq
Hi Rajesh, To begin with I would suggest you to rethink about your key design. If you are getting a lot of data(which I assume you do) then this type of keys could lead you to region hotspotting . Coming back to your original question you could a

Re: Type of filter to use

2016-01-04 Thread Rajeshkumar J
Hi, For the above query i decided to use fuzzy rowfilter as of now. we can use fuzzy row filter when rowkey has fixed length. For the above query i decided to design row key as -mm-dd| platform id | control id where platform id and control id are a two digit nos. So now actions can be perfor

Re: MultiRowRangeFilter in hbase shell

2016-01-04 Thread Ted Yu
You can find the usage in the unit test at the end of the patch: https://issues.apache.org/jira/secure/attachment/12691923/HBASE_11144_V18.patch Cheers > On Jan 3, 2016, at 9:32 PM, Rajeshkumar J wrote: > > Hi Ted, > > Also I do want to scan multiple row key ranges and I came across this >