make it slower. Just to get an idea, if you skip
> doing the Gets, how fast does it became?
>
> J-D
>
>
> On Fri, Sep 27, 2013 at 10:33 AM, Pavan Sudheendra >wrote:
>
> > Hi everyone,
> > I posted this question many time before and i've given full details o
Hi everyone,
I posted this question many time before and i've given full details on
stackoverflow..
http://stackoverflow.com/q/19056712/938959
Please i need someone to guide me in the right direction here.
Help much appreciated!
--
Regards-
Pavan
Hi all,
How do i scan all the rows of HBase with only the Column Family?
Coumn Family --> cf
Column Qualifier --> \x00\x00\x06T,\x00\x00\x05d,\x00\x00\x00\x00 etc.,
Column Qualifier would be random so i won't know beforehand..Any idea of
how i can do this in Java API?
--
Regards-
Pavan
Sep 3, 2013 at 12:33 PM, Anoop John wrote:
> You are using Scan caching in ur MR java code? How many mapper and
> reducers in case of pig? How is ur Java MR job written.. a bit more on
> its logic pls.
>
> -Anoop-
>
> On Tue, Sep 3, 2013 at 11:33 AM, Pavan Sudheendra >wrote
Hi all,
I'm doing a kind of table join across 3 tables in the MR job ( plus doing
some computation).. It took nearly 19 hours to run with 21 mappers and 21
reducers.. But with pig it ran in less than 2 hours..
We are using HBase both as source and sink.. Is this normal?
On Mon, Sep 2, 2013 at 7:2
Hi all,
I have a question which is bugging me for more than a week.
I'm doing some computation across 3 tables in hbase where 1 table is around
25m rows, 2nd table 5m rows and 3rd table 1m rows..
My Java MR job takes a long time to execute.. (in terms of hours) but a pig
script does the same task
Async Operation.
>
>
> On Wed, Aug 28, 2013 at 3:41 PM, Pavan Sudheendra >wrote:
>
> > Hi all,
> > I know what we can go over to the HBase UI and make a split on our table
> so
> > that it will be distributed over the cluster.. Is there a way to know it
> >
Hi all,
I know what we can go over to the HBase UI and make a split on our table so
that it will be distributed over the cluster.. Is there a way to know it
via an API and to possibly change it? This is to know how many map tasks
run on our table before we actually run the MR job..
--
Regards-
Pav
of the ScannerTimeoutException that
> would arise due to very high caching values. You might want to increase the
> scanner timeout value in that case.
>
>
>
> On Mon, Aug 26, 2013 at 2:42 PM, Pavan Sudheendra wrote:
>
>> Hi Ashwanth, thanks for the reply..
>>
>&g
Further more, what can we do if a table has 25 online regions? Can we
safely set caching to a bigger number? Is a split necessary as well?
On Mon, Aug 26, 2013 at 2:42 PM, Pavan Sudheendra wrote:
> Hi Ashwanth, thanks for the reply..
>
> I went to the HBase Web UI and saw that my ta
with that 1500 value.
>
> Word of caution, if you increase it too much, you might see
> ScannerTimeoutException in your TT Logs.
>
>
> On Mon, Aug 26, 2013 at 2:29 PM, Pavan Sudheendra wrote:
>
>> Hi Ashwanth,
>> My caching is set to 1500 ..
>>
>> scan.se
nitiate a manual
> split on the table.
>
> But, before you do that. May be you can try increasing your client caching
> value (hbase.client.scanner.caching) in your Job.
>
>
> On Mon, Aug 26, 2013 at 2:09 PM, Pavan Sudheendra >wrote:
>
> > What is the input s
What is the input split of the HBase Table in this job status?
map() completion: 0.0
reduce() completion: 0.0
Counters: 24
File System Counters
FILE: Number of bytes read=0
FILE: Number of bytes written=216030
FILE: Number of read operations=
Hi all,
How to make use of a TableSplit or a Region Split? How is it used in
TableInputFormatBase#
getSplits() ?
I have 6 Region Servers across the cluster for the map-reduce task which i
am using, How to leverage this so that the table is split across the
clusters and the map-reduce application
source?
>
> On Thu, Aug 22, 2013 at 6:30 PM, Pavan Sudheendra
> wrote:
> > All the zookeeper warnings are coming from takstracker logs.
> >
> >
> > On Thu, Aug 22, 2013 at 5:10 PM, Jean-Marc Spaggiari <
> > jean-m...@spaggiari.org> wrote:
> >
>
HBase are you using ?
>
> Was hbase-site.xml in the classpath of your reducer ?
>
> Cheers
>
>
> On Sat, Aug 24, 2013 at 6:07 PM, Pavan Sudheendra >wrote:
>
> > This was the Namenode log generated while running MR applications.. What
> do
> > they indicate
This was the Namenode log generated while running MR applications.. What do
they indicate where the problem is?
2013-08-25 00:53:11,840 ERROR
>> org.apache.hadoop.hbase.zookeeper.RecoverableZooKeeper: ZooKeeper exists
>> failed after 3 retries
>> 2013-08-25 00:53:11,841 ERROR
>> org.apach
FYI i'm here to just getting other views on how much would it run in their
system compared to mine?
because just to process 600,000 map input records in an hour is just
wrong.. And it doesn't even show any map % increase.. Its at 0% throughout.
On Thu, Aug 22, 2013 at 9:18 PM, Pavan
>
> See my response in your other thread...
>
> On Aug 22, 2013, at 10:41 AM, Pavan Sudheendra
> wrote:
>
> > scan.setCaching(500);
> >
> > I really don't understand this purpose though..
> >
> >
> > On Thu, Aug 22, 2013 at 9:09 PM, Kevin O&#x
Hmmm. I'm not sure about this.. How do i check Jean?
On Thu, Aug 22, 2013 at 9:12 PM, Jean-Marc Spaggiari <
jean-m...@spaggiari.org> wrote:
> And size of the rows... can you load the 1m rows table in memory?
> Le 2013-08-22 11:41, "Pavan Sudheendra" a écrit
scan.setCaching(500);
I really don't understand this purpose though..
On Thu, Aug 22, 2013 at 9:09 PM, Kevin O'dell wrote:
> QQ what is your caching set to?
> On Aug 22, 2013 11:25 AM, "Pavan Sudheendra" wrote:
>
> > Hi all,
> >
> > A serious que
Hi all,
A serious question.. I know this isn't one of the best hbase practices but
I really want to know..
I am doing a join across 3 table in hbase.. One table contain 19m records,
one contains 2m and another contains 1m records.
I'm doing this inside the mapper function.. I know this can be do
M/R. It all depends on your
> requirements and then accordingly designing your set of jobs (application).
> This might require some creative thinking at your part.
>
> These are just my 2 cents.
>
> Regards,
> Shahab
>
>
> On Mon, Aug 19, 2013 at 10:22 AM, Pavan Sudhe
And just to be clear, sorry if this is a dumb question.. after updating the
/etc/hosts file are we supposed to restart hbase?
On Thu, Aug 22, 2013 at 8:03 PM, Pavan Sudheendra wrote:
> Isn't hbase.zookeeper.quorum suppose to contain only the address of the
> HBase master instead
Isn't hbase.zookeeper.quorum suppose to contain only the address of the
HBase master instead of all the region servers?
On Thu, Aug 22, 2013 at 8:01 PM, Pavan Sudheendra wrote:
> Vamshi and Jay .. Can you both share your /etc/hosts file?
>
> I have the exact same problem .. A
Vamshi and Jay .. Can you both share your /etc/hosts file?
I have the exact same problem .. All my namenode cluster just log this
connection refused when they are to log something useful for de-bugging..
But for me HBase region server tries to connect to localhost when i want it
to connect it to i
socket connection to server localhost/127.0.0.1:2181. Will not attempt to
> authenticate using SASL (Unable to locate a login configuration)
> java.net.ConnectException: Connection refused
>
>
> 2013/8/21 Pavan Sudheendra
>
> > Sorry what files are you talking about?
> &g
Sorry what files are you talking about?
Regards,
Pavan
On Aug 22, 2013 12:04 AM, "Jean-Marc Spaggiari"
wrote:
> TaskTrackers and Job trackers are MR nodes. You also have HDFS nodes and
> HBase nodes.
>
> What's the file name where you got that from?
>
>
to the other one using the name only, if
> it works, they you don't need to update the hosts file.
>
> JM
>
> 2013/8/21 Pavan Sudheendra
>
> > I should update the /etc/hosts file on every namenode correct?
> >
> >
> >
> >
> > On Wed, Aug
ote:
> I'm running with this INFO for more than a year now ;) So no, I don't think
> this is going to pose any real threats. You have everything configured
> correctly and everything seems to be working fine.
>
> JM
>
> 2013/8/21 Pavan Sudheendra
>
> > it doesn&
I should update the /etc/hosts file on every namenode correct?
On Wed, Aug 21, 2013 at 7:09 PM, Pavan Sudheendra wrote:
> But Jean all my namenodes log the same thing..
> 2013-08-21 13:38:55,815 INFO org.apache.zookeeper.ClientCnxn: Opening
> socket connection to server localhost/
orrect?
>
> I think you should simply ignore it.
>
> If it's annoying you, just turn org.apache.zookeeper.ClientCnxn loglevel to
> WARN on log4j.properties. (It's the setting I have on my own cluster).
>
> JM
>
> 2013/8/21 Pavan Sudheendra
>
> > @J
21, 2013 at 6:23 PM, Pavan Sudheendra wrote:
> Yes .. The zookeeper server is also 10.34.187.170 ..
>
>
> On Wed, Aug 21, 2013 at 6:21 PM, Jean-Marc Spaggiari <
> jean-m...@spaggiari.org> wrote:
>
>> Are you able to connect to your ZK server shell and list th
Yes .. The zookeeper server is also 10.34.187.170 ..
On Wed, Aug 21, 2013 at 6:21 PM, Jean-Marc Spaggiari <
jean-m...@spaggiari.org> wrote:
> Are you able to connect to your ZK server shell and list the nodes?
>
> 2013/8/21 Pavan Sudheendra
>
> > Yes.. I can do everyt
-Marc Spaggiari <
jean-m...@spaggiari.org> wrote:
> Sound correct. You are able to start the shell and scan the few first line
> of the tables, right?
>
> 2013/8/21 Pavan Sudheendra
>
> > This is my hbase-site.xml file if it helps:
> >
> >
>
_NAME"));
>
> And see if it works?
>
> JM
>
> 2013/8/21 Pavan Sudheendra
>
> > Is this a zookeeper specific error or something?
> >
> >
> > On Wed, Aug 21, 2013 at 6:06 PM, Pavan Sudheendra > >wrote:
> >
> > > Hi Jean,
> > &g
Is this a zookeeper specific error or something?
On Wed, Aug 21, 2013 at 6:06 PM, Pavan Sudheendra wrote:
> Hi Jean,
>
> ubuntu@ip-10-34-187-170:~$ cat /etc/hostname
> ip-10-34-187-170
> ubuntu@ip-10-34-187-170:~$ hostname
> ip-10-34-187-170
>
>
>
> On Wed, Aug
e
>
> ?
>
> 2013/8/21 Pavan Sudheendra
>
> > Sure..
> > /etc/hosts file:
> >
> > 127.0.0.1 localhost
> > 10.34.187.170 ip-10-34-187-170
> > # The following lines are desirable for IPv6 capable hosts
> > ::1 ip6-localhost ip6-loopback
> >
Jean-Marc Spaggiari <
jean-m...@spaggiari.org> wrote:
> Can you past you host file here again with the modification you have done?
>
> Also, can you share a big more of you code? What are you doing with the
> config object after, how do you create your table object, etc.?
>
ng there, then you most probably have a
> DHCP. Just give a try to what I told you on the other message.
>
> JM
>
> 2013/8/21 Pavan Sudheendra
>
> > @Manoj i have set hbase.zookeeper.quorum in my M-R application..
> >
> > @Jean The cloudera manager picks up t
> If this does'nt work either please check if HBASE_HOME and HBASE_CONF_DIR
> is set correctly.
>
> BR/Manoj
>
>
> On Wed, Aug 21, 2013 at 4:48 PM, Pavan Sudheendra >wrote:
>
> > Yes. My /etc/hosts have the correct mapping to localhost
> >
> &g
localhost
> > for 127.0.0.1. Also ensure that if you have hbase.zookeeper.quorum in
> your
> > configuration and also check if HBase classpath is appended to Hadoop
> > classpath.
> >
> >
> > BR/Manoj
> >
> >
> > On Wed, Aug 21, 2013 at 4:10 PM
Hadoop Namenode reports the following error which is unusual :
013-08-21 09:21:12,328 INFO org.apache.zookeeper.ClientCnxn: Opening socket
connection to server localhost/127.0.0.1:2181. Will not attempt to
authenticate using SASL (Unable to locate a login configuration)
java.net.ConnectException:
> > create the HTable object again and again in the mapper? Where is
> > > ContentidxTable
> > > (the name of the table, I believe?) defined? What is your actually
> > > requirement?
> > >
> > > Also, have you looked into this, the api for wiri
e is
> ContentidxTable
> (the name of the table, I believe?) defined? What is your actually
> requirement?
>
> Also, have you looked into this, the api for wiring HBase tables with M/R
> jobs?
> http://hbase.apache.org/book/mapreduce.example.html
>
> Regards,
> Shahab
>
Also, the same code works perfectly fine when i run it in single node
cluster. I've added the hbase classpath to HADOOP_CLASSPATH and have set
all the other env variables also..
On Mon, Aug 19, 2013 at 6:33 PM, Pavan Sudheendra wrote:
> Hi all,
> I'm getting the followin
Hi all,
I'm getting the following error messages everytime i run the map-reduce job
across multiple hadoop clusters:
java.lang.NullPointerException
at org.apache.hadoop.hbase.util.Bytes.toBytes(Bytes.java:414)
at org.apache.hadoop.hbase.client.HTable.(HTable.java:170)
at com.company$Analyz
I am using hbase with hadoop and i haven't seen this error until now..
I just switched to cloudera and now this happened.
java.io.IOException: java.lang.RuntimeException: java.io.IOException:
No such file or directory
at
org.apache.hadoop.hbase.mapreduce.TableMapReduceUtil.findOrCreateJar(Ta
@Jonathan Ted Yu is right! Ignore my mail :)
On Thu, Aug 1, 2013 at 10:46 PM, Ted Yu wrote:
> Take a look at ColumnPaginationFilter.java and its unit test.
>
> Cheers
>
> On Thu, Aug 1, 2013 at 10:01 AM, Jonathan Cardoso
> wrote:
>
>> Hi!
>>
>> Is there a way to scan a HBase table getting, for ex
Use
http://hbase.apache.org/apidocs/org/apache/hadoop/hbase/client/Scan.html#getStartRow()
and
http://hbase.apache.org/apidocs/org/apache/hadoop/hbase/client/Scan.html#getStopRow()
i=1; //start row
j=100; //end row
while(i wrote:
> Hi!
>
> Is there a way to scan a HBase table getting, for examp
tion is atomicity and consistency of the two tables.
>
> Regards,
> Shahab
>
>
> On Wed, Jul 31, 2013 at 9:37 AM, Pavan Sudheendra >wrote:
>
> > Yes perfect. Could i get few examples on how to create a secondary lookup
> > table?
> >
> >
> > On
your table into a secondary index lookup
> table where the value will be the key, and the key will be the value. That
> way you just to a get on this secondary index lookup table instead of doing
> a scan...
>
> Make sense?
>
> JM
>
> 2013/7/31 Pavan Sudheendra
>
&
docs/org/apache/hadoop/hbase/filter/FamilyFilter.html
> .
>
> Basically, you have all the filters listed there:
> http://hbase.apache.org/apidocs/org/apache/hadoop/hbase/filter/package-summary.html
>
> You should be able to find one which match your needs.
>
> JM
>
> 2013
secondary index table for direct access...
>
> JM
>
> 2013/7/31 Pavan Sudheendra
>
>> How do i get the row key by specifying a row value assuming row values
>> are unique?
>>
>> What API should i use for this purpose?
>> --
>> Regards-
>> Pavan
>>
--
Regards-
Pavan
How do i get the row key by specifying a row value assuming row values
are unique?
What API should i use for this purpose?
--
Regards-
Pavan
Just make sure JAVA_HOME is set in the current shell. Also, check jps after
you start HBase.
It should indicate a process HMaster as running.
Regards,
Pavan
On Jul 24, 2013 8:41 PM, "Ted Yu" wrote:
> Did you install HBase in distributed mode ?
> Which release do you use ?
>
> Are you able to see
your own M/R job (assuming that
> you don't have complex or custom requirements.)
>
> Regards,
> Shahab
>
>
> On Fri, Jul 19, 2013 at 12:24 AM, Pavan Sudheendra wrote:
>
>> Hi,
>>
>> I know that HBase by default doesn't support table joins like
Hi,
I know that HBase by default doesn't support table joins like RDBMS..
But anyway, I have a table who value contains a json with a particular
ID in it..
This id references another table where it is a key..
I want to fetch the id first from table A , query table 2 and get its
corresponding valu
you have to do to test the standalone version. Don't
> touch any config file, don't modify anything.
>
> JM
>
> 2013/7/16 Pavan Sudheendra
>
>> Jean, I'm getting a whole bunch of errors if i try your method!
>> I don't why i'm the only one w
:0 overruns:0 frame:0
> TX packets:237 errors:0 dropped:0 overruns:0 carrier:0
> collisions:0 lg file transmission:1000
> RX bytes:43079 (42.0 KiB) TX bytes:32650 (31.8 KiB)
> Interruption:40 Adresse de base:0xa000
>
> And I do everything u
at your /etc/hosts file says
>>
>> On Tuesday, July 16, 2013, Pavan Sudheendra wrote:
>>
>> > Ah. The value of quorun should be localhost or my ip address?
>> >
>> > On Tue, Jul 16, 2013 at 5:32 PM, Rohit Kelkar
>> > wrote:
>> > &g
, July 16, 2013, Pavan Sudheendra wrote:
>
>> Ah. The value of quorun should be localhost or my ip address?
>>
>> On Tue, Jul 16, 2013 at 5:32 PM, Rohit Kelkar
>> wrote:
>> > I suggested that because you were able to use the shell but not the
>> client
>>
Ah. The value of quorun should be localhost or my ip address?
On Tue, Jul 16, 2013 at 5:32 PM, Rohit Kelkar wrote:
> I suggested that because you were able to use the shell but not the client
> code.
>
> - R
>
> On Tuesday, July 16, 2013, Pavan Sudheendra wrote:
>
>> Y
rc Spaggiari wrote:
>
>> Hi Pavan,
>>
>> You should try to avoid localhost. Try to prefers your host name. Is
>> http://ubuntu:60010/master-status?filter=all<
>> http://localhost:60010/master-status?filter=all>working
>> fine?
>>
>> JM
&g
t;:-1},{"statustimems":-1,"status":"Waiting
for a call","starttimems":1373948304755,"description":"REPL IPC Server
handler 0 on 38039","state":"WAITING","statetimems":-1},
all the rpc calls gets stuck on wa
@Kim how is that possible? I'm using the stable release.
On Tue, Jul 16, 2013 at 8:37 AM, Pavan Sudheendra wrote:
> @Jean, Nope, it didn't work. I even re-installed hbase.
>
> Has anyone worked with the Hbase-The definitive guide examples with
> standalone mode?
>
> O
an-m...@spaggiari.org> wrote:
>
>> You should change our hosts file to something like that:
>>
>> 127.0.0.1 localhost
>> 192.168.XXX.YYY ubuntu.ubuntu-domain ubuntu
>>
>> Where 192.168.XXX.YYY is your local host IP address
>>
>> Can you change
The fact of the matter is i'm able to anything in the shell. It works
fine, but i'm not able to execute normal CRUD operations with the JAVA
API programatically.
On Mon, Jul 15, 2013 at 5:48 PM, Pavan Sudheendra wrote:
> Hi Jean,
> Here is my /etc/hosts/
>
> 127.0.0.1
owser to see if your master is
> running;
> 3) What is into your log file?
>
> JM
>
> 2013/7/15 Pavan Sudheendra
>
>> I've been trying to run examples from HBase-The definitve guide and
>> i've been encountering with this error and i'm not able to get pa
I've been trying to run examples from HBase-The definitve guide and
i've been encountering with this error and i'm not able to get past
it. I'm running in Stand alone mode if that helps.
Exception in thread "main" org.apache.hadoop.hbase.MasterNotRunningException: �
17136@ubuntulocalhost,32992,13
70 matches
Mail list logo
