Re: CDH3 U1 Hive Job-commit very slow

2011-08-11 Thread air
cleus.store.rdbms.RDBMSStoreManager$ClassAdder.performTablesValidation(RDBMSStoreManager.java:2702) > at > org.datanucleus.store.rdbms.RDBMSStoreManager$ClassAdder.addClassTablesAndValidate(RDBMSStoreManager.java:2503) > at > org.datanucleus.store.rdbms.RDBMSStoreManager$ClassA

Re: CDH3 U1 Hive Job-commit very slow

2011-08-10 Thread air
it would submit the map reduce job. > > ** ** > > *From:* air [mailto:cnwe...@gmail.com] > *Sent:* Wednesday, August 10, 2011 3:40 AM > > *To:* user@hive.apache.org > *Subject:* Re: CDH3 U1 Hive Job-commit very slow > > ** ** > > there is only 10186 par

RE: CDH3 U1 Hive Job-commit very slow

2011-08-10 Thread Aggarwal, Vaibhav
] Sent: Wednesday, August 10, 2011 3:40 AM To: user@hive.apache.org Subject: Re: CDH3 U1 Hive Job-commit very slow there is only 10186 partitions in the metadata store (select count(1) from PARTITIONS; in mysql), I think it is not the problem. 2011/8/10 Aggarwal, Vaibhav mailto:vagg...@amazon.com

Re: CDH3 U1 Hive Job-commit very slow

2011-08-10 Thread air
there is only 10186 partitions in the metadata store (select count(1) from PARTITIONS; in mysql), I think it is not the problem. 2011/8/10 Aggarwal, Vaibhav > Do you have a lot of partitions in your table? > > Time taken to process the partitions before submitting the job is > proportional t

RE: CDH3 U1 Hive Job-commit very slow

2011-08-09 Thread Steven Wong
You can tail the Hive log and see what it is doing at the time. From: air [mailto:cnwe...@gmail.com] Sent: Tuesday, August 09, 2011 1:19 AM To: user@hive.apache.org Subject: Fwd: CDH3 U1 Hive Job-commit very slow -- Forwarded message -- From: air mailto:cnwe...@gmail.com>> Date:

RE: CDH3 U1 Hive Job-commit very slow

2011-08-09 Thread Aggarwal, Vaibhav
Do you have a lot of partitions in your table? Time taken to process the partitions before submitting the job is proportional to number of partitions. There is a patch I submitted recently as an attempt to alleviate this problem: https://issues.apache.org/jira/browse/HIVE-2299 If that is not th