This is still not working as in the XML file the *final* property has been
set as true so that means I cannot override it.
And this below simple query is also throwing same exception-
*SELECT event.app_payload ['n'] AS changed_cguid*
* FROM soj_session_container a LATERAL VIEW explode(a.events) t AS event*
* WHERE a.dt = '20120918'*
* AND unix_timestamp(SojTimestampToDate(event.event_timestamp), 'yyyy/MM/dd
HH:mm:ss') >= unix_timestamp('2012/09/18 00:00:00', 'yyyy/MM/dd HH:mm:ss')*
* AND unix_timestamp(SojTimestampToDate(event.event_timestamp), 'yyyy/MM/dd
HH:mm:ss') <= unix_timestamp('2012/09/18 02:00:00', 'yyyy/MM/dd HH:mm:ss')*
Exception I am getting:-
*Job Submission failed with exception
'org.apache.hadoop.ipc.RemoteException(java.io.IOException:
java.io.IOException: The number of tasks for this job 2070929 exceeds the
configured limit 200000*
* *
Any other suggestion what should I do to overcome this problem? May be any
changes in the query can overcome this problem?
*Raihan Jamal*
On Wed, Oct 3, 2012 at 2:59 PM, Chalcy Raja
<chalcy.r...@careerbuilder.com>wrote:
> Hi Raihan,****
>
> ** **
>
> You can set it in hive prompt like below,****
>
> set mapred.jobtracker.maxtasks.per.job=7777777; ****
>
> ** **
>
> To see if it is set just type in hive prompt, *set; * and you’ll see this
> parameter in the output.****
>
> ** **
>
> Hope this helps,****
>
> Chalcy****
>
> ** **
>
> *From:* Raihan Jamal [mailto:jamalrai...@gmail.com]
> *Sent:* Wednesday, October 03, 2012 5:51 PM
> *To:* user@hive.apache.org
> *Subject:* Re: org.apache.hadoop.ipc.RemoteException(java.io.IOException:
> java.io.IOException****
>
> ** **
>
> Ok. Found the issue I guess.****
>
> ****
>
> This is the below settings we have in the *mapred-site.xml *for the
> site-specific configuration in Hadoop. And that is the reason exception is
> getting thrown.****
>
> ****
>
> <property>****
>
> <!-- 10,000 is 100 tasks per node on a 100-node cluster -->****
>
> <name>mapred.jobtracker.maxtasks.per.job</name>****
>
> <value>200000</value>****
>
> <final>true</final>****
>
> </property>****
>
> ** **
>
> How can I override these changes manually from the Hive prompt? Any
> suggestions?
>
>
>
> *Raihan Jamal*****
>
>
>
> ****
>
> On Wed, Oct 3, 2012 at 2:19 PM, Raihan Jamal <jamalrai...@gmail.com>
> wrote:****
>
> Can anyone help me out here? What does the below error means? And this is
> the query I am using-****
>
> ** **
>
> *SELECT cguid,*****
>
> *event_item,*****
>
> *event_timestamp,*****
>
> *event_site_id*****
>
> *FROM (*****
>
> *SELECT event.app_payload ['n'] AS cguid,*****
>
> *event.app_payload ['itm'] AS event_item,*****
>
> *max(event.event_timestamp) AS event_timestamp,*****
>
> *event.site_id AS event_site_id*****
>
> *FROM soj_session_container a LATERAL VIEW explode(a.events) t AS event***
> **
>
> *WHERE a.dt = '20120917'*****
>
> *AND event.app_payload ['n'] IS NOT NULL*****
>
> *AND instr(event.app_payload ['itm'], '%') = 0*****
>
> *AND event.app_payload ['itm'] IS NOT NULL*****
>
> *AND (*****
>
> *event.page_type_id = '4340'*****
>
> *OR event.page_type_id = '2047675'*****
>
> *)*****
>
> *GROUP BY event.app_payload ['n'],*****
>
> *event.site_id,*****
>
> *event.app_payload ['itm']*****
>
> *ORDER BY cguid,*****
>
> *event_timestamp DESC*****
>
> *) m*****
>
> *LEFT OUTER JOIN (*****
>
> *SELECT event.app_payload ['n'] AS changed_cguid*****
>
> *FROM soj_session_container a LATERAL VIEW explode(a.events) t AS event***
> **
>
> *WHERE a.dt = '20120918'*****
>
> *AND unix_timestamp(SojTimestampToDate(event.event_timestamp),
> 'yyyy/MM/dd HH:mm:ss') >= unix_timestamp('2012/09/18 00:00:00', 'yyyy/MM/dd
> HH:mm:ss')*****
>
> *AND unix_timestamp(SojTimestampToDate(event.event_timestamp),
> 'yyyy/MM/dd HH:mm:ss') <= unix_timestamp('2012/09/18 02:00:00', 'yyyy/MM/dd
> HH:mm:ss')*****
>
> *) n ON m.cguid = n.changed_cguid*****
>
> *WHERE n.changed_cguid IS NULL*****
>
>
>
>
> *Raihan Jamal*****
>
>
>
> ****
>
> On Wed, Oct 3, 2012 at 11:05 AM, Raihan Jamal <jamalrai...@gmail.com>
> wrote:****
>
> I am running a Hive query and I am getting this exception below-****
>
> ** **
>
> *Job Submission failed with exception
> 'org.apache.hadoop.ipc.RemoteException(java.io.IOException:
> java.io.IOException: The number of tasks for this job 2072020 exceeds the
> configured limit 200000*****
>
> ** **
>
> I am not sure what does this error means? Can anyone help me out here?
>
>
>
> *Raihan Jamal*****
>
> ** **
>
> ** **
>
> ** **
>
