Pig does not depends on MR historyserver directly, but MR framework does. Pig needs to pull counter/diagnose information for each MR job it submits from AM, but if AM dies, MR client will redirect the request to history server. Note in Pig 0.13 and prior, there is a bug which keep Pig trying history server for a long time (http://issues.apache.org/jira/browse/PIG-4164).
Daniel On 1/27/15, 6:07 PM, "canan chen" <[email protected]> wrote: >Has anyone experience this ? Thanks > >On Fri, Jan 16, 2015 at 7:28 PM, canan chen <[email protected]> wrote: > >> I try to run pig on mapreduce, but would meet the following error if I >> didn't start mr history server. >> But I can run hive on mapreduce without MR historyserver, not sure why >>pig >> on mapreduce on MR historyserver. As my understanding historyserver is >>used >> for recovery, so it should be OK without historyserver, right ? >> >> >> 2015-01-16 18:38:48,642 [main] INFO >> org.apache.hadoop.yarn.client.api.impl.TimelineClientImpl - Timeline >> service address: http://0.0.0.0:8188/ws/v1/timeline/ >> 2015-01-16 18:38:48,643 [main] INFO >>org.apache.hadoop.yarn.client.RMProxy >> - Connecting to ResourceManager at /0.0.0.0:8032 >> 2015-01-16 18:38:48,650 [main] INFO >> org.apache.hadoop.mapred.ClientServiceDelegate - Application state is >> completed. FinalApplicationStatus=SUCCEEDED. Redirecting to job history >> server >> 2015-01-16 18:38:49,653 [main] INFO org.apache.hadoop.ipc.Client - >> Retrying connect to server: localhost/0.0.0.0:10020. Already tried 0 >> time(s); retry policy is >>RetryUpToMaximumCountWithFixedSleep(maxRetries=10, >> sleepTime=1000 MILLISECONDS) >> 2015-01-16 18:38:50,656 [main] INFO org.apache.hadoop.ipc.Client - >> Retrying connect to server: localhost/0.0.0.0:10020. Already tried 1 >> time(s); retry policy is >>RetryUpToMaximumCountWithFixedSleep(maxRetries=10, >> sleepTime=1000 MILLISECONDS) >> 2015-01-16 18:38:51,658 [main] INFO org.apache.hadoop.ipc.Client - >> Retrying connect to server: localhost/0.0.0.0:10020. Already tried 2 >> time(s); retry policy is >>RetryUpToMaximumCountWithFixedSleep(maxRetries=10, >> sleepTime=1000 MILLISECONDS) >> 2015-01-16 18:38:52,661 [main] INFO org.apache.hadoop.ipc.Client - >> Retrying connect to server: localhost/0.0.0.0:10020. Already tried 3 >> time(s); retry policy is >>RetryUpToMaximumCountWithFixedSleep(maxRetries=10, >> sleepTime=1000 MILLISECONDS) >> >> >> >>
