Hi William,
because it is the only job that is running I don't think it is resource
contention. We have configured capacity scheduler which means using yarn
queues. As it is the only job I cant see that it is waiting somehow in the
queue.
Br,
Dennis
Von meinem iPhone gesendet
> Am 20.07.20
Dennis, do you know what’s taking the additional time? Is it the Spark Job,
or oozie waiting for allocation from YARN? Do you have resource contention
issue in YARN?
On Fri, Jul 19, 2019 at 12:24 AM Bartek Dobija
wrote:
> Hi Dennis,
>
> Oozie jobs shouldn't take that long in a well configured cl
Hi Dennis,
Oozie jobs shouldn't take that long in a well configured cluster. Oozie
allocates it's own resources in Yarn which may require fine tuning. Check
if YARN gives resources to the Oozie job immediately which may be one of
the reasons and change jobs priorities in YARN scheduling configurat
Dear experts,
I am using Spark for processing data from HDFS (hadoop). These Spark
application are data pipelines, data wrangling and machine learning
applications. Thus Spark submits its job using YARN.
This also works well. For scheduling I am now trying to use Apache Oozie, but I
am facin
