Hi Batch interval is 5min. I actually managed to fix the issue by turning off dynamic allocation and the external shuffle service.
This seems to have helped and now the scheduling delay is between 0-5ms and processing time is about 2.8min which is lower than my batch interval. I also noticed that enabling dynamic allocation and the external shuffle service had a high impact on cpu usage. Thanks Juan On Wed, Mar 9, 2016 at 6:00 AM, Matthias Niehoff < [email protected]> wrote: > hi, > > What’s your batch interval? if the processing time is constantly bigger > than your batch interval it is totally normal that your scheduling delay is > going up. > > 2016-03-08 23:28 GMT+01:00 jleaniz <[email protected]>: > >> Hi, >> >> I have a streaming application that reads batches from Flume, does some >> transformations and then writes parquet files to HDFS. >> >> The problem I have right now is that the scheduling delays are really >> really >> high, and get even higher as time goes. Have seen it go up to 24 hours. >> The >> processing time for each batch is usually steady at 50s or less. >> >> The workers and master are pretty much idle most of the time. Any ideas >> why >> the scheduling time would be so high when the processing time is low? >> >> Thanks >> >> Juan >> >> >> >> -- >> View this message in context: >> http://apache-spark-user-list.1001560.n3.nabble.com/Streaming-job-delays-tp26433.html >> Sent from the Apache Spark User List mailing list archive at Nabble.com. >> >> --------------------------------------------------------------------- >> To unsubscribe, e-mail: [email protected] >> For additional commands, e-mail: [email protected] >> >> > > > -- > Matthias Niehoff | IT-Consultant | Agile Software Factory | Consulting > codecentric AG | Zeppelinstr 2 | 76185 Karlsruhe | Deutschland > tel: +49 (0) 721.9595-681 | fax: +49 (0) 721.9595-666 | mobil: +49 (0) > 172.1702676 > www.codecentric.de | blog.codecentric.de | www.meettheexperts.de | > www.more4fi.de > > Sitz der Gesellschaft: Solingen | HRB 25917| Amtsgericht Wuppertal > Vorstand: Michael Hochgürtel . Mirko Novakovic . Rainer Vehns > Aufsichtsrat: Patric Fedlmeier (Vorsitzender) . Klaus Jäger . Jürgen Schütz > > Diese E-Mail einschließlich evtl. beigefügter Dateien enthält vertrauliche > und/oder rechtlich geschützte Informationen. Wenn Sie nicht der richtige > Adressat sind oder diese E-Mail irrtümlich erhalten haben, informieren Sie > bitte sofort den Absender und löschen Sie diese E-Mail und evtl. > beigefügter Dateien umgehend. Das unerlaubte Kopieren, Nutzen oder Öffnen > evtl. beigefügter Dateien sowie die unbefugte Weitergabe dieser E-Mail ist > nicht gestattet >
