I just created a fix. https://github.com/apache/zeppelin/pull/1019 Please try it when you have time and let me know if it does not solve your problem.
Thanks, moon On Wed, Jun 15, 2016 at 4:05 PM Jose Rivera-Rubio < jose.riv...@internavenue.com> wrote: > Not at all, thank you!! > > *Jose Rivera* > p.s. We're Hiring <https://www.internavenue.com/about/jobs/>! > > *Data Engineer Lead **Intern Avenue <http://www.internavenue.com/>* > Watch Intern Avenue on: BBC Dragons’ Den > <http://www.internavenue.com/dragons-den> > e: jose.riv...@internavenue.com <janos.fe...@internavenue.com> | > www.internavenue.com | Great Interns Online > > On 16 June 2016 at 00:04, moon soo Lee <m...@apache.org> wrote: > >> Thanks for share the problem. >> Let me try create a fix. >> >> Best, >> moon >> >> >> On Wed, Jun 15, 2016 at 3:56 PM Jose Rivera-Rubio < >> jose.riv...@internavenue.com> wrote: >> >>> Hi, thanks for the quick reply. >>> >>> I do have 'auto-restart interpreter' after the cron job is executed, as >>> my task is quite memory intensive. >>> >>> Apart from the multiple-interpreter issue, I think there's also the case >>> of concurrency being the instigator of the SIGTERM kill signal. >>> >>> Thanks, >>> >>> Jose >>> >>> *Jose Rivera* >>> p.s. We're Hiring <https://www.internavenue.com/about/jobs/>! >>> >>> *Data Engineer Lead **Intern Avenue <http://www.internavenue.com/>* >>> Watch Intern Avenue on: BBC Dragons’ Den >>> <http://www.internavenue.com/dragons-den> >>> e: jose.riv...@internavenue.com <janos.fe...@internavenue.com> | >>> www.internavenue.com | Great Interns Online >>> >>> On 15 June 2016 at 21:29, moon soo Lee <m...@apache.org> wrote: >>> >>>> Thanks for good finding. Do you use cron scheduling with 'auto-restart >>>> interpreter on cron execution' checked? >>>> >>>> 'Notebook.CronJob.execute' checks only last paragraph. Therefore if >>>> multiple type of interpreter is being used in a note and If 'auto-restart >>>> interpreter on cron execution' is checked, there is possibility to restart >>>> interpreter process while some paragraph is still running. >>>> >>>> Thanks, >>>> moon >>>> >>>> On Wed, Jun 15, 2016 at 9:38 AM Jose Rivera-Rubio < >>>> jose.riv...@internavenue.com> wrote: >>>> >>>>> It seems to be working if I only have one paragraph (or one >>>>> interpreter type) in the notebook (note) with the cron job. Can this be >>>>> related with the `Notebook.CronJob.execute` checks on the very last >>>>> paragraph of the notebook, creating erratic behaviour when running >>>>> multiple >>>>> paragraphs in parallel or paragraphs use different interpreters? >>>>> >>>>> *Jose Rivera* >>>>> p.s. We're Hiring <https://www.internavenue.com/about/jobs/>! >>>>> >>>>> *Data Engineer Lead **Intern Avenue <http://www.internavenue.com/>* >>>>> Watch Intern Avenue on: BBC Dragons’ Den >>>>> <http://www.internavenue.com/dragons-den> >>>>> e: jose.riv...@internavenue.com <janos.fe...@internavenue.com> | >>>>> www.internavenue.com | Great Interns Online >>>>> >>>>> On 15 June 2016 at 17:27, Jose Rivera-Rubio < >>>>> jose.riv...@internavenue.com> wrote: >>>>> >>>>>> I have tried increasing this timeout but still get the SIGTERM (kill) >>>>>> signal. >>>>>> >>>>>> How can I debug this? >>>>>> >>>>>> I have timed the script in the terminal, wit `time sh script.sh` , >>>>>> then modifying the timeouts to be well above this number and tried the >>>>>> cron >>>>>> job without success. >>>>>> >>>>>> Thank you >>>>>> >>>>>> *Jose Rivera* >>>>>> p.s. We're Hiring <https://www.internavenue.com/about/jobs/>! >>>>>> >>>>>> *Data Engineer Lead **Intern Avenue <http://www.internavenue.com/>* >>>>>> Watch Intern Avenue on: BBC Dragons’ Den >>>>>> <http://www.internavenue.com/dragons-den> >>>>>> e: jose.riv...@internavenue.com <janos.fe...@internavenue.com> | >>>>>> www.internavenue.com | Great Interns Online >>>>>> >>>>>> On 18 February 2016 at 17:57, moon soo Lee <m...@apache.org> wrote: >>>>>> >>>>>>> Hi Jose Rivera, >>>>>>> >>>>>>> If your script takes more than the timeout defined here >>>>>>> https://github.com/apache/incubator-zeppelin/blob/master/shell/src/main/java/org/apache/zeppelin/shell/ShellInterpreter.java#L49, >>>>>>> the process will be terminated. >>>>>>> Otherwise, it looks like something send SIGTERM (kill) signal to >>>>>>> your script. >>>>>>> >>>>>>> Or another case? >>>>>>> >>>>>>> Thanks, >>>>>>> moon >>>>>>> >>>>>>> On Wed, Feb 17, 2016 at 10:22 AM Jose Rivera-Rubio < >>>>>>> jose.riv...@internavenue.com> wrote: >>>>>>> >>>>>>>> Hi, >>>>>>>> >>>>>>>> I have a paragraph that runs a shell script to import data from a >>>>>>>> MySQL db to Parquet files using Python and Pyspark (it's legacy import >>>>>>>> code, that's why it's in a shell script instead of the python code >>>>>>>> directly >>>>>>>> in the paragraph). >>>>>>>> >>>>>>>> The issue is that if I manually run the paragraph it does the >>>>>>>> import correctly, no issues at all. However, if I set a cron job so it >>>>>>>> runs >>>>>>>> periodically it fails with this cryptic error: >>>>>>>> >>>>>>>> Paragraph received a SIGTERM. >>>>>>>> ExitValue: 143 >>>>>>>> >>>>>>>> Any ideas? >>>>>>>> >>>>>>>> Thanks! >>>>>>>> >>>>>>>> *Jose Rivera* >>>>>>>> p.s. We're Hiring <https://www.internavenue.com/about/jobs/>! >>>>>>>> >>>>>>>> *Data Engineer Lead **Intern Avenue >>>>>>>> <http://www.internavenue.com/>* >>>>>>>> Watch Intern Avenue on: BBC Dragons’ Den >>>>>>>> <http://www.internavenue.com/dragons-den> >>>>>>>> e: jose.riv...@internavenue.com <janos.fe...@internavenue.com> | >>>>>>>> www.internavenue.com | Great Interns Online >>>>>>>> >>>>>>> >>>>>> >>>>> >>> >
