Re: Local spark interpreter with extra java options

2021-07-25 Thread Lior Chaga
Awesome, thanks Jeff. On Sun, Jul 25, 2021 at 11:24 AM Jeff Zhang wrote: > Hi Lior, > > It would be fixed in https://github.com/apache/zeppelin/pull/4127 > > > Lior Chaga 于2021年7月25日周日 下午3:58写道: > >> After a couple of attempts of code fixes, when every time I seemed to >> make things work just

Re: Local spark interpreter with extra java options

2021-07-25 Thread Jeff Zhang
Hi Lior, It would be fixed in https://github.com/apache/zeppelin/pull/4127 Lior Chaga 于2021年7月25日周日 下午3:58写道: > After a couple of attempts of code fixes, when every time I seemed to make > things work just to find out the next step in the process breaks, I've > found the most simple solution -

Re: Local spark interpreter with extra java options

2021-07-25 Thread Lior Chaga
After a couple of attempts of code fixes, when every time I seemed to make things work just to find out the next step in the process breaks, I've found the most simple solution - put them extraJavaOptions in spark-defaults.conf (instead of keeping them in interpreter settings) On Sun, Jul 11, 20

Re: Local spark interpreter with extra java options

2021-07-11 Thread Lior Chaga
Thanks Jeff, So I should escape the whitespaces? Is there a ticket for it? couldn't find one On Sun, Jul 11, 2021 at 1:10 PM Jeff Zhang wrote: > I believe this is due to SparkInterpreterLauncher doesn't support > parameters with whitespace. (It would use whitespace as delimiter to > separate par

Re: Local spark interpreter with extra java options

2021-07-11 Thread Jeff Zhang
I believe this is due to SparkInterpreterLauncher doesn't support parameters with whitespace. (It would use whitespace as delimiter to separate parameters), this is a known issue Lior Chaga 于2021年7月11日周日 下午4:14写道: > So after adding the quotes in both SparkInterpreterLauncher > and interpreter.sh

Re: Local spark interpreter with extra java options

2021-07-11 Thread Lior Chaga
So after adding the quotes in both SparkInterpreterLauncher and interpreter.sh, interpreter is still failing with same error of Unrecognized option. But the weird thing is that if I copy the command supposedly executed from zeppelin (as it is printed to log) and run it directly in shell, the interp

Re: Local spark interpreter with extra java options

2021-07-08 Thread Jeff Zhang
Thanks Lior for the investigation. Lior Chaga 于2021年7月8日周四 下午8:31写道: > Ok, I think I found the issue. It's not only that the quotations are > missing from the --conf param, they are also missing from > the --driver-java-options, which is concatenated to > the INTERPRETER_RUN_COMMAND in interpre

Re: Local spark interpreter with extra java options

2021-07-08 Thread Lior Chaga
Ok, I think I found the issue. It's not only that the quotations are missing from the --conf param, they are also missing from the --driver-java-options, which is concatenated to the INTERPRETER_RUN_COMMAND in interpreter.sh I will fix it in my build, but would like a confirmation that this is ind

Local spark interpreter with extra java options

2021-07-08 Thread Lior Chaga
I'm trying to run zeppelin using local spark interpreter. Basically everything works, but if I try to set `spark.driver.extraJavaOptions` or `spark.executor.extraJavaOptions` containing several arguments, I get an exception. For instance, for providing `-DmyParam=1 -DmyOtherParam=2`, I'd get: Error