Hi Jon,
sorry for the late replay.
A while ago I was struggling with as well. Unfortunately, there’s no direct way
to do this per pipeline.
However, you can set default arguments by passing them to the job service
container using the environment variable _JAVA_OPTIONS.
I hope this still helps!
Hi,
I'm having trouble trying to figure out where to pass "conf" spark-submit
arguments to a spark job service. I don't particularly care at this point
whether the job service uses the same set of args for all jobs passed to
the service, I'm just having trouble finding where I can send these args
