;
>
> On Fri, Mar 30, 2018 at 11:41 AM, Koert Kuipers wrote:
> > does anyone know why all spark settings end up being system properties,
> and
> > where this is done?
> >
> > for example when i pass "--conf spark.foo=bar" into spark-submit then
> > Syste
her tricks, but user code can always break those.
Where: this is done by the SparkSubmit class (look for the Scala
version, "sys.props").
On Fri, Mar 30, 2018 at 11:41 AM, Koert Kuipers wrote:
> does anyone know why all spark settings end up being system properties, and
> where
does anyone know why all spark settings end up being system properties, and
where this is done?
for example when i pass "--conf spark.foo=bar" into spark-submit then
System.getProperty("spark.foo") will be equal to "bar"
i grepped the spark codebase for System.setPr
