That's the whole reason why some of the intended configuration changes were backed out just before the 0.9.0 release. It's a well-known issue, even if a completely satisfactory solution isn't as well-known and is probably something which should do another iteration on.
On Wed, Mar 12, 2014 at 9:10 AM, Koert Kuipers <ko...@tresata.com> wrote: > i am reading the spark configuration params from another configuration > object (typesafe config) before setting them as system properties. > > i noticed typesafe config has trouble with settings like: > spark.speculation=true > spark.speculation.interval=0.5 > > the issue seems to be that if spark.speculation is a "container" that has > more values inside then it cannot be also a value itself, i think. so this > would work fine: > spark.speculation.enabled=true > spark.speculation.interval=0.5 > > just a heads up. i would probably suggest we avoid this situation. >
