Hey Marcelo, When we did the configuration pull request, we actually avoided having a big list of defaults in one class file, because this creates a file that all the components in the project depend on. For example, since we have some settings specific to streaming and the REPL, do we want those settings to appear in a file that’s in “core”? It might be better to just make sure we use constants for defaults in the code, or maybe have a separate class per project.
The other problem with this kind of change is that it’s disruptive to all the other ongoing patches, so I wouldn’t consider it high-priority right now. We haven’t had a ton of problems with settings being mistyped. If you do want to do something like this though, apart from the comment above about modules, please make sure this is not a public API. As soon as we add it to the API, it means we can’t change or remove those config settings. I’d also suggest giving each config setting a single name instead of having “ui”, “shuffle”, etc objects, since the chained calls to conf.ui.port.value look somewhat confusing. Matei On Jun 23, 2014, at 3:57 PM, Marcelo Vanzin <van...@cloudera.com> wrote: > I started with some code to implement an idea I had for SPARK-529, and > before going much further (since it's a large and kinda boring change) > I'd like to get some feedback from people. > > Current code it at: > https://github.com/vanzin/spark/tree/SPARK-529 > > There are still some parts I haven't fully fleshed out yet (see TODO > list in the commit message), but that's the basic idea. Let me know if > you have any feedback or different ideas. > > Thanks! > > > -- > Marcelo
