Re: Providing a namespace for third-party configurations

My 2 cents, I second with Sean, as it's not necessary for downstream projects to have a rule for config name starting with "spark". I guess I know why they want it, but then it should be clear what's the goal. To differentiate with existing spark configs? Using another prefix would help. To smooth

Re: Providing a namespace for third-party configurations

It's possible, but pretty unlikely to have an exact namespace collision. It's probably a best practice to clearly separates settings, etc that are downstream add-ons into a separate namespace, and I don't mind a sentence in a doc somewhere suggesting a convention, but I really think it's up to down

Providing a namespace for third-party configurations

I discovered today that EMR provides its own optimizations for Spark . Some of these optimizations are controlled by configuration settings with names like `spark.sql.dynamicPartitionPruning.enabled` or `spark.sql.optim