Re: [DISCUSS] Versionless Spark Programming Guide Proposal

Just FYI, the Hive languages manual is also version-less: https://cwiki.apache.org/confluence/display/Hive/LanguageManual It's not a strong data point as this doc is not actively updated, but my personal feeling is that it's nice to see the history of a feature: when it was introduced, when it got

Re: [DISCUSS] Versionless Spark Programming Guide Proposal

There are two issues and one main benefit that I see with versioned programming guides: - *Issue 1*: We often retroactively realize that code snippets have bugs and explanations are confusing (see examples: dropDuplicates , dropDuplicatesWith

Re: [DISCUSS] Versionless Spark Programming Guide Proposal

Shall we decouple these two decisions? - Move the programming guide to the spark-website repo, to allow faster iterations and releases - Make programming guide version-less I think the downside of moving the programming guide to the spark-website repo is almost negligible: you may need t

Re: [DISCUSS] Versionless Spark Programming Guide Proposal

I think some of the issues raised here are not really common. Examples should follow best practice. It would be odd to have an example that exploits ansi.enabled=false to e.g. overflow an integer. Instead an example that works with ansi mode will typically work perfectly fine in an older version,