Devs, While investigating another issue, I came across this OOM error when using the Console Sink with any source that can be larger than the available driver memory. In my case, I was using the File source and I had a 14G file in the monitored dir.
I traced back the issue to a `df.collect` in the Console Sink code. I created a JIRA for it: https://issues.apache.org/jira/browse/SPARK-21710 and a PR is available: https://github.com/apache/spark/pull/18923 I hope a committer can check it out. -kr, Gerard.