It looks like taskContext.attemptId doesn't mean what one thinks it might mean, based on
http://apache-spark-developers-list.1001551.n3.nabble.com/Get-attempt-number-in-a-closure-td8853.html and the unresolved https://issues.apache.org/jira/browse/SPARK-4014 Is there any alternative way to tell if compute is being called from a retry? Barring that, does anyone have any tips on how it might be possible to get the attempt count propagated to executors? It would be extremely useful for the kafka rdd preferred location awareness.