Should 3.1.0 config props be 3.1.1 (as s.k.executor.missingPodDetectDelta)?

2021-01-23 Thread Jacek Laskowski
Hi, Just noticed a couple of configuration properties marked as version("3.1.0") in Spark on Kubernetes' Config. There's one spark.kubernetes.executor.missingPodDetectDelta property marked as 3.1.1. That triggered this question in my mind whether all together should be 3.1.1? (We could leave it

Re: Should 3.1.0 config props be 3.1.1 (as s.k.executor.missingPodDetectDelta)?

2021-01-23 Thread Hyukjin Kwon
I think we can just leave it as is. We have the unofficial 3.1.0 release with its corresponding git tag so 3.1.0 mark isn't completely useless. Also changing means we should go through JIRAs and change the version properties, fixing conflict, etc. which I don't think is worthwhile. On Sat, 23 Jan

Re: Should 3.1.0 config props be 3.1.1 (as s.k.executor.missingPodDetectDelta)?

2021-01-23 Thread Jacek Laskowski
Hi Hyukjin, Agreed. I asked to see if I'm not missing anything. Thank you. Pozdrawiam, Jacek Laskowski https://about.me/JacekLaskowski "The Internals Of" Online Books Follow me on https://twitter.com/jaceklaskowski On Sat, Ja

[K8S] ExecutorPodsWatchSnapshotSource with no spark-exec-inactive label in 3.1?

2021-01-23 Thread Jacek Laskowski
Hi, spark-exec-inactive label is not used to watch executor pods in ExecutorPodsWatchSnapshotSource [1] as it happens in PollRunnable [2]. Any reasons? Am I correct to consider it a bug? [1] https://github.com/apache/spark/blob/270a9a3cac25f3e799460320d0fc94ccd7ecfaea/resource-managers/kubernete