Hi, Yes, correct. I was too forceful in discouraging people using it. I think @deprecated would be a right direction.
What should be the next step? I think I should file an JIRA so it's in a release notes. Correct? I was very surprised to have noticed its resurrection in the very latest module of Spark - Structured Streaming - that will be an inspiration for others to learn Spark. Jacek On 14 Jan 2017 12:48 p.m., "Mridul Muralidharan" <mri...@gmail.com> wrote: > Since TaskContext.getPartitionId is part of the public api, it cant be > removed as user code can be depending on it (unless we go through a > deprecation process for it). > > Regards, > Mridul > > > On Sat, Jan 14, 2017 at 2:02 AM, Jacek Laskowski <ja...@japila.pl> wrote: > > Hi, > > > > Just noticed that TaskContext#getPartitionId [1] is not used and > > moreover the scaladoc is incorrect: > > > > "It will return 0 if there is no active TaskContext for cases like > > local execution." > > > > since there are no local execution. (I've seen the comment in the code > > before but can't find it now). > > > > The reason to remove it is that Structured Streaming is giving new > > birth to the method in ForeachSink [2] which may look like a > > "resurrection". > > > > There's simply TaskContext.get.partitionId. > > > > What do you think? > > > > [1] https://github.com/apache/spark/blob/master/core/src/ > main/scala/org/apache/spark/TaskContext.scala#L41 > > [2] https://github.com/apache/spark/blob/master/sql/core/ > src/main/scala/org/apache/spark/sql/execution/streaming/ > ForeachSink.scala#L50 > > > > Pozdrawiam, > > Jacek Laskowski > > ---- > > https://medium.com/@jaceklaskowski/ > > Mastering Apache Spark 2.0 https://bit.ly/mastering-apache-spark > > Follow me at https://twitter.com/jaceklaskowski > > > > --------------------------------------------------------------------- > > To unsubscribe e-mail: dev-unsubscr...@spark.apache.org > > >
