Hi all, I'm reading the code in spark-sql-execution-stat-FrequentItems.scala, and I'm a little confused about the "add" method in the FreqItemCounter class. Please see the link here: https://github.com/apache/spark/blob/master/sql/core/src/main/scala/org/apache/spark/sql/execution/stat/FrequentItems.scala
My question is when the baseMap does not contain the key, and the size of the baseMap is not less than size, why should we just keep the key/value pairs whose value is greater than count? Just like this example: Now the baseMap is Map(1 -> 3, 2 -> 3, 3 -> 4), and the size is 3. I want to add Map(4 -> 25) into this baseMap, so it will retain the key/values whose value is greater than 25, and in that way, the baseMap will be null. However, I think we should at least add 4 -> 25 into the baseMap. Could anybody help me with this problem? Best, Yucheng -- View this message in context: http://apache-spark-developers-list.1001551.n3.nabble.com/FrequentItems-in-spark-sql-execution-stat-tp13527.html Sent from the Apache Spark Developers List mailing list archive at Nabble.com. --------------------------------------------------------------------- To unsubscribe, e-mail: dev-unsubscr...@spark.apache.org For additional commands, e-mail: dev-h...@spark.apache.org
