Github user tdas commented on a diff in the pull request:
https://github.com/apache/spark/pull/1723#discussion_r17196753
--- Diff: core/src/main/scala/org/apache/spark/util/Utils.scala ---
@@ -856,13 +859,27 @@ private[spark] object Utils extends Logging {
* finding the call site of a method.
*/
private val SPARK_CLASS_REGEX =
"""^org\.apache\.spark(\.api\.java)?(\.util)?(\.rdd)?\.[A-Z]""".r
+ private val SCALA_CLASS_REGEX = """^scala""".r
+ private val SPARK_STREAMING_CLASS_REGEX = """^org\.apache\.spark""".r
+ private val SPARK_EXAMPLES_CLASS_REGEX =
"""^org\.apache\.spark\.examples""".r
+
+ private def defaultRegexFunc(className: String): Boolean = {
+ SPARK_CLASS_REGEX.findFirstIn(className).isDefined ||
+ SCALA_CLASS_REGEX.findFirstIn(className).isDefined
+ }
+
+ def streamingRegexFunc(className: String): Boolean = {
--- End diff --
@mubarak Sorry if I wasnt clear in my previous comment, but I wanted the
streaming regex to be added to spark.streaming.util.Utils (new file needed).
Logically, Spark code should not have any reference to higher level stuff as
Spark Streaming.
---
If your project is set up for it, you can reply to this email and have your
reply appear on GitHub as well. If your project does not have this feature
enabled and wishes so, or if the feature is enabled but not working, please
contact infrastructure at [email protected] or file a JIRA ticket
with INFRA.
---
---------------------------------------------------------------------
To unsubscribe, e-mail: [email protected]
For additional commands, e-mail: [email protected]
