Not sure if that's documented in terms of Spark but this is a fairly common pattern in scala known as "pimp my library" pattern, you can easily find many generic example of using this pattern.
If you want I can quickly cook up a short conplete example with rdd(although there is nothing really more to my example in earlier mail) ? Thanks Tarun Kumar On Mon, 5 Dec 2016 at 7:15 AM, long <longteng...@gmail.com> wrote: > So is there documentation of this I can refer to? > > On Dec 5, 2016, at 1:07 AM, Tarun Kumar [via Apache Spark Developers List] > <[hidden email] <http:///user/SendEmail.jtp?type=node&node=20104&i=0>> > wrote: > > Hi Tenglong, > > In addition to trsell's reply, you can add any method to an rdd without > making changes to spark code. > > This can be achieved by using implicit class in your own client code: > > implicit class extendRDD[T](rdd: RDD[T]){ > > def foo() > > } > > Then you basically nees to import this implicit class in scope where you > want to use the new foo method. > > Thanks > Tarun Kumar > > On Mon, 5 Dec 2016 at 6:59 AM, <<a > href="x-msg://19/user/SendEmail.jtp?type=node&node=20102&i=0" > target="_top" rel="nofollow" link="external" class="">[hidden email]> wrote: > > How does your application fetch the spark dependency? Perhaps list your > project dependencies and check it's using your dev build. > > On Mon, 5 Dec 2016, 08:47 tenglong, <<a > href="x-msg://19/user/SendEmail.jtp?type=node&node=20102&i=1" > target="_top" rel="nofollow" link="external" class="">[hidden email]> wrote: > > Hi, > > Apparently, I've already tried adding a new method to RDD, > > for example, > > class RDD { > def foo() // this is the one I added > > def map() > > def collect() > } > > I can build Spark successfully, but I can't compile my application code > which calls rdd.foo(), and the error message says > > value foo is not a member of org.apache.spark.rdd.RDD[String] > > So I am wondering if there is any mechanism prevents me from doing this or > something I'm doing wrong? > > > > > -- > View this message in context: > http://apache-spark-developers-list.1001551.n3.nabble.com/Can-I-add-a-new-method-to-RDD-class-tp20100.html > Sent from the Apache Spark Developers List mailing list archive at > Nabble.com. > > --------------------------------------------------------------------- > > To unsubscribe e-mail: <a > href="x-msg://19/user/SendEmail.jtp?type=node&node=20102&i=2" > target="_top" rel="nofollow" link="external" class="">[hidden email] > > > > ------------------------------ > If you reply to this email, your message will be added to the discussion > below: > > http://apache-spark-developers-list.1001551.n3.nabble.com/Can-I-add-a-new-method-to-RDD-class-tp20100p20102.html > To unsubscribe from Can I add a new method to RDD class?, click here. > NAML > <http://apache-spark-developers-list.1001551.n3.nabble.com/template/NamlServlet.jtp?macro=macro_viewer&id=instant_html%21nabble%3Aemail.naml&base=nabble.naml.namespaces.BasicNamespace-nabble.view.web.template.NabbleNamespace-nabble.view.web.template.NodeNamespace&breadcrumbs=notify_subscribers%21nabble%3Aemail.naml-instant_emails%21nabble%3Aemail.naml-send_instant_email%21nabble%3Aemail.naml> > > > > ------------------------------ > View this message in context: Re: Can I add a new method to RDD class? > <http://apache-spark-developers-list.1001551.n3.nabble.com/Can-I-add-a-new-method-to-RDD-class-tp20100p20104.html> > Sent from the Apache Spark Developers List mailing list archive > <http://apache-spark-developers-list.1001551.n3.nabble.com/> at > Nabble.com. >
