RE: Actors and sparkcontext actions

[Suraj Satishkumar Sheth]

Hi Ognen,
See if this helps. I was working on this :

class MyClass[T](sc : SparkContext, flag1 : Boolean, rdd : RDD[T], hdfsPath : 
String) extends Actor {

  def act(){
    if(flag1) this.process()
    else this.count
  }
  
  private def process(){
    println(sc.textFile(hdfsPath).count)
    //do the processing
  }
  
  private def count(){
   println(rdd.count)
   //do the counting
  }

}

Thanks and Regards,
Suraj Sheth


-----Original Message-----
From: Ognen Duzlevski [mailto:og...@nengoiksvelzud.com] 
Sent: 27 February 2014 01:09
To: u...@spark.incubator.apache.org
Subject: Actors and sparkcontext actions

Can someone point me to a simple, short code example of creating a basic Actor 
that gets a context and runs an operation such as .textFile.count? 
I am trying to figure out how to create just a basic actor that gets a message 
like this:

case class Msg(filename:String, ctx: SparkContext)

and then something like this:

class HelloActor extends Actor {
     import context.dispatcher

     def receive = {
         case Msg(fn,ctx) => {
             // get the count here!
             // cts.textFile(fn).count
         }
         case _ => println("huh?")
     }
}

Where I would want to do something like:

val conf = new
SparkConf().setMaster("spark://192.168.10.29:7077").setAppName("Hello").setSparkHome("/Users/maketo/plainvanilla/spark-0.9")
val sc = new SparkContext(conf)
val system = ActorSystem("mySystem")

val helloActor1 = system.actorOf( Props[ HelloActor], name = "helloactor1")
helloActor1 ! new Msg("test.json",sc)

Thanks,
Ognen