Does SparkContext exists when this part (AskDriverWithReply()) of the scheduler code gets executed?
On Sun, Oct 12, 2014 at 1:54 PM, rapelly kartheek <kartheek.m...@gmail.com> wrote: > Hi Sean, > I tried even with sc as: sc.parallelize(data). But. I get the error: value > sc not found. > > On Sun, Oct 12, 2014 at 1:47 PM, sowen [via Apache Spark User List] < > ml-node+s1001560n16233...@n3.nabble.com> wrote: > >> It is a method of the class, not a static method of the object. Since a >> SparkContext is available as sc in the shell, or you have perhaps created >> one similarly in your app, write sc.parallelize(...) >> On Oct 12, 2014 7:15 AM, "rapelly kartheek" <[hidden email] >> <http://user/SendEmail.jtp?type=node&node=16233&i=0>> wrote: >> >>> Hi, >>> >>> I am trying to write a String that is not an rdd to HDFS. This data is a >>> variable in Spark Scheduler code. None of the spark File operations are >>> working because my data is not rdd. >>> >>> So, I tried using SparkContext.parallelize(data). But it throws error: >>> >>> [error] >>> /home/karthik/spark-1.0.0/core/src/main/scala/org/apache/spark/storage/BlockManagerMaster.scala:265: >>> not found: value SparkContext >>> [error] SparkContext.parallelize(result) >>> [error] ^ >>> [error] one error found >>> >>> I realized that this data is part of the Scheduler. So, the Sparkcontext >>> would not have got created yet. >>> >>> Any help in "writing scheduler variable data to HDFS" is appreciated!! >>> >>> -Karthik >>> >> >> >> ------------------------------ >> If you reply to this email, your message will be added to the >> discussion below: >> >> http://apache-spark-user-list.1001560.n3.nabble.com/How-to-convert-a-non-rdd-data-to-rdd-tp16230p16233.html >> To unsubscribe from How to convert a non-rdd data to rdd., click here >> <http://apache-spark-user-list.1001560.n3.nabble.com/template/NamlServlet.jtp?macro=unsubscribe_by_code&node=16230&code=a2FydGhlZWsubWJtc0BnbWFpbC5jb218MTYyMzB8LTE1NjA1NDM4NDM=> >> . >> NAML >> <http://apache-spark-user-list.1001560.n3.nabble.com/template/NamlServlet.jtp?macro=macro_viewer&id=instant_html%21nabble%3Aemail.naml&base=nabble.naml.namespaces.BasicNamespace-nabble.view.web.template.NabbleNamespace-nabble.view.web.template.NodeNamespace&breadcrumbs=notify_subscribers%21nabble%3Aemail.naml-instant_emails%21nabble%3Aemail.naml-send_instant_email%21nabble%3Aemail.naml> >> > > -- View this message in context: http://apache-spark-user-list.1001560.n3.nabble.com/How-to-convert-a-non-rdd-data-to-rdd-tp16230p16235.html Sent from the Apache Spark User List mailing list archive at Nabble.com.
