Re: Spark Streaming + Kafka + Mesos/Marathon strangeness

2014-03-28 Thread Tathagata Das
know >> what value to use in a mesos cluster, since the processes are running in >> lxc >> containers they wont be sharing a filesystem (or machine for that matter) >> >> I cant use an s3n:// url for local dir can I? >> >> >> >> -- >> Vi

Re: Spark Streaming + Kafka + Mesos/Marathon strangeness

2014-03-27 Thread Evgeny Shishkin
they wont be sharing a filesystem (or machine for that matter) > > I cant use an s3n:// url for local dir can I? > > > > -- > View this message in context: > http://apache-spark-user-list.1001560.n3.nabble.com/Spark-Streaming-Kafka-Mesos-Marathon-strangeness-tp3285p3373.html > Sent from the Apache Spark User List mailing list archive at Nabble.com. >

Re: Spark Streaming + Kafka + Mesos/Marathon strangeness

2014-03-27 Thread Tathagata Das
alue to use in a mesos cluster, since the processes are running in > lxc > containers they wont be sharing a filesystem (or machine for that matter) > > I cant use an s3n:// url for local dir can I? > > > > -- > View this message in context: > http://apache-spark-user-li

Re: Spark Streaming + Kafka + Mesos/Marathon strangeness

2014-03-27 Thread Scott Clasen
View this message in context: http://apache-spark-user-list.1001560.n3.nabble.com/Spark-Streaming-Kafka-Mesos-Marathon-strangeness-tp3285p3373.html Sent from the Apache Spark User List mailing list archive at Nabble.com.

Re: Spark Streaming + Kafka + Mesos/Marathon strangeness

2014-03-27 Thread Tathagata Das
ce the tasks are not running on the same machine, the file is not found > when the second task takes over. > > How do you set spark.local.dir appropriately when running on mesos? > > > > -- > View this message in context: > http://apache-spark-user-list.1001560.n3.nabble.c

Re: Spark Streaming + Kafka + Mesos/Marathon strangeness

2014-03-27 Thread Scott Clasen
://apache-spark-user-list.1001560.n3.nabble.com/Spark-Streaming-Kafka-Mesos-Marathon-strangeness-tp3285p3356.html Sent from the Apache Spark User List mailing list archive at Nabble.com.

Re: Spark Streaming + Kafka + Mesos/Marathon strangeness

2014-03-26 Thread Scott Clasen
. Thanks! -- View this message in context: http://apache-spark-user-list.1001560.n3.nabble.com/Spark-Streaming-Kafka-Mesos-Marathon-strangeness-tp3285p3312.html Sent from the Apache Spark User List mailing list archive at Nabble.com.

Re: Spark Streaming + Kafka + Mesos/Marathon strangeness

2014-03-26 Thread Tathagata Das
jetty-7.x.y-SNAPSHOT > 14/03/26 19:06:48 INFO server.AbstractConnector: Started > SocketConnector@0.0.0.0:45286 > > > > Any pointers as to un-stick this?? > > > > > > > -- > View this message in context: > http://apache-spark-user-list.1001560.n3.nabble.com/Spark-Streaming-Kafka-Mesos-Marathon-strangeness-tp3285.html > Sent from the Apache Spark User List mailing list archive at Nabble.com. >

Spark Streaming + Kafka + Mesos/Marathon strangeness

2014-03-26 Thread Scott Clasen
-stick this?? -- View this message in context: http://apache-spark-user-list.1001560.n3.nabble.com/Spark-Streaming-Kafka-Mesos-Marathon-strangeness-tp3285.html Sent from the Apache Spark User List mailing list archive at Nabble.com.