Re: Spark and Flume integration - do I understand this correctly?

2014-07-29 Thread dapooley
Great, thanks guys, that helped a lot and I've got a sample working. As a follow up, when do worker/masters become necessity? -- View this message in context: http://apache-spark-user-list.1001560.n3.nabble.com/Spark-and-Flume-integration-do-I-understand-this-correctly-tp10879p10908.html Sent

Spark and Flume integration - do I understand this correctly?

2014-07-29 Thread dapooley
Hi, I am trying to integrate Spark onto a Flume log sink and avro source. The sink is on one machine (the application), and the source is on another. Log events are being sent from the application server to the avro source server (a log directory sink on the arvo source prints to verify) The aim