Re: Using Apache Spark Streaming - how to handle changing data format within stream

2016-11-09 Thread coolgar
uot;).foreach { line => if // it's a header parser = someParserBasedOn(line) else items += parser.parse(line) } items.iterator } -- View this message in context: http://apache-spark-user-list.1001560.n3.nabble.com/Using-Apache-Spark-Streaming-how-to-handle-changi

Re: Using Apache Spark Streaming - how to handle changing data format within stream

2016-11-07 Thread Cody Koeninger
a new #fields comes along, rinse, repeat? > > > > > > -- > View this message in context: > http://apache-spark-user-list.1001560.n3.nabble.com/Using-Apache-Spark-Streaming-how-to-handle-changing-data-format-within-stream-tp28037.html > Sent from the Apache Spark User L

Using Apache Spark Streaming - how to handle changing data format within stream

2016-11-07 Thread coolgar
s the messages that follow the header until a new #fields comes along, rinse, repeat? -- View this message in context: http://apache-spark-user-list.1001560.n3.nabble.com/Using-Apache-Spark-Streaming-how-to-handle-changing-data-format-within-stream-tp28037.html Sent from the Apache Spark Use