RE: FW: Websphere MQ as a data source for Apache Spark Streaming

2015-06-01 Thread Chaudhary, Umesh
issue. From: Tathagata Das [mailto:t...@databricks.com] Sent: Friday, May 29, 2015 3:30 AM To: Chaudhary, Umesh Cc: Arush Kharbanda; user@spark.apache.org Subject: Re: FW: Websphere MQ as a data source for Apache Spark Streaming Are you sure that the data can be saved as strings? Another, more

Re: FW: Websphere MQ as a data source for Apache Spark Streaming

2015-05-29 Thread Tathagata Das
Are you sure that the data can be saved as strings? Another, more controlled approach is use DStream.foreachRDD , which takes a Function2 parameter, RDD and Time. There you can explicitly do stuff with the RDD, save it to separate files (separated by time), or whatever. Might help you to debug wha