Hello,
I really wonder that whether i can stream compressed data with using
KafkaUtils.createDirectStream(...) or not.
This is formal code that i use ;
JavaPairInputDStream<String, String> messages =
KafkaUtils.createStream(javaStreamingContext, zookeeperConfiguration,
groupName, topicMap, StorageLevel.MEMORY_AND_DISK_SER());
final JavaDStream<String> lines = messages.map(new
Function<Tuple2<String, String>, String>() {
@Override
public String call(Tuple2<String, String> tuple2) {
System.out.println("Stream Received: " + tuple2._2());
return tuple2._2();
}
});
in this code i am consuming string message but in my case i need to consume
compressed data from stream then uncompress it and finally read string
message.
Could you please asist me how i can go in right way ? ı am a bit confused
that whether spark streaming has this ability or not.
--
View this message in context:
http://apache-spark-user-list.1001560.n3.nabble.com/Stream-compressed-data-from-KafkaUtils-createDirectStream-tp28010.html
Sent from the Apache Spark User List mailing list archive at Nabble.com.
---------------------------------------------------------------------
To unsubscribe e-mail: [email protected]
