Re: kafka direct streaming python API fromOffsets

2016-05-03 Thread Tigran Avanesov
ctStream's fromOffsets parameter takes a map from TopicAndPartition to Long. There is documentation for a python constructor for TopicAndPartition: http://spark.apache.org/docs/latest/api/python/_modules/pyspark/streaming/kafka.html#TopicAndPartition On Mon, May 2, 2016 at 5:54 AM, Tigran Avane

kafka direct streaming python API fromOffsets

2016-05-02 Thread Tigran Avanesov
Hi, I'm trying to start consuming messages from a kafka topic (via direct stream) from a given offset. The documentation of createDirectStream says: :param fromOffsets: Per-topic/partition Kafka offsets defining the (inclusive) starting point of the stream. However it expects a dictionary o