[
https://issues.apache.org/jira/browse/SPARK-18386?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=15653821#comment-15653821
]
Ofir Manor commented on SPARK-18386:
------------------------------------
BTW [[email protected]] - I think that filtering (by timestamp) can be done
today "the hard way" if the Kafka broker is 0.10.1.
The user could use the 0.10.1 client to get a list of offsets for his requested
timestamp, then submit a job to spark using explicit offsets to be used by
Spark's 0.10.0 client (quite ugly but should work).
> Batch mode SQL source for Kafka
> -------------------------------
>
> Key: SPARK-18386
> URL: https://issues.apache.org/jira/browse/SPARK-18386
> Project: Spark
> Issue Type: Improvement
> Components: SQL
> Reporter: Cody Koeninger
>
> An SQL equivalent to the DStream KafkaUtils.createRDD would be useful for
> querying over a defined batch of offsets.
> The possibility of Kafka 0.10.1 time indexing (e.g. a batch from timestamp X
> to timestamp Y) should be taken into account, even if not available in the
> initial implementation.
--
This message was sent by Atlassian JIRA
(v6.3.4#6332)
---------------------------------------------------------------------
To unsubscribe, e-mail: [email protected]
For additional commands, e-mail: [email protected]
