[
https://issues.apache.org/jira/browse/SPARK-15406?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=15553328#comment-15553328
]
Michael Armbrust commented on SPARK-15406:
------------------------------------------
This missed 2.0.1, but I'm hopeful it will be in 2.0.2
(https://github.com/apache/spark/pull/15367). I think its possible that offset
assignment could also make 2.0.2. Please let us know more about your use case
on the JIRA (SPARK-17812). We internally are ensuring exactly once, so I'd
love to understand your case better.
While the initial PR doesn't handle every case, I think it does have pretty
easy to understand semantics and I have been using it in production. We were
also careful to only expose APIs that I do not expect to break in future
releases.
> Structured streaming support for consuming from Kafka
> -----------------------------------------------------
>
> Key: SPARK-15406
> URL: https://issues.apache.org/jira/browse/SPARK-15406
> Project: Spark
> Issue Type: New Feature
> Reporter: Cody Koeninger
>
> This is the parent JIRA to track all the work for the building a Kafka source
> for Structured Streaming. Here is the design doc for an initial version of
> the Kafka Source.
> https://docs.google.com/document/d/19t2rWe51x7tq2e5AOfrsM9qb8_m7BRuv9fel9i0PqR8/edit?usp=sharing
> ================== Old description =========================
> Structured streaming doesn't have support for kafka yet. I personally feel
> like time based indexing would make for a much better interface, but it's
> been pushed back to kafka 0.10.1
> https://cwiki.apache.org/confluence/display/KAFKA/KIP-33+-+Add+a+time+based+log+index
--
This message was sent by Atlassian JIRA
(v6.3.4#6332)
---------------------------------------------------------------------
To unsubscribe, e-mail: [email protected]
For additional commands, e-mail: [email protected]
