The main thing is picking up new partitions. You can't do that
without reimplementing portions of the consumer rebalance. The
low-level consumer is really low level, and the old high-level
consumer is basically broken (it might have been fixed by the time
they abandoned it, I dunno)
On Fri, Oct
>
> Without a hell of a lot more work, Assign would be the only strategy
> usable.
How would the current "subscribe" break?
Without a hell of a lot more work, Assign would be the only strategy usable.
On Fri, Oct 7, 2016 at 3:25 PM, Michael Armbrust wrote:
>> The implementation is totally and completely different however, in ways
>> that leak to the end user.
>
>
> Can you elaborate? Especially in the context of the i
> 0.10 consumers won't work on an earlier broker.
> Earlier consumers will (should?) work on a 0.10 broker.
>
This lines up with my testing. Is there a page I'm missing that describes
this? Like does a 0.9 client work with 0.8 broker? Is it always old
clients can talk to new brokers but not vi
>
> The implementation is totally and completely different however, in ways
> that leak to the end user.
Can you elaborate? Especially in the context of the interface provided by
structured streaming.
0.10 consumers won't work on an earlier broker.
Earlier consumers will (should?) work on a 0.10 broker.
The main things earlier consumers lack from a user perspective is
support for SSL, and pre-fetching messages. The implementation is
totally and completely different however, in ways that leak
Does Kafka 0.10 work on a Kafka 0.8/0.9 cluster?
On Fri, Oct 7, 2016 at 1:14 PM, Jeremy Smith
wrote:
> +1
>
> We're on CDH, and it will probably be a while before they support Kafka
> 0.10. At the same time, we don't use their Spark and we're looking forward
> to upgrading to 2.0.x and using st
+1
We're on CDH, and it will probably be a while before they support Kafka
0.10. At the same time, we don't use their Spark and we're looking forward
to upgrading to 2.0.x and using structured streaming.
I was just going to write our own Kafka Source implementation which uses
the existing KafkaRD
