Thanks, I checked a consumer group using the below command on old and new
clusters but the new cluster shows error consumer group doesn't exist. Am I
missing something?
As per the properties all groups should be included except the backlisted
ones.
kafka-consumer-groups.sh --bootstrap-server <> -
As Ryanne said,
MM2 "syncs" offsets - in that it maintains a mapping of "cluster A offsets"
to "cluster B offsets" in cluster B, so if you have to move a consumer
group from A to B, you can relatively easily point the consumer group at
the offsets on B that map to its offsets on A.
On Mon, Mar 8,
MirrorMaker v1 does not sync offsets, but MM2 does!
Ryanne
On Sun, Mar 7, 2021, 10:02 PM Pushkar Deole wrote:
> Thanks you all!
>
> Blake, for your comment:
>
> It'll require having a HA cluster running in another region, of course.
> One other caveat is that it doesn't preserve the offsets of
You can see the replicated consumer group offsets using the
kafka-consumer-groups.sh tool. Make sure the consumer group is in the
groups allowlist.
Ryanne
On Sun, Mar 7, 2021, 11:09 PM Navneeth Krishnan
wrote:
> Hi Ryanne,
>
> I commented out the alias config and set target to source as false.
Hi Ryanne,
I commented out the alias config and set target to source as false. Now I
don't see the error anymore and everything looks good. Thanks a lot for the
feedback. One more question, how can I check if the consumer group offsets
are replicated. If I switch over an application to this new cl
Thanks you all!
Blake, for your comment:
It'll require having a HA cluster running in another region, of course.
One other caveat is that it doesn't preserve the offsets of the records
-> I believe I can't afford to keep another cluster running due to cost
reasons.Can you elaborate on the offset
Navneeth, it looks like you are trying to override the cluster aliases to
the empty string. If you look closely, these properties are defined using
incorrect syntax. I'm not sure how the properties would be parsed, but you
should fix that to rule it out as a source of confusion.
Then, be aware tha
Hi,
To stream data from kafka topic to Google BigQuery, i see some references
to
wepay-kafka-connect-bigquery-2.1.0
However, this sounds like being depreciated.
Currently we can stream directory through Spark Structured Streaming with
writeStream. \
ou
