Unable to reset kafka offset in a consumer group

Hi All, I'm trying to reset a kafka offset for a topic in a consumer group, i have stopped all the consumers using the consumer group but i'm still receiving the message that the current state is stable. Is there a way to put the consumer group to an inactive state after stopping all the c

Re: Unable to reset kafka offset in a consumer group

t forward by an increment of 1 just for > this topic? > kafka-consumer-groups.sh script has an option `--shift-by` to shift current > offset by 'n'. > I think this is what you are looking for. > > Thank you > Luke > > On Thu, Aug 25, 2022 at 4:25 PM Chris Pear

Re: Unable to reset kafka offset in a consumer group

:57 PM Chris Peart wrote: Thanks Luke, The problem I was having was that the consumer group was not inactive even with all the consumers stopped. Managed to work around this now. Many Thanks Chris On 25 Aug 2022, at 10:12 am, Luke Chen wrote: Hi Chris, is there a way to force the offset

Help with log.dirs please

Hi All, I have a 4 node kafka cluster running version 2.8.1, we have started pushing data to the cluster but can only see one disk being used. We had 6 disk configured as non-raid and 1 partition per disk, we have the following in fstab: /dev/mapper/VolGroup01-data/data/1xfsno

Re: Help with log.dirs please

> it may be entirely normal that his has happened. > > > — > Peter Bukowinski > >> On Sep 15, 2022, at 3:35 AM, Chris Peart wrote: >> >> >> >> Hi All, >> >> I have a 4 node kafka cluster running version 2.8.1, we have started pushing

Re: Help with log.dirs please

s up. Search the logs for > ‘log.dirs’ and ensure the correct configs are loaded. > > -- > Peter > >> On Sep 15, 2022, at 11:10 PM, Chris Peart wrote: >> >> Hi Peter, >> >> Thanks for your response, we have the following configuration: >> >&

Re: Help with log.dirs please

figuration somewhere, maybe in zookeeper? Any help here would be much appreciated :) Many Thanks, Chris On 2022-09-16 08:22, Chris Peart wrote: Thanks Peter, I'll check the logs next week and let you know my findings. Many Thanks Chris On 16 Sep 2022, at 7:45 am, Peter Bukowinski w

Re: Help with log.dirs please

resetting Kafka would be great if ok please. Many Thanks Chris > On 20 Sep 2022, at 3:37 pm, Chris Peart wrote: > >  > Hi Peter, > > I have checked the logs on all 4 brokers and could only see > /data/1/data/kafka being used, log.dirs config in the logs showed al

Re: Help with log.dirs please

h, you have the steps correct. An alternative to > changing the zk path is to use zkCli to remove the paths. If you use a > zookeeper chroot, just delete everything from that chroot down from zkCli, > e.g. `rmr /[kafka-chroot]` > > — > Peter > >> On Sep 20, 2022, at 11:56

Re: Help with log.dirs please

i've exhausted all my options. Many Thanks, Chris On 2022-09-20 21:43, Chris Peart wrote: Thanks Peter, I'll give this a go tomorrow and let you know how I get on. Many Thanks, Chris On 20 Sep 2022, at 9:32 pm, Peter Bukowinski wrote: Hi Chris, If the configs are correct and the permissi

Re: Help with log.dirs please

ote: > > Hi Chris, > > Are you sure kafka is starting up (and remains up) successfully? From the > error, it seems like none of the brokers are online. > > — > Peter > > >> On Sep 21, 2022, at 4:02 AM, Chris Peart wrote: >> >> >>

Re: Help with log.dirs please

ep ‘>’ | sort -n -k3`, do you get a list of all your brokers? > > — > Peter > >> On Sep 21, 2022, at 10:26 AM, Chris Peart wrote: >> >> Hi Peter, >> Kafka brokers are staying up and show when I query zookeeper brokers/ids >> Is the any other files that require delet

Re: Help with log.dirs please

? -- Peter On Sep 21, 2022, at 10:26 AM, Chris Peart wrote: Hi Peter, Kafka brokers are staying up and show when I query zookeeper brokers/ids Is the any other files that require deletion as it doesn't make sense I have 0 brokers, when trying to create a topic. I get no errors when listing

Re: Help with log.dirs please

t; > — > Peter > >> On Sep 22, 2022, at 1:08 AM, Chris Peart wrote: >> >> Hi Peter, >> >> >> >> Thanks for getting back to me on this, i have done the following checks as >> suggested: >> >> Kafka version: 2.8.1 >>

Help with MM2 active/passive configuration

Hi All, I'm new to MirrorMaker and have a production cluster running version 2.8.1 and have a development cluster running the same version. Our prod cluster has 4 brokers and our dev cluster has 3 brokers, both have a replication factor of 3. I would like to setup an active/passive replic

Re: Help with MM2 active/passive configuration

's > mirror maker is configured. If you've got a Connect cluster already, I > recommend using that and manually configuring the MirrorSourceConnector. > > Ryanne > >> On Wed, Oct 19, 2022, 5:26 AM Chris Peart wrote: >> >> >> >> Hi All, >

Re: Help with MM2 active/passive configuration

Hi Ryanne, I cannot find what you mentioned, all i'm looking for are some example configurations for and active/passive setup. Many Thanks, Chris On 2022-10-19 18:32, Chris Peart wrote: Thanks Ryan, can you provide a link to the readme please. Many Thanks Chris On 19 Oct 2022, at

__consumer_offsets & __transaction_state topics have ReplicationFactor: 1

Hi, We have a kafka production cluster that was setup with the defaults for __consumer_offsets & __transaction_state topics. Is there a way to increase the replication factor from 1 to 3 using the kafka-reassign-partitions tool? We are also replacing our 4 brokers with new brokers, this h

Re: __consumer_offsets & __transaction_state topics have ReplicationFactor: 1

t;,"partition":49,"replicas":[1],"log_dirs":["any"]}]} > > Save this to use as the --reassignment-json-file option during rollback > Successfully started partition reassignment for __consumer_offsets-49 > kafka % ./bin/kafka-topics.sh --

Re: __consumer_offsets & __transaction_state topics have ReplicationFactor: 1

ions in it. > > Yeah I'm pretty sure you'd do basically the same for __transaction_state. I > havent tested that myself locally so might be worth doing so on your end. > > Hope that helps a bit. > > Andrew > >> On Fri, Dec 16, 2022 at 11:41 AM Chris Peart w

RE: __consumer_offsets & __transaction_state topics have ReplicationFactor: 1

> "replicas": [ 2, 3 ] > }, > { > "topic": "__consumer_offsets", > "partition": 3, > "replicas": [ 3, 4 ] > }, > { > "topic": "__consumer_offsets", > "p

RE: __consumer_offsets & __transaction_state topics have ReplicationFactor: 1

ly it is to lose replicas at the same time). But I’d start with 3 unless you have a reason to go higher. Andrew Sent from my iPhone > On Dec 18, 2022, at 6:57 AM, Chris Peart wrote: > > Hi Andrew & Alex, > > Thanks for all your help on this, it's much appreciated.

Please unsubscribe ch...@peart.me.uk

Please unsubscribe ch...@peart.me.uk, I am unable to send from this address anymore but it does forward to chris-pe...@outlook.com Many Thanks, Chris