Hi Kamal, On Mon, Jun 17, 2024 at 11:44 AM Kamal Chandraprakash <kamal.chandraprak...@gmail.com> wrote: > > We can use the console-consumer to read the contents of the > `__remote_log_metadata` topic. Why are we proposing a new tool? > > sh kafka-console-consumer.sh --bootstrap-server localhost:9092 --topic > __remote_log_metadata --consumer-property exclude.internal.topics=false > --formatter > org.apache.kafka.server.log.remote.metadata.storage.serialization.RemoteLogMetadataSerde\$RemoteLogMetadataFormatter > --from-beginning >
Thanks from bringing this up. It works fine but a running broker is required, so it would make it inconvenient for a remote support engineer. Also you may have to deal with client security configuration, and it would be complicated to only dump specific segments. I'm adding to the rejected alternative for now, but I'm open to changes. > > > On Mon, Jun 17, 2024 at 12:53 PM Federico Valeri <fedeval...@gmail.com> > wrote: > > > Hi Divij, > > > > On Sun, Jun 16, 2024 at 7:38 PM Divij Vaidya <divijvaidy...@gmail.com> > > wrote: > > > > > > Hello Federico > > > > > > Please note that the topic-based RLMM is one of the possible > > > implementations of RLMM. Hence, whatever solution we design here should: > > 1\ > > > be explicit that this tooling only works for topic based RLMM 2\ specify > > > the handling of the failure mode when topic based RLMM is not being used. > > > > > > > That's true, thanks for pointing out. > > > > > I would argue that Topic based RLMM cannot be treated the same as other > > > internal topics. Topic based RLMM topic is an optional topic which can > > have > > > any possible schema (depending on plugin implementation) whereas > > > other internal topics are always guaranteed to be present with a fixed > > > schema. > > > > > > > Right, I updated the KIP with an improved option description. > > > > > In light of the above statements, the rejected alternative sounds better > > to > > > me because: > > > 1\ it provides the ability to dump logs for "any" RLMM implementation and > > > not just topic based RLMM. > > > 2\ we don't have to deal with schema evolution of topic based RLMM in > > this > > > tool. That responsibility will be delegated to the decoder class which > > the > > > operator can define using the flag "--value-decoder-class". > > > > > > Is there a reason that you are unable to use the rejected solution (which > > > requires no changes) for debugging purposes? > > > > > > > The rejected alternative will still be available, but I thought that > > having a dedicated flag would make debugging easier, as I guess most > > people will use the default RLMM implementation. I would be happy to > > hear other opinions on this. > > > > > -- > > > Divij Vaidya > > > > > > > > > > > > On Sat, Jun 15, 2024 at 4:43 PM Federico Valeri <fedeval...@gmail.com> > > > wrote: > > > > > > > Hi all, > > > > > > > > I'd like to kick off a discussion for KIP-1057, that proposes to add > > > > remote log metadata flag to the dump log tool, which is useful when > > > > debugging. > > > > > > > > > > > > > > https://cwiki.apache.org/confluence/display/KAFKA/KIP-1057%3A+Add+remote+log+metadata+flag+to+the+dump+log+tool > > > > > > > > Thanks, > > > > Fede > > > > > >
