Get last snapshot from compacted topic

I'd like to save a snapshot of a processing node's state in a compacted kafka topic. A large number of nodes would save their snapshots in the same partition. What is an efficient way for a (restarted) node to find the offset of its latest snapshot? Using just Kafka (no database, local file, e

ZK path name mentioned in the online Replication Design document seems to be incorrect

ent was not updated when the path changed? regards, aki

swagger/openapi spec for Kafka-Connect REST API?

I am looking for the openapi spec for Kafka-Connect API but I couldn't find one. If it is available somewhere, could someone tell me where I can find it? regards, aki

Re: swagger/openapi spec for Kafka-Connect REST API?

API? > > Hi, > > I'm also searching for this. If you find it, please let me know. > Thank you. > -- > Miguel Silvestre > > > On Mon, May 18, 2020 at 8:59 AM Aki Yoshida wrote: > > > I am looking for the openapi spec for Kafka-Connect API but I couldn'

Using FileConfigProvider with restricted path?

Hi, I am wondering if there is a possibility to restrict the path used for the FileConfigProvider's value in the configuration e.g., ${file:/xxxpath/xxxfile.properties:xxxname}. I am looking for a way to restrict this path. Thanks regards, aki

When using MM2, how should the consumer be switched from source to target?

ere also migrated to the target and the consumer could simply start subscribing from the mirrored topic from the continued offset. Did I miss some configuration parameters or does the consumer need to perform some actions to be able to consume the records seamlessly? I appreciate for your help. Thanks. regards, aki

Re: When using MM2, how should the consumer be switched from source to target?

there is no way to use the console client but use java API RemoteClusterUtils.translateOffsets() to get the offset and re-consuming from the new topic. Is this correct? El jue, 17 dic 2020 a las 21:18, Aki Yoshida () escribió: > > Hi, > I have a question regarding how to migrate a con

MirrorConnectorsIntegrationTest's testOneWayReplicationWithAutoOffsetSync

pposed to provide? Any help to understand this matter would be greatly appreciated. Regards, aki

--override option for bin/connect-distributed.sh

/kafka/commit/1e54536598d1ce328d0aee10edb728270cc04af1 Could someone tell me if this is a good idea or a bad idea? If bad, is there some alternative way to customise the properties? If good, can I create a PR? I would appreciate for your suggestion. Thanks. regards, aki

Re: --override option for bin/connect-distributed.sh

Hi Tom, thank for your suggestion. I'll follow the KIP process. regards, aki El mar, 22 dic 2020 a las 10:18, Tom Bentley () escribió: > > Hi Aki, > > Since this is a change to a public API of the project it would need to be > done through the KIP process [1]. Since writing

Re: When using MM2, how should the consumer be switched from source to target?

2ee963c%40%3Cusers.kafka.apache.org%3E Could you or someone comment on this? Thanks. Regards, aki El sáb, 2 ene 2021 a las 20:31, Ryanne Dolan () escribió: > > Aki, that's right. Prior to 2.7, you can use the translateOffsets method > from within your client code, or you can write a

Re: When using MM2, how should the consumer be switched from source to target?

fka/commit/9063db0f8c4a53f5d9764612af898981d499a7b7 regards, aki El mié, 13 ene 2021 a las 3:17, Ning Zhang () escribió: > > Hello Aki, > > Can you elaborate on "when there are some messages left in the original > cluster" ? > > On 2021/01/12 13:01:40, Aki Yoshida

Question about MM2's source-cluster.checkpoints.internal topic

rrent offset value. But this is a bit of pain. Thank you for any help. Regards, aki

question about property advertised.listeners

actual broker port within the container so that the broker itself can also find itself, right? thanks. regards, aki

Looking for some info on kafka homepage

o find this info) Could someone point me to the pages or if no such page is available, could we make one? Thanks regards, aki

Re: question about property advertised.listeners

Hi Kaufman, I wasn't thinking of using listeners to switch the running port itself. That makes it very simple. It was stupid of me. thank you. regards, aki 2016-11-30 17:43 GMT+01:00 Kaufman Ng : > The advertised.listeners property would not help you to map a different > port. If