Hi, Omnia,
Thanks for the updated KIP. A few more comments.
JR11. value.serializers and value.deserializers: Should they be of type
List? Also, where are key.serializers and key.deserializers?
JR12. Do we still need ComposableSerializer and ComposableDeserializer?
JR13. large.message.payload.st
Hi Omnia, nice design. I have few questions:
FV1: Looking at PayloadStore and PayloadResponse, "id" and "path"
seems to be the same thing, which is a reference to the data in the
external store. If my interpretation is correct, could we converge to
using one of the two terms?
FV2: I think Payload
Hi Jun, thanks for having the time to review this
> JR1. While the KIP is potentially useful, I am wondering who is responsible
> for retention for the objects in the payload store. Once a message with a
> reference is deleted, the key of the external object is lost and the object
> may never be
Hi, Omnia,
Thanks for the KIP. A few comments.
JR1. While the KIP is potentially useful, I am wondering who is responsible
for retention for the objects in the payload store. Once a message with a
reference is deleted, the key of the external object is lost and the object
may never be deleted.
J
Hi everyone,
If there is no any more feedback need to be addressed then I will open voting
on this KIP early next week.
Regards
Omnia
> On 28 May 2025, at 11:13, Omnia Ibrahim wrote:
>
> Hi Luke, I had another round of updating the KIP.
> I now made `PayloadStore::publish` returns a simplifi
Hi Luke, I had another round of updating the KIP.
I now made `PayloadStore::publish` returns a simplified version of
“PayloadResponse” which will now will simply have the "fullPayloadPath", and
“PayloadException”.
If PayloadResponse
- contains PayloadStoreException with `isRetriable` flag set
Hi Luke, sorry for late response.
> IMO, the retry should be the logic inside the
> "large.message.payload.store.class"
> implementation. If we really want it, I think we need to make it clear in
> which circumstance we will retry. For example, if it's an unknown exception
> thrown from S3 API, wh
Hi Omnia,
Thanks for the explanation and update.
It's better now.
Questions:
> 2. It's not clear how the "retry count" comes into play in the KIP. It
> needs more explanation.
My initial thinking is the retry configs are a must for all blob stores, so
we can provide them, and validate them for fr
Hi Kirk, thanks for having time to look into this,
> KT1: In PayloadReferenceValue, Is "payloadStoreClass" intended to be a
> fully-qualified Java class name, or something else? I'm considering the case
> where a Producer and/or Consumer are using a non-Java-based client.
This would be any stri
Hi Omnia,
A very interesting KIP! Thanks for the write up and discussion thus far!
A few questions:
KT1: In PayloadReferenceValue, Is "payloadStoreClass" intended to be a
fully-qualified Java class name, or something else? I'm considering the case
where a Producer and/or Consumer are using a n
Hi Luke,
>> 3. What does "LargeMessageFormatter" do in the process?
>> I thought all we want to do is to replace the "large value data" into a
>> path, and consumers will read the path via blob store class.
>> All these should be done in serializer/deserializer, so why do we need the
>> formatter?
Hi Luke, thanks for having the time to look into the KIP
> 2. It's not clear how the "retry count" comes into play in the KIP. It
> needs more explanation.
My initial thinking is the retry configs are a must for all blob stores, so we
can provide them, and validate them for free for all blob store
Hi Omnia,
Thanks for proposing this feature that many users expected.
Some comments:
1. It's quite interesting to see the idea of chained
serializer/deserializer used here. I like it.
2. It's not clear how the "retry count" comes into play in the KIP. It
needs more explanation.
3. What does "La
Pump this thread up
> On 11 Apr 2025, at 16:32, Omnia Ibrahim wrote:
>
> Thanks PoAn for having a look into the KIP
>> PY_1: Does LargeMessageDeserializer need
>> large.message.blob.store.blob.id_generator
>> configuration? The blob id is encapsulated in Kafka event by
>> LargeMessageSerialize
Thanks PoAn for having a look into the KIP
> PY_1: Does LargeMessageDeserializer need
> large.message.blob.store.blob.id_generator
> configuration? The blob id is encapsulated in Kafka event by
> LargeMessageSerializer.
> The LargeMessageDeserializer can get the id from the event.
You are right,
Hi Omnia,
Thanks for the KIP. It’s good to integrate blob store for large objects.
PY_1: Does LargeMessageDeserializer need
large.message.blob.store.blob.id_generator
configuration? The blob id is encapsulated in Kafka event by
LargeMessageSerializer.
The LargeMessageDeserializer can get the id
16 matches
Mail list logo
