It's either meaningless and ignored (for uncompressed messages) or a relative
offset for the messages in the set (for compressed messages).
Field
Description
Offset
This is the offset used in kafka as the log sequence number. When the producer
is sending non compressed messages, it can set the offsets to anything. When
the producer is sending compressed messages, to avoid server side
recompression, each compressed message should have offset starting from 0 and
increasing by one for each inner message in the compressed message. (see more
details about compressed messages in Kafka below)
-hans
> On Jan 18, 2017, at 11:17 PM, Paolo Patierno <ppatie...@live.com> wrote:
>
> Yes I know so ... what's the value of the Offset field in the MessageSet when
> producer sends messages ?
>
> ________________________________
> From: Hans Jespersen <h...@confluent.io>
> Sent: Wednesday, January 18, 2017 11:38:16 PM
> To: users@kafka.apache.org
> Subject: Re: Kafka Protocol : about "Offset" in the MessageSet
>
> Producer will not know the offset of the message(s) at the time they send to
> the broker but they can receive that information back as confirmation of
> successful publish.
>
> -hans
>
>
>
>
>
>> On Jan 18, 2017, at 1:25 AM, Paolo Patierno <ppatie...@live.com> wrote:
>>
>> Hi,
>>
>>
>> reading about the Kafka protocol specification here
>> (https://cwiki.apache.org/confluence/display/KAFKA/A+Guide+To+The+Kafka+Protocol)
>> I see that the producer and fetch requests share a MessageSet (for messages
>> sent and received).
>>
>> As it is defined, the MessageSet is [Offset MessageSize Message].
>>
>>
>> I'm ok on having the offset when client receives a message (knowing its
>> offset) but what about producer ? Does the producer send the "offset" as
>> destination of the message ? Or the "Offset" is empty in case of producer ?
>>
>>
>> Thanks,
>>
>> Paolo
>>
>>
>>
>> Paolo Patierno
>> Senior Software Engineer (IoT) @ Red Hat
>> Microsoft MVP on Windows Embedded & IoT
>> Microsoft Azure Advisor
>>
>> Twitter : @ppatierno<http://twitter.com/ppatierno>
>> Linkedin : paolopatierno<http://it.linkedin.com/in/paolopatierno>
>> Blog : DevExperience<http://paolopatierno.wordpress.com/>
>
