Hi Guozhang, I checked the code again. Actually CRC check probably won't fail. The newly added timestamp field might be treated as keyLength instead, so we are likely to receive an IllegalArgumentException when try to read the key. I'll update the KIP.
Thanks, Jiangjie (Becket) Qin On Thu, Sep 3, 2015 at 12:48 PM, Jiangjie Qin <j...@linkedin.com> wrote: > Hi, Guozhang, > > Thanks for reading the KIP. By "old consumer", I meant the > ZookeeperConsumerConnector in trunk now, i.e. without this bug fixed. If we > fix the ZookeeperConsumerConnector then it will throw exception complaining > about the unsupported version when it sees message format V1. What I was > trying to say is that if we have some ZookeeperConsumerConnector running > without the fix, the consumer will complain about CRC mismatch instead of > unsupported version. > > Thanks, > > Jiangjie (Becket) Qin > > On Thu, Sep 3, 2015 at 12:15 PM, Guozhang Wang <wangg...@gmail.com> wrote: > >> Thanks for the write-up Jiangjie. >> >> One comment about migration plan: "For old consumers, if they see the new >> protocol the CRC check will fail".. >> >> Do you mean this bug in the old consumer cannot be fixed in a >> backward-compatible way? >> >> Guozhang >> >> >> On Thu, Sep 3, 2015 at 8:35 AM, Jiangjie Qin <j...@linkedin.com.invalid> >> wrote: >> >> > Hi, >> > >> > We just created KIP-31 to propose a message format change in Kafka. >> > >> > >> > >> https://cwiki.apache.org/confluence/display/KAFKA/KIP-31+-+Message+format+change+proposal >> > >> > As a summary, the motivations are: >> > 1. Avoid server side message re-compression >> > 2. Honor time-based log roll and retention >> > 3. Enable offset search by timestamp at a finer granularity. >> > >> > Feedback and comments are welcome! >> > >> > Thanks, >> > >> > Jiangjie (Becket) Qin >> > >> >> >> >> -- >> -- Guozhang >> > >
