> On Aug. 6, 2014, 5:32 a.m., Timothy Chen wrote: > > core/src/main/scala/kafka/server/TransactionManager.scala, line 300 > > <https://reviews.apache.org/r/23568/diff/6/?file=653418#file653418line300> > > > > I'm just trying to think if there is a chance to have overlapping tx > > ids especially around failover. I'm not familiar with the tx changes so > > just wondering if the range size can cause issues. Seems like you're > > incrementing the zk version on each id fetch and allow a range of the batch > > size of them. I was thinking if I sent 999 messages and then failed over, > > and the new broker that took over has a small batch size now, even though > > the version has incremented the new range now overlaps then right? It might > > already been handled but just want to throw it out there. > > Dong Lin wrote: > Sure. Thanks for taking time to help think about it. > > Here I assume the batch size should be the same across producers > regardless of when they start. If a producer sends 999 messages, say with > txid in range 1000 to 1999, and fails over, the next producer will get the > txid in range 2000 to 2999. The range is guaranteed to be disjoint from > ranges used by previous/existing producers. Does this make sense? > > Dong Lin wrote: > Correction: I actually mean broker where I say producer in my response. > > Timothy Chen wrote: > The next producer will do so by looking at the default batch size with > the version, which is what I was thinking there really isn't anything > guarding aganist the range being changed from the code. > The safest thing to do is perhaps write the range into zk node data and > when the current range is smaller have a big warning. > But minimally we will want to leave a comment in the code on the batch > size to have this warning so this isn't something to experiment with as it > can mess up existing transactions.
It is the broker who looks at the default batch size with the version, and allocate unique txId to producer upon request. In other words, this code on the server side and we don't need to worry about Kafka user changing this code. Am I right? - Dong ----------------------------------------------------------- This is an automatically generated e-mail. To reply, visit: https://reviews.apache.org/r/23568/#review49696 ----------------------------------------------------------- On Aug. 6, 2014, 4:25 a.m., Dong Lin wrote: > > ----------------------------------------------------------- > This is an automatically generated e-mail. To reply, visit: > https://reviews.apache.org/r/23568/ > ----------------------------------------------------------- > > (Updated Aug. 6, 2014, 4:25 a.m.) > > > Review request for kafka. > > > Bugs: KAFKA-1523 > https://issues.apache.org/jira/browse/KAFKA-1523 > > > Repository: kafka > > > Description > ------- > > KAFKA-1523 Transaction manager and its failover handling. > > > Diffs > ----- > > core/src/main/scala/kafka/admin/TopicCommand.scala > 003a09c6160618bc94858ebc0d806b2aa4158e0a > core/src/main/scala/kafka/cluster/Partition.scala > 134aef9c88068443d4d465189f376dd78605b4f8 > core/src/main/scala/kafka/common/Topic.scala > ad759786d1c22f67c47808c0b8f227eb2b1a9aa8 > core/src/main/scala/kafka/controller/ControllerChannelManager.scala > ecbfa0f328ba6a652a758ab20cacef324a8b2fb8 > core/src/main/scala/kafka/message/Message.scala > d2a7293c7be4022af30884330924791340acc5c1 > core/src/main/scala/kafka/server/KafkaApis.scala > fd5f12ee31e78cdcda8e24a0ab3e1740b3928884 > core/src/main/scala/kafka/server/KafkaConfig.scala > 1a45f8716ccc0398cf9395d91d66199d16882aae > core/src/main/scala/kafka/server/KafkaServer.scala > 28711182aaa70eaa623de858bc063cb2613b2a4d > core/src/main/scala/kafka/server/ReplicaManager.scala > 897783cb756de548a8b634876f729b63ffe9925e > core/src/main/scala/kafka/server/RequestPurgatory.scala > 3d0ff1e2dbd6a5c3587cffa283db70415af83a7b > core/src/main/scala/kafka/server/TransactionManager.scala PRE-CREATION > core/src/main/scala/kafka/utils/ZkUtils.scala > dcdc1ce2b02c996294e19cf480736106aaf29511 > > Diff: https://reviews.apache.org/r/23568/diff/ > > > Testing > ------- > > > Thanks, > > Dong Lin > >
