> On March 8, 2014, 12:48 a.m., Guozhang Wang wrote: > > core/src/main/scala/kafka/utils/Utils.scala, line 545 > > <https://reviews.apache.org/r/18774/diff/3/?file=513166#file513166line545> > > > > Since this logging entry is from the Utils we will not know the id of > > the producer either from the logging head. Could we also add this along > > with key/values?
Will the producer id be useful? If it's for debugging, ProducerPerformance can generate different msg ids in different producer instances. For MM, you probably just want to know if there is a failure during send. - Jun ----------------------------------------------------------- This is an automatically generated e-mail. To reply, visit: https://reviews.apache.org/r/18774/#review36588 ----------------------------------------------------------- On March 7, 2014, 1:42 a.m., Jun Rao wrote: > > ----------------------------------------------------------- > This is an automatically generated e-mail. To reply, visit: > https://reviews.apache.org/r/18774/ > ----------------------------------------------------------- > > (Updated March 7, 2014, 1:42 a.m.) > > > Review request for kafka. > > > Bugs: KAFKA-1240 > https://issues.apache.org/jira/browse/KAFKA-1240 > > > Repository: kafka > > > Description > ------- > > 1. Added a errorLoggingCallback and used it in ProducerPerformance and new > MirrorMaker. 2. Set retry.backoff.ms in ProducerPerformance. > > > Log error when running in async mode. > > > 1. Converted all existing testcases in replication_testsuite and > mirror_maker_testsuite to use the new producer. 2. Minor cleanup to > ProducerPerformance and MirrorMaker. 3. Fixed a few places in python script > where we forget to release the lock, which can cause the system test to hang > when stopping all entities. 4. Left migration_testsuite as it is. We probably > can just delete the suite when we phase out the old producer. 5. There are 4 > testcase failures in replication_testsuite and 2 testcase failsures in > mirror_maker. These may be fixed with the configrable retry backoff time. > Will rerun the tests. 6. To void having to duplicate too many test cases > exercising both the old and the new producer, I propose that we just > duplicate the following testcases to run with the old producer (1) all > testcases in mirrormaker_testsuite and (2) testcases 0101-0110 and 4011-4018 > in replication_testsuite. Most testcases in replication_testsuite are really > for testing the replication logic on the br okers. From the producer perspective, as long as we can exercise the leader failure case, it doesn't really matter how the leader fails. > > > Diffs > ----- > > core/src/main/scala/kafka/tools/newproducer/MirrorMaker.scala > d23ef9a31aa0cc38b60e7651ae85ad6ad7dfcbf4 > core/src/main/scala/kafka/utils/Utils.scala > a89b0463685e6224d263bc9177075e1bb6b93d04 > perf/src/main/scala/kafka/perf/ProducerPerformance.scala > 5d399d94eafad7e3e50c538810f0aff53c59073f > system_test/migration_tool_testsuite/migration_tool_test.py > ce6f4f684beaa272409f6e25b74c957a9591206b > system_test/mirror_maker_testsuite/config/mirror_producer.properties > b2bf2c23b6b397571113fba7e75cead6404561a4 > system_test/mirror_maker_testsuite/mirror_maker_test.py > dfffb4e977ef22f48c6e503687b89ac33edc5338 > > system_test/mirror_maker_testsuite/testcase_5001/testcase_5001_properties.json > 287cab9a9b6b9e9d117905a1ac06708a74dfadcd > > system_test/mirror_maker_testsuite/testcase_5002/testcase_5002_properties.json > 5457eb1d26526759dcdd69895be633f76c01506a > > system_test/mirror_maker_testsuite/testcase_5003/testcase_5003_properties.json > 98fefee5838a2c7a1255042ac637dae46d72f51c > > system_test/mirror_maker_testsuite/testcase_5004/testcase_5004_properties.json > 6067b1263d0a5ebad0bd2846da89262c79d72f26 > > system_test/mirror_maker_testsuite/testcase_5005/testcase_5005_properties.json > 58483ad375cd1c7ec8dd8586af142f1b9c1bf437 > > system_test/mirror_maker_testsuite/testcase_5006/testcase_5006_properties.json > 1d9190ce7c1b5eb72b43d669e777a6398c25eed6 > system_test/replication_testsuite/replica_basic_test.py > e20130b85aecdacc8473f3e16abba007a3eeb3eb > > system_test/replication_testsuite/testcase_0001/testcase_0001_properties.json > 9bc164bee1dd0ad64904598fed555c9229cf82a4 > > system_test/replication_testsuite/testcase_0002/testcase_0002_properties.json > c90d7536a75641322562a018dd089081d5f5788f > > system_test/replication_testsuite/testcase_0003/testcase_0003_properties.json > b62b8aa50a38cb20906781815ca050b3a43fa172 > > system_test/replication_testsuite/testcase_0004/testcase_0004_properties.json > b91cef88533cb5e7b70a222fd924cd49b786bc93 > > system_test/replication_testsuite/testcase_0005/testcase_0005_properties.json > 4b3f76a21642dc568cc343f5ffb3364968859503 > > system_test/replication_testsuite/testcase_0006/testcase_0006_properties.json > b9b3485c46b6193575873e50143b712fcfc45bd0 > > system_test/replication_testsuite/testcase_0007/testcase_0007_properties.json > 5c4351fad98ef0a81e7dd93f397cb8cd3e3439f8 > > system_test/replication_testsuite/testcase_0008/testcase_0008_properties.json > 79cfed8bc63e429ce4875e68e85e2ca72fb2f7e9 > > system_test/replication_testsuite/testcase_0009/testcase_0009_properties.json > a52b709579f58c6c7766b07691c41a971bb4dd95 > > system_test/replication_testsuite/testcase_0010/testcase_0010_properties.json > 8d4b5fea2d1f91bced397af14d47b0e24ee04872 > > system_test/replication_testsuite/testcase_0011/testcase_0011_properties.json > b03f9cf059899ce42e0250f084b5a445ec4aa8bc > > system_test/replication_testsuite/testcase_0021/testcase_0021_properties.json > 40c2f8df13051ed90eb1ddb6a2005b43518213f5 > > system_test/replication_testsuite/testcase_0022/testcase_0022_properties.json > 7a3046035329f8986339fe25c0da4c2ecaa9b1ea > > system_test/replication_testsuite/testcase_0023/testcase_0023_properties.json > d921f01f332a4ed810b3ce12540a52892f89f178 > > system_test/replication_testsuite/testcase_0024/testcase_0024_properties.json > 839eb1476981ead1a82ecc4c00703eb1eec208ef > > system_test/replication_testsuite/testcase_0101/testcase_0101_properties.json > 85e4b61751b12dc4a9092097c1ae13a5d8947dee > > system_test/replication_testsuite/testcase_0102/testcase_0102_properties.json > 0d2f59fbcaa30159e26e01e83d9957039702a902 > > system_test/replication_testsuite/testcase_0103/testcase_0103_properties.json > 34acfa9d5c684e6221f54ee32ce89f47088bbd5a > > system_test/replication_testsuite/testcase_0104/testcase_0104_properties.json > 4145345fc433a294ae4094b698bf8d1f3776367f > > system_test/replication_testsuite/testcase_0105/testcase_0105_properties.json > 2eecc768e7e9711dd733819d7111090137435783 > > system_test/replication_testsuite/testcase_0106/testcase_0106_properties.json > 744174eab1c82984f92bd8f120e1df2a4892cef0 > > system_test/replication_testsuite/testcase_0107/testcase_0107_properties.json > e881b138c1fc042666e99187270e851f821e84ee > > system_test/replication_testsuite/testcase_0108/testcase_0108_properties.json > 7b48fdb441ad5931c1146d3797a9e5dc46d27b94 > > system_test/replication_testsuite/testcase_0109/testcase_0109_properties.json > a98ae0390bf556357b867a3d0c50022406f78a63 > > system_test/replication_testsuite/testcase_0110/testcase_0110_properties.json > f51abc156ae634b268213e9d97a96565484b5269 > > system_test/replication_testsuite/testcase_0111/testcase_0111_properties.json > fff0d689eba5f31248be1f972e5481ac8acc3438 > > system_test/replication_testsuite/testcase_0112/testcase_0112_properties.json > 636f02a2342d8e9d35c14b321e61f76c74b07339 > > system_test/replication_testsuite/testcase_0113/testcase_0113_properties.json > bdb885e43829e3075c875ec587dbc78d01e1da87 > > system_test/replication_testsuite/testcase_0114/testcase_0114_properties.json > ca51c5f5f183fccb8e3400a429533bb66547eeb7 > > system_test/replication_testsuite/testcase_0115/testcase_0115_properties.json > 459becf251d7cf18eb60c9da7473abc5515c5706 > > system_test/replication_testsuite/testcase_0116/testcase_0116_properties.json > c9471bc62ac6ac02022a53738dfb15fa65105c71 > > system_test/replication_testsuite/testcase_0117/testcase_0117_properties.json > 8159464b16c2c478f7c34e567e7dc83ab0d89545 > > system_test/replication_testsuite/testcase_0118/testcase_0118_properties.json > f73f0e43a3cf7176202232e768005c1279775e47 > > system_test/replication_testsuite/testcase_0119/testcase_0119_properties.json > 442a481230b151cb267991d4d6bdacee2d49344f > > system_test/replication_testsuite/testcase_0121/testcase_0121_properties.json > 606aad399dd3a69f80f86284c5cc3854fc3d237d > > system_test/replication_testsuite/testcase_0122/testcase_0122_properties.json > ea475361c8c5493c9bd07a0bc0ec32b711bc3adb > > system_test/replication_testsuite/testcase_0123/testcase_0123_properties.json > f9c92733cf4656e621260f58a4ee6b3f4422158e > > system_test/replication_testsuite/testcase_0124/testcase_0124_properties.json > 02f8506b20d345c3d61c8deb2a8fd1bdccc86f28 > > system_test/replication_testsuite/testcase_0125/testcase_0125_properties.json > 00482791aa9699f8d4f7d0b8edfdd01fd2fbef36 > > system_test/replication_testsuite/testcase_0126/testcase_0126_properties.json > 5119e61bbfca3ae4a7d028d1a39150d422d3c900 > > system_test/replication_testsuite/testcase_0127/testcase_0127_properties.json > 8b53fa7df5056e3d1b8ff76e5615059e5932d833 > > system_test/replication_testsuite/testcase_0128/testcase_0128_properties.json > e8edb9f4c70099c89172beb7672f23ab7cfcb2b6 > > system_test/replication_testsuite/testcase_0131/testcase_0131_properties.json > a140882287b9cf3453167b01255b949a705a9f6e > > system_test/replication_testsuite/testcase_0132/testcase_0132_properties.json > 48b30c7e014f0b5d504af1a760599525863e8895 > > system_test/replication_testsuite/testcase_0133/testcase_0133_properties.json > 8276aae0aa8226461e4595a17068e63442ddbcf5 > > system_test/replication_testsuite/testcase_0134/testcase_0134_properties.json > 73bb85995226230bcef47d890f72ec1436f24878 > > system_test/replication_testsuite/testcase_0151/testcase_0151_properties.json > eebba4db5d978267daf3ebe00615ac7711fb29d3 > > system_test/replication_testsuite/testcase_0152/testcase_0152_properties.json > debf544b3541a7297a42a4aae478a0c1d315f216 > > system_test/replication_testsuite/testcase_0153/testcase_0153_properties.json > 57b7d985236c83d4405f4f6a5a492022364c0b64 > > system_test/replication_testsuite/testcase_0154/testcase_0154_properties.json > c09fab719459d4692832f269898162438f3de69d > > system_test/replication_testsuite/testcase_0155/testcase_0155_properties.json > dd5ac52f86b023682b437c1d934373f827e05bcd > > system_test/replication_testsuite/testcase_0156/testcase_0156_properties.json > 8236ca58994b5e81f477c9ac6df837694e7967a9 > > system_test/replication_testsuite/testcase_0157/testcase_0157_properties.json > a28bf81e2b79e61b0dc7ca3fcce4e1dd592b9007 > > system_test/replication_testsuite/testcase_0158/testcase_0158_properties.json > 3d6edbdb16c5838b3f1d12a20c5d200d03668c63 > > system_test/replication_testsuite/testcase_0159/testcase_0159_properties.json > 030c9e8f0560f49291bf426b99a5a691caff210c > > system_test/replication_testsuite/testcase_0201/testcase_0201_properties.json > c6f8a23f358c5f7ab23ac48ce9b78062188b3f48 > > system_test/replication_testsuite/testcase_0202/testcase_0202_properties.json > 7b1a4c25c73fb274dfa8aba190c079330af8998f > > system_test/replication_testsuite/testcase_0203/testcase_0203_properties.json > 47276a85a038e278de75f1775da9ec9ce7912695 > > system_test/replication_testsuite/testcase_0204/testcase_0204_properties.json > 3742cfad8f25149accb062d4f4b7d23de484bcf3 > > system_test/replication_testsuite/testcase_0205/testcase_0205_properties.json > d41672d0b281810fb0b2355a77433287b6f9d4ca > > system_test/replication_testsuite/testcase_0206/testcase_0206_properties.json > a32d8882deb14fe26e8bc42ba500c7ca465f0dc3 > > system_test/replication_testsuite/testcase_0207/testcase_0207_properties.json > 573787099f92c50d629358a94ca2fb7e7ae4c264 > > system_test/replication_testsuite/testcase_0208/testcase_0208_properties.json > c3d1d2cb90ce21f4fe268dfabe731ea76308652a > > system_test/replication_testsuite/testcase_0209/testcase_0209_properties.json > 8d2cebafa0fb33c6fc729eacbb83d02ae45132a4 > > system_test/replication_testsuite/testcase_0251/testcase_0251_properties.json > 76ea6775657c916b543d36882b35fae3be757841 > > system_test/replication_testsuite/testcase_0252/testcase_0252_properties.json > 672b3ff883f2705d204981bf4a5140f333e345f5 > > system_test/replication_testsuite/testcase_0253/testcase_0253_properties.json > 19bd096b837353fae59a133feb413b5589ddb385 > > system_test/replication_testsuite/testcase_0254/testcase_0254_properties.json > 8d6add098da11324c065c856700c1296b827297a > > system_test/replication_testsuite/testcase_0255/testcase_0255_properties.json > 2a7e7776214d6071b492353c28023754ec815d6c > > system_test/replication_testsuite/testcase_0256/testcase_0256_properties.json > 91fbc9da75e2998ae75fbec140301c671ff92631 > > system_test/replication_testsuite/testcase_0257/testcase_0257_properties.json > 5594a9bc76d1ce50ddbf1668549dadca8eea8e84 > > system_test/replication_testsuite/testcase_0258/testcase_0258_properties.json > 6eabd3772dbe643ebca1495f174d2a228eccda02 > > system_test/replication_testsuite/testcase_0259/testcase_0259_properties.json > 3bbeeb10adfd8153b99f24fb55896b08de917974 > > system_test/replication_testsuite/testcase_0301/testcase_0301_properties.json > 496340c574ec35f0d5bd82e072692aee8221f05c > > system_test/replication_testsuite/testcase_0302/testcase_0302_properties.json > beabc86e2cdbac9cd8c5770f1e8924fbcc3dbab4 > > system_test/replication_testsuite/testcase_0303/testcase_0303_properties.json > 0e8f5a45895a32a780096aa2de8f619221989f7c > > system_test/replication_testsuite/testcase_0304/testcase_0304_properties.json > 98a385556c7819eb846182ae1df45b7d652caf86 > > system_test/replication_testsuite/testcase_0305/testcase_0305_properties.json > e09ac9c09b971e5ddf5b70181f6a2b75f3f22c02 > > system_test/replication_testsuite/testcase_0306/testcase_0306_properties.json > bcfe91b08c7e2424b2e8f012591caa47c1004387 > > system_test/replication_testsuite/testcase_0307/testcase_0307_properties.json > da07a9e4751b53269942cb984f67c2d0f1f46cbc > > system_test/replication_testsuite/testcase_0308/testcase_0308_properties.json > 0fecafc1a49c6aae3d171ab9d8f08944f7e8af06 > > system_test/replication_testsuite/testcase_0309/testcase_0309_properties.json > 90bd40442cc95ba049d4d11a91547de92b1ee9ff > system_test/replication_testsuite/testcase_1/testcase_1_properties.json > 7e1b3fbf2d22bcb33cc2f8dd52060f3f728eec0a > > system_test/replication_testsuite/testcase_4001/testcase_4001_properties.json > d2ffd952416e7822d2be4e171b5b5f951759408b > > system_test/replication_testsuite/testcase_4002/testcase_4002_properties.json > c86525db84ad0f926b96cfc93a0f8508d3e59981 > > system_test/replication_testsuite/testcase_4003/testcase_4003_properties.json > b77e4fdf400172a0b1bca4436223f9fd3928d705 > > system_test/replication_testsuite/testcase_4004/testcase_4004_properties.json > e753327795162b07b628b620e0a8b9e42d955062 > > system_test/replication_testsuite/testcase_4005/testcase_4005_properties.json > 5468401c6b41d280185d005d132c7a045ef50620 > > system_test/replication_testsuite/testcase_4006/testcase_4006_properties.json > e5ab0a0e8698133037cbf879873fc57a864211d4 > > system_test/replication_testsuite/testcase_4007/testcase_4007_properties.json > 7aa6e9a72ca74858e824abe251c666606fe10e4d > > system_test/replication_testsuite/testcase_4008/testcase_4008_properties.json > 08aa108f4f43a02424780cef4285383978ec8642 > > system_test/replication_testsuite/testcase_4011/testcase_4011_properties.json > 512fafb949c0d4b3a54030a5b87faeefca8c7ed2 > > system_test/replication_testsuite/testcase_4012/testcase_4012_properties.json > 9b711af013994357f2cc3a3046ba319326f9c1d8 > > system_test/replication_testsuite/testcase_4013/testcase_4013_properties.json > 3836366b3b19e5ab48c2e101c23978a165dbe4ba > > system_test/replication_testsuite/testcase_4014/testcase_4014_properties.json > 86ab75a46a48e1d99e42d8a9fc920b77d570da0b > > system_test/replication_testsuite/testcase_4015/testcase_4015_properties.json > 82d51b6072c6d6aaec827e357c82d4932ee60f96 > > system_test/replication_testsuite/testcase_4016/testcase_4016_properties.json > 31c1be0d73393c6e5fe72afff8494030f6d61eb9 > > system_test/replication_testsuite/testcase_4017/testcase_4017_properties.json > 72f78b074243ffabefe59c7158d3b45cda8593ea > > system_test/replication_testsuite/testcase_4018/testcase_4018_properties.json > ee459f4e35d682811a89bb8952d8d1afbff8e62d > > system_test/replication_testsuite/testcase_9051/testcase_9051_properties.json > 958eef7df0be16b5a3d6f126a5272b3dc3bffe1f > system_test/utils/kafka_system_test_utils.py > 5d2b7dfd46b07ab8749b1b2293e35183f14ae9af > > Diff: https://reviews.apache.org/r/18774/diff/ > > > Testing > ------- > > Rerun all system test. There is only one test in replication_suite that's > failing. Will file a separate jira. > > > Thanks, > > Jun Rao > >
