----------------------------------------------------------- This is an automatically generated e-mail. To reply, visit: https://reviews.apache.org/r/18774/#review36245 -----------------------------------------------------------
core/src/main/scala/kafka/tools/newproducer/MirrorMaker.scala <https://reviews.apache.org/r/18774/#comment67198> I had initially added multiple producer support to the mirror maker until we are fully convinced that a single producer's throughput with the highest durability setting can match the consumer's throughput. I haven't seen us publish these performance results yet. So to allow configuring MirrorMaker with multiple producers, I suggest we keep it as it at least for a while - Neha Narkhede On March 5, 2014, 5:56 p.m., Jun Rao wrote: > > ----------------------------------------------------------- > This is an automatically generated e-mail. To reply, visit: > https://reviews.apache.org/r/18774/ > ----------------------------------------------------------- > > (Updated March 5, 2014, 5:56 p.m.) > > > Review request for kafka. > > > Bugs: KAFKA-1240 > https://issues.apache.org/jira/browse/KAFKA-1240 > > > Repository: kafka > > > Description > ------- > > 1. Converted all existing testcases in replication_testsuite and > mirror_maker_testsuite to use the new producer. 2. Minor cleanup to > ProducerPerformance and MirrorMaker. 3. Fixed a few places in python script > where we forget to release the lock, which can cause the system test to hang > when stopping all entities. 4. Left migration_testsuite as it is. We probably > can just delete the suite when we phase out the old producer. 5. There are 4 > testcase failures in replication_testsuite and 2 testcase failsures in > mirror_maker. These may be fixed with the configrable retry backoff time. > Will rerun the tests. 6. To void having to duplicate too many test cases > exercising both the old and the new producer, I propose that we just > duplicate the following testcases to run with the old producer (1) all > testcases in mirrormaker_testsuite and (2) testcases 0101-0110 and 4011-4018 > in replication_testsuite. Most testcases in replication_testsuite are really > for testing the replication logic on the br okers. From the producer perspective, as long as we can exercise the leader failure case, it doesn't really matter how the leader fails. > > > Diffs > ----- > > core/src/main/scala/kafka/tools/newproducer/MirrorMaker.scala > d23ef9a31aa0cc38b60e7651ae85ad6ad7dfcbf4 > perf/src/main/scala/kafka/perf/ProducerPerformance.scala > 5d399d94eafad7e3e50c538810f0aff53c59073f > system_test/migration_tool_testsuite/migration_tool_test.py > ce6f4f684beaa272409f6e25b74c957a9591206b > system_test/mirror_maker_testsuite/config/mirror_producer.properties > b2bf2c23b6b397571113fba7e75cead6404561a4 > system_test/mirror_maker_testsuite/mirror_maker_test.py > dfffb4e977ef22f48c6e503687b89ac33edc5338 > > system_test/mirror_maker_testsuite/testcase_5001/testcase_5001_properties.json > 287cab9a9b6b9e9d117905a1ac06708a74dfadcd > > system_test/mirror_maker_testsuite/testcase_5002/testcase_5002_properties.json > 5457eb1d26526759dcdd69895be633f76c01506a > > system_test/mirror_maker_testsuite/testcase_5003/testcase_5003_properties.json > 98fefee5838a2c7a1255042ac637dae46d72f51c > > system_test/mirror_maker_testsuite/testcase_5004/testcase_5004_properties.json > 6067b1263d0a5ebad0bd2846da89262c79d72f26 > > system_test/mirror_maker_testsuite/testcase_5005/testcase_5005_properties.json > 58483ad375cd1c7ec8dd8586af142f1b9c1bf437 > > system_test/mirror_maker_testsuite/testcase_5006/testcase_5006_properties.json > 1d9190ce7c1b5eb72b43d669e777a6398c25eed6 > system_test/replication_testsuite/replica_basic_test.py > e20130b85aecdacc8473f3e16abba007a3eeb3eb > > system_test/replication_testsuite/testcase_0001/testcase_0001_properties.json > 9bc164bee1dd0ad64904598fed555c9229cf82a4 > > system_test/replication_testsuite/testcase_0002/testcase_0002_properties.json > c90d7536a75641322562a018dd089081d5f5788f > > system_test/replication_testsuite/testcase_0003/testcase_0003_properties.json > b62b8aa50a38cb20906781815ca050b3a43fa172 > > system_test/replication_testsuite/testcase_0004/testcase_0004_properties.json > b91cef88533cb5e7b70a222fd924cd49b786bc93 > > system_test/replication_testsuite/testcase_0005/testcase_0005_properties.json > 4b3f76a21642dc568cc343f5ffb3364968859503 > > system_test/replication_testsuite/testcase_0006/testcase_0006_properties.json > b9b3485c46b6193575873e50143b712fcfc45bd0 > > system_test/replication_testsuite/testcase_0007/testcase_0007_properties.json > 5c4351fad98ef0a81e7dd93f397cb8cd3e3439f8 > > system_test/replication_testsuite/testcase_0008/testcase_0008_properties.json > 79cfed8bc63e429ce4875e68e85e2ca72fb2f7e9 > > system_test/replication_testsuite/testcase_0009/testcase_0009_properties.json > a52b709579f58c6c7766b07691c41a971bb4dd95 > > system_test/replication_testsuite/testcase_0010/testcase_0010_properties.json > 8d4b5fea2d1f91bced397af14d47b0e24ee04872 > > system_test/replication_testsuite/testcase_0011/testcase_0011_properties.json > b03f9cf059899ce42e0250f084b5a445ec4aa8bc > > system_test/replication_testsuite/testcase_0021/testcase_0021_properties.json > 40c2f8df13051ed90eb1ddb6a2005b43518213f5 > > system_test/replication_testsuite/testcase_0022/testcase_0022_properties.json > 7a3046035329f8986339fe25c0da4c2ecaa9b1ea > > system_test/replication_testsuite/testcase_0023/testcase_0023_properties.json > d921f01f332a4ed810b3ce12540a52892f89f178 > > system_test/replication_testsuite/testcase_0024/testcase_0024_properties.json > 839eb1476981ead1a82ecc4c00703eb1eec208ef > > system_test/replication_testsuite/testcase_0101/testcase_0101_properties.json > 85e4b61751b12dc4a9092097c1ae13a5d8947dee > > system_test/replication_testsuite/testcase_0102/testcase_0102_properties.json > 0d2f59fbcaa30159e26e01e83d9957039702a902 > > system_test/replication_testsuite/testcase_0103/testcase_0103_properties.json > 34acfa9d5c684e6221f54ee32ce89f47088bbd5a > > system_test/replication_testsuite/testcase_0104/testcase_0104_properties.json > 4145345fc433a294ae4094b698bf8d1f3776367f > > system_test/replication_testsuite/testcase_0105/testcase_0105_properties.json > 2eecc768e7e9711dd733819d7111090137435783 > > system_test/replication_testsuite/testcase_0106/testcase_0106_properties.json > 744174eab1c82984f92bd8f120e1df2a4892cef0 > > system_test/replication_testsuite/testcase_0107/testcase_0107_properties.json > e881b138c1fc042666e99187270e851f821e84ee > > system_test/replication_testsuite/testcase_0108/testcase_0108_properties.json > 7b48fdb441ad5931c1146d3797a9e5dc46d27b94 > > system_test/replication_testsuite/testcase_0109/testcase_0109_properties.json > a98ae0390bf556357b867a3d0c50022406f78a63 > > system_test/replication_testsuite/testcase_0110/testcase_0110_properties.json > f51abc156ae634b268213e9d97a96565484b5269 > > system_test/replication_testsuite/testcase_0111/testcase_0111_properties.json > fff0d689eba5f31248be1f972e5481ac8acc3438 > > system_test/replication_testsuite/testcase_0112/testcase_0112_properties.json > 636f02a2342d8e9d35c14b321e61f76c74b07339 > > system_test/replication_testsuite/testcase_0113/testcase_0113_properties.json > bdb885e43829e3075c875ec587dbc78d01e1da87 > > system_test/replication_testsuite/testcase_0114/testcase_0114_properties.json > ca51c5f5f183fccb8e3400a429533bb66547eeb7 > > system_test/replication_testsuite/testcase_0115/testcase_0115_properties.json > 459becf251d7cf18eb60c9da7473abc5515c5706 > > system_test/replication_testsuite/testcase_0116/testcase_0116_properties.json > c9471bc62ac6ac02022a53738dfb15fa65105c71 > > system_test/replication_testsuite/testcase_0117/testcase_0117_properties.json > 8159464b16c2c478f7c34e567e7dc83ab0d89545 > > system_test/replication_testsuite/testcase_0118/testcase_0118_properties.json > f73f0e43a3cf7176202232e768005c1279775e47 > > system_test/replication_testsuite/testcase_0119/testcase_0119_properties.json > 442a481230b151cb267991d4d6bdacee2d49344f > > system_test/replication_testsuite/testcase_0121/testcase_0121_properties.json > 606aad399dd3a69f80f86284c5cc3854fc3d237d > > system_test/replication_testsuite/testcase_0122/testcase_0122_properties.json > ea475361c8c5493c9bd07a0bc0ec32b711bc3adb > > system_test/replication_testsuite/testcase_0123/testcase_0123_properties.json > f9c92733cf4656e621260f58a4ee6b3f4422158e > > system_test/replication_testsuite/testcase_0124/testcase_0124_properties.json > 02f8506b20d345c3d61c8deb2a8fd1bdccc86f28 > > system_test/replication_testsuite/testcase_0125/testcase_0125_properties.json > 00482791aa9699f8d4f7d0b8edfdd01fd2fbef36 > > system_test/replication_testsuite/testcase_0126/testcase_0126_properties.json > 5119e61bbfca3ae4a7d028d1a39150d422d3c900 > > system_test/replication_testsuite/testcase_0127/testcase_0127_properties.json > 8b53fa7df5056e3d1b8ff76e5615059e5932d833 > > system_test/replication_testsuite/testcase_0128/testcase_0128_properties.json > e8edb9f4c70099c89172beb7672f23ab7cfcb2b6 > > system_test/replication_testsuite/testcase_0131/testcase_0131_properties.json > a140882287b9cf3453167b01255b949a705a9f6e > > system_test/replication_testsuite/testcase_0132/testcase_0132_properties.json > 48b30c7e014f0b5d504af1a760599525863e8895 > > system_test/replication_testsuite/testcase_0133/testcase_0133_properties.json > 8276aae0aa8226461e4595a17068e63442ddbcf5 > > system_test/replication_testsuite/testcase_0134/testcase_0134_properties.json > 73bb85995226230bcef47d890f72ec1436f24878 > > system_test/replication_testsuite/testcase_0151/testcase_0151_properties.json > eebba4db5d978267daf3ebe00615ac7711fb29d3 > > system_test/replication_testsuite/testcase_0152/testcase_0152_properties.json > debf544b3541a7297a42a4aae478a0c1d315f216 > > system_test/replication_testsuite/testcase_0153/testcase_0153_properties.json > 57b7d985236c83d4405f4f6a5a492022364c0b64 > > system_test/replication_testsuite/testcase_0154/testcase_0154_properties.json > c09fab719459d4692832f269898162438f3de69d > > system_test/replication_testsuite/testcase_0155/testcase_0155_properties.json > dd5ac52f86b023682b437c1d934373f827e05bcd > > system_test/replication_testsuite/testcase_0156/testcase_0156_properties.json > 8236ca58994b5e81f477c9ac6df837694e7967a9 > > system_test/replication_testsuite/testcase_0157/testcase_0157_properties.json > a28bf81e2b79e61b0dc7ca3fcce4e1dd592b9007 > > system_test/replication_testsuite/testcase_0158/testcase_0158_properties.json > 3d6edbdb16c5838b3f1d12a20c5d200d03668c63 > > system_test/replication_testsuite/testcase_0159/testcase_0159_properties.json > 030c9e8f0560f49291bf426b99a5a691caff210c > > system_test/replication_testsuite/testcase_0201/testcase_0201_properties.json > c6f8a23f358c5f7ab23ac48ce9b78062188b3f48 > > system_test/replication_testsuite/testcase_0202/testcase_0202_properties.json > 7b1a4c25c73fb274dfa8aba190c079330af8998f > > system_test/replication_testsuite/testcase_0203/testcase_0203_properties.json > 47276a85a038e278de75f1775da9ec9ce7912695 > > system_test/replication_testsuite/testcase_0204/testcase_0204_properties.json > 3742cfad8f25149accb062d4f4b7d23de484bcf3 > > system_test/replication_testsuite/testcase_0205/testcase_0205_properties.json > d41672d0b281810fb0b2355a77433287b6f9d4ca > > system_test/replication_testsuite/testcase_0206/testcase_0206_properties.json > a32d8882deb14fe26e8bc42ba500c7ca465f0dc3 > > system_test/replication_testsuite/testcase_0207/testcase_0207_properties.json > 573787099f92c50d629358a94ca2fb7e7ae4c264 > > system_test/replication_testsuite/testcase_0208/testcase_0208_properties.json > c3d1d2cb90ce21f4fe268dfabe731ea76308652a > > system_test/replication_testsuite/testcase_0209/testcase_0209_properties.json > 8d2cebafa0fb33c6fc729eacbb83d02ae45132a4 > > system_test/replication_testsuite/testcase_0251/testcase_0251_properties.json > 76ea6775657c916b543d36882b35fae3be757841 > > system_test/replication_testsuite/testcase_0252/testcase_0252_properties.json > 672b3ff883f2705d204981bf4a5140f333e345f5 > > system_test/replication_testsuite/testcase_0253/testcase_0253_properties.json > 19bd096b837353fae59a133feb413b5589ddb385 > > system_test/replication_testsuite/testcase_0254/testcase_0254_properties.json > 8d6add098da11324c065c856700c1296b827297a > > system_test/replication_testsuite/testcase_0255/testcase_0255_properties.json > 2a7e7776214d6071b492353c28023754ec815d6c > > system_test/replication_testsuite/testcase_0256/testcase_0256_properties.json > 91fbc9da75e2998ae75fbec140301c671ff92631 > > system_test/replication_testsuite/testcase_0257/testcase_0257_properties.json > 5594a9bc76d1ce50ddbf1668549dadca8eea8e84 > > system_test/replication_testsuite/testcase_0258/testcase_0258_properties.json > 6eabd3772dbe643ebca1495f174d2a228eccda02 > > system_test/replication_testsuite/testcase_0259/testcase_0259_properties.json > 3bbeeb10adfd8153b99f24fb55896b08de917974 > > system_test/replication_testsuite/testcase_0301/testcase_0301_properties.json > 496340c574ec35f0d5bd82e072692aee8221f05c > > system_test/replication_testsuite/testcase_0302/testcase_0302_properties.json > beabc86e2cdbac9cd8c5770f1e8924fbcc3dbab4 > > system_test/replication_testsuite/testcase_0303/testcase_0303_properties.json > 0e8f5a45895a32a780096aa2de8f619221989f7c > > system_test/replication_testsuite/testcase_0304/testcase_0304_properties.json > 98a385556c7819eb846182ae1df45b7d652caf86 > > system_test/replication_testsuite/testcase_0305/testcase_0305_properties.json > e09ac9c09b971e5ddf5b70181f6a2b75f3f22c02 > > system_test/replication_testsuite/testcase_0306/testcase_0306_properties.json > bcfe91b08c7e2424b2e8f012591caa47c1004387 > > system_test/replication_testsuite/testcase_0307/testcase_0307_properties.json > da07a9e4751b53269942cb984f67c2d0f1f46cbc > > system_test/replication_testsuite/testcase_0308/testcase_0308_properties.json > 0fecafc1a49c6aae3d171ab9d8f08944f7e8af06 > > system_test/replication_testsuite/testcase_0309/testcase_0309_properties.json > 90bd40442cc95ba049d4d11a91547de92b1ee9ff > system_test/replication_testsuite/testcase_1/testcase_1_properties.json > 7e1b3fbf2d22bcb33cc2f8dd52060f3f728eec0a > > system_test/replication_testsuite/testcase_4001/testcase_4001_properties.json > d2ffd952416e7822d2be4e171b5b5f951759408b > > system_test/replication_testsuite/testcase_4002/testcase_4002_properties.json > c86525db84ad0f926b96cfc93a0f8508d3e59981 > > system_test/replication_testsuite/testcase_4003/testcase_4003_properties.json > b77e4fdf400172a0b1bca4436223f9fd3928d705 > > system_test/replication_testsuite/testcase_4004/testcase_4004_properties.json > e753327795162b07b628b620e0a8b9e42d955062 > > system_test/replication_testsuite/testcase_4005/testcase_4005_properties.json > 5468401c6b41d280185d005d132c7a045ef50620 > > system_test/replication_testsuite/testcase_4006/testcase_4006_properties.json > e5ab0a0e8698133037cbf879873fc57a864211d4 > > system_test/replication_testsuite/testcase_4007/testcase_4007_properties.json > 7aa6e9a72ca74858e824abe251c666606fe10e4d > > system_test/replication_testsuite/testcase_4008/testcase_4008_properties.json > 08aa108f4f43a02424780cef4285383978ec8642 > > system_test/replication_testsuite/testcase_4011/testcase_4011_properties.json > 512fafb949c0d4b3a54030a5b87faeefca8c7ed2 > > system_test/replication_testsuite/testcase_4012/testcase_4012_properties.json > 9b711af013994357f2cc3a3046ba319326f9c1d8 > > system_test/replication_testsuite/testcase_4013/testcase_4013_properties.json > 3836366b3b19e5ab48c2e101c23978a165dbe4ba > > system_test/replication_testsuite/testcase_4014/testcase_4014_properties.json > 86ab75a46a48e1d99e42d8a9fc920b77d570da0b > > system_test/replication_testsuite/testcase_4015/testcase_4015_properties.json > 82d51b6072c6d6aaec827e357c82d4932ee60f96 > > system_test/replication_testsuite/testcase_4016/testcase_4016_properties.json > 31c1be0d73393c6e5fe72afff8494030f6d61eb9 > > system_test/replication_testsuite/testcase_4017/testcase_4017_properties.json > 72f78b074243ffabefe59c7158d3b45cda8593ea > > system_test/replication_testsuite/testcase_4018/testcase_4018_properties.json > ee459f4e35d682811a89bb8952d8d1afbff8e62d > > system_test/replication_testsuite/testcase_9051/testcase_9051_properties.json > 958eef7df0be16b5a3d6f126a5272b3dc3bffe1f > system_test/utils/kafka_system_test_utils.py > 5d2b7dfd46b07ab8749b1b2293e35183f14ae9af > > Diff: https://reviews.apache.org/r/18774/diff/ > > > Testing > ------- > > > Thanks, > > Jun Rao > >
