Yes exactly, I am just trying to rebalance partitions to the new node. I just want to balance it evently across the all cluster.
Am I doing wrong? On Thu, Jul 10, 2014 at 9:58 AM, Clark Haskins < chask...@linkedin.com.invalid> wrote: > I am confused as to exactly what you are trying to accomplish. Are you > trying to evenly distribute partitions across your newly added nodes? > > -Clark > > Clark Elliott Haskins III > LinkedIn DDS Site Reliability Engineer > Kafka, Zookeeper, Samza SRE > Mobile: 505.385.1484 > BlueJeans: https://www.bluejeans.com/chaskins > > > chask...@linkedin.com > https://www.linkedin.com/in/clarkhaskins > There is no place like 127.0.0.1 > > > > > On 7/8/14, 11:11 AM, "Florian Dambrine" <flor...@gumgum.com> wrote: > > >I realized that I did not respond to you Clark, > > > >Here is the entire Json that I sliced in multiple pieces: > > > >{"version":1,"partitions":[{"topic":"SLOTS","partition":20,"replicas":[101 > >421743,105114483,101461702]},{"topic":"RTB","partition":12,"replicas":[101 > >671664,101812541,101862816]},{"topic":"B_IMPRESSION","partition":25,"repli > >cas":[101461782,101812541,101862816]},{"topic":"RTB","partition":19,"repli > >cas":[101461702,101811991,101812541]},{"topic":"B_IMPRESSION","partition": > >18,"replicas":[101811991,102311671,105114483]},{"topic":"AD_EVENTS","parti > >tion":16,"replicas":[102311671,101862816,105114483]},{"topic":"ASSETS","pa > >rtition":28,"replicas":[101671664,101862816,102311671]},{"topic":"AD_EVENT > >S","partition":34,"replicas":[102311671,101421743,101461702]},{"topic":"AD > >_EVENTS","partition":43,"replicas":[102311671,101461702,101461782]},{"topi > >c":"D_STATISTICS","partition":4,"replicas":[101811991,101461782,101671664] > >},{"topic":"ASSETS","partition":41,"replicas":[102311671,101461782,1016716 > >64]},{"topic":"SLOTS","partition":16,"replicas":[101812541,101671664,10181 > >1991]},{"topic":"ASSETS","partition":44,"replicas":[101461702,101812541,10 > >1862816]},{"topic":"ASSETS","partition":6,"replicas":[105114483,102311671, > >101421743]},{"topic":"RTB","partition":31,"replicas":[101811991,105114483, > >101421743]},{"topic":"B_IMPRESSION","partition":29,"replicas":[101862816,1 > >01461702,101461782]},{"topic":"SLOTS","partition":21,"replicas":[101461702 > >,101421743,101461782]},{"topic":"ASSETS","partition":39,"replicas":[101812 > >541,101421743,101461702]},{"topic":"ASSETS","partition":49,"replicas":[101 > >862816,101461782,101671664]},{"topic":"AD_EVENTS","partition":32,"replicas > >":[101812541,102311671,105114483]},{"topic":"SLOTS","partition":31,"replic > >as":[101461782,101671664,101811991]},{"topic":"RTB","partition":23,"replic > >as":[101812541,105114483,101421743]},{"topic":"ASSETS","partition":23,"rep > >licas":[102311671,101421743,101461702]},{"topic":"B_IMPRESSION","partition > >":26,"replicas":[101671664,101862816,102311671]},{"topic":"RTB","partition > >":47,"replicas":[101461782,105114483,101421743]},{"topic":"AD_EVENTS","par > >tition":26,"replicas":[105114483,101421743,101461702]},{"topic":"B_IMPRESS > >ION","partition":40,"replicas":[105114483,101811991,101812541]},{"topic":" > >D_STATISTICS","partition":36,"replicas":[101421743,101671664,101811991]},{ > >"topic":"SLOTS","partition":28,"replicas":[105114483,101421743,101461702]} > >,{"topic":"AD_EVENTS","partition":36,"replicas":[101421743,101671664,10181 > >1991]},{"topic":"D_STATISTICS","partition":49,"replicas":[101811991,105114 > >483,101421743]},{"topic":"ASSETS","partition":11,"replicas":[101811991,101 > >812541,101862816]},{"topic":"AD_EVENTS","partition":24,"replicas":[1018628 > >16,102311671,105114483]},{"topic":"RTB","partition":3,"replicas":[10167166 > >4,101811991,101812541]},{"topic":"RTB","partition":41,"replicas":[10181254 > >1,101461702,101461782]},{"topic":"ASSETS","partition":14,"replicas":[10231 > >1671,105114483,101421743]},{"topic":"RTB","partition":30,"replicas":[10167 > >1664,102311671,105114483]},{"topic":"SLOTS","partition":26,"replicas":[101 > >862816,101812541,102311671]},{"topic":"D_STATISTICS","partition":0,"replic > >as":[101421743,102311671,105114483]},{"topic":"D_STATISTICS","partition":8 > >,"replicas":[105114483,101862816,102311671]},{"topic":"B_IMPRESSION","part > >ition":36,"replicas":[101811991,101421743,101461702]},{"topic":"SLOTS","pa > >rtition":49,"replicas":[101461782,101812541,101862816]},{"topic":"D_STATIS > >TICS","partition":22,"replicas":[101811991,101812541,101862816]},{"topic": > >"B_IMPRESSION","partition":38,"replicas":[101862816,101461782,101671664]}, > >{"topic":"AD_EVENTS","partition":11,"replicas":[101461782,101461702,101671 > >664]},{"topic":"AD_EVENTS","partition":39,"replicas":[101671664,101862816, > >102311671]},{"topic":"SLOTS","partition":25,"replicas":[101812541,10181199 > >1,101862816]},{"topic":"D_STATISTICS","partition":44,"replicas":[105114483 > >,101461782,101671664]},{"topic":"ASSETS","partition":47,"replicas":[101811 > >991,101421743,101461702]},{"topic":"AD_EVENTS","partition":12,"replicas":[ > >101671664,101461782,101811991]},{"topic":"SLOTS","partition":4,"replicas": > >[101461782,105114483,101421743]},{"topic":"SLOTS","partition":47,"replicas > >":[101421743,101671664,101811991]},{"topic":"ASSETS","partition":29,"repli > >cas":[101811991,102311671,105114483]},{"topic":"B_IMPRESSION","partition": > >22,"replicas":[105114483,101461782,101671664]},{"topic":"RTB","partition": > >14,"replicas":[101812541,102311671,105114483]},{"topic":"D_STATISTICS","pa > >rtition":39,"replicas":[101671664,101862816,102311671]},{"topic":"ASSETS", > >"partition":12,"replicas":[101812541,101862816,102311671]},{"topic":"D_STA > >TISTICS","partition":31,"replicas":[101811991,101862816,102311671]},{"topi > >c":"SLOTS","partition":0,"replicas":[102311671,101811991,101812541]},{"top > >ic":"RTB","partition":22,"replicas":[101811991,102311671,105114483]},{"top > >ic":"D_STATISTICS","partition":2,"replicas":[101461782,101421743,101461702 > >]},{"topic":"SLOTS","partition":43,"replicas":[101812541,102311671,1051144 > >83]},{"topic":"ASSETS","partition":36,"replicas":[101461782,101862816,1023 > >11671]},{"topic":"B_IMPRESSION","partition":1,"replicas":[101812541,101862 > >816,102311671]},{"topic":"ASSETS","partition":34,"replicas":[101421743,101 > >671664,101811991]},{"topic":"AD_EVENTS","partition":40,"replicas":[1018119 > >91,102311671,105114483]},{"topic":"AD_EVENTS","partition":5,"replicas":[10 > >1812541,101671664,101811991]},{"topic":"ASSETS","partition":25,"replicas": > >[101421743,101461782,101671664]},{"topic":"RTB","partition":8,"replicas":[ > >105114483,101421743,101461702]},{"topic":"B_IMPRESSION","partition":48,"re > >plicas":[102311671,101811991,101812541]},{"topic":"RTB","partition":2,"rep > >licas":[101461782,101671664,101811991]},{"topic":"AD_EVENTS","partition":1 > >5,"replicas":[101862816,101812541,102311671]},{"topic":"RTB","partition":4 > >,"replicas":[101811991,101812541,101862816]},{"topic":"SLOTS","partition": > >22,"replicas":[101461782,101461702,101671664]},{"topic":"SLOTS","partition > >":19,"replicas":[105114483,102311671,101421743]},{"topic":"AD_EVENTS","par > >tition":21,"replicas":[101671664,101811991,101812541]},{"topic":"D_STATIST > >ICS","partition":26,"replicas":[105114483,101421743,101461702]},{"topic":" > >ASSETS","partition":32,"replicas":[102311671,101461702,101461782]},{"topic > >":"RTB","partition":38,"replicas":[101461782,102311671,105114483]},{"topic > >":"D_STATISTICS","partition":6,"replicas":[101862816,101811991,101812541]} > >,{"topic":"B_IMPRESSION","partition":28,"replicas":[101812541,101421743,10 > >1461702]},{"topic":"RTB","partition":36,"replicas":[101421743,101812541,10 > >1862816]},{"topic":"AD_EVENTS","partition":49,"replicas":[101811991,105114 > >483,101421743]},{"topic":"SLOTS","partition":17,"replicas":[101862816,1018 > >11991,101812541]},{"topic":"D_STATISTICS","partition":24,"replicas":[10186 > >2816,102311671,105114483]},{"topic":"D_STATISTICS","partition":37,"replica > >s":[101461702,101811991,101812541]},{"topic":"AD_EVENTS","partition":46,"r > >eplicas":[101461702,101812541,101862816]},{"topic":"B_IMPRESSION","partiti > >on":6,"replicas":[101461702,101461782,101671664]},{"topic":"ASSETS","parti > >tion":4,"replicas":[101862816,101812541,102311671]},{"topic":"ASSETS","par > >tition":0,"replicas":[101461782,101461702,101671664]},{"topic":"ASSETS","p > >artition":30,"replicas":[101812541,105114483,101421743]},{"topic":"RTB","p > >artition":18,"replicas":[101421743,101671664,101811991]},{"topic":"D_STATI > >STICS","partition":29,"replicas":[101461782,101811991,101812541]},{"topic" > >:"RTB","partition":9,"replicas":[101421743,101461782,101671664]},{"topic": > >"AD_EVENTS","partition":48,"replicas":[101671664,102311671,105114483]},{"t > >opic":"D_STATISTICS","partition":47,"replicas":[101461782,101862816,102311 > >671]},{"topic":"SLOTS","partition":46,"replicas":[105114483,101461782,1016 > >71664]},{"topic":"RTB","partition":48,"replicas":[101671664,101421743,1014 > >61702]},{"topic":"ASSETS","partition":35,"replicas":[101461702,101811991,1 > >01812541]},{"topic":"B_IMPRESSION","partition":34,"replicas":[101461782,10 > >1862816,102311671]},{"topic":"SLOTS","partition":1,"replicas":[105114483,1 > >01812541,101862816]},{"topic":"AD_EVENTS","partition":18,"replicas":[10142 > >1743,101461702,101461782]},{"topic":"D_STATISTICS","partition":38,"replica > >s":[101461782,101812541,101862816]},{"topic":"ASSETS","partition":8,"repli > >cas":[101461702,101421743,101461782]},{"topic":"ASSETS","partition":26,"re > >plicas":[101461702,101671664,101811991]},{"topic":"D_STATISTICS","partitio > >n":21,"replicas":[101671664,101811991,101812541]},{"topic":"B_IMPRESSION", > >"partition":10,"replicas":[101812541,102311671,105114483]},{"topic":"AD_EV > >ENTS","partition":2,"replicas":[101461782,101421743,101461702]},{"topic":" > >ASSETS","partition":19,"replicas":[101671664,101812541,101862816]},{"topic > >":"D_STATISTICS","partition":48,"replicas":[101671664,102311671,105114483] > >},{"topic":"RTB","partition":32,"replicas":[101812541,101421743,101461702] > >},{"topic":"D_STATISTICS","partition":28,"replicas":[101461702,101671664,1 > >01811991]},{"topic":"B_IMPRESSION","partition":35,"replicas":[101671664,10 > >2311671,105114483]},{"topic":"SLOTS","partition":3,"replicas":[101461702,1 > >02311671,105114483]},{"topic":"D_STATISTICS","partition":41,"replicas":[10 > >1812541,105114483,101421743]},{"topic":"B_IMPRESSION","partition":17,"repl > >icas":[101671664,101812541,101862816]},{"topic":"B_IMPRESSION","partition" > >:4,"replicas":[105114483,101421743,101461702]},{"topic":"RTB","partition": > >11,"replicas":[101461782,101811991,101812541]},{"topic":"SLOTS","partition > >":41,"replicas":[101671664,101812541,101862816]},{"topic":"B_IMPRESSION"," > >partition":27,"replicas":[101811991,105114483,101421743]},{"topic":"RTB"," > >partition":17,"replicas":[105114483,101461702,101461782]},{"topic":"D_STAT > >ISTICS","partition":15,"replicas":[101862816,101812541,102311671]},{"topic > >":"RTB","partition":33,"replicas":[101862816,101461702,101461782]},{"topic > >":"B_IMPRESSION","partition":0,"replicas":[101811991,101812541,101862816]} > >,{"topic":"ASSETS","partition":2,"replicas":[101811991,101671664,101812541 > >]},{"topic":"AD_EVENTS","partition":28,"replicas":[101461702,101671664,101 > >811991]},{"topic":"D_STATISTICS","partition":3,"replicas":[101671664,10146 > >1702,101461782]},{"topic":"RTB","partition":13,"replicas":[101811991,10186 > >2816,102311671]},{"topic":"D_STATISTICS","partition":46,"replicas":[101461 > >702,101812541,101862816]},{"topic":"AD_EVENTS","partition":38,"replicas":[ > >101461782,101812541,101862816]},{"topic":"RTB","partition":39,"replicas":[ > >101671664,105114483,101421743]},{"topic":"RTB","partition":16,"replicas":[ > >102311671,101421743,101461702]},{"topic":"AD_EVENTS","partition":33,"repli > >cas":[101862816,105114483,101421743]},{"topic":"B_IMPRESSION","partition": > >20,"replicas":[101862816,101421743,101461702]},{"topic":"AD_EVENTS","parti > >tion":30,"replicas":[101671664,101812541,101862816]},{"topic":"B_IMPRESSIO > >N","partition":9,"replicas":[101811991,101862816,102311671]},{"topic":"RTB > >","partition":5,"replicas":[101812541,101862816,102311671]},{"topic":"AD_E > >VENTS","partition":25,"replicas":[102311671,105114483,101421743]},{"topic" > >:"AD_EVENTS","partition":4,"replicas":[101811991,101461782,101671664]},{"t > >opic":"D_STATISTICS","partition":42,"replicas":[101862816,101421743,101461 > >702]},{"topic":"SLOTS","partition":32,"replicas":[101671664,101811991,1018 > >12541]},{"topic":"SLOTS","partition":48,"replicas":[101461702,101811991,10 > >1812541]},{"topic":"D_STATISTICS","partition":27,"replicas":[101421743,101 > >461782,101671664]},{"topic":"ASSETS","partition":46,"replicas":[101671664, > >105114483,101421743]},{"topic":"RTB","partition":42,"replicas":[101862816, > >101461782,101671664]},{"topic":"D_STATISTICS","partition":5,"replicas":[10 > >1812541,101671664,101811991]},{"topic":"RTB","partition":15,"replicas":[10 > >1862816,105114483,101421743]},{"topic":"AD_EVENTS","partition":13,"replica > >s":[101811991,101671664,101812541]},{"topic":"SLOTS","partition":12,"repli > >cas":[101461702,105114483,101421743]},{"topic":"ASSETS","partition":7,"rep > >licas":[101421743,105114483,101461702]},{"topic":"ASSETS","partition":48," > >replicas":[101812541,101461702,101461782]},{"topic":"B_IMPRESSION","partit > >ion":2,"replicas":[101862816,102311671,105114483]},{"topic":"D_STATISTICS" > >,"partition":23,"replicas":[101812541,101862816,102311671]},{"topic":"ASSE > >TS","partition":3,"replicas":[101812541,101811991,101862816]},{"topic":"SL > >OTS","partition":33,"replicas":[101811991,101812541,101862816]},{"topic":" > >ASSETS","partition":5,"replicas":[102311671,101862816,105114483]},{"topic" > >:"SLOTS","partition":11,"replicas":[101421743,102311671,105114483]},{"topi > >c":"SLOTS","partition":35,"replicas":[101862816,102311671,105114483]},{"to > >pic":"ASSETS","partition":22,"replicas":[101862816,105114483,101421743]},{ > >"topic":"B_IMPRESSION","partition":32,"replicas":[101421743,101811991,1018 > >12541]},{"topic":"AD_EVENTS","partition":1,"replicas":[101461702,105114483 > >,101421743]},{"topic":"ASSETS","partition":33,"replicas":[105114483,101461 > >782,101671664]},{"topic":"AD_EVENTS","partition":6,"replicas":[101862816,1 > >01811991,101812541]},{"topic":"ASSETS","partition":17,"replicas":[10146170 > >2,101461782,101671664]},{"topic":"B_IMPRESSION","partition":49,"replicas": > >[105114483,101812541,101862816]},{"topic":"RTB","partition":6,"replicas":[ > >101862816,102311671,105114483]},{"topic":"RTB","partition":35,"replicas":[ > >105114483,101671664,101811991]},{"topic":"SLOTS","partition":27,"replicas" > >:[102311671,105114483,101421743]},{"topic":"SLOTS","partition":34,"replica > >s":[101812541,101862816,102311671]},{"topic":"SLOTS","partition":30,"repli > >cas":[101461702,101461782,101671664]},{"topic":"D_STATISTICS","partition": > >10,"replicas":[101461702,101421743,101461782]},{"topic":"AD_EVENTS","parti > >tion":45,"replicas":[101421743,101811991,101812541]},{"topic":"SLOTS","par > >tition":8,"replicas":[101862816,101671664,101811991]},{"topic":"D_STATISTI > >CS","partition":33,"replicas":[101862816,105114483,101421743]},{"topic":"A > >D_EVENTS","partition":14,"replicas":[101812541,101811991,101862816]},{"top > >ic":"SLOTS","partition":13,"replicas":[101461782,101421743,101461702]},{"t > >opic":"AD_EVENTS","partition":17,"replicas":[105114483,102311671,101421743 > >]},{"topic":"ASSETS","partition":24,"replicas":[105114483,101461702,101461 > >782]},{"topic":"ASSETS","partition":20,"replicas":[101811991,101862816,102 > >311671]},{"topic":"RTB","partition":43,"replicas":[102311671,101671664,101 > >811991]},{"topic":"RTB","partition":44,"replicas":[105114483,101811991,101 > >812541]},{"topic":"SLOTS","partition":15,"replicas":[101811991,101461782,1 > >01671664]},{"topic":"ASSETS","partition":18,"replicas":[101461782,10181199 > >1,101812541]},{"topic":"AD_EVENTS","partition":7,"replicas":[102311671,101 > >812541,101862816]},{"topic":"RTB","partition":27,"replicas":[101421743,101 > >811991,101812541]},{"topic":"RTB","partition":21,"replicas":[101671664,101 > >862816,102311671]},{"topic":"B_IMPRESSION","partition":33,"replicas":[1014 > >61702,101812541,101862816]},{"topic":"RTB","partition":46,"replicas":[1014 > >61702,102311671,105114483]},{"topic":"RTB","partition":10,"replicas":[1014 > >61702,101671664,101811991]},{"topic":"AD_EVENTS","partition":41,"replicas" > >:[101812541,105114483,101421743]},{"topic":"RTB","partition":49,"replicas" > >:[101811991,101461702,101461782]},{"topic":"D_STATISTICS","partition":19," > >replicas":[101461702,101461782,101671664]},{"topic":"AD_EVENTS","partition > >":10,"replicas":[101461702,101421743,101461782]},{"topic":"SLOTS","partiti > >on":10,"replicas":[105114483,101862816,102311671]},{"topic":"ASSETS","part > >ition":42,"replicas":[105114483,101671664,101811991]},{"topic":"B_IMPRESSI > >ON","partition":14,"replicas":[101421743,101461782,101671664]},{"topic":"R > >TB","partition":7,"replicas":[102311671,105114483,101421743]},{"topic":"SL > >OTS","partition":42,"replicas":[101811991,101862816,102311671]},{"topic":" > >B_IMPRESSION","partition":8,"replicas":[101671664,101811991,101812541]},{" > >topic":"SLOTS","partition":18,"replicas":[102311671,101862816,105114483]}, > >{"topic":"B_IMPRESSION","partition":23,"replicas":[101421743,101671664,101 > >811991]},{"topic":"B_IMPRESSION","partition":31,"replicas":[105114483,1016 > >71664,101811991]},{"topic":"D_STATISTICS","partition":1,"replicas":[101461 > >702,105114483,101421743]},{"topic":"B_IMPRESSION","partition":44,"replicas > >":[101671664,105114483,101421743]},{"topic":"RTB","partition":28,"replicas > >":[101461702,101812541,101862816]},{"topic":"ASSETS","partition":31,"repli > >cas":[101862816,101421743,101461702]},{"topic":"RTB","partition":25,"repli > >cas":[102311671,101461702,101461782]},{"topic":"B_IMPRESSION","partition": > >37,"replicas":[101812541,101461702,101461782]},{"topic":"B_IMPRESSION","pa > >rtition":13,"replicas":[105114483,101461702,101461782]},{"topic":"B_IMPRES > >SION","partition":24,"replicas":[101461702,101811991,101812541]},{"topic": > >"AD_EVENTS","partition":8,"replicas":[105114483,101862816,102311671]},{"to > >pic":"ASSETS","partition":9,"replicas":[101461782,101671664,101811991]},{" > >topic":"ASSETS","partition":43,"replicas":[101421743,101811991,101812541]} > >,{"topic":"SLOTS","partition":37,"replicas":[105114483,101461702,101461782 > >]},{"topic":"B_IMPRESSION","partition":19,"replicas":[101812541,105114483, > >101421743]},{"topic":"AD_EVENTS","partition":3,"replicas":[101671664,10146 > >1702,101461782]},{"topic":"SLOTS","partition":2,"replicas":[101421743,1018 > >62816,102311671]},{"topic":"B_IMPRESSION","partition":12,"replicas":[10231 > >1671,101421743,101461702]},{"topic":"SLOTS","partition":29,"replicas":[101 > >421743,101461702,101461782]},{"topic":"ASSETS","partition":38,"replicas":[ > >101811991,105114483,101421743]},{"topic":"ASSETS","partition":45,"replicas > >":[101461782,102311671,105114483]},{"topic":"B_IMPRESSION","partition":47, > >"replicas":[101862816,101671664,101811991]},{"topic":"B_IMPRESSION","parti > >tion":46,"replicas":[101812541,101461782,101671664]},{"topic":"RTB","parti > >tion":29,"replicas":[101461782,101862816,102311671]},{"topic":"RTB","parti > >tion":26,"replicas":[105114483,101461782,101671664]},{"topic":"B_IMPRESSIO > >N","partition":21,"replicas":[102311671,101461702,101461782]},{"topic":"B_ > >IMPRESSION","partition":45,"replicas":[101811991,101461702,101461782]},{"t > >opic":"B_IMPRESSION","partition":5,"replicas":[101421743,101461702,1014617 > >82]},{"topic":"RTB","partition":45,"replicas":[101421743,101862816,1023116 > >71]},{"topic":"B_IMPRESSION","partition":15,"replicas":[101461702,10167166 > >4,101811991]},{"topic":"D_STATISTICS","partition":7,"replicas":[102311671, > >101812541,101862816]},{"topic":"D_STATISTICS","partition":17,"replicas":[1 > >05114483,102311671,101421743]},{"topic":"SLOTS","partition":6,"replicas":[ > >101811991,101461702,101461782]},{"topic":"SLOTS","partition":24,"replicas" > >:[101811991,101671664,101812541]},{"topic":"SLOTS","partition":9,"replicas > >":[102311671,101812541,101862816]},{"topic":"D_STATISTICS","partition":25, > >"replicas":[102311671,105114483,101421743]},{"topic":"AD_EVENTS","partitio > >n":44,"replicas":[105114483,101461782,101671664]},{"topic":"AD_EVENTS","pa > >rtition":37,"replicas":[101461702,101811991,101812541]},{"topic":"ASSETS", > >"partition":16,"replicas":[101421743,101461702,101461782]},{"topic":"SLOTS > >","partition":44,"replicas":[101862816,105114483,101421743]},{"topic":"B_I > >MPRESSION","partition":16,"replicas":[101461782,101811991,101812541]},{"to > >pic":"ASSETS","partition":10,"replicas":[101671664,101811991,101812541]},{ > >"topic":"B_IMPRESSION","partition":11,"replicas":[101862816,105114483,1014 > >21743]},{"topic":"AD_EVENTS","partition":20,"replicas":[101461782,10167166 > >4,101811991]},{"topic":"ASSETS","partition":1,"replicas":[101671664,101461 > >782,101811991]},{"topic":"SLOTS","partition":45,"replicas":[102311671,1014 > >61702,101461782]},{"topic":"SLOTS","partition":38,"replicas":[101421743,10 > >1461782,101671664]},{"topic":"AD_EVENTS","partition":29,"replicas":[101461 > >782,101811991,101812541]},{"topic":"RTB","partition":24,"replicas":[101862 > >816,101421743,101461702]},{"topic":"RTB","partition":40,"replicas":[101811 > >991,101421743,101461702]},{"topic":"B_IMPRESSION","partition":30,"replicas > >":[102311671,101461782,101671664]},{"topic":"D_STATISTICS","partition":18, > >"replicas":[101421743,101461702,101461782]},{"topic":"D_STATISTICS","parti > >tion":20,"replicas":[101461782,101671664,101811991]},{"topic":"ASSETS","pa > >rtition":13,"replicas":[101862816,102311671,105114483]},{"topic":"B_IMPRES > >SION","partition":42,"replicas":[101461702,101862816,102311671]},{"topic": > >"AD_EVENTS","partition":23,"replicas":[101812541,101862816,102311671]},{"t > >opic":"RTB","partition":34,"replicas":[102311671,101461782,101671664]},{"t > >opic":"D_STATISTICS","partition":16,"replicas":[102311671,101862816,105114 > >483]},{"topic":"SLOTS","partition":23,"replicas":[101671664,101461782,1018 > >11991]},{"topic":"RTB","partition":0,"replicas":[101421743,101461702,10146 > >1782]},{"topic":"AD_EVENTS","partition":19,"replicas":[101461702,101461782 > >,101671664]},{"topic":"ASSETS","partition":37,"replicas":[101671664,102311 > >671,105114483]},{"topic":"SLOTS","partition":39,"replicas":[101461702,1016 > >71664,101811991]},{"topic":"AD_EVENTS","partition":9,"replicas":[101421743 > >,105114483,101461702]},{"topic":"D_STATISTICS","partition":43,"replicas":[ > >102311671,101461702,101461782]},{"topic":"RTB","partition":37,"replicas":[ > >101461702,101862816,102311671]},{"topic":"ASSETS","partition":15,"replicas > >":[105114483,101421743,101461702]},{"topic":"B_IMPRESSION","partition":7," > >replicas":[101461782,101671664,101811991]},{"topic":"ASSETS","partition":2 > >7,"replicas":[101461782,101812541,101862816]},{"topic":"SLOTS","partition" > >:40,"replicas":[101461782,101811991,101812541]},{"topic":"AD_EVENTS","part > >ition":0,"replicas":[101421743,102311671,105114483]},{"topic":"D_STATISTIC > >S","partition":12,"replicas":[101671664,101461782,101811991]},{"topic":"D_ > >STATISTICS","partition":45,"replicas":[101421743,101811991,101812541]},{"t > >opic":"B_IMPRESSION","partition":41,"replicas":[101421743,101812541,101862 > >816]},{"topic":"SLOTS","partition":36,"replicas":[102311671,101421743,1014 > >61702]},{"topic":"AD_EVENTS","partition":22,"replicas":[101811991,10181254 > >1,101862816]},{"topic":"RTB","partition":1,"replicas":[101461702,101461782 > >,101671664]},{"topic":"B_IMPRESSION","partition":43,"replicas":[101461782, > >102311671,105114483]},{"topic":"D_STATISTICS","partition":9,"replicas":[10 > >1421743,105114483,101461702]},{"topic":"D_STATISTICS","partition":11,"repl > >icas":[101461782,101461702,101671664]},{"topic":"D_STATISTICS","partition" > >:32,"replicas":[101812541,102311671,105114483]},{"topic":"D_STATISTICS","p > >artition":34,"replicas":[102311671,101421743,101461702]},{"topic":"RTB","p > >artition":20,"replicas":[101461782,101812541,101862816]},{"topic":"SLOTS", > >"partition":7,"replicas":[101812541,101461782,101671664]},{"topic":"AD_EVE > >NTS","partition":47,"replicas":[101461782,101862816,102311671]},{"topic":" > >D_STATISTICS","partition":14,"replicas":[101812541,101811991,101862816]},{ > >"topic":"D_STATISTICS","partition":35,"replicas":[105114483,101461702,1014 > >61782]},{"topic":"AD_EVENTS","partition":35,"replicas":[105114483,10146170 > >2,101461782]},{"topic":"ASSETS","partition":21,"replicas":[101812541,10231 > >1671,105114483]},{"topic":"D_STATISTICS","partition":30,"replicas":[101671 > >664,101812541,101862816]},{"topic":"AD_EVENTS","partition":31,"replicas":[ > >101811991,101862816,102311671]},{"topic":"B_IMPRESSION","partition":39,"re > >plicas":[102311671,101671664,101811991]},{"topic":"D_STATISTICS","partitio > >n":13,"replicas":[101811991,101671664,101812541]},{"topic":"D_STATISTICS", > >"partition":40,"replicas":[101811991,102311671,105114483]},{"topic":"SLOTS > >","partition":14,"replicas":[101671664,101461702,101461782]},{"topic":"ASS > >ETS","partition":40,"replicas":[101862816,101461702,101461782]},{"topic":" > >SLOTS","partition":5,"replicas":[101671664,101421743,101461702]},{"topic": > >"AD_EVENTS","partition":27,"replicas":[101421743,101461782,101671664]},{"t > >opic":"B_IMPRESSION","partition":3,"replicas":[102311671,105114483,1014217 > >43]},{"topic":"AD_EVENTS","partition":42,"replicas":[101862816,101421743,1 > >01461702]}]} > > > > > > > >On Tue, Jul 8, 2014 at 10:59 AM, Florian Dambrine <flor...@gumgum.com> > >wrote: > > > >> Here is the entire logic to rebalance the cluster which is done by this > >> groovy script ( > >> > >> > https://github.com/Lowess/Kafka/blob/master/KafkaPartitionRebalancer.groo > >>vy > >> ) > >> > >> #1: Request the zookeeper and get the broker id list > >> #2: Request zookeeper and get the list of topic > >> #3: Generate the topic-to-move.json which looks like: > >> > >> { > >> "version": 1, > >> "topics": [ > >> { > >> "topic": "SLOTS" > >> }, > >> { > >> "topic": "ASSETS" > >> }, > >> { > >> "topic": "AD_EVENTS" > >> }, > >> { > >> "topic": "B_IMPRESSION" > >> }, > >> { > >> "topic": "B_STATISTICS" > >> }, > >> { > >> "topic": "PAGES" > >> }, > >> { > >> "topic": "RTB" > >> }, > >> { > >> "topic": "D_STATISTICS" > >> }, > >> { > >> "topic": "D_REPORTING" > >> } > >> ] > >> } > >> > >> #4: Upload this file on the kafka node (/tmp/topics-to-move.json) and > >>run the following command: > >> bin/kafka-reassign-partitions.sh --zookeeper ZK_IP:2181 > >>--topics-to-move-json-file /tmp/topics-to-move.json --generate > >>--broker-list "ALL_BROKERS_THAT_ARE_RETURNED_BY_ZOOKEEPER_ON_STEP_#1" > >> > >> #5: Parse the json returned by the previous step and slice it into > >>smaller json (the number of partitions contained in a Json is limited by > >>the groovy script (10 partitions in this example))that look like: > >> > >> { > >> "version": 1, > >> "partitions": [ > >> { > >> "topic": "D_REPORTING", > >> "partition": 2, > >> "replicas": [ > >> 102311671, > >> 10517222, > >> 102311679 > >> ] > >> }, > >> { > >> "topic": "AD_EVENTS", > >> "partition": 48, > >> "replicas": [ > >> 102311671, > >> 109715277, > >> 101531906 > >> ] > >> }, > >> { > >> "topic": "D_STATISTICS", > >> "partition": 47, > >> "replicas": [ > >> 109715277, > >> 10517222, > >> 102311679 > >> ] > >> }, > >> { > >> "topic": "SLOTS", > >> "partition": 46, > >> "replicas": [ > >> 101131445, > >> 102336284, > >> 10517222 > >> ] > >> }, > >> { > >> "topic": "RTB", > >> "partition": 48, > >> "replicas": [ > >> 101021441, > >> 102311671, > >> 102336284 > >> ] > >> }, > >> { > >> "topic": "PAGES", > >> "partition": 14, > >> "replicas": [ > >> 102311679, > >> 102311671, > >> 102336284 > >> ] > >> }, > >> { > >> "topic": "ASSETS", > >> "partition": 35, > >> "replicas": [ > >> 10517222, > >> 101131445, > >> 102311679 > >> ] > >> }, > >> { > >> "topic": "B_IMPRESSION", > >> "partition": 34, > >> "replicas": [ > >> 101131445, > >> 102311672, > >> 102311671 > >> ] > >> }, > >> { > >> "topic": "B_STATISTICS", > >> "partition": 19, > >> "replicas": [ > >> 109715277, > >> 101531906, > >> 102311672 > >> ] > >> }, > >> { > >> "topic": "AD_EVENTS", > >> "partition": 18, > >> "replicas": [ > >> 109715277, > >> 102311671, > >> 102336284 > >> ] > >> } > >> ] > >> } > >> > >> #6: Upload the previous Json on the kafka node > >> (/tmp/expand-cluster-reassignment.json) and run the following command: > >> bin/kafka-reassign-partitions.sh --zookeeper ZK_IP:2181 > >>--reassignment-json-file > >> /tmp/expand-cluster-reassignment.json --execute --broker-list > >> "ALL_BROKERS_THAT_ARE_RETURNED_BY_ZOOKEEPER_ON_STEP_#1" > >> > >> #7: Loop on the verification step while the json returned by the > >>following > >> command contains failed partitions: > >> bin/kafka-reassign-partitions.sh --zookeeper ZK_IP:2181 > >>--reassignment-json-file > >> /tmp/expand-cluster-reassignment.json --verify --broker-list > >> "ALL_BROKERS_THAT_ARE_RETURNED_BY_ZOOKEEPER_ON_STEP_#1" > >> > >> #8 Execute a new json part file similar as step #5 util all of them have > >> ran. > >> > >> > >> Hope that will help you guys. > >> > >> > >> On Tue, Jul 8, 2014 at 10:31 AM, Clark Haskins < > >> chask...@linkedin.com.invalid> wrote: > >> > >>> Can you copy/paste the json you are passing to the reassignment tool? > >>>Plus > >>> the commands. Also do a describe on your topics. > >>> > >>> -Clark > >>> > >>> Clark Elliott Haskins III > >>> LinkedIn DDS Site Reliability Engineer > >>> Kafka, Zookeeper, Samza SRE > >>> Mobile: 505.385.1484 > >>> BlueJeans: https://www.bluejeans.com/chaskins > >>> > >>> > >>> chask...@linkedin.com > >>> https://www.linkedin.com/in/clarkhaskins > >>> There is no place like 127.0.0.1 > >>> > >>> > >>> > >>> > >>> On 7/8/14, 10:26 AM, "Florian Dambrine" <flor...@gumgum.com> wrote: > >>> > >>> >I let the tool running for an entire weekend on the test cluster and > >>>on > >>> >Monday it was still saying "failed"... > >>> > > >>> >I have 500 Go per Kafka node and it is a 8 nodes cluster. > >>> > > >>> >I am also wondering if I am using the tool correctly. Currently I am > >>> >running the tool to rebalance everything across the entire cluster. > >>>As I > >>> >have 3 replicas the tool requires at least 3 brokers. > >>> > > >>> >Should I add 3 new Kafka nodes and rebalance some topics to these new > >>> >nodes > >>> >only? I am afraid to unbalance the cluster with this option. > >>> > > >>> >Any suggestions? > >>> > > >>> >Thanks for your help. > >>> > > >>> > > >>> >On Mon, Jul 7, 2014 at 9:29 PM, Jun Rao <jun...@gmail.com> wrote: > >>> > > >>> >> The failure could mean that the reassignment is still in progress. > >>>If > >>> >>you > >>> >> have lots of data, it may take some time to move the data to new > >>> >>brokers. > >>> >> You could observe the max lag in each broker to see how far behind > >>>new > >>> >> replicas are (see > >>> >>http://kafka.apache.org/documentation.html#monitoring). > >>> >> > >>> >> Thanks, > >>> >> > >>> >> Jun > >>> >> > >>> >> > >>> >> On Mon, Jul 7, 2014 at 4:42 PM, Florian Dambrine > >>><flor...@gumgum.com> > >>> >> wrote: > >>> >> > >>> >> > When I run the tool with the --verify option it says failed for > >>>the > >>> >>some > >>> >> > partitions. > >>> >> > > >>> >> > The problem is I do not know if it is a zookeeper issue or if the > >>> tool > >>> >> > really failed. > >>> >> > > >>> >> > I faced one time the zookeeper issue ( > >>> >> > https://issues.apache.org/jira/browse/KAFKA-1382) and by killing > >>>the > >>> >> > responsible Kafka the partition switched from failed to completed > >>> >> > successfully. > >>> >> > > >>> >> > What should I do when the Kafka tool says that it failed to move > >>>the > >>> >> > partition? > >>> >> > > >>> >> > > >>> >> > > >>> >> > > >>> >> > On Mon, Jul 7, 2014 at 4:33 PM, Clark Haskins > >>> >> > <chask...@linkedin.com.invalid > >>> >> > > wrote: > >>> >> > > >>> >> > > How does it get stuck? > >>> >> > > > >>> >> > > -Clark > >>> >> > > > >>> >> > > Clark Elliott Haskins III > >>> >> > > LinkedIn DDS Site Reliability Engineer > >>> >> > > Kafka, Zookeeper, Samza SRE > >>> >> > > Mobile: 505.385.1484 > >>> >> > > BlueJeans: https://www.bluejeans.com/chaskins > >>> >> > > > >>> >> > > > >>> >> > > chask...@linkedin.com > >>> >> > > https://www.linkedin.com/in/clarkhaskins > >>> >> > > There is no place like 127.0.0.1 > >>> >> > > > >>> >> > > > >>> >> > > > >>> >> > > > >>> >> > > On 7/7/14, 3:49 PM, "Florian Dambrine" <flor...@gumgum.com> > >>>wrote: > >>> >> > > > >>> >> > > >Hi, > >>> >> > > > > >>> >> > > >I am trying to add new brokers to an existing 8 nodes Kafka > >>> >>cluster. > >>> >> We > >>> >> > > >have around 10 topics and the number of partition is set to > >>>50. In > >>> >> order > >>> >> > > >to > >>> >> > > >test the reassgin-partitions scripts, I tried on a sandbox > >>>cluster > >>> >>the > >>> >> > > >following steps. > >>> >> > > > > >>> >> > > >I developed a script which is able to parse the reassignment > >>> >>partition > >>> >> > > >plan > >>> >> > > >given by the Kafka tool in smaller pieces (reassigning maximum > >>>10 > >>> >> > > >partitions at a time). > >>> >> > > > > >>> >> > > >Unfortunately I faced some issues with the tool that sometimes > >>>get > >>> >> stuck > >>> >> > > >on > >>> >> > > >one partition. In this case I have to kill and restart the > >>>three > >>> >> Kafkas > >>> >> > on > >>> >> > > >which the partition has been relocated to unlock the process > >>>(One > >>> >> kafka > >>> >> > at > >>> >> > > >a time). > >>> >> > > > > >>> >> > > >Moreover, I have also faced these two issues that are already > >>>on > >>> >>Jira: > >>> >> > > > > >>> >> > > >https://issues.apache.org/jira/browse/KAFKA-1382 > >>> >> > > >https://issues.apache.org/jira/browse/KAFKA-1479 > >>> >> > > > > >>> >> > > >We really need to add new nodes to our Kafka cluster, does > >>>anybody > >>> >> have > >>> >> > > >already rebalance a Kafka 0.8.1.1? What could you advise me? > >>> >> > > > > >>> >> > > >Thanks, and feel free to ask me if you need more details. > >>> >> > > > > >>> >> > > > > >>> >> > > > > >>> >> > > >-- > >>> >> > > >*Florian Dambrine* | Intern, Big Data > >>> >> > > >*GumGum* <http://www.gumgum.com/> | *Ads that stick* > >>> >> > > >209-797-3994 | flor...@gumgum.com > >>> >> > > > >>> >> > > > >>> >> > > >>> >> > > >>> >> > -- > >>> >> > *Florian Dambrine* | Intern, Big Data > >>> >> > *GumGum* <http://www.gumgum.com/> | *Ads that stick* > >>> >> > 209-797-3994 | flor...@gumgum.com > >>> >> > > >>> >> > >>> > > >>> > > >>> > > >>> >-- > >>> >*Florian Dambrine* | Intern, Big Data > >>> >*GumGum* <http://www.gumgum.com/> | *Ads that stick* > >>> >209-797-3994 | flor...@gumgum.com > >>> > >>> > >> > >> > >> -- > >> *Florian Dambrine* | Intern, Big Data > >> *GumGum* <http://www.gumgum.com/> | *Ads that stick* > >> 209-797-3994 | flor...@gumgum.com > >> > > > > > > > >-- > >*Florian Dambrine* | Intern, Big Data > >*GumGum* <http://www.gumgum.com/> | *Ads that stick* > >209-797-3994 | flor...@gumgum.com > > -- *Florian Dambrine* | Intern, Big Data *GumGum* <http://www.gumgum.com/> | *Ads that stick* 209-797-3994 | flor...@gumgum.com
