We have partitions that are in the 100s of GBs. It shouldn't have to shuffle around GB chunks of data unless you have done partition moves, or had a broker offline for a while. Is that the case?
If not your ISR problem is probably related to something else other than retention size. On Fri, Sep 7, 2018 at 7:25 AM Emmett Butler <emm...@parsely.com> wrote: > Hi users, > > What's the biggest topic you've seen in production use? The factors I'm > interested in are log.retention.* config parameters and number of > partitions. > > We run several 200-partition topics that have log retention set to roughly > 1GB per partition. Is this big by common usage standards, or not so big? > > I ask because we've had some situations in which ISRs appeared to take an > inordinately long time to synchronize, and I'm guessing this is partially > related to the need to shuttle around 1GB hunks of data. > > -- > Emmett Butler | Senior Software Engineer > < > http://www.parsely.com/?utm_source=Signature&utm_medium=emmett-butler&utm_campaign=Signature > <http://www.parsely.com/?utm_source=Signature&utm_medium=emmett-butler&utm_campaign=Signature>> >
