The average observed per broker write bandwidth is roughly 5 MB/s and read bandwidth is roughly 20 MB/s. But these statistics are from clusters that are no where close to maxing out the write/read bandwidth available. In most cases though, you will hit the network bottleneck (~125 MB/s) first.
Thanks, Neha On Thu, Oct 3, 2013 at 11:08 AM, Graeme Wallace < graeme.wall...@farecompare.com> wrote: > Jun, > > Just to follow up - have you any stats on what disk read + write throughput > you are getting in production at LinkedIn ? > > regards, > > Graeme > > > > On Thu, Oct 3, 2013 at 7:17 AM, Graeme Wallace < > graeme.wall...@farecompare.com> wrote: > > > Its whatever dstat reports - i think it must be MBytes ? > > > > > > On Wed, Oct 2, 2013 at 10:29 PM, Jun Rao <jun...@gmail.com> wrote: > > > >> If you use a reasonable fetch size in the consumer (e.g. 100+KB) or > above, > >> you could probably get most of the sequential scan performance from > those > >> disks. Do you mean 250 Mbits or MBytes? > >> > >> Thanks, > >> > >> Jun > >> > >> > >> On Wed, Oct 2, 2013 at 4:35 PM, Graeme Wallace < > >> graeme.wall...@farecompare.com> wrote: > >> > >> > We notice that we get a lot higher performance from Kafka up until the > >> > point where disk reads are taking place. (which is to be expected as > its > >> > reading from filesystem cache). > >> > > >> > However, given that we have 12x7200rpm SATA disks configured as jBOD - > >> what > >> > sort of disk read performance should i see reported by dstat when > things > >> > are running flat out ? > >> > > >> > I'm seeing 250-340Mb sustained. > >> > > >> > -- > >> > Graeme Wallace > >> > CTO > >> > FareCompare.com > >> > O: 972 588 1414 > >> > M: 214 681 9018 > >> > > >> > > > > > > > > -- > > Graeme Wallace > > CTO > > FareCompare.com > > O: 972 588 1414 > > M: 214 681 9018 > > > > > > > -- > Graeme Wallace > CTO > FareCompare.com > O: 972 588 1414 > M: 214 681 9018 >
