> For example in my case running 'upgradesstables' on ~300GB node takes about > 30+ hours. +1 And repair.
>> 1 TB at 1Gb/s will take roughly 2.2hrs assume we stream from say 100 nodes… All of these suggestions on node size boil down to: make sure compaction, repair, node replacement and node moves complete in a time that is acceptable to you. So would suggest you run some tests. Cheers ----------------- Aaron Morton Freelance Cassandra Developer New Zealand @aaronmorton http://www.thelastpickle.com On 7/03/2013, at 5:06 AM, Michal Michalski <mich...@opera.com> wrote: > I think it's still true, but not because of network-related issues, but > because of the maintenance problems it will cause during per-node operations. > For example in my case running 'upgradesstables' on ~300GB node takes about > 30+ hours. The other IO-intensive operations will probably be a pain too. > > M. > > W dniu 07.03.2013 13:59, Kanwar Sangha pisze: >> <Do you have 9.5TB on a single node ?> >> In the normal case it's best to have around 300 to 500GB per node. With that >> much data is will take a week to run repair or replace a failed node. >> >> Hi Aaron - This was true for pre 1.2 but with 1.2 and virtual nodes, does >> this still hold ? 1 TB at 1Gb/s will take roughly 2.2hrs assume we stream >> from say 100 nodes... >> >> >> From: aaron morton [mailto:aa...@thelastpickle.com] >> Sent: 06 March 2013 23:47 >> To: user@cassandra.apache.org >> Subject: Re: Hinted handoff >> >> Check the IO utilisation using iostat >> >> You *really* should not need to make HH run faster, if you do there is some >> thing bad going on. I would consider dropping the hints and running repair. >> >> Data is ~9.5 TB >> Do you have 9.5TB on a single node ? >> In the normal case it's best to have around 300 to 500GB per node. With that >> much data is will take a week to run repair or replace a failed node. >> >> Cheers >> >> ----------------- >> Aaron Morton >> Freelance Cassandra Developer >> New Zealand >> >> @aaronmorton >> http://www.thelastpickle.com >> >> On 6/03/2013, at 1:22 PM, Kanwar Sangha >> <kan...@mavenir.com<mailto:kan...@mavenir.com>> wrote: >> >> >> Is this correct ? >> >> I have Raid 0 setup for 16 TB across 8 disks. Each disk is 7.2kRPM with IOPS >> of 80 per disk. Data is ~9.5 TB >> >> So 4K * 80 * 9.5 = 3040 KB ~ 23.75 Mb/s. >> >> So basically I am limited at the disk rather than the n/w >> >> From: Kanwar Sangha [mailto:kan...@mavenir.com<http://mavenir.com>] >> Sent: 06 March 2013 15:11 >> To: user@cassandra.apache.org<mailto:user@cassandra.apache.org> >> Subject: RE: Hinted handoff >> >> After trying to bump up the "hinted_handoff_throttle_in_kb" to 1G/b per sec, >> It still does not go above 25Mb/s. Is there a limitation ? >> >> >> >> From: Kanwar Sangha [mailto:kan...@mavenir.com] >> Sent: 06 March 2013 14:41 >> To: user@cassandra.apache.org<mailto:user@cassandra.apache.org> >> Subject: RE: Hinted handoff >> >> Got the param. thanks >> >> From: Kanwar Sangha [mailto:kan...@mavenir.com] >> Sent: 06 March 2013 13:50 >> To: user@cassandra.apache.org<mailto:user@cassandra.apache.org> >> Subject: Hinted handoff >> >> Hi - Is there a way to increase the hinted handoff throughput ? I am seeing >> around 8Mb/s (bits). >> >> Thanks, >> Kanwar >> >> >
