Ok, it took me a long time to get py_stress working. I didn't have thrift / boost / gcc on my debian box :) ....
I'm using this command line believing it's similar to my c# tests from a remote box: cnb:~/apache-cassandra-0.6.0-src/contrib/py_stress# python stress.py -o insert -n 1000000 -d 10.113.0.195 -y super its wicked fast, as you'd expect. total,interval_op_rate,avg_latency,elapsed_time 61765,6176,0.00804715166591,10 126667,6490,0.00763260502319,20 184952,5828,0.00798357854098,30 249697,6474,0.00813590502387,40 310190,6049,0.00820316256729,50 370394,6020,0.00823702464729,60 431557,6116,0.00811347469409,70 492085,6052,0.00818783322119,80 558895,6681,0.00740800772112,90 620426,6153,0.00805101232533,100 685168,6474,0.00766180823714,110 748768,6360,0.00779282277485,120 811008,6224,0.00797537056523,130 867327,5631,0.00881408287019,140 This leaves me further stumped. I guess i will try running py_stress from a remote box, because I've got to believe it has something to do with the connection. Thanks for nudging me toward py_stress. I'm no closer to understanding, but I have more info! Stu On Thu, Apr 15, 2010 at 1:57 PM, Jonathan Ellis <jbel...@gmail.com> wrote: > What kind of numbers do you get from contrib/py_stress? > > (that's located somewhere else in 0.5, but you should really be using > 0.6 anyway.) > > On Thu, Apr 15, 2010 at 12:53 PM, Heath Oderman <he...@526valley.com> > wrote: > > So checking it out quickly: > > vmstat - > > Never swaps. si and so stay at 0 during the load. > > iostat -x > > the %util never climbs above 0.00, but the avgrg-sz jumps bewteen samples > > from 0 - 30 - 90 - 0 (5 second intervals) > > top shows the cpu barely working and mem utilization is below 20%. > > Still slow. :( > > Thanks for the suggestions. In your article on your blog it'd be awesome > to > > include some implications, like "avgrg-sz over 250 may mean XXX" Even if > > it's utterly hardware and system dependent it'd give a guy like me an > idea > > if what I was seeing was bad or good. :D > > Thanks again, > > Heath > > > > On Thu, Apr 15, 2010 at 1:34 PM, Heath Oderman <he...@526valley.com> > wrote: > >> > >> Thanks Jonathan, I'll check this out right away. > >> > >> On Thu, Apr 15, 2010 at 1:32 PM, Jonathan Ellis <jbel...@gmail.com> > wrote: > >>> > >>> You're right, to get those numbers on debian something is very wrong. > >>> > >>> Have you looked at > >>> http://spyced.blogspot.com/2010/01/linux-performance-basics.html ? > >>> What is the bottleneck on the linux machines? > >>> > >>> With the kind of speed you are seeing I wouldn't be surprised if it is > >>> swapping. > >>> > >>> -Jonathan > >>> > >>> On Tue, Apr 13, 2010 at 11:38 PM, Heath Oderman <he...@526valley.com> > >>> wrote: > >>> > Hi, > >>> > I wrote a few days ago and got a few good suggestions. I'm still > >>> > seeing > >>> > dramatic differences between Cassandra 0.5.0 on OSX vs. Debian Linux. > >>> > I've tried on Debian with the Sun JRE and the Open JDK with nearly > >>> > identical > >>> > results. I've tried a mix of hardware. > >>> > Attached are some graphs I've produced of my results which show that > in > >>> > OSX, > >>> > Cassandra takes longer with a greater load but is wicked fast > >>> > (expected). > >>> > In the SunJDK or Open JDK on Debian I get amazingly consistent time > >>> > taken to > >>> > do the writes, regardless of the load and the times are always > >>> > ridiculously > >>> > high. It's insanely slow. > >>> > I genuinely believe that I must be doing something very wrong in my > >>> > Debian > >>> > setups, but they are all vanilla installs, both 64 bit and 32 bit > >>> > machines, > >>> > 64bit and 32 bit installs. Cassandra packs taken from > >>> > http://www.apache.org/dist/cassandra/debian. > >>> > I am using Thrift, and I'm using a c# client because that's how I > >>> > intend to > >>> > actually use Cassandra and it seems pretty sensible. > >>> > An example of what I'm seeing is: > >>> > 5 Threads Each writing 100,000 Simple Entries > >>> > OSX: 1 min 16 seconds ~ 6515 Entries / second > >>> > Debian: 1 hour 15 seconds ~ 138 Records / second > >>> > 15 Threads Each writing 100,000 Simple Entries > >>> > OSX: 2min 30 seconds seconds writing ~10,000 Entries / second > >>> > Debian: 1 hour 1.5 minutes ~406 Entries / second > >>> > 20 Threads Each Writing 100,000 Simple Entries > >>> > OSX: 3min 19 seconds ~ 10,050 Entries / second > >>> > Debian: 1 hour 20 seconds ~ 492 Entries / second > >>> > If anyone has any suggestions or pointers I'd be glad to hear them. > >>> > Thanks, > >>> > Stu > >>> > Attached: > >>> > 1. CassLoadTesting.ods (all my results and graphs in OpenOffice > format > >>> > downloaded from Google Docs) > >>> > 2. OSX Records per Second - a graph of how many entries get written > per > >>> > second for 10,000 & 100,000 entries as thread count is increased in > >>> > OSX. > >>> > 3. Open JDK Records per Second - the same graph but of Open JDK on > >>> > Debian > >>> > 4. Open JDK Total Time By Thread - the total time taken from test > start > >>> > to > >>> > finish (all threads completed) to write 10,000 & 100,000 entries as > >>> > thread > >>> > count is increased in Debian with Open JDK > >>> > 5. OSX Total time by Thread - same as 4, but for OSX. > >>> > > >>> > > >> > > > > >
