> Dear colleagues, > > any hints to tune rrdtool with ~30k rrd files (approx 2k target > devices)? > > machine is mostly IO-bound, showing 100% disk load with 8 or sometimes > even 3 mB/s, 300-400 tps (it's 2 SATA300 disks in gmirror)
Long and short of it, rrd sucks. I'm not sure whats sucks worse though, the architecture, the code, the data format, the fact that you have to exec open seek seek seek seek close for every one of those rrd files, or the 10000 character long command line you get to throw at it to generate complex graphs. :) I was working on a complete rewrite of the rrd update code a couple months back, but got distracted by other bigger projects and haven't had time to finish it up. If anyone is interested, there is some alpha code (aka do not run this on any .rrd files you value) at: http://sourceforge.net/projects/librrd/ The goal was to implement an efficient C API which wouldn't open/close the file with every cycle, use entirely mmap() (in stock code even when you say to use mmap, there is still a ton of legacy code which seeks around unnecessarily), and implement fine grained read/write locking for concurrency. Personally though, I suspect the vast majority of the speed improvements would come from just general code improvement rather than any specific technique (e.g. removing 5 levels of indirection which are completely unnecessary to accomplish a step, and which only exist because the code is a running hack based on small contributions that "make it work" from 100 different people who primarily write perl, all without any overall design). Pretty sure the remaining bug is in the CDP (consolidated data points) code which creates missing data points in the event you've waited longer than a PDP interval between updates. But more to the point, everything in rrd_update_cdp() is something I haven't yet reverse engineered to figure out what the actual goal is. If you can figure this out, you can probably rewrite that entire block in 1/5th the code, like everywhere else. :) Please let me know if there is any interest in this, I'll be happy to help provide info on what I've done so far, until I find more free time. :) -- Richard A Steenbergen <[EMAIL PROTECTED]> http://www.e-gerbil.net/ras GPG Key ID: 0xF8B12CBC (7535 7F59 8204 ED1F CC1C 53AF 4C41 5ECA F8B1 2CBC) _______________________________________________ freebsd-stable@freebsd.org mailing list http://lists.freebsd.org/mailman/listinfo/freebsd-stable To unsubscribe, send any mail to "[EMAIL PROTECTED]"
