FYI: MySQL performs fine with 100G files if you set it up correctly. I haven't used the dashboard or looked at the source code, but with that kind of storage I'd say you have a write-heavy application. You can tune for that quite easily, although scaling beyond a single master will be a bit more tricky as opposed to write-heavy apps where you can just add slaves.
innodb_file_per_table should imho be set for every mysql server in existence for many reasons, but it will only work if you have innodb tables and not MyISAM (duh to me, not so duh to others maybe ;)). Don't go and delete binlogs at random if you love your data and want to be able to do proper backups. If you see too many binlogs, just set expire-logs-days to something sane (read: larger then the time between your backups). If you really want to get rid of some binlogs, purge them usign mysql, don't just delete the files: http://dev.mysql.com/doc/refman/5.0/en/purge-binary-logs.html Just saying: mysql is not as bad as people make it seem :) Walter On Tue, Jan 10, 2012 at 04:06, Jo Rhett <jrh...@netconsonance.com> wrote: > On Jan 9, 2012, at 3:31 PM, Daniel Pittman wrote: > > Is there any reasonable estimate for what amount of space you expect one > > system to use? I realize this likely varies with the report size, but the > > rate of growth seems high enough that I'm surprised it wasn't mentioned in > > the installation docs. I mean, it's grown half a gigabyte in the last 6 > > hours. With that kind of growth rate, you'd expect a warning to provide > > enough space for it and how to estimate your needs. > > > That growth rate seems ... excessive. Ultimately, the size of the > stored data is pretty directly related to the size of your YAML > reports; can you capture one of those and see how big it is on disk? > > > FYI, in 10 hours the database has grown slightly more than 1G. That's an > extensive growth rate. > > Looking at the yaml files, I'm seeing 410k per file * 400 nodes = 160Mb per > 30 minutes. > > Is there really no optimization that is performed on the data stored in the > database? Coming up with a few hundred gigabytes of file storage is one > thing. Trying to make mysql perform well with 100Gb database is an entirely > different matter. > > -- > Jo Rhett > Net Consonance : consonant endings by net philanthropy, open source and > other randomness > > -- > You received this message because you are subscribed to the Google Groups > "Puppet Users" group. > To post to this group, send email to puppet-users@googlegroups.com. > To unsubscribe from this group, send email to > puppet-users+unsubscr...@googlegroups.com. > For more options, visit this group at > http://groups.google.com/group/puppet-users?hl=en. -- Walter Heck -- follow @walterheck on twitter to see what I'm up to! -- Check out my new startup: Server Monitoring as a Service @ http://tribily.com Follow @tribily on Twitter and/or 'Like' our Facebook page at http://www.facebook.com/tribily -- You received this message because you are subscribed to the Google Groups "Puppet Users" group. To post to this group, send email to puppet-users@googlegroups.com. To unsubscribe from this group, send email to puppet-users+unsubscr...@googlegroups.com. For more options, visit this group at http://groups.google.com/group/puppet-users?hl=en.
