GitHub user daviftorres created a discussion: [MySQL] Pruning / Off-loading Database
As environments grow, both the number of database transactions and the overall database size increase exponentially. Some data must remain in the main (hot) database at all times, while other data, useful mainly for historical purposes (investigations, data analysis, etc.), can safely be moved to colder storage to offload the database. I’m tired of dealing with DBs that grow out of control and teams who just throw CPU and RAM at the problem to buy time. The end of that story always looks like Jurassic Park (lol). ## Benefits - Faster execution of complex transactions - More responsive WebUI - Faster backup and restore operations - Lower load during frequent snapshots (e.g., every 15 minutes), especially if they complete in under 10 seconds - And more ## Observations - Tables such as `cloud.alert` and `cloud.usage_event` currently have no retention policy. - The Usage service cannot yet be configured to read from the `cloud` database on the hot MySQL host while writing to a `cloud_usage` database on a colder MySQL host. ## Idea / Proposal - Periodically move entries older than 90 days into a separate `usage_archive` database on a colder MySQL host, and also store them in an S3 bucket for DR. ## Next I’d love to hear your thoughts and different perspectives. GitHub link: https://github.com/apache/cloudstack/discussions/12159 ---- This is an automatically sent email for [email protected]. To unsubscribe, please send an email to: [email protected]
