John, Don't forget two huge additional benefits to it being external: Scalability and distribution. The service handling this can be clustered and distributed if it is external.
Aimon On 2/7/11 3:12 PM, John Purrier wrote: > > There are a lot of benefits to having an external system be > responsible for handling usage data from Nova, Swift, or other > OpenStack services. I would call out: > > > > 1. Simplification of code within each service. The collection > and publication of usage data is "dumb"... store the service usage > data in a service defined schema (could just be logfiles) until it is > fetched/pushed and then delete. No additional service API's required > beyond that to efficiently move the data out of the service. > > > > 2. Simplification of hardware required for each service. If the > service is required to do analytics/processing on the usage data this > will drive more powerful hardware solutions. > > > > 3. Single target for organizational usage data. This allows the > best selection of hardware and software for collecting and maintaining > large datasets. This also allows the > analytic/billing/auditing/warehousing service to scale independently > of any particular service. > > > > 4. Data warehousing for compliance and billing repeatability. It > is a likely scenario that billing and usage disputes will occur, > sometimes months after the fact. It is imperative that the billing and > compliance reports be able to be recreated and analyzed. Holding all > of this data inside the individual services for long retention periods > doesn't make sense. > > > > 5. Data warehousing for cold storage. The ability to tier > storage for long term retention, at the cheapest possible cost. > > > > 6. Being able to do joins effectively across different service > usages to create consolidated billing. > > > > 7. In addition to billing and auditing use cases, being able to > make the data available to scheduled and ad hoc reporting. This > involved analytic processing, data reduction, and other processing > that is not appropriate on the provisioning and real-time service > controllers. > > > > 8. Specialized data processing to allow real-time updates to > system behaviors. This is getting into the future, but creating a feed > forward analytic process may allow some really smart auto-scaling and > dynamic provisioning scenarios. > > > > John > > > > -----Original Message----- > From: openstack-bounces+john=openstack....@lists.launchpad.net > [mailto:openstack-bounces+john=openstack....@lists.launchpad.net] On > Behalf Of Monsyne Dragon > Sent: Monday, February 07, 2011 2:58 PM > To: openstack@lists.launchpad.net > Subject: Re: [Openstack] Pondering multi-tenant needs in nova. > > > > On 2/7/11 2:49 PM, Eric Day wrote: > > > Thanks for explaining things further, Jay. > > > > > > I agree if we want external systems poking into Nova for audit/billing > > > queries, then yes, this gets inefficient. My assumption is that Nova > > > specific DBs only contain operational data required for production and > > > it would push billing/audit events to some external system that can > > > collect, aggregate, and answer those queries efficiently. Trying to > > > design a common data store that fits both use cases of provisioning > > > instances/networks/volumes along with handling queries for > > > billing/audit would be difficult (as we are seeing). Pushing > > > billing/audit data to another system gives us the flexibility to > > > choose the most suitable data store and querying abilities for each > > > use case without making sacrifices for the other. > > > > > > > Yes this is the model proposed with the system-usage blueprint. Nova > > publishes usage data, via a pub/sub interface, and a separate > > billing/audit system subscribes to those events, and builds it's > > datastore as it sees fit. > > > > > > > _______________________________________________ > Mailing list: https://launchpad.net/~openstack > Post to : openstack@lists.launchpad.net > Unsubscribe : https://launchpad.net/~openstack > More help : https://help.launchpad.net/ListHelp -- Aimon Bustardo | Morph Labs | +1.310.625.0608 (mobile) | +1.310.437.4898 (office) | www.morphlabs.com | ai...@mor.ph
_______________________________________________ Mailing list: https://launchpad.net/~openstack Post to : openstack@lists.launchpad.net Unsubscribe : https://launchpad.net/~openstack More help : https://help.launchpad.net/ListHelp
