On Thu, Apr 3, 2014 at 6:24 AM, Yuanjing (D) <yj.y...@huawei.com> wrote: > Hi: > > > > I have deployed a RDO havana version's openstack environment for a period. > > > > By chance I tested APIs of ceilometer, in the first I defined a new meter > like meter_test, then used 'ceilometer sample-list -m yjmeter' command to > check the result. > > I found that the query cost about 13 seconds, why it was so slow with one > record? > > > > Now that RDO used mongodb as default DB for ceilometer, I logging into it > and found that the meter collection had about 2,700,000 records. Maybe the > mass records slowed the query. > > > > Take production environemnt into account, If there are 100 hosts , each host > contains 30 VM. > > CeilometerI use default configuration of 11 compute pollsters and the > pipeline interval is 600s. > > Then one day system will generate 4,752,000 record, the formula is: > 11(meter) * 30(VM) * 6(system run pollsters 6 times an hour) * 24(a day) * > 100(host) = 4,752,000 records. > > > > For above case I think it is necessary to restrict number of records by some > mechanism, below are my immature idea: > > 1. Make restriction on max supported records by time or number of samples, > discard old records.
There is a mechanism to delete old data in place using an age. > > 2. Providing API to delete samples by resource_id or some other conditions, > so the third integration system may call this API to delete related samples > when delete a resource. That's not a bad idea, although you wouldn't always want to call it. Metering data should live longer than the resource being billed. > > 3. Running period task of accounting on raw samples, using 1min samples to > generate 5min statistics samples, using 5min statistics to generate 30min > statistics samples, and so on. Every period of sample has individual data > table and has resriction on max supported records . This sort of roll-up would be useful for monitoring, but would break the audit trail for metering. So it might be useful, but may not solve the whole problem. > I am not a ceilometer programmer and I apologize if I am missing something > very obvious. The rate of collection, meters kept, and retention time are all configurable using the main configuration file or the pipeline YAML file. That's not to say this isn't a real problem -- I see one or two summit sessions that may cover performance issues like this, and I know it is a high priority for both of our PTL candidates. Doug > > Can you give me some help to make me clear about them and how to implement > my requirement? > > > > Thanks > > > _______________________________________________ > OpenStack-dev mailing list > OpenStack-dev@lists.openstack.org > http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev > _______________________________________________ OpenStack-dev mailing list OpenStack-dev@lists.openstack.org http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev
