On 06/10/2013 11:44 PM, Aaron Rosen wrote:
Hi Gary,
The motivation for this patch was to remove the unnecessary querying
that was being done to quantum during the initial sync, previously a
get_subnets/_ports was done for each network in addition to an update
for the dhcp port even if it didn't need to). In my opinion the
amount of data being sent to the agents is not a big deal. Even in
extremely large deployments we are talking about on order of megs of
data that are sent and this is only done on initial sync. That said,
I'll update the patch to remove subnets that are not dhcp enabled as
you are right we don't need to send these down.
Great.
In regards to rewriting the hosts file for each update I don't think
this is really a big deal either. Implementing logic to hash the files
to see if it needs to be updated seems like overkill as it will only
save us on the initial sync. Currently, with the code on review i can
restart the dhcp and have it complete sync_state() with 2k networks 8k
ports in ~20 seconds where it previously took 30+ minutes.
20 seconds is a very long time. :)
On the same subject though another performance issue that I think we
should fix is the agent currently updates the ipallocation table for
each renewed lease which happens every 120 seconds by the default
value. This generates a ton of API calls to quantum and grows the more
vms that are deployed(!) In my opinion it would be worth while to
batch these and provide a bulk update method that is done every 60
seconds rather than immediately when it gets a renew.
Nice idea
I'm not sure about the left over port issue you are seeing. I've not
noticed this issue on my end. I'll keep an eye out though as well.
I'll try and provide some details tomorrow (I have ben unable to
reproduce with rabbitmq on a VM but it reproduces with qpid on baremetal)
Best,
Aaron
On Sun, Jun 9, 2013 at 12:10 AM, Gary Kotton <gkot...@redhat.com
<mailto:gkot...@redhat.com>> wrote:
Hi,
In addition to Aarons changes there are still a lot of issues
which I think are problematic with the current implementation. The
problems that I see at the moment are as follows:
1. The amount of data that sent on the network to the agents
2. The way in which the data is parsed on the agent:
i. Networks that are disabled are sent to the agent
ii. Subnets that have DHCP disabled are sent to the agent
iii. The host file is rebuilt with each and every change. This
means looping over all of the ports in the network each time there
is an update.
3. Message reliability. At the moment I am investigating a bug
where deleting 10 networks leaves the host file with details of a
port that has already been deleted. (
I think that a way of addressing the above may be as follows:
1. Enable the plugin to prepare the information for the agents.
This can leverage the database filtering and reduce the amount of
data on the network. In addition to this the plugin can/may have
drivers that are specific to the agents implementation - for
example in the case of dnsmasq, it will be preparing the host
file. In the case of the dnsmasq we can send host file deltas. A
further reduction can be compressing the host file.
2. In order to prevent very heavy operations when either
agents/services are restarted we can have a hash of the above
file. This can ensure that it is synced (this is something that is
specific to the driver implementations).
Thoughts?
Thanks
Gary
--
Mailing list: https://launchpad.net/~quantum-core
<https://launchpad.net/%7Equantum-core>
Post to : quantum-core@lists.launchpad.net
<mailto:quantum-core@lists.launchpad.net>
Unsubscribe : https://launchpad.net/~quantum-core
<https://launchpad.net/%7Equantum-core>
More help : https://help.launchpad.net/ListHelp
--
Mailing list: https://launchpad.net/~quantum-core
Post to : quantum-core@lists.launchpad.net
Unsubscribe : https://launchpad.net/~quantum-core
More help : https://help.launchpad.net/ListHelp
