>
> Furthermore, every interaction with the IPAM requires a cluster-wide
> lock on the IPAM. Having a central cluster-wide lock on every VM
> start/stop/migrate will significantly limit parallel operations.
> Event
> starting two VMs in parallel will be limited by this central lock. At
> boot trying to start many VMs (ideally as much in parallel as
> possible)
> is limited by the central IPAM lock even further.
>>Cluster wide locks are relatively cheap, especially if one avoids
>>having
>>a long critical section, i.e., query IPAM while still unlocked, then
>>read and update the state locked, if the newly received IP is already
>>in there then simply give up lock again and repeat.
>>We also have a clusters wide lock for starting HA guests, to set the
>>wanted ha-resource state, that is no issue at all, you can start/stop
>>many orders of magnitudes more VMs than any HW/Storage could cope
>>with.
You also need to think about external ipam, where maybe it'll take some
seconds to find an available ip and allocate it. (it's depend of size
of the subnet, could have also dns update, ...)
so, it'll really limit the parallelism of vm start.
(Personnaly, If we have choice between reserved at vm/nic create &&
ephemeral at vm start, it's ok me).
> Once an IP is reserved via IPAM, the dnsmasq config can be generated
> stateless and idempotent from the pve IPAM and is identical on all
nodes
> regardless if a VM/CT actually resides on that node or is running or
> stopped. This is especially useful for VM migration because the IP
> stays consistent without spacial considering.
>>
>>That should be orthogonal to the feature set, if we have all the info
>>saved somewhere else
>>But this also speaks against having it in the VM config, as that
>>would
>>mean that every node needs to parse every guests' config
>>periodically,
>>which is way worse than some cluster lock and breaks with our base
>>axiom that guests are owned by their current node, and only by that,
>>and a node should not really alter behavior dependent on some
>>"foreign"
>>guest.
I think that is really more simple to add ip in local dnsmasq at vm
start
dnsmasq --dhcp-hostsfile=/var/lib/reservation.txt
echo "mac ip" >> /var/lib/reservation.txt
SIGUP dnsmasq
for persistant ip, we just need search previously allocated ip-mac in
ipam, then write reservation to dnsmasq and start vm
for epheral ip, we need to find && allocated a free ip in ipam, then
write the ip/mac in dnsmasq and start vm
and for external ipam, I had proposed to use local ipam as read cache.
When allocating a new ip (persistent or ephemeral):
search mac/ip exist in external ipam
true: write it to local pve ipam cache
false: allocate a new free ip in external ipam && write it
to local pve ipam cache
Like this, for persistant ips, we don't care if external ipam is down
at vm start.
and we can also reuse local ipam ips list for firewall or other stuff,
without need to call the external ipam api.
_______________________________________________
pve-devel mailing list
pve-devel@lists.proxmox.com
https://lists.proxmox.com/cgi-bin/mailman/listinfo/pve-devel
