Roman, great thread, thanks for posting! Comment inline :)
On 07/19/2016 02:03 PM, Roman Dobosz wrote:
<snip>
It can identified 3 levels of FPGA resources, which can be nested one
on the others:
1. Whole FPGA. If used discrete FPGA, than even today it might be pass
through to the VM.
2. Region in FPGA. Some of the FPGA models can be divided into regions
or slots. Also, for some model it is possible to (re)program such
region individually - in this case there is a possibility to pass
entire slot to the VM, so that it might be possible to reprogram
such slot, and utilize the algorithm within the VM.
3. Accelerator in region/FPGA. If there is an accelerator programmed
in the slot, it is possible, that such accelerator provides us with
Virtual Functions (similar to the SR-IOV), than every available VF
can be treated as a resource.
4. It might be also necessary to track every VF individually, although
I didn't assumed it will be needed, nevertheless with nested
resources it should be easy to handle it.
Correlation between such resources are a bit different from NUMA -
while in NUMA case there is a possibility to either schedule a VM with
some memory specified, or request memory within NUMA cell, in FPGA if
there is slot taken, or accelerator already programmed and used, there
is no way to offer FPGA as a whole to the tenant, until all
accelerators and slots are free.
I've followed Jay idea about nested resources and having in mind
blueprint[2] regarding dynamic resources I've prepared how it fit in.
<snip>
To get id of resource of type acceleratorX to allocate 8 VF:
SELECT rp.id
FROM resource_providers rp
LEFT JOIN allocations al ON al.resource_provider_id = rp.id
LEFT JOIN inventories iv ON iv.resource_provider_id = rp.id
WHERE al.resource_class_id = 1668
AND (iv.total - COALESCE(al.used, 0)) >= 8;
Right idea, yes, but you would need to INNER JOIN inventories and LEFT
JOIN from the winnowed set of inventory records to a grouped projection
of allocations. :)
The SQL would be this:
SELECT rp.id
FROM resource_providers rp
INNER JOIN inventories iv
ON rp.id = iv.resource_provider_id
AND iv.resource_class_id = 1688
LEFT JOIN (
SELECT resource_provider_id, SUM(used) as used
FROM allocations
WHERE resource_class_id = 1688
GROUP BY resource_provider_id
) AS al
ON iv.resource_provider_id = al.id
WHERE (iv.total - COALESCE(al.used, 0)) >= 8;
The other SQL queries you listed had a couple errors, but the ideas were
mostly sound. I'll include the FPGA use cases when I write up the nested
resource providers spec proposal.
The only thing I'd say is that I was envisioning the dynamic resource
classes for FPGAs to be the resource context to an already-flashed
algorithm, not to the FPGA root device (or a region even). But, who
knows, perhaps we can work something out. More discussion on the spec...
Best,
-jay
__________________________________________________________________________
OpenStack Development Mailing List (not for usage questions)
Unsubscribe: openstack-dev-requ...@lists.openstack.org?subject:unsubscribe
http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev
