On 08-Feb-20 8:09 PM, Dmitry Kozlyuk wrote:
The main reason DPDK memory management works the way it does is because
of need to support multiprocess. In order to map memory in all
processes, we need that space reserved (otherwise there's no guarantee
that the newly mapped memory segment will be mapped in all processes,
and it'll cause runtime failure). If it wasn't for that, we could
allocate memory arbitrarily and as needed. Windows should either follow
this model, or drop secondary support and go its own way - the internals
are OS-specific anyway.
I think Windows should support multi-process, because there is a demand and
an ongoing design effort for multi-tenancy and resource arbitration [0].
Until Windows kernel implements "secure API" for the architecture proposed by
[0] (if it does at all), DPDK multi-process model can to some point support
the features desired. For example, a primary process may be a service
performing resource arbitration for applications being secondary processes.
Bear in mind that DPDK also supports external memory, you might
need to make some allowances for that too.
I haven't considered external memory yet. Does it need anything beyond
mapping VA to IOVA?
No, just a few checks here and there. The brunt of the mapping is on the
shoulders of the user, and "external memory API" is really just
registering this memory with DPDK so that calls like rte_virt2memseg()
work correctly.
As for IOMMU - we don't support IOVA as VA addressing on FreeBSD, so if
Windows port can only work with IOVA as PA, that's fine too. The
question of IOVA mode really boils down to, do we control the DMA
addresses (IOVA as VA mode), or does the system (IOVA as PA). I'm not
familiar with how IOMMU works on Windows, but as long as it fits into
that model and we keep the API, it should also be OK :)
AFAIK, Windows doesn't expose IOMMU either to applications or drivers. Do I
understand correctly that implies only IOVA as PA can be supported, because
mappings can't be set up?
In general, yes. The name probably doesn't match very well, admittedly,
but that's the equivalent as far as software is concerned. That's not a
problem - FreeBSD doesn't support IOVA as VA either :)
The trouble is, PA cannot generally be used if IOMMU is present, but there
is no way to tell if it is. Windows kernel offers API to allocate buffers for
DMA [1], but MM doesn't know if it allocates memory for DMA or not, even if
that kernel API would be exposed. If I got it right, DPDK just can't be used
on Windows with IOMMU enabled (can't tell for VMs that don't see IOMMU).
[0]:
https://www.dpdk.org/wp-content/uploads/sites/35/2018/12/RMenonOCardona_Improving-Security-in-Windows-DPDK.pdf
[1]:
https://docs.microsoft.com/en-us/windows-hardware/drivers/ddi/wdm/nc-wdm-pallocate_common_buffer_ex
--
Thanks,
Anatoly
