On 4/14/2020 12:44 PM, Dmitry Kozlyuk wrote:
Note: no changes in cover letter since v2.
This patchset implements basic MM with the following features:
* Hugepages are dynamically allocated in user-mode.
* Only 2MB hugepages are supported.
* IOVA is always PA, obtained through kernel-mode driver.
* No 32-bit support (presumably not demanded).
* Ni multi-process support (it is forcefully disabled).
* No-huge mode for testing with IOVA unavailable.
The first commit introduces a new kernel-mode driver, virt2phys.
It translates user-mode virtual addresses into physical addresses.
On Windows community call 2020-04-01 it was decided this driver can be
used for now, later netUIO may pick up its code/interface or not.
New EAL public functions for memory mapping are introduced
to mitigate OS differences in DPDK libraries and applications:
* rte_mem_map
* rte_mem_unmap
* rte_mem_lock
To support common MM routines, internal wrappers for low-level
memory reservation and file management are introduced. These changes
affect Linux and FreeBSD EAL. Shared code is placed unded /unix/
subdirectory (suggested by Thomas).
Also, entire <sys/queue.h> is imported from FreeBSD, replacing existing
partial import. There is already a license exception for this file.
Windows MM duplicates quite a lot of code from Linux EAL:
* eal_memalloc_alloc_seg_bulk
* eal_memalloc_free_seg_bulk
* calc_num_pages_per_socket
* rte_eal_hugepage_init
Perhaps this should be left as-is until Windows MM evolves into having
some specific requirements for these parts.
Notes on checkpatch warnings:
* No space after comma / no space before closing parent in macros---
definitely a false-positive, unclear how to suppress this.
* Issues from imported BSD code---probably should be ignored?
* Checkpatch is not run against dpdk-kmods (Windows drivers).
---
v3:
* Fix Linux build on and aarch64 and 32-bit x86 (reported by CI).
* Fix logic and error handling while allocating segments.
* Fix Unix rte_mem_map(): return NULL on failure.
* Fix some checkpatch.sh issues:
* Do not return positive errno, use DWORD for GetLastError().
* Make dpdk-kmods source files non-executable.
* Improve GSG for Windows Server (suggested by Ranjit Menon).
v2:
* Rebase on ToT. Move all new code shared between Linux and FreeBSD
to /unix/ subdirectory, also factor out some existing code there.
* Improve description of Clang issue with rte_page_sizes on Windows.
Restore -fstrict-enum for EAL. Check running, not target compiler.
* Use EAL prefix for private facilities instead if RTE.
* Improve documentation comments for new functions.
* Remove co-installer for virt2phys. Add a typecast for clarity.
* Document virt2phys in user guide, improve its own README.
* Explicitly and forcefully disable multi-process.
Dmitry Kozlyuk (9):
eal/windows: do not expose private EAL facilities
eal/windows: improve CPU and NUMA node detection
eal/windows: initialize hugepage info
eal: introduce internal wrappers for file operations
eal: introduce memory management wrappers
eal: extract common code for memseg list initialization
eal/windows: fix rte_page_sizes with Clang on Windows
eal/windows: replace sys/queue.h with a complete one from FreeBSD
eal/windows: implement basic memory management
config/meson.build | 12 +-
doc/guides/windows_gsg/build_dpdk.rst | 20 -
doc/guides/windows_gsg/index.rst | 1 +
doc/guides/windows_gsg/run_apps.rst | 84 ++
lib/librte_eal/common/eal_common_fbarray.c | 57 +-
lib/librte_eal/common/eal_common_memory.c | 104 +-
lib/librte_eal/common/eal_private.h | 134 +-
lib/librte_eal/common/malloc_heap.c | 1 +
lib/librte_eal/common/meson.build | 9 +
lib/librte_eal/freebsd/eal_memory.c | 55 +-
lib/librte_eal/include/rte_memory.h | 74 ++
lib/librte_eal/linux/eal_memory.c | 68 +-
lib/librte_eal/meson.build | 4 +
lib/librte_eal/rte_eal_exports.def | 119 ++
lib/librte_eal/rte_eal_version.map | 4 +
lib/librte_eal/unix/eal.c | 47 +
lib/librte_eal/unix/eal_memory.c | 113 ++
lib/librte_eal/unix/meson.build | 7 +
lib/librte_eal/windows/eal.c | 160 +++
lib/librte_eal/windows/eal_hugepages.c | 108 ++
lib/librte_eal/windows/eal_lcore.c | 187 ++-
lib/librte_eal/windows/eal_memalloc.c | 418 ++++++
lib/librte_eal/windows/eal_memory.c | 1141 +++++++++++++++++
lib/librte_eal/windows/eal_mp.c | 103 ++
lib/librte_eal/windows/eal_thread.c | 1 +
lib/librte_eal/windows/eal_windows.h | 129 ++
lib/librte_eal/windows/include/meson.build | 2 +
lib/librte_eal/windows/include/pthread.h | 2 +
lib/librte_eal/windows/include/rte_os.h | 48 +-
.../windows/include/rte_virt2phys.h | 34 +
lib/librte_eal/windows/include/rte_windows.h | 43 +
lib/librte_eal/windows/include/sys/queue.h | 663 +++++++++-
lib/librte_eal/windows/include/unistd.h | 3 +
lib/librte_eal/windows/meson.build | 6 +
34 files changed, 3611 insertions(+), 350 deletions(-)
create mode 100644 doc/guides/windows_gsg/run_apps.rst
create mode 100644 lib/librte_eal/unix/eal.c
create mode 100644 lib/librte_eal/unix/eal_memory.c
create mode 100644 lib/librte_eal/unix/meson.build
create mode 100644 lib/librte_eal/windows/eal_hugepages.c
create mode 100644 lib/librte_eal/windows/eal_memalloc.c
create mode 100644 lib/librte_eal/windows/eal_memory.c
create mode 100644 lib/librte_eal/windows/eal_mp.c
create mode 100644 lib/librte_eal/windows/eal_windows.h
create mode 100644 lib/librte_eal/windows/include/rte_virt2phys.h
create mode 100644 lib/librte_eal/windows/include/rte_windows.h
Tested-by: Pallavi Kadam <pallavi.ka...@intel.com>
