On Tue, Jul 19, 2016 at 12:16:52PM -0700, Brenden Blanco wrote:
> The mlx4 driver by default allocates order-3 pages for the ring to
> consume in multiple fragments. When the device has an xdp program, this
> behavior will prevent tx actions since the page must be re-mapped in
> TODEVICE mode, which cannot be done if the page is still shared.
>
> Start by making the allocator configurable based on whether xdp is
> running, such that order-0 pages are always used and never shared.
>
> Since this will stress the page allocator, add a simple page cache to
> each rx ring. Pages in the cache are left dma-mapped, and in drop-only
> stress tests the page allocator is eliminated from the perf report.
>
> Note that setting an xdp program will now require the rings to be
> reconfigured.
>
> Before:
> 26.91% ksoftirqd/0 [mlx4_en] [k] mlx4_en_process_rx_cq
> 17.88% ksoftirqd/0 [mlx4_en] [k] mlx4_en_alloc_frags
> 6.00% ksoftirqd/0 [mlx4_en] [k] mlx4_en_free_frag
> 4.49% ksoftirqd/0 [kernel.vmlinux] [k] get_page_from_freelist
> 3.21% swapper [kernel.vmlinux] [k] intel_idle
> 2.73% ksoftirqd/0 [kernel.vmlinux] [k] bpf_map_lookup_elem
> 2.57% swapper [mlx4_en] [k] mlx4_en_process_rx_cq
>
> After:
> 31.72% swapper [kernel.vmlinux] [k] intel_idle
> 8.79% swapper [mlx4_en] [k] mlx4_en_process_rx_cq
> 7.54% swapper [kernel.vmlinux] [k] poll_idle
> 6.36% swapper [mlx4_core] [k] mlx4_eq_int
> 4.21% swapper [kernel.vmlinux] [k] tasklet_action
> 4.03% swapper [kernel.vmlinux] [k] cpuidle_enter_state
> 3.43% swapper [mlx4_en] [k] mlx4_en_prepare_rx_desc
> 2.18% swapper [kernel.vmlinux] [k] native_irq_return_iret
> 1.37% swapper [kernel.vmlinux] [k] menu_select
> 1.09% swapper [kernel.vmlinux] [k] bpf_map_lookup_elem
>
> Signed-off-by: Brenden Blanco <bbla...@plumgrid.com>
...
> +#define MLX4_EN_CACHE_SIZE (2 * NAPI_POLL_WEIGHT)
> +struct mlx4_en_page_cache {
> + u32 index;
> + struct mlx4_en_rx_alloc buf[MLX4_EN_CACHE_SIZE];
> +};
amazing that this tiny recycling pool makes such a huge difference.
Acked-by: Alexei Starovoitov <a...@kernel.org>
