> On 11 Aug 2016, at 19:51, Ben RUBSON <ben.rub...@gmail.com> wrote: > > >> On 11 Aug 2016, at 18:36, Adrian Chadd <adrian.ch...@gmail.com> wrote: >> >> Hi! > > Hi Adrian, > >> mlx4_core0: <mlx4_core> mem >> 0xfbe00000-0xfbefffff,0xfb000000-0xfb7fffff irq 64 at device 0.0 >> numa-domain 1 on pci16 >> mlx4_core: Initializing mlx4_core: Mellanox ConnectX VPI driver v2.1.6 >> (Aug 11 2016) >> >> so the NIC is in numa-domain 1. Try pinning the worker threads to >> numa-domain 1 when you run the test: >> >> numactl -l first-touch-rr -m 1 -c 1 ./test-program > > # numactl -l first-touch-rr -m 1 -c 1 /usr/local/bin/iperf -c 192.168.2.1 -l > 128KB -P 16 -i 2 -t 6000 > Could not parse policy: '128KB' > > I did not manage to give arguments to command. Any idea ?
I answer to myself, this should do the trick : numactl -l first-touch-rr -m 1 -c 1 -- /usr/local/bin/iperf -c 192.168.2.1 -l 128KB -P 16 -i 2 -t 6000 However of course it still gives the error below : > # numactl -l first-touch-rr -m 1 -c 1 /usr/local/bin/iperf > > numactl: numa_setaffinity: Invalid argument > > And sounds like -m is not allowed with first-touch-rr. > What should I use ? > > Thank you ! > >> You can also try pinning the NIC threads to numa-domain 1 versus 0 (so >> the second set of CPUs, not the first set.) >> >> vmstat -ia | grep mlx (get the list of interrupt thread ids) >> then for each: >> >> cpuset -d 1 -x <irq id> >> >> Run pcm-memory.x each time so we can see the before and after effects >> on local versus remote memory access. >> >> Thanks! >> >> >> >> -adrian > _______________________________________________ freebsd-net@freebsd.org mailing list https://lists.freebsd.org/mailman/listinfo/freebsd-net To unsubscribe, send any mail to "freebsd-net-unsubscr...@freebsd.org"
