Related to change 18278[1], I was wondering if there is really a benefit of dealing with 128-byte cachelines like we do today. Compiling VPP with cacheline size set to 128 will basically just add 64 bytes of unused space at the end of each cacheline so vlib_buffer_t for example will grow from 128 bytes to 256 bytes, but we will still need to prefetch 2 cachelines like we do by default.
Whta will happen if we just leave that to be 64? [Honnappa] Currently, ThunderX1 and Octeon TX have 128B cache line. What I have heard from Marvel folks is 64B cache line setting in DPDK does not work. I have not gone into details on what does not work exactly. May be Nitin can elaborate. 1. sometimes (and not very frequently) we will issue 2 prefetch instructions for same cacheline, but I hope hardware is smart enough to just ignore 2nd one 2. we may face false sharing issues if first 64 bytes is touched by one thread and another 64 bytes are touched by another one Second one sounds to me like a real problem, but it can be solved by aligning all per-thread data structures to 2 x cacheline size. [Honnappa] Sorry, I don’t understand you here. Even if the data structure is aligned on 128B (2 X 64B), 2 contiguous blocks of 64B data would be on a single cache line. Actually If i remember correctly, even on x86 some of hardware prefetchers are dealing with blocks of 2 cachelines. So unless I missed something, my proposal here is, instead of maintaining special 128 byte images for some ARM64 machines, let’s just align all per-thread data structures to 128 and have just one ARM image. [Honnappa] When we run VPP compiled with 128B cache line size on platforms with 64B cache line size, there is a performance degradation. Hence the proposal is to make sure the distro packages run on all platforms. But one can get the best performance when compiled for a particular target. Thoughts? -- Damjan [1] https://gerrit.fd.io/r/#/c/18278/ -=-=-=-=-=-=-=-=-=-=-=- Links: You receive all messages sent to this group. View/Reply Online (#12532): https://lists.fd.io/g/vpp-dev/message/12532 Mute This Topic: https://lists.fd.io/mt/30426937/675477 Group Owner: vpp-dev+ow...@lists.fd.io<mailto:vpp-dev+ow...@lists.fd.io> Unsubscribe: https://lists.fd.io/g/vpp-dev/unsub [honnappa.nagaraha...@arm.com] -=-=-=-=-=-=-=-=-=-=-=- IMPORTANT NOTICE: The contents of this email and any attachments are confidential and may also be privileged. If you are not the intended recipient, please notify the sender immediately and do not disclose the contents to any other person, use it for any purpose, or store or copy the information in any medium. Thank you.
-=-=-=-=-=-=-=-=-=-=-=- Links: You receive all messages sent to this group. View/Reply Online (#12551): https://lists.fd.io/g/vpp-dev/message/12551 Mute This Topic: https://lists.fd.io/mt/30426937/21656 Group Owner: vpp-dev+ow...@lists.fd.io Unsubscribe: https://lists.fd.io/g/vpp-dev/unsub [arch...@mail-archive.com] -=-=-=-=-=-=-=-=-=-=-=-
