> The sensible thing to do in nvme is to use different paths for > different queues. That is e.g. in the RDMA case use the HCA closer > to a given CPU by default. We might allow to override this for > cases where the is a good reason, but what I really don't want is > configurability for configurabilities sake.
That makes sense but I'm not sure it covers everything. Probably the most common way to do NVMe/RDMA will be with a single HCA that has multiple ports, so there's no sensible CPU locality. On the other hand we want to keep both ports to the fabric busy. Setting different paths for different queues makes sense, but there may be single-threaded applications that want a different policy. I'm not saying anything very profound, but we have to find the right balance between too many and too few knobs. - R.
