Hello folks. I have a rare situation that I’m told some centers are successfully pulling off and am looking for guidance - multi-hop lnet routing. In short, I have 2 distinct o2ib fabrics at disparate geo sites joined by a routed ethernet fabric. I’m looking to use a 2-lnet-router chain to plumb the two o2ib fabrics together.
servers on the left, clients on the right o2ib0(10.5.0.0/16) <-> router(o2ib0,tcp0) <-> routed eth (10.37.0.0/16, 10.38.0.0/16) <-> router(tcp0,o2ib2) <-> o2ib2(10.6.0.0/16) I have both sets of routers up but traffic absolutely fails the 2nd hop in either direction (I can `lctl ping` tcp0 from o2ib2 and o2ib0 but no further). I’ve tried adding a route ON the routers, that didn’t help. I’ve tried defining the 2nd hop on the client: options lnet routes="tcp0 10.6.0.[250-251]@o2ib2;\ o2ib0 10.37.250.[162-163]@tcp0” but that failed with the following kern message on lnet load: 74067:0:(router.c:644:lnet_add_route()) Cannot add route with gateway 10.37.250.162@tcp. There is no local interface configured on LNet tcp Does anyone have any hints here? It feels like I’m a syntax change or a routing hint away from getting this working. _______________________________________________ lustre-discuss mailing list lustre-discuss@lists.lustre.org http://lists.lustre.org/listinfo.cgi/lustre-discuss-lustre.org
