hi all. i'm currently using two openbsd boxes to provide redundant site-to-site vpn connectivity between two geographically distant sites. each has both private (mpls ethernet handoff) and public (standard internet) connectivity. i have a pair of gif tunnels, one over each network, ospf to decide which route to take, and ipsec laid on top. this works so well that on more than one occasion we've been called by our provider (at&t for mpls) and given an apology for their circuit being down - our primary circuit - and we didn't even realize it had been down. we are thrilled with this.
we are on the verge of rolling out several more remote sites, each with multiple network connections (one has three - one mpls and two separate isps). i want to cluster openbsd at each site for redundancy. i believe i can do this with relative ease by just extending my current method (doubling the gifs at each site). but i just received a request to allow the remote sites to talk directly with each other as well. that points to gifs between each site, adding further complexity. now this is starting to look like a mesh, and i've never attempted that before - so i'm here asking for recommendations from those of you who are experienced with this and have the time to help. just trying to extend my current method ... each cluster will have two openbsd boxes. with main office having a single isp and mpls, and one remote site having two isps and mpls - that alone yields 12 gif tunnels. and that's just two sites. suddenly i'm not so sure i'm headed in the right direction. so now i have lots of questions ... am i going about this all wrong? do i need separate route tables for each isp connection at the remote site so that the gif tunnels are assured of going out the correct path? should i be using (private) bgp instead of ospf? and if bgp, should i be using it to push out the ipsec stuff? (i just read that openbgp has ipsec-related capability, but haven't dug into that yet.) specific recommendations will be warmly welcomed, as i have a short timetable and a lot of these new requirements just came to light (scope creep at the last minute - gotta love that). thanks in advance!
