On Fri, 3 Apr 2009 13:52:28 -0500 Marco Peereboom <sl...@peereboom.us> wrote:
> That said I can guarantee that the OpenBSD project pays more attention > to its users then other OS'. This does not mean that the users get to > set the road-map. When an idea is not good the author is told so, > usually, in strong language. The opposite is Linux and other unnamed > BSDs where everyone agrees with each other paralyzing proper > development. A stupid idea is still a stupid idea and it isn't > magically going to mature like a good wine. Over the last handful of days, I've been trying to figure out whether or not the the way of doing things proposed by my work is actually a stupid idea. Though I've been sent out to investigate status on all UNIX variants, both open and closed source, I'm pushing for using OpenBSD in one of their new designs. The design involves a technology called "Express Ether" though it is typically written as "ExpEther," and it is basically a way to run a PCIe bus over ethernet. Though this might be the first you've heard of it, ExpEther has been in development at NEC for the last five years, and yes, I'm currently working on getting the documentation released for the existing silicon. http://www.nec.co.jp/press/en/0702/0801.html http://www.expether.org/ In short, you can think of ExpEther as something between a bus extender and a bridge (PCIe<->ethernet), so basically anything you can plug into a PCIe slot can be made available to a remote machine. Yep, you can even partition attached devices into VLANs and basically "build" a computer on the fly out of available parts attached to the network. For example if your VPN or secure website is running a little slow, you would usually halt the machine and add a crypto accelerator, but with ExpEther, you just export a crypto accelerator device on another system to the system that needs it and the recipient system assumes the device is attached to it's local PCIe bus. One of the first applications I'm working on is exporting a softraid volume over ExpEther. I was asked if it was possible to build a shim that makes a block device like a softraid sd0a look like an ATA device sitting on a (fictitious) ATA controller on the PCIe bus? Though it's certainly an uncommon thing to try to do, there's just something about this approach that makes me wonder if it's a crazy/stupid idea, or absolutely brilliant? To *me* (complete idiot), I'm wondering if this is being approached at the wrong level, namely shimming a block device like sd0a to be seen as a ata/scsi device on a fictitious controller, versus shimming something below it, i.e. scsibus0 at softraid0 sd0 at scsibus0 The *consumer* of the resource is expecting to see a disk attached to a (fictitious) scsi/ata controller on it's local PCIe bus (which is imported via ExpEther). The *provider* of the resource needs to take a softraid volume and make it look like just a (fictitious) disk attached to a (fictitious) scsi/ata controller on a (fictitious) PCIe bus (which is exported via ExpEther). Whether or not the shimming is done below partitioning on the provider side is yet to be determined. If it is done above partitioning on the provider side (i.e. block devices like sd0a), the result will be two layers of partitioning (both provider and consumer) since sd0a on the provider-system would become the (fictitious) sd0 on the consuming-system. The thing to remember is we're talking *below* the file system, so well intended suggestions of NFS, ZFS, or file-system-de-jour are not at all relevant. As for the vast number of different types of potential failure modes, the PCIe spec includes hot-plug requirements (yet who knows if your $VENDOR implemented them properly), but on top of that, the ExpEther spec also has it's own hot-plug requirements, and they've been implemented. Even with all this, getting the potential failure modes correctly handled at the various levels will take a lot of effort. Yes Nick, you can show up with your nail-gun (ramset). (; Though this is for my work, I'm quietly doing my best to make it benefit the project in various ways (including docs, code, ...). If any of you would be kind enough to drop kick me in the direction of finding a clue, or even want to voice an opinion about the bleeding edge (pink elephants, vaporware, etc.) stuff I'm working on, it would be much appreciated. I'm *way* over my head on a lot of this stuff, but I'm learning it as fast as I can. Thanks, jcr -- J.C. Roberts
