Reading old emails is always interesting. Turns out I was in discussions with a company building a CPU that was a very good fit to NIX. I was trying to get that company to ship a research system to lsub.
They were initially very agreeable but, finally, stopped talking about Plan 9 on their system. What stopped them? The Lucent Public License. That license blocked a number of potential partnerships with companies back then. I am glad Nokia helped us resolve that problem. ron P.S. that CPU never went very far. The GPU tsunami killed it. On Thu, Dec 26, 2024 at 10:24 PM Ron Minnich <rminn...@p9f.org> wrote: > > OK, I got curious about when NIX started to happen. Basically, in 2011 > or so, we had wrapped up the Blue Gene work, the last Blue Gene > systems having been shipped, and jmk and I were thinking about what to > do; there was still DOE money left.. We decided to revive the k10 work > from 2005 or so. We had stopped the k10 work in 2006, when Fred > Johnson, DOE program manager of FAST-OS, asked the FAST-OS researchers > to start focusing on the upcoming petaflop HPC systems, which were not > going to be x86 clusters, and (so long ago!) were not going to run > Linux. So we went full circle: DOE funded Plan 9 on k10, then we > shifted gears in 2006 to Blue Gene (Power PC), then in 2011, it was > back to ... K10. > > I wrote a note summarizing what jmk and I came up with and sent it out > to the lsub folks and jmk on April 21. A lot of it is not what we did > :-) but the core idea, of application cores, we did do. > > So, below, the note, showing the core idea in april 2021. The "in May" > reference is lsub's kind offer to fly me out and give me a place to > stay for May 2011. The result was, for me, a chance to work with and > learn from very smart researchers! The 'we' mentioned in the note is > jmk and me. Note that the idea is very unformed. By the time I got > there Nemo had figured out the core operation of switching between AC > and TC, and Charles had convinced me that, since we're running on a > shared memory machine, we might want to take advantage of that fact. > > I pushed hard on having only 2M pages, which we later continued on > Harvey. A standard HPC noise benchmark (rminnich/github.com/ftq) > showed this worked very well. Nemo came up with a very nice idea; once > the break got above 1G, just use 1G pages, b/c only 1 or 2 programs > would need it, but we'd save lots of page table pages in doing this. > It worked well. > > In retrospect, it's not clear that just having 2M pages is a good > idea. 4K seems clearly too small, but 64K seems a better size all > around. > > What was really incredible was just how little of the kernel we had to > change to get it to work, and just how quickly we had the basic system > (about 2 weeks). And it worked so well. We made a minor change to exec > and to rc to make it trivially easy to schedule processes on an AC. > > Finally, why did something like this not ever happen? Because GPUs > came along a few years later and that's where all the parallelism in > HPC is nowadays. NIX was a nice idea, but it did not survive in the > GPU era. > > " > I think we came to a good conclusion about what to do in May. > > The idea is to base our work on the Plan 9 k8 port for the 9k kernel. > I would like to explore the concept of application cores. An > application core is a core that only runs user mode programs. Right > now there are lots of questions about how to do this, but application > cores are seen as a next step in manycore systems. In a system of N^2 > cores,vendors are telling me that something like N of then will be > able to run a kernel, and that N^2-N will not be able to. Application > cores save power, heat, money, and die space. > > The idea is that to prototype we can run a full-up Plan 9 kernel on > core 0, then have a driver (/dev/apcore) with clone file > (/dev/apcore/clone) that a process can open to gain access to a core. > The application core process can be assembled by writing to a ctl file > to do such operations as allocating and writing memory, setting > registers, etc., then launched via write to the ctl file. The > application core process talks to the kernel via typed ipc channels > such as we have today -- it will look kind of like an ioproc but the > channels will be highly optimized like the ones in Barrelfish. > > All the models we need for this mode exist in Plan 9. > > Here's what's neat. You can have application cores, but we can also > have core 0 running a traditional time-shared kernel (Plan 9 in this > case). That way, if you run out of application cores, the traditional > time-shared model is there on core 0. I think this hybrid model is > going to be very powerful. > > So I will be booting the 9k/k8 kernel to make sure I know how. That > way, when I get there, we can get a quick start. > > thanks > " > > On Thu, Dec 26, 2024 at 9:54 PM Ron Minnich <rminn...@p9f.org> wrote: > > > > Hello, I more or less started that project with a white paper early in > > 2011 so may be able to help. NIX was inspired by what we learned from > > the Blue Gene work and other Plan 9 work sponsored by DOE FAST-OS, > > which ran from 2005-2011. During those years, DOE FAST-OS sponsored > > the amd64 compiler, k10 kernel, blue gene port, and NIX, to name a few > > things. I was at both LANL and SNL over that time period. > > > > A group of us spent May of 2011 at lsub getting the initial NIX system > > to work. It was a very productive month :-) The group at lsub were as > > good as it gets, and then we had jmk and Charles there too. Quite the > > Dream Team. > > > > What would you like to know? I also have an initial broken port to > > 9front if you'd like to try to bring it to life. > > > > ron > > > > On Thu, Dec 26, 2024 at 9:13 PM Andreas.Elding via 9fans > > <9fans@9fans.net> wrote: > > > > > > Hello, > > > > > > I was wondering if anyone has any experience using the NIX HPC > > > environment? Traditionally, there's a scheduler that keeps track of the > > > resources in the system, what nodes are busy and with which jobs, how > > > much ram is in use and such. > > > > > > I'm finding very sparse information on the NIX project, so I turn here to > > > ask if anyone has actually used it and can share some details? > > > > > > The site with the most information on it seems to be > > > https://lsub.org/nix/ but the research papers that I have found there > > > are not too detailed (perhaps I've only found previews?). > > > > > > Any extra information would be appreciated. > > > ------------------------------------------ 9fans: 9fans Permalink: https://9fans.topicbox.com/groups/9fans/T7692a612f26c8ec5-Mba95700949bfb1f32c295b1f Delivery options: https://9fans.topicbox.com/groups/9fans/subscription
