Hypervisor
Hi everyone, I will create the network virtualization in Hurd, so I need to implement a mechanism which help pfinet servers communicate with each other. That is, for example, if pfinet 1 has IP A and pfinet 2 has IP B, the packet sent by pfinet 1 with destination address IP B should be received by pfinet 2. Sub-hurd should also be able to use this mechanism to communicate with each other. Meanwhile this mechanism should allow non-privileged the user to start his own pfinet. The possible approach is to create a hypervisor whose role is: 1. to create some virtual network interface, so pfinet can send packets to it. 2. to receive the packet from pfinet, and forward the packet to other pfinets in hurd 3. or forward the packet to the real network device in the kernel and send it to the network. 4. to guard the network traffic and filter illegal packets (forged by some malicious users) from pfinet or some other programs. To implement it, I have to solve some problems: 1. What kind of programming interface should the hypervisor provide to pfinet or other programs? In the current implementation of pfinet, the pfinet server gets the privileged port, opens the network interface by calling device_open() and set the filter rules by calling device_set_filter(). My solution is: Since we want every user to be able to run pfinet, we should modify the code of pfinet: pfinet explicitly get the send right to hypervisor by opening some files created by hypervisor or getting it from the proc server. I'm not sure whether the pfinet still needs to set filter rules when it sends packets to the hypervisor. 2. How does the hypervisor route packets to the right pfinet server or the network device in the kernel? As my understanding, if there are two pfinet servers running in the same network interface and a packet comes, both pfinet servers should be able to receive the packet in the current hurd. This is right because the pfinet server can drop the packet it doesn't want. The hypervisor can do the same thing: If it receives a packet from a pfinet server or from the network device in the kernel, it can send the packet to all pfinet servers that are connected to the hypervisor. (maybe the hypervisor should make a choice to send the packet to the real network interface or to pfinet servers when it receives a packet from a pfinet server) But the problem is that every user can run pfinet now, so one user may be able to watch other users' packets if every pfinet packets can get the packet. It should be a secure issue. 3. How does the hypervisor guard the network traffic and filter illegal packets? I have no idea how to do it. and I'm not very sure I should do it. That's all what I think should be done. Could someone give me some advices or comments? Thank you in advance. Best, Zheng Da
Re: Possible bug in name of command 'settrans'
A.Salatov ha scritto: No, you correct me if I'm wrong, but when I think about 'settrans' my mind always going to compare it to 'umount' and I started to think about a reasons why 'umount' is 'umount' and not 'unmount'. I've always found *that* a bug. umount means nothing. uNmount has a meaning. If it was somehow all abbreviated, like "mnt" and "umnt" it could have had a sense. But "umount" has no sense whatsoever. However I live with it. The simplest reason for it, that I could imagine, it is so because it less typing to do. Maybe, but it's silly the same. Typing a 'n' more is a small price for the sake of clarity. You ever try to type 'setrans' instead 'settrans'. No, *you* ever try that. Don't tell me what *I* do, because you don't know me. *I* can cope pretty well with double letters, thanks. I see that you have some problems with that (see your spelling below), but that's *your* problem, not our. You remind me of a collegue of mine that used a cli-interface-based software I wrote in my lab. One of the commands was "genlist", and he always mistyped it "getlist". Instead of learning the right command, he complained with me, asking to alias the command to "getlist", because that's what he often typed (even if "getlist" made no sense semantically for what the command did). I complied for the sake of living quietly, but it was extremly annoying. If it is not enough, may be it is beter to have even shorter form, like - 'strans' :) Fine. I'd also like "st", that would make even more sense. In this case 's' would actualy stand for "SetTRANSlator", and it OK? Ok, all this topic is a kinda joke, but with an sense in it as I supose. If no one would treat it in real, well then it is complitly a joke. :( Probably it is. :) PS. Of course I remeber about aliases :) But may be I just misunderstud some thing? :) Don't know. I'd say just to use aliases and live happy. :) m.
Hypervisor
Hi everyone, I will create the network virtualization in Hurd, so I need to implement a mechanism which help pfinet servers communicate with each other. That is, for example, if pfinet 1 has IP A and pfinet 2 has IP B, the packet sent by pfinet 1 with destination address IP B should be received by pfinet 2. Sub-hurd should also be able to use this mechanism to communicate with each other. Meanwhile this mechanism should allow non-privileged the user to start his own pfinet. The possible approach is to create a hypervisor whose role is: 1. to create some virtual network interface, so pfinet can send packets to it. 2. to receive the packet from pfinet, and forward the packet to other pfinets in hurd 3. or forward the packet to the real network device in the kernel and send it to the network. 4. to guard the network traffic and filter illegal packets (forged by some malicious users) from pfinet or some other programs. To implement it, I have to solve some problems: 1. What kind of programming interface should the hypervisor provide to pfinet or other programs? In the current implementation of pfinet, the pfinet server gets the privileged port, opens the network interface by calling device_open() and set the filter rules by calling device_set_filter(). My solution is: Since we want every user to be able to run pfinet, we should modify the code of pfinet: pfinet explicitly get the send right to hypervisor by opening some files created by hypervisor or getting it from the proc server. I'm not sure whether the pfinet still needs to set filter rules when it sends packets to the hypervisor. 2. How does the hypervisor route packets to the right pfinet server or the network device in the kernel? As my understanding, if there are two pfinet servers running in the same network interface and a packet comes, both pfinet servers should be able to receive the packet in the current hurd. This is right because the pfinet server can drop the packet it doesn't want. The hypervisor can do the same thing: If it receives a packet from a pfinet server or from the network device in the kernel, it can send the packet to all pfinet servers that are connected to the hypervisor. (maybe the hypervisor should make a choice to send the packet to the real network interface or to pfinet servers when it receives a packet from a pfinet server) But the problem is that every user can run pfinet now, so one user may be able to watch other users' packets if every pfinet packets can get the packet. It should be a secure issue. 3. How does the hypervisor guard the network traffic and filter illegal packets? I have no idea how to do it. and I'm not very sure I should do it. Could someone give me some advices or comments? Thank you in advance. Best, Zheng Da
Re: Revision control
Hi, On Sun, Jun 08, 2008 at 09:29:00AM +0200, Arne Babenhauserheide wrote: > Am Samstag 07 Juni 2008 17:47:28 schrieb [EMAIL PROTECTED]: > If git isn't readily usable for me, then it is quite probable, that it > is unusable for many programmers, including some who might be > interested in contributing to the Hurd. It doesn't seem to detain any of the Linux contributors, nor the Xorg ones... In the latter case, I know that there was some grunting in the beginnig, but it soon died out; git problems are hardly ever mentioned now. (And again, I think most of them were only because some people didn't bother to read up on the fundamentals in the first place.) > Why do I _have to_ understand its basic concepts to use it without > running into problems (as opposed to wanting to understand them, > because they interest me)? No powerful tool can be properly used without understanding it. Unless perhaps you put a restrictive interface on top of it. And in fact, the git developers never cease to point out that such interfaces can be easily created for git. The fact that very little actually exists in this way, only proves that in spite of any initial grunting, people are quite fine using the less shiny but fully powerful standard interface in the end. > UNIX usability is great. I didn't understand most of its basic > concepts until I learned them in my informatics class at university, > but I could use it quite well, because current tools provide a system > which (mostly) just works. Without knowing about the file system, about pipes, about processes, about signals, about scripts, about the init system? I have a hard time believing this. Unless perhaps you mean using GNOME. That is not what I was talking about. > But I think going for the simpler tool is useful, if the more complex > tool doesn't offer significant advantages for the project. Well, I still believe git *does* offer significant advantages, for any project. But again, I'm not really qualified to argue this point... > And in my eyes the potential of Mercurial and git is about the same. > What mostly differs is the order in which their features were (and, I > assume, will be) implemented. Well, for all I know, git got almost all of its features within the first few months. Mercurial on the other hand has been in the works for several years by now (isn't it actually older than git?), and still seems to miss features that I consider crucial. That's hardly the same level... Unless by "potential", you mean "we should use it now, as it might catch up some day" -- I hope you will understand that I'm not convinced by that :-) > For Linux development, where (as I now know) 40% of the sourcefiles > are changed from one version to the next, and where most people who > dare touch the code are longterm kernel hackers, git is the better > choice, I think. Linux has fast changing parts as well as relatively stable ones. I don't see why this would be any different with the Hurd. (Making corrections for overall activity, of course...) And anyways, I don't see how this is related to the choice of revision control system. -antrik-
Re: Namespace-based translator selection; project details
Hi, On Sat, Jun 07, 2008 at 11:21:10PM +0300, Sergiu Ivanov wrote: > On Sat, Jun 7, 2008 at 3:13 PM, <[EMAIL PROTECTED]> wrote: > > Nevertheless, there is a lot of merit in your observation: I realise > > now that there is really no reason to externally classify the > > translators, and attach them them to a different node up front. > > Instead, all translators could just be attached to the node with all > > static translators present; and the proxy would provide an > > additional interface an all the virtual nodes, explicitely allowing > > to ask for the same node, but untranslated. > > > Normal translators like "gunzip" would just use the underlying node > > directly, forming an ordinary translator stack. Filter translators > > on the other hand would use the additional interface to got the > > untranslated node corresponding to the translated one they are > > attached to, and work on that. > > Yes, that was my idea. Ah. Seems I'm a bit slow :-) > > The communication channel is established the moment a translator is > > launched: The translator is attached to the underlying node upon > > startup. Part of this process involves the translator getting a port > > to the node on the parent FS it is attached to. (The virtual node on > > the main namespace proxy, in our case.) > > > > Most of the requests on this port are simply forwarded by the proxy > > to the underlying "real" nodes, so it behaves just like the > > translator had been indeed attached directly to the "real" nodes. > > Some requests however are more explicitely handled by the proxy -- > > like for instance providing an untranslated (virtual) node > > corresponding to the translated one... > > Yes, it seems clear for me that a simple translator does not need to > know whether it is set upon a virtual node created by the proxy or > upon a "real" node. The more specific requests, like asking for the > untranslated node, bother me. I'll try to figure out how to organize > such communication. I don't really understand. What is there to figure out? > > I wouldn't write it as "file,,x", though -- this is too confusing. > > The result of actually opening "file,,x" would be quite different: > > It would stack a new dynamic "x" translator on top of the "file" > > node with all it's static traslators present -- "x", "u", "y", "z", > > and now another "x"... > > OK, I will avoid such notation. I just wanted to use less words, but > lost accuracy :-) Well, I tend to think along these lines as well... But nevertheless it's wrong :-) > Note that the virtual node mirroring the original "file" node with the > > static "x" translator applied (what you call "file,,x", though as I > > explained above this is confusing), is actually created by the main > > proxy, upon request from "-u". "-u" first opens a mirror of the > > untranslated node, and then follows only the "x" translator, which > > causes the main proxy to create another mirror, this time with the > > "x" applied. "-u" then returns that as the result to be passed to > > the client. > > > > When another filtering translator is applied, say "-foo", this one > > will again ask for a mirror of the untranslated node, and check what > > translators exist on it. It gets only "x" this time, as "x" is the > > only translator present on the virtual node previously created. If > > the filter rule of "-foo" says that "x" should be filtered, foo will > > return a mirror of "file" without any translators; otherwise, it > > will just again return the mirror of "file" with "x" applied -- the > > very same (virtual) node it is attached to. In other words, "-foo" > > won't change anything if the filter rule doesn't require skipping > > "x"; it just passes through the previous node. > > Do I understand it correctly that after applying the '-u' translator, > there will be two virtual nodes in the proxy filesystem, of which one > will be mirroring the untranslated version of 'file' and the other > will be 'file,,x'? Not really. The one mirroring the pure untranslated "file" is used only temporarily. All in all, three virtual nodes are involved: The one with all translators present, to which "-u" is attached; the completely untranslated one used by "-u" temporarily; and finally the one with "x" attached, which is returned to the client. Oh, and "file,,x" is still wrong :-) > In other words, the user of the proxy will think that translators 'y' > and 'z' are gone for good, right? The proxy won't really care that in > the underlying filesystem the node 'file' is translated by static > translators 'x', 'u', 'y', and 'z', will it? Yes, so to say. The client doesn't see the "u", "y", and "z" translators at all -- it sees the node as if the translator never were present. > > But if there is some standard way to ask any translator for the > > underlying node, we wouldn't need the special interface. It would > > simplify things. > > I'll try to find out whether such a possibility exists, but I'm afraid > I'll fai
Re: Updating the progress
Hi Madhusudan, On Mon, Jun 09, 2008 at 08:45:06PM +0530, Madhusudan C.S wrote: > Yes, some parts of the code is a template of other netfs translators. > But if seen keenly, nearly almost all the translators I am referring > follows the same template or rather have copied the template from one > another. Which is perfectly resonable... > To be on safer side I thought of using the template of ftpfs which is > in hurd main source only, since it is copyrighted to FSF. Indeed, that's what I would suggest myself. Both for licensing reasons, and for correctness -- after all, this code is written by the same people who created the libraries :-) > I hope that will not have any adverse effect on my code in terms of > copyright and license issues. No, not at all. But I do think you should mention where the code comes from, and more importantly, the copyright dates of the original code. > > BTW I wonder, why are you keeping build logs in git?... > > Aah I wanted to tell about that myself, when we discuss among our > friends about the progress, I usually used to tell about the silliest > programming errors we make during programming which leads to hours of > work to find and fix it. So many of them asked me to maintain a build > log so they can see how things are going, and also I thought it would > be nice for me to go through such things, because this is the first > big free software project I am doing that too from scratch. I > understand your concern that it would make the repository very big > unnecessarily. I wont be transferring that to Hurd CVS you provide us. > Is that ok or should I discontinue that from git? Well, it's a bit unusual, but I have no real objections -- keep it if you want :-) -antrik-
Re: Fwd: Re: Revision control
Hi, On Sun, Jun 08, 2008 at 01:39:51AM +0200, Arne Babenhauserheide wrote: > Am Samstag 07 Juni 2008 18:19:29 schrieb [EMAIL PROTECTED]: > > Err... Does that mean that Mercurial doesn't even offer rebase (and > > it can't be implemented trivially)?! > > That means, that the Mercurial people say, that history should be > preserved in most cases, so it wasn't first priority. This is precisely the sort of policy decisions that the software should not impose upon users. In fact, changing history is useful in many many situations, and one of the main reasons I consider git so powerful. Surely you will say now that Linux perhaps needs to change history, but other projects don't... :-P -antrik-
Re: Revision control
Hi, On Sun, Jun 08, 2008 at 01:18:25AM +0200, Arne Babenhauserheide wrote: > [...] and because of some glitches like having to garbage collect > regularly instead of having a lean implementation in the first place. Again, this is not a glitch, but a conscious decision -- and a brilliant one at that IMHO. I mentioned only some of the many advantages of this approach. > Would you use a file system you have to repack regularly? Just think > about what would happen, if we build a server on top of git, which > used it as filesystem backend. With Mercurial as backend it would just > work. A revision control system is not a file system though. And running a filesystem on it is not exactly the common use case... Certainly not what we want to do with the Hurd repository. > This was one of the things which helped to convince me, that Mercurial > is the right choice for me: > > """Unlike many revision control systems, the concepts upon which > Mercurial is built are simple enough that it???s easy to understand > how the software really works. Knowing this certainly isn???t > necessary, but I find it useful to have a ???mental model??? of > what???s going on. > > This understanding gives me confidence that Mercurial has been > carefully designed to be both safe and efficient. And just as > importantly, if it???s easy for me to retain a good idea of what the > software is doing when I perform a revision control task, I???m less > likely to be surprised by its behaviour. > > In this chapter, we???ll initially cover the core concepts behind > Mercurial???s design, then continue to discuss some of the interesting > details of its implementation.""" It's funny that you quote this passage: It fully supports my argument that you need to understand the fundamentals -- much better than I could ever argue it myself :-) I wonder whether you were aware, while quoting it, how perfectly it fits git... > And you can easily access all the core functionality of Mercurial > using either the default Python shell (just do "from mercurial import > ", or advanced shells like ipython (which I very much > appreciate). Except if I don't know Python... And that is only one of the many many reasons why libraries are usually inferior to the traditional UNIX approach of reusing code by invoking other processes. But that's a totally different discussion... Though it gets us back to my original point about following UNIX philosophy. -antrik-
Re: GSoC: the plan for the project network virtualization
Hi, On Sat, Jun 07, 2008 at 10:12:21PM +0200, zhengda wrote: > 1) hypervisor creates a virtual network interface >How is the programming interface like for the virtual network >interface? I mean, is the pfinet still able to open it with >device_open()? Well, as I mentioned in the original discussion, both approaches are possible: Either there is a special para-virtualized interface which pfinets must explicitely implement so they can talk to the hypervisor. Or the hypervisor provides a virtual network interface that looks just like the actual Mach-provided one. The first approach is simpler on the hypervisor side, but requires changes in pfinet, and isn't fully transparent in usage. Para-virtualization also can be more efficient -- but I'm not sure it would make any difference here. I personally think the fully virtualized approach is more elegant, but YMMV. >If pfinet can open the interface with device_open(), I think we >need to write another program like boot to give pfinet the pseudo >master device port and help pfinet open the virtual network >interface. Why another program? I'm pretty sure "boot" is the right place to handle this. >The second question is: who can create the virtual network >interface? By default, the user who invokes the hypervisor. (I.e. the one running "boot".) More sophisticated policy control is certainly possible, but I can't think of any use case right now. >I still don't understand who should check and control the packet. I'm not sure I understand the question. I thought it is rather clear that the hypervisor does the checks?... -antrik-
