Hello, On Mon, Dec 29, 2008 at 8:25 AM, <olafbuddenha...@gmx.net> wrote:
> On Mon, Dec 22, 2008 at 07:19:50PM +0200, Sergiu Ivanov wrote: > > I'm rather inclined to view the node concept introduced in libnetfs as > > a rather generic concept, and my question arose from such > > understanding. Still, I realize that such use will be abusive, so I'd > > like to ask now how would you suggest to implement this feature? > > Not knowing libnetfs, I can only make rather wild guesses... > > In a sense, we are implementing multiple filesystems: One for each > filesystem in the mirrored directory tree. > > The most radical approach would be to actually start a new nsmux > instance for each filesystem in the mirrored tree. This might in fact be > easiest to implement, though I'm not sure about other consequences... > What do you think? Do you see how it could work? Do you consider it a > good idea? > I'm not really sure about the details of such implementation, but when I consider the recursive magic stuff, I'm rather inclined to come to the conclusion that this will be way too much... I'd rather suggest implementing something thinner than nsmux to proxy specifically filesystems or sticking with just a single instance of nsmux. Probably, this variant could work, but I'll tell you frankly: I cannot imagine how to do that, although I feel that it's possible. I could probably dig a bit deeper and come up with more detailed conclusions, but I'm not sure whether it is worth it. What do you say? > But let's assume for now we stick with one nsmux instance for the whole > tree. With trivfs, it's possible for one translator to serve multiple > filesystems -- I guess netfs can do that too... Could you please explain what do you mean by saying this in a more detailed way? Unfourtunately, I can but very vaguely imagine what are you trying to tell me... > Though it might be > tricky, because here we don't attach the translator to multiple > locations in the "outside" filesystem, but rather attach to one outside > location, while the other filesystems are in turn attached to nodes > served by nsmux itself -- I wonder whether this recursion would cause > any complications. I'm sorry, I don't get it :-( > Even if it works, it might be too much overkill, though. The alternative > would be to override the default implementations of some of the fsys and > other RPCs dealing with control ports, so we would only serve one > filesystem from the library's point of view, but still be able to return > different control ports. As we override the standard implementations, it would be up to us how we > handle things in this case. Easiest probably would be to store a control > port to the respective real filesystem in the port structure of every > proxy control port we return to clients. This is variant I was thinking about: custom implementations of some RPCs is the fastest way. At least I can imagine quite well what is required to do and I can tell that this variant will be, probably, the least resource consuming of all. > > > The file_getcontrol()->fsys_getroot() sequence is a very odd one -- > > > what would you provide as the "dotdot" node for fsys_getroot()?... I > > > don't think this possibility was really intended. > > > Hm... I'm not sure I can understand properly what you are talking > > about. Why would this sequence be odd? I cannot think of any other > > possibilities to traverse a translator stack than using this sequence. > > Eh? Traversing translator stacks -- just like normal lookup -- uses > file_get_translator_cntl(), not file_getcontrol(), right? > Yes, yes, that's right... I've mixed up function names. > As for the ``dotdot'' node, nsmux usually knows who is the parent of > > the current node; if we are talking about a client using nsmux, it is > > their responsibility to know who is the parent of the current node. > > > > OTOH, I am not sure at all about the meaning of this argument, > > especially since it is normally provided in an unauthenticated > > version. > > AIUI it is returned when doing a lookup for ".." on the node returned by > fsys_getroot(). In other words, it normally should be the directory in > which the translated node resides. Yep, this is my understanding, too. I guess I have to take a glimpse into the source to figure out *why* this argument is required... As the authentication is always specific to the client, there is no > point in the translator holding anything but an unauthenticated port for > "..". Sorry for the offtopic, but could be please explain what do you mean by authentication here? (I would just like to clear out some issues in my understanding of Hurd concepts) > My point was that if we obtain the control port with file_getcontrol(), > rather than file_get_translator_cntl() as in a normal lookup, we lack > the context: We can use file_getcontrol() to check what translator > serves a given file, but we don't know where the translator resides in > the directory tree. Thus, calling fsys_getroot() and doing further > lookups from there doesn't really seem to make much sense in this > situation... Which is why I think we might be able to get away just > ignoring this case. > Aha, I see. Probably, we can indeed forget about this situation. > > If I understand things correctly, then unionfs does *not* proxy all > > nodes. Similarly to nsmux, unionfs only proxies directory nodes. > > Oh, I thought you said at some point that unionfs proxies all nodes... > Guess I mixed it up. If unionfs doesn't, I'm pretty confident we do not > need to do it either :-) > Aha, ok :-) I've got it :-) > > > > Hm... Suppose nsmux is asked to fetch 'file,,x,y'. > > > > > > You mean "file,,x,,y" I assume?... > > > > > This syntax looks new to me :-) I don't remember seeing or using it > > before. > > That's pretty strange... I indeed can't find me mentioning this syntax > anywhere in the list archives. But I'm almost sure I must have used it > on IRC at least... I don't really remember, but that's not a problem. > Anyways, even if I never stated it explicitely, I don't see any reason > to consider using any *different* syntax. This is really the obvious > one. We use one suffix to set a dynamic translator, and then another > suffix to set a second one on top of it... No need to special-case this > in any way. I see... This syntax indeed wants a new shadow node for each new translator in a dynamic translator stack. > Well, setting translators in a simple loop is a bit faster, since, for > > example, you don't have to consider the possibility of an escaped > > ``,,'' every time > > Totally neglectible... > Of course :-) I've got a strange habit of trying to reduce the number of string operations... > (and this is not the only reason). > > What are the others? > The main reason that makes me feel uneasy is the fact that retries actually involve lookups. I cannot really figure out for now what should be looked up if I want to add a new translator in the dynamic translator stack... Although I cannot deny the fact that using some smart retry mechanism would be much more readable and extensible than a loop. > > Unfourtunately, I cannot really feel the advantage of using retries in > > the case of creating dynamic translator stacks. > > Uniformity for one: You agreed that we should forward the retries when > crossing translator boundaries on the mirrored filesystem, and also when > continuing the lookup after starting a dynamic translator. Why should we > do differently when starting multiple dynamic translators, conflating > the lookup steps into one, instead of forwarding the retries after > starting each individual translator?... Hm, this does sound very reasonable, and it's a pity I've never looked at this matter from this point of view... I think I've got this thing sorted out. I'm just hesitating about what should be looked up after starting up an individual dynamic translator. As soon as you give me a hint, I'll put my hand on that code :-) > > BTW, could you please expound on the question why would we need extra > > shadow nodes in a dynamic translator stack? I fail to see how this > > would make our life simpler :-) > > It's not even about simplicity: It's about correctness. Well, correctness does make life simpler ;-) At least for me :-) It's harder to come up with a good example when dealing only with > single-node translators; but when considering translators serving > directories, it really becomes quite obvious: The first dynamic > translator exports a directory. Now in the original lookup we have > another magic suffix, so this directory is further translated. However, > further lookups can be performed on the directory, and such further > lookups might want to skip the second translator. They can actually use > another magic suffix to invoke a different second translator on top of > the same first one. Or someone might even set a static translator on the > directory provided by the first dynamic one... > Yes, I can see now... Thank you for explanation :-) > In these cases it is crucial that we do *not* see the second dynamic > translator resulting from the original lookup, while accessing the > directory provided by the first dynamic translator. In other words, the > second dynamic translator must sit on a shadow node, just like the first > one. That's the definition of dynamic translators: They are visible only > for their own clients, but not from the underlying node. (No matter > whether this underlying node is served by another dynamic translator.) That seems clear. What makes we wonder, however, is how a filter will traverse a dynamic translator stack, if it will not be able to move from a dynamic translator to the one which is next in the dynamic translator stack? > Having said that, I still believe that this also simplyfies things, as > this way we don't need any special-casing when setting more than one > dynamic translator. We just start the first one, and push the ball back > to the client -- the resulting retry will make us start the second one > in exactly the same manner as the first. > This is indeed a good strategy, as soon as all *_S_dir_lookup routines do likewise when they encounter a translator. Regards, scolobb
