On Wed, Apr 01, 2009 at 09:38:35PM -0400, Sagie <[email protected]> was heard to say: > Bellow is the gist of my proposal. Any and all feedback much > appreciated.
No, not another great proposal! :-P > Add remote host package management through Aptitude GTK. The current > local interaction with apt will be extended to span across SSH > connections to remote hosts. In addition, the UI will be extended to > incorporate a comparison view similar in nature to SVN code merge views. > This will list differences in package configurations & versions between > hosts while allowing to selectively synchronize differences. Design will > aim at allowing future non GTK front-end developers to reuse > functionality. Sagie, This is something I really have wanted to do for a long time. However, getting aptitude to work over a network could be really, really hard; I think you'll have to use some sort of a nasty hack to make it even tractable at all. Before I even began to consider this project I would need to be convinced that you've thought through how to approach this aspect of it and that you can produce something basically functional in three months. (more on this below) Also, we already have a lot of proposals for aptitude projects, and I only feel comfortable trying to mentor one of them. I'm just saying that to warn you that there's a lot of competition here; if your main goal is to participate in SOC, I would recommend filing a second proposal alongside this one in case I end up picking a different aptitude project. (note that the deadline is *Friday*) However, your proposal sounds very well thought through and I am going to take it seriously. If you aren't picked, I would encourage you to work on this anyway; I would be happy to work with you to integrate patches that work in this direction. > --- Sought Features & Implementation --- > Basically the idea is simple : abstract the fact we are administering > apt remotely - this will be achieved by generating an interface > containing a most of the functionality present in 'apt.cc' which > 'local_apt' & 'remote_apt' classes will implement. All remote apt > functions will be piped through the SSH channel. This paragraph makes me wonder if you've contemplated the full scope of what you need to do. apt.cc is only the tip of the iceberg: it handles initializing the standard apt structures and loading them into memory. It does **NOT** provide access to the apt database; that is done in aptcache.cc, which extends some core apt classes found in /usr/include/apt-pkg. The apt database class, in turn, is linked to a number of other classes (such as the raw package cache structures, as well as package and version iterators) which expose the contents of the cache. In short: you would have to essentially write a fully abstracted apt-like library on top of apt for this to work. And that would be terrible for the performance of a local aptitude (many things that are simple inline lookups would turn into function calls), while not really working over the network (because there are just too many calls to apt functions to tunnel efficiently over the network). I think a more pragmatic, albeit also uglier, approach is to simply lock the package database and cache on the remote side, then *copy* all the apt state structures over the network, copying them back when you're done (of course, you can use rsync for this instead). The advantage of this is that then you only have to deal with making most of aptitude stop using the global apt cache pointer (which it shouldn't be using anyway) and start using some wrapper class instead. It can crawl all over the package cache however much it wants without worrying that it's a remote cache. You may also need to fix some stuff in apt; I think it tends to assume that there's only one copy of everything and that it's in the standard places (but my RootDir patches, which are integrated in sid, should help a bit here). > [-] modular design > All functionality related to package configuration comparison will be > extracted to a separate module, possible compiled as a standalone > library. This may be used by network management tools such as nagios. I'm not quite sure what this means in practical terms. > [-] SSH communication channels with remote apt hosts > SSH interaction may use pssh, openssh or similar. Although ultimately > apt would adopt a full client-server model this is beyond the scope of > this project. I am still investigating the applicability of using RPC to > remotely invoke libapt. Output parsing is another option, though one > which should be avoided if possible. Establishing exactly which files > should be imported via the network is to be established after consulting > the Debian community. The thought of trying to run apt calls through RPC makes my hair stand on end. aptitude probably makes, oh, a few million or so calls to apt methods before it even shows you its main interface window. I would be shocked if the number is below several hundred thousand. Can you tunnel that over the network as synchronous RPC calls without having the latency kill you? (I'm pretty sure the answer is "no", but I haven't used RPC lately) The other stuff is easy once you solve these problems; I won't comment on it. > 1. SSH based communication with remote apt based hosts, preferable as > a standalone library > 2. An abstraction layer between local & remote apt instances If you could get these two working reliably and not implement anything else, I would consider your project a huge success (although in reality you would have to write at least a little bit of frontend code to test the backend). The other stuff is not that hard once these two are implemented, and if you submit a proposal I would recommend marking 3-6 as "reach" goals, to be dealt with if you manage to complete the meat of your proposal ahead of schedule. Obviously it would be even better to have a strong UI for this, but these two are the hard sticking points. Right now I think your proposal is probably too much for you to bite off over the summer, particularly since you don't seem to be very familiar with apt's guts (something that you'll have to deal closely with in order to carry the project out) but I am willing to be convinced otherwise. Daniel _______________________________________________ Soc-coordination mailing list [email protected] http://lists.alioth.debian.org/mailman/listinfo/soc-coordination
