Replying to three in one. >> Well, this would mean: >> a) double ftpmaster. Just as a rough number, the new machine that is >> currently "in progress" has an estimated cost of 20000 Dollar (if you >> take prices from HP Website. This is not what it will cost in the >> end, but it shows what category of stuff we have to get) >> b) Have the double space at our hoster. >> c) Run it with heartbeat, drbd and all that.
>> Point c) is actually easy enough, even though im not DSA and can't decide >> for them to do it. But technically it would be a working setup, provided >> b) works out, as you really want a *FAST* connection between the >> two. Which means local. > Is there any way to build a distributed service instead of relying on > one central machine (or two machines sitting next to each other)? Not currently, no. The actions in the archive can not split in a way this would make sense to place on various machines. That is, not those that are the important ones here (upload processing, mirrortree updating, that parts all around it). > I don't know exactly what services are involved, but typically and > generally, when I deploy a server infrastructure, I try to setup (at > least) two machines at different geographical places that each can > provide the entire service. ... > The complexity in getting a heartbeat, drbd, etc solution to work can > easily eat up any downtime savings you plan to get. It is actually a very easy setup and I am running multiple of them. The complexity is mainly in the double rackspace, double internet connection, *FAST* inter-machine connection, not in the bit drbd stuff. On 12071 March 1977, Guus Sliepen wrote: > What is it that ftp-master does that can only run on one computer at a time? > If > most of the services it provides could be distributed, you could spread the > load to > multiple machines, and get redundancy at the same time. Thank you, we never thought of that... On 12071 March 1977, Thomas Koch wrote: > Joerg Jaspert: > <SNIP> >> The only trouble this setup has is that you have a pretty huge expensive >> machine always on and running, but not actually doing stuff for >> 99.999999999999% of the time. > </SNIP> > Hadoop is now in Debian: > http://packages.qa.debian.org/h/hadoop.htmlHadoop is an Open Source > implementation of Google's File System, MapReduce and > BigTable (HBase, not yet packaged). > The idea behind Google's infrastructure and therefor Hadoop is: Have many > cheap comodity servers that together form a powerful cluster. Each node of > the > cluster is redundant and can be replaced without downtime. > I believe, but can't know for sure, that everything what FTP-Master does, > could be implemented on top of hadoop. > However it means for sure a lot of work and many hardcore sysadmins will feel > very uncomfortable to use Java, the language Hadoop is written in. > I'm planning to give a presentation of hadoop at the DebConf in Bosnia and > maybe then we may discuss, if hadoop should have a place in Debian's > infrastructure. - For now I'm happy, if somebody became curious. :-) The idea behind hadoop and stuff is certainly a nice one. Yet, I do believe it has an entirely different target and can not easily be adopted to the tasks ftp-master does. There simply are things you can not, in a sensible way, make distributed. On 12071 March 1977, Obey Arthur Liu wrote: > Isn't there /some/ python/jython support ? > Would you co-mentor such a project as part of a Summer of Code project > ? Do you know someone who would ? > It need not be ftpmaster. There are probably other critical debian > infrastructure which could use this. I do not think changing dak and all it does to hadoop and distributed is possible within gsoc. Nor within ten of it. I wont block any who tries though, so have fun. :) -- bye, Joerg <andreasj> Also diese neuen Spam-Mails muten an wie Blog-Posts von Clint Adams <andreasj> irgendwie ist es eine Geschichte, aber ich versteh sie nicht -- To UNSUBSCRIBE, email to debian-devel-requ...@lists.debian.org with a subject of "unsubscribe". Trouble? Contact listmas...@lists.debian.org Archive: http://lists.debian.org/87bpe471t0....@gkar.ganneff.de
