On Tue, 19 Apr 2005 22:26:23 +0300 Eran Tromer <[EMAIL PROTECTED]> wrote:
> Hi, > > On 19/04/05 21:13, Gilad Ben-Yossef wrote: > > MOSIX/OpenMOSIX is a great > > academic excersize - a working academic excersize, but not something I > > would use except for very specific and narrow taks in controled conditions. > > That's consistent with my experience. Here at the Weizmann Institute, > the IT department built a MOSIX-based cluster out of a dozen high-end > machines. It failed miserably. AFAIK, the main problem was that > migration just never happened for most user processes (even after fixing > the default setup which disallows migration of anything invoked via ssh, > which wasn't documented anywhere). To start with, anything that used > shared memory and (IIRC) threads couldn't migrate. Also, anything that > did noticable amounts of I/O got locked to its home node, even though > everything was running on an NFS-mounted filesystem anyway [1]. Since > all processes on the cluster had the same home node (i.e., the formal > gateway to the cluster which everybody sshed to), they ended up having > one overloaded node and 11 nearly idle machines. > both mosix and openmosix don't migrate (posix) threads. They both didn't migrate processes using shared memory. openmosix has a patch that allows migrating processes using shared memory (migshm) which I believe has been incorporated into the mainstream code by now. I don't have experience with mosix but for openmosix io intensive processes stay on the home node by default. You can override the decision though and tell openmosix to treat the program as processor intensive and not io intensive. You can also tell it whether the activity checking should be averaged over short or long term. BTW there are some system calls that cause problems also as they have to be executed on the home node (timeofday mainly) > Eran > > [1] In theory it might have been possible to work around that using the > distributed FS that comes with MOSIX/OpenMOSIX, but I wouldn't bet on > it. I wildly guess it would require a major migration and have some > funny stuff non-Unix semantics, and my general impression was that the > FS is half-baked. > The specialized file system was removed from openmosix (forgot its name) as it gave some trouble. They now recommend using another dedicated shared file system not developed by the same team (sorry, forgot its name but the mailing list has some references to it). NFS is not good for file sharing using openmosix, IIRC there is some problem with syncing file changes between system which can cause file corruptions. Another option is kerrighed, developed at INRIA. The downside is that it only handled up to 32 CPUs last time I checked and can't add and remove nodes dynamically, but it does support migrating posix threads and shared memory and at least according to a benchmark done by someone affiliated with them, network traffic and performance is better on migrated process. with openmosix (and another option that I don't recall at the moment) pipe and socket performance dropped dramatically even after migrating back. BTW there is a way to allow matlab to migrate over openmosix (and probably mosix) although you need to disable the gui interface (JAVA) using the - nojvm flag (it can still open image windows) and disable the licence manager heartbeat (something like TWM_HEARTBEAT_INTERVAL=-1 matlab -nojvm). It doesn't migrate by default since the gui uses java which in turn uses threads. > ================================================================= > To unsubscribe, send mail to [EMAIL PROTECTED] with > the word "unsubscribe" in the message body, e.g., run the command > echo unsubscribe | mail [EMAIL PROTECTED] > > > +++++++++++++++++++++++++++++++++++++++++++ > This Mail Was Scanned By Mail-seCure System > at the Tel-Aviv University CC. > +++++++++++++++++++++++++++++++++++++++++++ This Mail Was Scanned By Mail-seCure System at the Tel-Aviv University CC. ================================================================= To unsubscribe, send mail to [EMAIL PROTECTED] with the word "unsubscribe" in the message body, e.g., run the command echo unsubscribe | mail [EMAIL PROTECTED]
