On Sep 16, 12:16 pm, Yann Le Du <[EMAIL PROTECTED]> wrote: > I use sage, v. 3.1.1, and am trying to build an application (Monte Carlo > stuff) and use dsage to parallelize the code : very easy stuff, just do a > series of jobs, done normally in sequence on a single computer, in > parallel on many.
Let me add what I can to this discussion. I used dsage a while back, and eventually found it unsuitable for the demands I had (I have yet to find a good alternative). I don't think we should wait for the parallel ipython to "swoop in and save us," since dsage is a nice application, and if it does have some hiccups, it is certainly worth improving and getting right. This will happen when someone gets frustrated and fixes what they don't like about it. A lot of the things I didn't like about dsage have been fixed, either by me bugging Yi about them in the past, or more recently. What I wanted to use dsage for was more comparable to one computer with lots of processsors than lots of machines with a few processors each, which is where dsage is valuable. > 6/ If you have any notes, drafts, illustrating some of dsage > functionalities, I'd be more than happy to check them out. I have left a patch (which probably no longer works out of the box--- this is Sage 2.5.3) which implements a pretty complicated distributed function, which is a good way to do what it sounds like you're doing. http://sage.math.washington.edu/home/rlmill/patches/dist_de_codes.patch The callback function gets called as soon as a job finishes, which answers: > 5/ What test can I apply to a dsage job to see if it's finished ? Say a > job outputs a list, and I want to plot it, can I say something like "If > there is some output, plot it, otherwise wait." ? I would consider both of these to be bugs: > 1/ Why isn't there a clear, publicized, illustrated description of how to > use dsage ? I managed to make it work, but only after googling hard. > > But I also noticed that if a worker gets killed, then its job gets lost. > Isn't it possible for the server to check if a worker is alive, > every once in a while, and if not requeue its job ? I had to bug Yi in person on a regular basis to learn dsage... If you have any documentation to contribute (esp. the stuff you had to work extra hard to figure out), I'm sure you would not have a hard time getting it reviewed ;-) > 2/ How can I send a job to a worker that will output intermediate values ? > I mean, say the job sent to a particular worker computes some value, and > that it takes 100 iterations, how can I output temporary values every 10 > iterations and have the server report those intermediate values ? I might be wrong, but I thought there was a mechanism to retrieve the output of stdout from a job that is still running. You could use this to transmit special values you find back early... > 4/ What is the function I can use to check which worker did what, and if > it's alive, and what job got interrupted. D.get_my_jobs() will give you a list of JobWrappers, I think. When you create jobs (see dist_de_codes for an example, because I do this there), before you submit them you can set any attribute you want, such as job.id = "this job is computing using parameters XXX and algorithm YYY", and I think the things you get back from D.get_my_jobs() will have these attributes. I really want to encourage you to submit any patches you think might be useful, either making dsage better, or more easy to use, or especially any documentation which fills in the gap that you had to do the hard way. The main problem with dsage as it is right now is that Yi didn't really work with many other people on the project, and now that he is gone, it is struggling. I think it is worth trying to salvage the situation. --~--~---------~--~----~------------~-------~--~----~ To post to this group, send email to sage-support@googlegroups.com To unsubscribe from this group, send email to [EMAIL PROTECTED] For more options, visit this group at http://groups.google.com/group/sage-support URLs: http://www.sagemath.org -~----------~----~----~----~------~----~------~--~---
