[sage-devel] Re: Sage + Hadoop

2012-02-24 Thread Keshav Kini
First we need to upgrade our IPython to 0.12! Is anyone working on it? -Keshav Join us in #sagemath on irc.freenode.net ! On Saturday, February 25, 2012 7:25:10 AM UTC+8, Harald Schilly wrote: > > personally, i'm a fan of IPython's parallelization framework. Maybe it > would be best to ext

[sage-devel] Re: Sage + Hadoop

2012-02-24 Thread Harald Schilly
personally, i'm a fan of IPython's parallelization framework. Maybe it would be best to extend their existing framework? this would benefit sage directly because ipython is included. implementing higher-level interfaces directly into sage will come handy later. H -- To post to this group, sen

[sage-devel] Re: Sage + Hadoop

2012-02-24 Thread Florent Hivert
Hi Andrew, On Thu, Feb 23, 2012 at 12:44:39PM -0800, akm wrote: > > I'm interested. But in the practical application I've in mind, the data > > isn't > > there it is generated on the fly by sage itself. The problem it to gather > > information (eg: the size) on huge sets from combinatorics. Th

Re: [sage-devel] Re: Sage + Hadoop

2012-02-24 Thread Florent Hivert
On Thu, Feb 23, 2012 at 12:52:38PM -0800, akm wrote: > > I'm interested. But in the practical application I've in mind, the data > > isn't > > there it is generated on the fly by sage itself. The problem it to gather > > information (eg: the size) on huge sets from combinatorics. Those sets are >

[sage-devel] Re: Sage + Hadoop

2012-02-24 Thread Andrew Musselman
> In any case the streaming utility seems quit like what would be good for > Sage. What do I have to do to take a file with, say, a lot of random > numbers and then use a Sage script to compute the square of each? Can you > give me a brief description? Here's a way to perform that task using sage

[sage-devel] Re: Sage + Hadoop

2012-02-23 Thread Martin Raum
I had look at http://wiki.apache.org/hadoop/PythonWordCount . I might have misconcluded that using Jython means that you have to use it. It was marked as a Python example, making me think it was a generic one. In any case the streaming utility seems quit like what would be good for Sage. What d

[sage-devel] Re: Sage + Hadoop

2012-02-23 Thread akm
> From having a short glance at the webpage I see that you use jpython. I > doubt that Sage builds with it, so I wonder whether this can become a > problem or not. Which webpage are you looking at? I haven't thought much yet about how to implement this integration, but there are some options. Ha

[sage-devel] Re: Sage + Hadoop

2012-02-23 Thread akm
> I'm interested. But in the practical application I've in mind, the data isn't > there it is generated on the fly by sage itself. The problem it to gather > information (eg: the size) on huge sets from combinatorics. Those sets are > generated by a choice tree. The exploration of the various branc

[sage-devel] Re: Sage + Hadoop

2012-02-22 Thread Martin Raum
Hi Andrew, I would be interested as well. It seems like a fabously efficient way of distributing jobs. >From having a short glance at the webpage I see that you use jpython. I doubt that Sage builds with it, so I wonder whether this can become a problem or not. Best, Martin -- To post to th