[Rpy] R console: long output
Hello,
I'm enjoying the fun (self-imposed) task of learning Python, RPy2 and
PyGTK all in one go, so this question may be partly due to my general
status as a newbie. My application will have an embedded console in it.
Everything is generally working nicely, however I've noticed that if I
load a particularly large data file via R's read.delim() command, Python
chokes on the output. The data is loaded as expected, but when I try to
print the output to the console, only a portion of the data.frame is
printed (~65000 characters), presumably because it reached the maximum
string size, and then a seemingly-unrelated GTK widget breaks (but we'll
ignore that part for now)
What seems ideal to me would be a way to grab the R console output one
line at a time rather than all at once so I can just append those lines
individually to my gtk text buffer. I can easily do this for a specific
task, such as only for a data.frame, programmatically break it apart by
rows. However, I'm hoping for something more general that could apply to
any output.
I have tried to set robjects.rinterface.setWriteConsole(f) to some
function f that just appends to a buffer, but that console callback
function doesn't seem to append anything to the buffer if I do a call to
robjects.r("some command here"). That said, I'm not sure how this method
works exactly and if I'd encounter the same problem because a single
string of the entire output would just be appended to the buffer.
Ok that was a bit long-winded. Is there an obvious solution to this
problem that I'm missing? Am I not being clever enough?
Thanks for your help,
brandon
--
Apps built with the Adobe(R) Flex(R) framework and Flex Builder(TM) are
powering Web 2.0 with engaging, cross-platform capabilities. Quickly and
easily build your RIAs with Flex Builder, the Eclipse(TM)based development
software that enables intelligent coding and step-through debugging.
Download the free 60 day trial. http://p.sf.net/sfu/www-adobe-com
___
rpy-list mailing list
rpy-list@lists.sourceforge.net
https://lists.sourceforge.net/lists/listinfo/rpy-list
Re: [Rpy] R console: long output
> >> What seems ideal to me would be a way to grab the R console output one
> >> line at a time rather than all at once so I can just append those lines
> >> individually to my gtk text buffer. I can easily do this for a specific
> >> task, such as only for a data.frame, programmatically break it apart by
> >> rows. However, I'm hoping for something more general that could apply to
> >> any output.
>
> Erh... did you check the example ?
> http://bitbucket.org/lgautier/rpy2/src/813a7bc798ae/demos/radmin.py
> It seems to be doing what you are looking for.
>
Yes, checked it before I sent the email (ah, but if only I found it
before I started coding...I wouldn't have reinvented the wheel so much;
though I guess it was better to learn on my own.). My code for adding
the result to the gtk.TextBuffer is essentially the same.
What I meant was having access to the lines of output in R before it's
converted to a Python stringbut see below for a workaround.
> >> I have tried to set robjects.rinterface.setWriteConsole(f) to some
> >> function f that just appends to a buffer, but that console callback
> >> function doesn't seem to append anything to the buffer if I do a call to
> >> robjects.r("some command here").
> >
> > robjects.r("foo") doesn't print anything itself. (Unless 'foo' calls
> > the print() function.) Try robjects.r("print(foo)") or
> > robjects.r["print"](robjects.r("foo")).
>
> The confusion might occur because genuine R console calls print on an
> object (a bit like a Python console calls the method __repr__() ).
I'm embarassed, because I read about using R's print() function in a
former email on this email list...but somehow I didn't connect the dots
for my own problem. Anyway, I'm happy to see that when I do it correctly
and the print() command's output is appended to the buffer, the output
is separated into many cells rather than as one big string.
I just whipped up a quick function to combine cells into lines:
def buf2linebuf(buf):
line = ""
line_buf = []
for cell in buf:
if cell[0] == '\n':
line_buf.append(line)
line = cell.lstrip()
else:
line = line + cell
return line_buf
When I then appended each of these lines one at a time to my TextBuffer,
the output was no longer truncated.
Thanks for your quick help. Once I get a bit more comfortable in Python,
I'll be happy to help out with RPy since my project will be relying so
heavily on it.
cheers,
brandon
--
Apps built with the Adobe(R) Flex(R) framework and Flex Builder(TM) are
powering Web 2.0 with engaging, cross-platform capabilities. Quickly and
easily build your RIAs with Flex Builder, the Eclipse(TM)based development
software that enables intelligent coding and step-through debugging.
Download the free 60 day trial. http://p.sf.net/sfu/www-adobe-com
___
rpy-list mailing list
rpy-list@lists.sourceforge.net
https://lists.sourceforge.net/lists/listinfo/rpy-list
Re: [Rpy] Pickling a Rpy2 object
I'm trying to do something similar: I'd like to create a save file that
includes the current state of the R environment. I don't want to create
a separate file for the R environment via one of R's built-in commands,
so I thought I'd pickle globalEnv alongside some other objects I'm
pickling. When I try to pickle the globalEnv, I get this error:
cPickle.PicklingError: Can't pickle :
import of module rinterface failed
How have other people gone about storing their R objects?
Thanks!
brandon invergo
On Mon, 2009-04-06 at 11:29 -0400, Réjean Ducharme wrote:
> Hi all,
I'm using Rpy2 to train a logistic regressor (using the "glm" function
in R), and I need to save the model for futher use.
I naively try to use pickle, but I have an error message:
import rpy2.robjects as robjects
fit = robjects.r.glm(formula=f, family=robjects.r.binomial(), data=data)
output = open('glm.pickle', 'w')
pickle.dump(fit, output)
>>> pickle.PicklingError: Can't pickle :
it's not found as rinterface.SexpVector
Is there another way to save a Rpy2 object???
Thanks,
Rejean Ducharme
--
Register Now & Save for Velocity, the Web Performance & Operations
Conference from O'Reilly Media. Velocity features a full day of
expert-led, hands-on workshops and two days of sessions from industry
leaders in dedicated Performance & Operations tracks. Use code vel09scf
and Save an extra 15% before 5/3. http://p.sf.net/sfu/velocityconf
___
rpy-list mailing list
rpy-list@lists.sourceforge.net
https://lists.sourceforge.net/lists/listinfo/rpy-list
Re: [Rpy] Pickling a Rpy2 object
FWIW, the solution I have for now is to use dump(ls(), file="") to get a
deparsed string representation of all the objects in memory, which can
then be pickled. I'm not having problems with it right now in any simple
cases, but I can imagine there are situations where deparsing gets
hairy, so being able to pickle the R globalEnv or somehow being able to
route the file output of R's save.image command to a python object that
can then be pickled would be the best....
Cheers,
-brandon invergo
On Thu, 2009-04-30 at 01:01 +0200, Brandon Invergo wrote:
> I'm trying to do something similar: I'd like to create a save file that
> includes the current state of the R environment. I don't want to create
> a separate file for the R environment via one of R's built-in commands,
> so I thought I'd pickle globalEnv alongside some other objects I'm
> pickling. When I try to pickle the globalEnv, I get this error:
> cPickle.PicklingError: Can't pickle :
> import of module rinterface failed
>
>
> How have other people gone about storing their R objects?
>
> Thanks!
> brandon invergo
>
>
>
> On Mon, 2009-04-06 at 11:29 -0400, Réjean Ducharme wrote:
> > Hi all,
>
> I'm using Rpy2 to train a logistic regressor (using the "glm" function
> in R), and I need to save the model for futher use.
> I naively try to use pickle, but I have an error message:
>
> import rpy2.robjects as robjects
> fit = robjects.r.glm(formula=f, family=robjects.r.binomial(), data=data)
> output = open('glm.pickle', 'w')
> pickle.dump(fit, output)
>
> >>> pickle.PicklingError: Can't pickle 'rinterface.SexpVector'>:
> it's not found as rinterface.SexpVector
>
>
> Is there another way to save a Rpy2 object???
>
> Thanks,
> Rejean Ducharme
>
>
> --
> Register Now & Save for Velocity, the Web Performance & Operations
> Conference from O'Reilly Media. Velocity features a full day of
> expert-led, hands-on workshops and two days of sessions from industry
> leaders in dedicated Performance & Operations tracks. Use code vel09scf
> and Save an extra 15% before 5/3. http://p.sf.net/sfu/velocityconf
> ___
> rpy-list mailing list
> rpy-list@lists.sourceforge.net
> https://lists.sourceforge.net/lists/listinfo/rpy-list
--
Register Now & Save for Velocity, the Web Performance & Operations
Conference from O'Reilly Media. Velocity features a full day of
expert-led, hands-on workshops and two days of sessions from industry
leaders in dedicated Performance & Operations tracks. Use code vel09scf
and Save an extra 15% before 5/3. http://p.sf.net/sfu/velocityconf
___
rpy-list mailing list
rpy-list@lists.sourceforge.net
https://lists.sourceforge.net/lists/listinfo/rpy-list
Re: [Rpy] py2 and threading
For the first problem, I would route all of the graphical output to files to
look at after running the program. For the second one, I would add the results
to the R environment after the subprocess completes. Would something like this
work for you?
import sys, os
from multiprocessing import Process, Queue
import rpy2.robjects as robs
from rpy2.robjects.packages import importr
grdevices = importr("grDevices")
def r_command(q, obj, cmd):
grdevices.pdf(file="{0}.pdf".format(obj))
q.put((obj,robs.r(str(cmd
print "Process %s output:" % os.getpid()
print robs.r.ls()
grdevices.dev_off()
if __name__ == "__main__":
q = Queue()
cmd = "c(1,2,3,4,5)"
obj = "test"
p = Process(target=r_command, args=(q, obj, cmd,))
p.start()
p.join()
res = q.get()
robs.globalenv[res[0]] = res[1]
print "Process %s output:" % os.getpid()
print robs.r[res[0]]
print robs.r.ls()
cmd = "plot(c(1,2,3,4,5))"
obj = "test2"
p = Process(target=r_command, args=(q, obj, cmd,))
p.start()
p.join()
res = q.get()
robs.globalenv[res[0]] = res[1]
print "Process %s output:" % os.getpid()
print robs.r[res[0]]
print robs.r.ls()
cmd = "plot(c(3,4,5,6,7))"
obj = "test3"
p = Process(target=r_command, args=(q, obj, cmd,))
p.start()
p.join()
res = q.get()
robs.globalenv[res[0]] = res[1]
print "Process %s output:" % os.getpid()
print robs.r[res[0]]
print robs.r.ls()
Cheers,
Brandon Invergo
- Original Message
From: Carson Farmer
To: Laurent Gautier
Cc: "RPy help, support and design discussion list"
Sent: Wed, March 31, 2010 2:02:13 AM
Subject: Re: [Rpy] py2 and threading
Thanks Laurent,
> Multiprocessing is way to go (pickling was implemented to make it easier,
> and there a google SoC project out bring storage-based arrays in).
I think you're likely right...
> R GUI builders could clearly and immediately benefit from having the GIL
> released, but in the meanwhile adopting a client-server strategy is the
> recommended way to go. That has limitations but also advantages; building a
> minimal server can be done in less than 20 lines of code (less than 10 lines
> if no comments).
> http://rpy.sourceforge.net/rpy2/doc-2.1/html/server.html#simple-socket-based-server-and-client
>
> Hoping this helps,
Indeed it has: I have experimented with both Multiprocessing and the
client-server implementation.
The multiprocessing route seems to work quite well, however, I'm
running into two problems, and I'm wondering if anyone else has
managed to get this working:
1) As expected, if a plot is generated in a separate process, it is
immediately destroyed once the process completes. I have tested this,
and basically I get a plot for about half a second, then it
disappears. Does anyone know of a way to get around this?
2) Assignments do not seem to span process' (again this is probably
expected, and I'm just simply doing something wrong). If I do
something like 'robjects.r("test <- c(1,2,3,4,5)")' in a subprocess,
then 'test' is *not* available in the parent process (whereas it *is*
obviously available in the subprocess). Does anyone know what I need
to do to make this work properly?
Here is a minimal example to illustrate issue 2 above (issue 1 can
similarly be tested by changing 'cmd' to "plot(c(1,2,3,4,5))") :
# -*- coding: utf-8 -*-
import sys, os
from multiprocessing import Process, Queue
import rpy2.robjects as robs
def r_command(q, cmd):
q.put(robs.r(str(cmd)))
print "Process %s output:" % os.getpid()
print robs.r.ls()
if __name__ == "__main__":
q = Queue()
cmd = "test <- c(1,2,3,4,5)"
p = Process(target=r_command, args=(q,cmd,))
p.start()
p.join()
print "Process %s output:" % os.getpid()
robs.r['print'](q.get())
print robs.r.ls()
--
Carson J. Q. Farmer
ISSP Doctoral Fellow
National Centre for Geocomputation
National University of Ireland, Maynooth,
http://www.carsonfarmer.com/
--
Download Intel® Parallel Studio Eval
Try the new software tools for yourself. Speed compiling, find bugs
proactively, and fine-tune applications for parallel performance.
See why Intel Parallel Studio got high marks during beta.
http://p.sf.net/sfu/intel-sw-dev
___
rpy-list mailing list
rpy-list@lists.sourceforge.net
https://lists.sourceforge.net/lists/listinfo/rpy-list
--
Download Intel® Parallel Studio Eval
Try the new software tools for yourself. Speed compiling, find bugs
proac
Re: [Rpy] export a package of R in python program
There's no way of specifying versions in R's library() function, so I'm
assuming that if you successfully have both versions of the package installed
in R, they must have different names (I'm pretty sure installing a new version
of a package overwrites the previous version). If that's the case, using the
latest version of rpy2 (2.1.0-rc):
from rpy2.robjects.packages import importr
new_package = importr("new.package.name")
old_package = importr("old.package.name")
And then just access the functions using standard python library syntax:
old_package.old_function() or new_package.some_other_function()
If you've somehow managed to have two different versions of the same package
installed in R with the same name, you'll have to find a more clever solution.
Cheers,
Brandon
From: zahra sheikhbahaee
To: "RPy help, support and design discussion list"
Sent: Wed, April 7, 2010 6:26:46 PM
Subject: [Rpy] export a package of R in python program
Hi,
I am using a package of R in my python code. I have to import two versions of
package in my program, because there is a function in one of them which does
not exist in the new version. The problem is that, afterward I want to use one
of the functions of new version but it looks for the function in old version
and ask the parameters of the other version which does not have the some
abilities of the new funcion. I wonder how I could export from the first
package and import in the new one in one program.
Zahra.
--
Download Intel® Parallel Studio Eval
Try the new software tools for yourself. Speed compiling, find bugs
proactively, and fine-tune applications for parallel performance.
See why Intel Parallel Studio got high marks during beta.
http://p.sf.net/sfu/intel-sw-dev___
rpy-list mailing list
rpy-list@lists.sourceforge.net
https://lists.sourceforge.net/lists/listinfo/rpy-list
Re: [Rpy] export a package of R in python program
Could you post the code that you've tried so we can see if this is reconcilable
within python/rpy2 or if it is, indeed, an R problem?
cheers,
brandon
From: zahra sheikhbahaee
To: "RPy help, support and design discussion list"
Sent: Wed, April 7, 2010 9:50:25 PM
Subject: Re: [Rpy] export a package of R in python program
Hi
I have already done this because each package has different suffix which shows
the version of the package. I imported in the program both of them but since
the name of the function is the same in both and it needs different parameters
in each one, it makes a big mistake. I figured out that even I call the
function with the prefix of the package it only realized the last one.
Zahra.
On Wed, Apr 7, 2010 at 6:56 PM, Brandon Invergo
wrote:
There's no way of specifying versions in R's library() function, so I'm
assuming that if you successfully have both versions of the package installed
in R, they must have different names (I'm pretty sure installing a new version
of a package overwrites the previous version). If that's the case, using the
latest version of rpy2 (2.1.0-rc):
>
>from rpy2.robjects.packages import importr
>new_package = importr("new.package.name")
>old_package = importr("old.package.name")
>
>And then just access the functions using standard python library syntax:
>old_package.old_function() or new_package.some_other_function()
>
>If you've somehow managed to have two different versions of the same package
>installed in R with the same name, you'll have to find a more clever
> solution.
>
>Cheers,
>Brandon
>
>
>
>
From: zahra sheikhbahaee
>To: "RPy help, support and design discussion list"
>
>Sent: Wed, April 7, 2010 6:26:46 PM
>Subject: [Rpy] export a package of R in python program
>
>
>>Hi,
>
>I am using a package of R in my python code. I have to import two versions of
>package in my program, because there is a function in one of them which does
>not exist in the new version. The problem is that, afterward I want to use one
>of the functions of new version but it looks for the function in old version
>and ask the parameters of the other version which does not have the some
>abilities of the new funcion. I wonder how I could export from the first
>package and import in the new one in one program.
>
>Zahra.
>
>--
>>Download Intel® Parallel Studio Eval
>>Try the new software tools for yourself. Speed compiling, find bugs
>>proactively, and fine-tune applications for parallel performance.
>>See why Intel Parallel Studio got high marks during beta.
>http://p.sf.net/sfu/intel-sw-dev
>___
>>rpy-list mailing list
>rpy-list@lists.sourceforge.net
>https://lists.sourceforge.net/lists/listinfo/rpy-list
>
>
--
Download Intel® Parallel Studio Eval
Try the new software tools for yourself. Speed compiling, find bugs
proactively, and fine-tune applications for parallel performance.
See why Intel Parallel Studio got high marks during beta.
http://p.sf.net/sfu/intel-sw-dev___
rpy-list mailing list
rpy-list@lists.sourceforge.net
https://lists.sourceforge.net/lists/listinfo/rpy-list
