Jason Roberts wrote: > Laurent, > > Thank you very much for the reply. > >> I am not certain of which way the risk probability stand (compile each >> time, or compile once and hope for the best). Time will tell. > > So rpy2 does not require recompilation every time R is released? How is it > binding to R then? (I have not looked at the C code yet. If you can just > point me in the right direction I can figure it out myself.)
The libR is used as a shared library. Under win32, and AFAIUI, that should translate as using an unbound DLL (otherwise the same version of libR will be required) and hold as long as the names used from the symbol table presented by libR.so do not change. In the case it does, then a new(er) version of rpy2 should be available. Admittedly not an absolute perfect options, but I wanted to avoid version-specific conditional definitions in the code; rpy had it, but I had to start from a simple base. This does not mean this aspect of rpy will not be added in the future, but I'd like to explore options first. On a related note, I'd like to offer the option to have R really embedded in rpy2 (with an R install inside the rpy2 installed module)... so if someone has the time... >> You could try with a dummy minimal extension to ArcGIS and tell us. > > I tried this out using ArcGIS 9.3 SP1, Python 2.5.1 (comes with ArcGIS 9.3), > and rpy2-2.0.3.win32-py2.5.exe. I created a Python-based geoprocessing tool > with the following code to exercise rpy2 in a minimal way: > > # Initialize the ArcGIS geoprocessor object, so we can communicate > # with ArcGIS. > > import arcgisscripting > gp = arcgisscripting.create() > > # Using rpy2, calculate the square root of the input parameter. If we > # catch an exception, report a traceback to ArcGIS. > > import os, traceback > try: > x = gp.GetParameter(0) > from rpy2 import robjects > sqrt_x = robjects.r.sqrt(x)[0] > except: > gp.AddError(traceback.format_exc()) > raise > > It worked (!!!) and the performance appeared to be quite good. I am running > it in a loop now to check for leaks. I'll send a followup on that later. If you are having an issue, check the following: http://www.mail-archive.com/rpy-list@lists.sourceforge.net/msg01696.html > There was one problem that I noticed immediately. Currently, line 37 of > rinterface/__init.py__ blindly adds R directories to the PATH: > > # Win32-specific code copied from RPy-1.x > if sys.platform == 'win32': > import win32api > os.environ['PATH'] += ';' + os.path.join(R_HOME, 'bin') > os.environ['PATH'] += ';' + os.path.join(R_HOME, 'modules') > os.environ['PATH'] += ';' + os.path.join(R_HOME, 'lib') I see. > The new PATH is persisted in the environment of the calling ArcGIS process. > When that process initializes the Python interpreter a second time, this > code is called again, adding duplicate entries to PATH. This can go on until > the PATH reaches 32767 characters, and then putenv will raise an OSError. In > my case, my tool ran 335 times before this occurred. I observed the problem > happen by adding additional logging statements to my minimal example above, > and watched the len(os.environ['PATH']) grow close to 32767 before putenv > failed. > > To fix, something like this is appropriate: > > # Win32-specific code copied from RPy-1.x > if sys.platform == 'win32': > import win32api > if os.path.join(R_HOME, 'bin') not in os.environ['PATH'].split(';'): > os.environ['PATH'] += ';' + os.path.join(R_HOME, 'bin') > if os.path.join(R_HOME, 'modules') not in os.environ['PATH'].split(';'): > os.environ['PATH'] += ';' + os.path.join(R_HOME, 'modules') > if os.path.join(R_HOME, 'lib') not in os.environ['PATH'].split(';'): > os.environ['PATH'] += ';' + os.path.join(R_HOME, 'lib') Thanks. An ultimate patch would likely be a little more complex (by checking that os.path.join(R_HOME, 'bin') is not the _first_ R found in the PATH... but I am sure of what PATH is needed for here - can someone with win32 try when just removing the PATH creation ?) > I'm currently running it 100000 times, monitoring memory and handles. I'll > let you know how it turns out. > > I'm pretty hopeful this will work out well. There could be problems with R > packages that do fancy things (like link to other C libraries) but even if > that's a problem, just having the ability to do basic R from ArcGIS 9.3 in a > performant manner will be very, very nice for us and our users. > > Jason > > > -----Original Message----- > From: Laurent Gautier [mailto:lgaut...@gmail.com] > Sent: Thursday, March 19, 2009 1:57 AM > To: RPy help, support and design discussion list > Cc: Jason Roberts > Subject: Re: FW: rpy2 in ArcGIS 9.3 > > Jason Roberts wrote: >> Greetings rpy2 developers, >> >> >> >> I am the primary developer of an open source Python package called >> Marine Geospatial Ecology Tools >> (http://code.env.duke.edu/projects/mget). These tools perform various >> jobs that are useful to marine ecologists. Many of the tools are >> designed to be invoked from ArcGIS, a desktop GIS application that runs >> on Windows. >> > > rpy2 works best on UNIX-alikes at the moment. > (features are not working on win32). > >> To date, we have had good success accessing R using rpy. Thank you very >> much for making this package freely available. > > I can't take those credits: > rpy is Walter and Greg's work, with the help of contributors. > >> But we noted last year >> that rpy is no longer being maintained, and rpy2 is the new replacement. > > Kind of. I started with rpy2 about a year ago, as what I was trying to > do did not appear possible with rpy. Rpy is still available, although > its development on the slow lane at the moment, I think. > >> It will be a big job for us to switch to rpy2, so we have been delaying >> the switch. In the interim, we've been compiling rpy every time a new R >> release has come out. This is probably increasingly risky, so we're >> becoming more motivated to make the switch. > > I am not certain of which way the risk probability stand (compile each > time, or compile once and hope for the best). Time will tell. > >> In addition, there is an >> ArcGIS 9.3 / rpy compatibility problem that is pretty inconvenient. >> Basically we are wondering if this problem exists with rpy2. >> >> >> >> The problem was discussed last year; see >> > http://sourceforge.net/tracker/?func=detail&atid=453021&aid=2062627&group_id > =48422 > <http://sourceforge.net/tracker/?func=detail&atid=453021&aid=2062627&group_i > d=48422>. >> In brief: Every time ArcGIS 9.3 runs a Python-based tool, it initializes >> a new instance of the Python interpreter in the ArcGIS process >> (typically ArcCatalog.exe or ArcMap.exe). The interpreter instance >> eventually loads the rpy extension module (e.g. _rpy2070.dll). The >> interpreter exits when the tool completes. But this does not cause the >> rpy extension module to be unloaded from the process, and when ArcGIS >> runs the tool a second time, creating a new Python interpreter, rpy >> fails to initialize. >> >> >> >> In last year's bug report, lgautier mentioned that "the problem was >> fixed a few weeks ago" (i.e. last summer). Is it correct then that this >> procedure of initializing the interpreter, using rpy2, shutting down the >> interpreter, and so on, can be done indefinitely from a single process >> without any ill effects? >> > > May be, may be not. > I have not looked at whether the C-level part of rpy2 does what it > should regarding the creating and destruction of Python interpreters. > > You could try with a dummy minimal extension to ArcGIS and tell us. > > > > Hoping this helps, > > > > L. > >> Thanks for your help! And thanks again to you guys for developing this >> great reusable software. >> >> >> >> Jason >> >> >> >> >> >> / / >> > > ------------------------------------------------------------------------------ Apps built with the Adobe(R) Flex(R) framework and Flex Builder(TM) are powering Web 2.0 with engaging, cross-platform capabilities. Quickly and easily build your RIAs with Flex Builder, the Eclipse(TM)based development software that enables intelligent coding and step-through debugging. Download the free 60 day trial. http://p.sf.net/sfu/www-adobe-com _______________________________________________ rpy-list mailing list rpy-list@lists.sourceforge.net https://lists.sourceforge.net/lists/listinfo/rpy-list
