2008/7/15 laurent oget <[EMAIL PROTECTED]>:
> can i get a quick hint on how i would go about calling --verbose through RPY
> ?
I suspect that the only way is to hack
line 93 of rpymodule.c
char *defaultargv[] = {"rpy", "-q", "--vanilla"};
and recompile/install rpy.
> I am pretty clueless about the way R does the garbage collection. One thing
> I know is there are columns that are shared between different linear
> regressions, so it might be that the garbage collector cleans up a column
> from iteration n-1 and breaks column n in the process.
> Is there a way to
> call the garbage collection explicitely before I create a new dataframe to
> test this hypothesis?
gc()
#possibly gc(verbose = TRUE)
I have made a toy example to see if I could reproduce it here (with rpy2 -
but there are similarities in the way R objects pointed at from Python objects
are protected from R's garbage collection)... and it seem that there
is something going on with garbage collection (depending on how the
python implementation, it either crashes after 10-50 iterations... or can
go through 1000 iterations.
> Thanks,
>
> Laurent
>
> 2008/7/14 Laurent Gautier <[EMAIL PROTECTED]>:
>>
>> Someone else reported on this list a similar sounding problem not so long
>> ago.
>>
>>
>> The problem might be caused by manipulating a stale pointer to an R object
>> (that is an object that was discarded during R's garbage collection),
>> and troubleshooting this will likely mean running things through a C
>> debugger.
>> You could try starting up you embedded R process with '--verbose' and see
>> if the problem happens right after garbage collection.
>> Without having further details on the exact code ran, it is difficult
>> to say more.
>>
>>
>>
>>
>>
>> 2008/7/14 laurent oget <[EMAIL PROTECTED]>:
>> > I am using rpy/R to perform linear regressions on a large number of
>> > datasets, in one python run, and am encountering segmentation faults
>> > after a
>> > large number of iteration, while handling cases which, taken on their
>> > own
>> > run without a problem. My intuition is that the previous iterations
>> > somehow
>> > corrupted the heap (or the stack). The code breaks in different places.
>> > One
>> > sample stack trace:
>> >
>> >
>> > *** caught segfault ***
>> > address 0x8, cause 'memory not mapped'
>> >
>> > Traceback:
>> > 1: unique.default(unlist(lapply(answer, length)))
>> > 2: unique(unlist(lapply(answer, length)))
>> > 3: sapply(xlev, is.null)
>> > 4: .getXlevels(mt, mf)
>> > 5: function (formula, data, subset, weights, na.action, method = "qr",
>> > model = TRUE, x = FALSE, y = FALSE, qr = TRUE, singular.ok = TRUE,
>> > contrasts = NULL, offset, ...) { ret.x <- x ret.y <- y cl <-
>> > match.call() mf <- match.call(expand.dots = FALSE) m <-
>> > match(c("formula", "data", "subset", "weights", "na.action",
>> > "offset"), names(mf), 0L) mf <- mf[c(1L, m)] mf$drop.unused.levels
>> > <-
>> > TRUE mf[[1L]] <- as.name("model.frame") mf <- eval(mf,
>> > parent.frame()) if (method == "model.frame") return(mf)
>> > else
>> > if (method != "qr") warning(gettextf("method = '%s' is not
>> > supported. Using 'qr'", method), domain = NA) mt <-
>> > attr(mf,
>> > "terms") y <- model.response(mf, "numeric") w <-
>> > as.vector(model.weights(mf)) if (!is.null(w) && !is.numeric(w))
>> > stop("'weights' must be a numeric vector") offset <-
>> > as.vector(model.offset(mf)) if (!is.null(offset)) { if
>> > (length(offset) == 1) offset <- rep(offset, NROW(y))
>> > else
>> > if (length(offset) != NROW(y)) stop(gettextf("number of
>> > offsets
>> > is %d, should equal %d (number of observations)",
>> > length(offset), NROW(y)), domain = NA) } if (is.empty.model(mt))
>> > { x <- NULL z <- list(coefficients = if (is.matrix(y))
>> > matrix(, 0, 3) else numeric(0), residuals = y, fitted.values
>> > = 0
>> > * y, weights = w, rank = 0L, df.residual = if (is.matrix(y))
>> > nrow(y) else length(y)) if (!is.null(offset)) {
>> > z$fitted.values <- offset z$residuals <- y - offset }
>> > } else { x <- model.matrix(mt, mf, contrasts) z <- if
>> > (is.null(w)) lm.fit(x, y, offset = offset, singular.ok =
>> > singular.ok, ...) else lm.wfit(x, y, w, offset =
>> > offset, singular.ok = singular.ok, ...) } class(z) <-
>> > c(if
>> > (is.matrix(y)) "mlm", "lm") z$na.action <- attr(mf, "na.action")
>> > z$offset <- offset z$contrasts <- attr(x, "contrasts") z$xlevels
>> > <-
>> > .getXlevels(mt, mf) z$call <- cl z$terms <- mt if (model)
>> > z$model <- mf if (ret.x) z$x <- x if (ret.y) z$y
>> > <-
>> > y if (!qr) z$qr <- NULL
>> > z}("sls_1133745~dp_1133745+dp_3124901", list(dp_3124901 = c(0, 0, 0, 0,
>> > 0,
>> > -0.328504066972036, 0, 0, 0, 0, 0, 0, 0, -0.328504066972036, 0, 0, 0, 0,
>> > 0,
>> > -0.328504066972036, -0.328504066972036, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0,
>> > -0.693147180559945, 0, -0.693147180559945, 0, 0, 0, -0.328504066972036,
>> > 0,
>> > 0, 0, 0, 0, 0, 0, 0, 0, -0.328504066972036, -0.328504066972036, 0, 0, 0,
>> > 0,
>> > 0, -0.693147180559945, 0, 0, 0, 0, -0.693147180559945, 0,
>> > -0.693147180559945, -0.693147180559945, 0, 0, 0, 0, 0, 0, 0,
>> > -0.248461359298500, 0, -0.248461359298500, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0,
>> > 0,
>> > -0.693147180559945, 0, 0, 0, -0.248461359298500, 0, 0,
>> > -0.248461359298500,
>> > 0, 0, -0.693147180559945, 0, 0, 0, 0, 0, 0), sls_1133745 =
>> > c(7.3487612610094, 7.30613534736707, 8.00046685932928, 7.39499697533915,
>> > 7.34321275228777, 7.88963522981753, 7.22973732372956, 8.18020882367827,
>> > 7.91558910409137, 7.26812538738378, 7.25818677308703, 7.3498801314529,
>> > 7.31767047076391, 7.65018297965786, 7.90898624450853, 8.19434808229588,
>> > 8.21457885218454, 7.3641669547084, 7.35953128995355, 7.7040815977251,
>> > 7.2031227430099, 7.31216649861154, 7.19473710631952, 7.29547045158008,
>> > 7.35154326140946, 7.17917838793797, 8.36984796851258, 7.1754285028756,
>> > 8.12093952739025, 7.687240693355, 7.2225733180656, 7.36405924490386,
>> > 7.3452615426351, 7.25804588634565, 7.23995411189234, 7.21269261931628,
>> > 7.3210695020337, 7.8386981421886, 7.373405696737, 7.17968137538913,
>> > 7.82070844738252, 7.43464714668485, 7.25244367037469, 7.23292820385588,
>> > 7.33945977029367, 7.2810277192681, 7.34753163260778, 7.94492823905035,
>> > 7.87310661464411, 7.14571650431343, 7.1568316491227, 8.04067559226017,
>> > 7.16502195742237, 7.27351582356697, 7.28802564845575, 8.77370188389706,
>> > 7.3224047480958, 6.4802899305943, 7.2810277192681, 7.26784638120198,
>> > 7.35638204246829, 7.20306320059336, 7.15358357480953, 8.01940553608164,
>> > 8.78195611526034, 7.24723014605046, 7.319056656224, 7.24884540813453,
>> > 7.25487774291456, 7.211054762329, 8.21131456124779, 7.19920773459773,
>> > 7.9786879985435, 7.28814186671376, 7.27840117311202, 7.2972806890663,
>> > 8.14356390282749, 7.34424090367824, 7.32067255061534, 7.34136070803461,
>> > 7.76719625525884, 7.20539015753155, 7.19294925869672, 8.12521670148863,
>> > 7.3678032648606, 7.32193563210652, 7.16022459567817, 7.34345210203388,
>> > 8.11895788519276, 7.81886461055585, 7.76421532379024, 7.93634917049156,
>> > 7.20526391225165, 7.2842720651344, 7.28607526956675, 7.29555188093676,
>> > 7.19784694602772, 7.18371895847972, 7.21964936074816, 7.87895077602558,
>> > 7.26955407354936), dp_1133745 = c(-0.223143551314210, 0,
>> > -0.287682072451781,
>> > 0, 0, -0.116533816255952, 0, -0.223143551314210, -0.116533816255952, 0,
>> > 0,
>> > 0, 0, -0.116533816255952, -0.116533816255952, -0.223143551314210,
>> > -0.287682072451781, 0, 0, -0.116533816255952, 0, 0, 0, 0, 0, 0,
>> > -0.287682072451781, 0, -0.287682072451781, -0.116533816255952, 0, 0, 0,
>> > 0,
>> > 0, 0, 0, -0.116533816255952, 0, 0, -0.248461359298500, 0, 0, 0, 0, 0, 0,
>> > -0.116533816255952, -0.116533816255952, 0, 0, -0.287682072451781, 0, 0,
>> > 0,
>> > -0.400477566597125, 0, 0, 0, 0, 0, 0, 0, -0.248461359298500,
>> > -0.400477566597125, 0, 0, 0, 0, 0, -0.287682072451781, 0,
>> > -0.287682072451781, 0, 0, 0, -0.223143551314210, 0, 0, 0,
>> > -0.116533816255952, 0, 0, -0.223143551314210, 0, 0, 0, 0,
>> > -0.287682072451781, -0.248461359298500, -0.116533816255952,
>> > -0.287682072451781, 0, 0, 0, 0, 0, 0, 0, -0.116533816255952, 0)))
>> >
>> >
>> > would anybody have a hint on how to troubleshoot this?
>> >
>> > Thanks,
>> >
>> > Laurent
>> >
>> >
>> >
>> > -------------------------------------------------------------------------
>> > Sponsored by: SourceForge.net Community Choice Awards: VOTE NOW!
>> > Studies have shown that voting for your favorite open source project,
>> > along with a healthy diet, reduces your potential for chronic lameness
>> > and boredom. Vote Now at http://www.sourceforge.net/community/cca08
>> > _______________________________________________
>> > rpy-list mailing list
>> > rpy-list@lists.sourceforge.net
>> > https://lists.sourceforge.net/lists/listinfo/rpy-list
>> >
>> >
>
>
-------------------------------------------------------------------------
Sponsored by: SourceForge.net Community Choice Awards: VOTE NOW!
Studies have shown that voting for your favorite open source project,
along with a healthy diet, reduces your potential for chronic lameness
and boredom. Vote Now at http://www.sourceforge.net/community/cca08
_______________________________________________
rpy-list mailing list
rpy-list@lists.sourceforge.net
https://lists.sourceforge.net/lists/listinfo/rpy-list
