On 10/16/2008 10:50 AM, culpritNr1 wrote:
Hello Gabor,
First of all, thanks for your reply.
Indeed, your strategy solves the problem of speed=). My question was,
however, more of an R design question.
Your suggestion of pre-allocating a big and continuous portion of memory is
probably the first manual work around to try. And it will work.
Now, modern high level languages like the continually improving R, Python
and also Matlab and Perl come with built-in goodies that run at C/C++ speed.
Those built-in functions are high level versions of functionalities that we
would code "manually" in low level languages.
Being relatively new to R, my expectations are that in a high level language
like this, I do not need to manually program a simple operation such as
in-place appending. Manipulation of rectangular data should be extremely
efficient. (I use it to make computation of DNA properties in organisms with
huge genomes like human and mouse.)
I think it's unlikely that this will soon be very efficient in R,
because the only way to make it so is to pre-allocate a lot of extra
space (so the appends don't grow the memory footprint). Generally
people run into space limitations that are harder to deal with than the
speed limitations, so inefficient allocations for the sake of speed
aren't going to happen unless the user asks for them (and Gabor told you
how to do that).
Duncan Murdoch
So, do you know whether or not such functions exists or is planned to be
incorporated in the near future? An optional module perhaps?
Again, thank you,
culpritNr1
Gabor Grothendieck wrote:
Create an empty matrix first and then fill it in. That
will avoid the overhead in repeatedly expanding it. If
you don't know how many rows then make it 1000 rows
and remove the unused ones once finished.
On Wed, Oct 15, 2008 at 4:05 PM, culpritNr1 <[EMAIL PROTECTED]>
wrote:
Hello fellow R sufferers,
Is there a way to perform an appending operation in place?
Currently, the way my pseudo-code goes is like this
for (i in 1:1000) {
if (some condition) {
newRow <- myFunction(myArguments)
X <- rbind(X, newRow) # <- this is the bottleneck!!
}
}
As you can see, it works but as the matrix X gets the size of a few
million
rows, the code runs very slow.
I am looking for something like the natively "in place" appending python
function called "append()" or the perl function "push". "In-place"
operations would allow me to do (in pseudocode)
for (i in 1:1000) {
if (some condition) {
newRow <- myFunction(myArguments)
append(X, newRow)
}
}
You see? I do not have to call and re-assign the giant X matrix every
loop
cycle.
Any help?
Thank you,
Your culpritNr1
--
View this message in context:
http://www.nabble.com/R%3A-%22in-place%22-appending-to-a-matrix.-tp20001258p20001258.html
Sent from the R help mailing list archive at Nabble.com.
______________________________________________
R-help@r-project.org mailing list
https://stat.ethz.ch/mailman/listinfo/r-help
PLEASE do read the posting guide
http://www.R-project.org/posting-guide.html
and provide commented, minimal, self-contained, reproducible code.
______________________________________________
R-help@r-project.org mailing list
https://stat.ethz.ch/mailman/listinfo/r-help
PLEASE do read the posting guide
http://www.R-project.org/posting-guide.html
and provide commented, minimal, self-contained, reproducible code.
______________________________________________
R-help@r-project.org mailing list
https://stat.ethz.ch/mailman/listinfo/r-help
PLEASE do read the posting guide http://www.R-project.org/posting-guide.html
and provide commented, minimal, self-contained, reproducible code.
