On 12 Oct 2012, at 09:46, Purna chander wrote:
> 4) scenario4:
>> x<-read.table("query.vec")
>> v<-read.table("query.vec2")
>> v<-as.matrix(v)
>> d<-dist(rbind(v,x),method="manhattan")
>> m<-as.matrix(d)
>> m2<-m[1:nrow(v),(nrow(v)+1):nrow(x)]
>> print(m2[1,1:10])
>
> time taken for running the code:
> real 0m0.445s
> user 0m0.401s
> sys 0m0.041s
> 1) Though scenario 4 is optimum, this scenario failed when matrix 'v'
> having more no. of rows. An error occurred while converting distance
> object 'd' to a matrix 'm'.
> For E.g: > m<-as.matrix(d)
> the above command resulted in error: "Error: cannot allocate
> vector of size 922.7 MB".
That's because you're calculating a full distance matrix with (10000+100) *
(10000+100) points and then extract the much smaller number of distance values
(10000 * 100) that you actually need.
I have a use case with similar requirements, so ...
> 3) Any other ideas to optimize the problem i'm facing with.
... my experimental "wordspace" package includes a function dist.matrix() for
calculating such cross-distance matrices. The function is written in C code
and doesn't handle NA's and NaN's properly, but it's considerably faster than
the current implementation of dist().
I haven't uploaded the package to CRAN yet, but you should be able to install
with
install.packages("wordspace", repos="http://R-Forge.R-project.org";)
Best,
Stefan
PS: Glad to see that daily builds on R-Forge work again -- that's an extremely
useful feature to get beta testers for experimental package versions. :-)
______________________________________________
R-help@r-project.org mailing list
https://stat.ethz.ch/mailman/listinfo/r-help
PLEASE do read the posting guide http://www.R-project.org/posting-guide.html
and provide commented, minimal, self-contained, reproducible code.
