Dear all, I am working with huge matrices (65600 x 65600) that represent the euclidean distances between sites. So, mat[i,j] = mat[j,i].
I would like to use a multinomial distribution to define to which site an element from site i can move: the closer is the target site j, the higher is the probability to move from i to j. For smaller matrices, I could do a cumsum like: cummat = cumsum(mat,2); and generate random numbers between 0 and maximum(cummat[i,:]). But with such huge matrices it is impossible to do this with the machines I have access to. I can load the matrices in a sparse-matrix format, that has 15,682,058 of non-zero elements (0.3% of the total elements of the matrix). I tried to run the cumsum in the sparse matrices, but it is still too slow. I can not figure out how to do a Multinomial Distribution with these huge matrices (sparse or not). I saw this https://groups.google.com/forum/#!topic/julia-dev/TLyWeDHGU6M and this http://dmbates.blogspot.ch/2012/03/julia-version-of-multinomial-sampler_12.html But it seems to work only when the matrix is already a probability matrix, with each of the 65600 rows summing to 1. But to do this I would need to do a cumsum first. Any idea about how to deal with such huge matrices? Thank you for your attention! Best, Charles -- Um axé! :) -- Charles Novaes de Santana, PhD http://www.imedea.uib-csic.es/~charles
