Helo TIm, yes beacause at that time l haven't discovered yet julia google group
On Mon, Jul 4, 2016 at 1:00 AM, Tim Holy <[email protected]> wrote: > Duplicated in > http://stackoverflow.com/questions/38147946/julia-how-to-store-tuples-in-different-files > > --Tim > > On Friday, July 1, 2016 7:48:34 AM CDT Ahmed Mazari wrote: > > Hello, > > > > l want to store each tuple X[p] in a different file > > # X[1] in mini_batch1.jld X[2] in mini_batch2..... > > # but my code below stores (duplicate) all the tuple of X[p] in the > files > > created. > > > > > > > > let's see an example : > > > > m= 100 k= 3 # number of tuples or partition > > y=rand_gen(m,k) > > > > (3,[[-1.0,1.0,1.0,1.0,-1.0,-1.0,1.0,-1.0,1.0,1.0,-1.0,-1.0, > > 1.0,1.0,-1.0,-1.0,-1.0,1.0,-1.0,1.0,1.0,1.0,1.0,-1.0,-1.0,- > > 1.0,1.0,1.0,1.0,-1.0,1.0,-1.0,1.0,-1.0],[1.0,-1.0,-1.0,-1.0, > > 1.0,1.0,1.0,1.0,-1.0,1.0,-1.0,1.0,-1.0,1.0,1.0,1.0,-1.0,-1. > > 0,1.0,1.0,-1.0,1.0,-1.0,1.0,-1.0,1.0,-1.0,1.0,1.0,-1.0,1.0, > > -1.0,-1.0],[1.0,-1.0,-1.0,1.0,1.0,-1.0,-1.0,-1.0,1.0,-1.0,1. > > 0,1.0,-1.0,-1.0,-1.0,-1.0,-1.0,1.0,1.0,-1.0,-1.0,1.0,1.0,1. > > 0,1.0,1.0,-1.0,-1.0,-1.0,1.0,1.0,-1.0,1.0]]) > > > > l want to have in : mini_batch1 the first tuple > > [-1.0,1.0,1.0,1.0,-1.0,-1.0,1.0,-1.0,1.0,1.0,-1.0,-1.0,1.0, > > 1.0,-1.0,-1.0,-1.0,1.0,-1.0,1.0,1.0,1.0,1.0,-1.0,-1.0,-1.0, > > 1.0,1.0,1.0,-1.0,1.0,-1.0,1.0,-1.0] > > > > > > mini_batch2 the second tuple [1.0,-1.0,-1.0,-1.0,1.0,1.0,1. > > 0,1.0,-1.0,1.0,-1.0,1.0,-1.0,1.0,1.0,1.0,-1.0,-1.0,1.0,1.0, > > -1.0,1.0,-1.0,1.0,-1.0,1.0,-1.0,1.0,1.0,-1.0,1.0,-1.0,-1.0] > > > > and so on. > > > > However my code do the job of creating the mini_batches files but fail to > > store one by one tuple. how can l fix that ? > > workspace() > > using JLD, HDF5 > > function gen_random(m,k) > > > > # m the length of the vector , for instance m=100000 and k the number of > > partitions let's set k=16 > > > > s = rand(m) > > # Pkg.add("JLD"), Pkg.add("HDF5") these two packages are needed in order > to > > store our vectors in files under the extension jld > > > > # allow to convert each random number to -1 or 1 > > > > X=float_to_binary(s) > > parts= kfoldperm(length(X),k) > > # l want to store each tuple X[p] in a different file > > # X[1] in mini_batch1.jld X[2] in mini_batch2..... > > # but my code below store all the tuple X[p] in the files created. > > for p in 1:length(parts) > > file =jldopen(@sprintf("my path to file/mini_batch%d.jld", p),"w") > > write(file, "X", [X[p] for p in parts]) > > close(file) > > end > > return [X[p] for p in parts] > > > > function float_to_binary(s,level=0.4) > > for i=1:length(s) > > s[i] = s[i] > level ? 1.0 : -1.0 > > end > > file = jldopen("/my path/mydata.jld", "w") > > write(file, "s", s) # alternatively, say "@write file A" > > close(file) > > return s > > end > > > > > > function kfoldperm(l,k) > > n,r = divrem(l,k) > > b = collect(1:n:l+1) > > for i in 1:length(b) > > b[i] += i > r ? r : i-1 > > end > > p = randperm(l) > > return [p[r] for r in [b[i]:b[i+1]-1 for i=1:k]] > > > > > > end > > >
