Java's System.arraycopy is the fastest you can get, since it delegates execution to a function implemented in C inside JVM. Simply, this is the fastest that your computer hardware can get. All in all Java arrays meet the same difficulties and implications as C arrays and that is why concationation of raw arrays is so "complex", in contrast to higher-level collections which use objects and pointers (e.g. LinkedList). In other words, difficulties you experience are natural outcome of how computer's memory management is made and there is no way around them. You get the most of the speed from arrays because they are solid (not fragmented) chunks of bytes allocated in memory in the moment of their creation. For that very reason you cannot extend an existing array (the size cannot be changed after creation) and you can't concatenate it with another array since first it would have to be concatenated.
The natural outcome also is that only arrays of same types can be concatenated with System.arraycopy since only array pointers store type data, and the contents are simply untyped bytes. And this is why it is byte-level and no type-checks are ever done besiedes the initial type-check. Again, higher-level pointer-based data structures like LinkedList or Queue can introduce boxed typed values, but that'd be waaay slower. Considering that only arrays of same type are concatenateable, creating a polymorphic function is easy - simply check the argument type like: ; first save types to use them later (def arr-type-int (class (ints 3))) ; ... same for other primitives... ; then in your func: (cond (= (class arr) arr-type-int) (do-int-concat) ...) For more reference: http://docs.oracle.com/javase/tutorial/java/nutsandbolts/arrays.html http://docstore.mik.ua/orelly/java-ent/jnut/ch02_09.htm As an alternative, try looking into Java NIO buffers - they too are fast and too have some limits. But maybe you could make good of them, depends on your use case. Although somewhat in another vein, but still relating fast data management is https://groups.google.com/forum/?hl=en#!topic/clojure/BayfuaqMzvs which brings in C-like structs in. On Sunday, July 21, 2013 2:39:38 AM UTC+4, Brian Craft wrote: > > Here are some experiments that aren't polymorphic. The System/arraycopy > version is fastest, by far. Is there any good way to make the other > versions faster, or make them handle any array type? > > (defn bconcat [& arrays] > (let [sizes (map count arrays) > sizes_r (vec (reductions + sizes)) > offsets (cons 0 (drop-last sizes_r)) > total (last sizes_r) > out (float-array total)] > (dorun (map #(System/arraycopy %2 0 out %1 %3) offsets arrays sizes)) > out)) > > (defn cconcat [& arrays] > (let [vs (map vec arrays) > cc (apply concat vs)] > (float-array cc))) > > (defn dconcat [& arrays] > (let [vs (map vec arrays) > cc (reduce into [] vs)] > (float-array cc))) > > (defn econcat [& arrays] > (let [cc (reduce into [] arrays)] > (float-array cc))) > > > On Saturday, July 20, 2013 2:24:14 PM UTC-7, Brian Craft wrote: >> >> Is there an easy, fast way to concat primitive arrays? I was hoping java >> arrays had some common interface for this, but I haven't found much of use. >> I mostly see code like this: >> >> byte[] c = new byte[a.length + b.length]; >> System.arraycopy(a, 0, c, 0, a.length); >> System.arraycopy(b, 0, c, a.length, b.length); >> >> which only works for bytes (in this case). >> > -- -- You received this message because you are subscribed to the Google Groups "Clojure" group. To post to this group, send email to clojure@googlegroups.com Note that posts from new members are moderated - please be patient with your first post. To unsubscribe from this group, send email to clojure+unsubscr...@googlegroups.com For more options, visit this group at http://groups.google.com/group/clojure?hl=en --- You received this message because you are subscribed to the Google Groups "Clojure" group. To unsubscribe from this group and stop receiving emails from it, send an email to clojure+unsubscr...@googlegroups.com. For more options, visit https://groups.google.com/groups/opt_out.
