Re: Transient Data Structures

2009-08-11 Thread Christophe Grand
On Tue, Aug 11, 2009 at 12:28 PM, Andy Fingerhut < andy_finger...@alum.wustl.edu> wrote: > > On Aug 10, 11:15 am, Christophe Grand wrote: > > Hi Andy, > > > > On Thu, Aug 6, 2009 at 7:40 PM, Andy Fingerhut < > > > > > > > > andy_finger...@alum.wustl.edu> wrote: > > > Thank you, Christophe! I've

Re: Transient Data Structures

2009-08-11 Thread Andy Fingerhut
On Aug 10, 11:15 am, Christophe Grand wrote: > Hi Andy, > > On Thu, Aug 6, 2009 at 7:40 PM, Andy Fingerhut < > > > > andy_finger...@alum.wustl.edu> wrote: > > Thank you, Christophe!  I've been wanting to try those out. > > > I made changes to 3 lines of my Clojure program for the k-nucleotide > >

Re: Transient Data Structures

2009-08-10 Thread samppi
Excellent, excellent. But I'm wondering, is it planned (or feasible) for structmap transients to be supported too? I often use and "modify" protean structmaps in loops, and I'd love to know if the concept of transients can be applied to them. On Aug 6, 4:53 am, Rich Hickey wrote: > On Aug 5, 10:

Re: Transient Data Structures

2009-08-10 Thread Christophe Grand
Hi Andy, On Thu, Aug 6, 2009 at 7:40 PM, Andy Fingerhut < andy_finger...@alum.wustl.edu> wrote: > Thank you, Christophe! I've been wanting to try those out. > > I made changes to 3 lines of my Clojure program for the k-nucleotide > benchmark, which spends most of its time in a function tally-dna

Re: Transient Data Structures

2009-08-07 Thread Patrick Sullivan
Ah hah, yeah I'm dumb, thanks to you and AlexK for catching my silliness. Funny how when I'm using normal clojure persistant structs I don't think about doing it the right way twice, but when doing it as a transient I slip into old imperative habits *headslap* ~Patrick On Aug 7, 8:20 am, John N

Re: Transient Data Structures

2009-08-07 Thread AlexK
On 7 Aug., 10:07, Patrick Sullivan wrote: > > Am I doing something silly here or is this a bug? You probably are using conj! for the side-effect, but after growing the hashmap to size 8 conj! returns a different map. user> (def foo (transient {1 1 2 2 3 3 4 4 5 5 6 6 7 7 8 8})) #'user/ foo us

Re: Transient Data Structures

2009-08-07 Thread John Newman
> > (def transhashmap (transient {}) (assoc transhashmap "a" 1) (assoc transhashmap "b" 2) etc Isn't that what Rich was talking about, about not bashing in place? On Fri, Aug 7, 2009 at 6:45 PM, Patrick Sullivan < wizardofwestma...@gmail.com> wrote: > > I don't have the EXACT code handy to c

Re: Transient Data Structures

2009-08-07 Thread Patrick Sullivan
Err assoc! obviously ;-) (Sorry for the double post, didn't want to confuse Cristophe). On Aug 7, 8:15 am, Patrick Sullivan wrote: > I don't have the EXACT code handy to c/p (at work now) but I did > something like the following. > (apologies for doing it such an iterative looking way, never go

Re: Transient Data Structures

2009-08-07 Thread Patrick Sullivan
I don't have the EXACT code handy to c/p (at work now) but I did something like the following. (apologies for doing it such an iterative looking way, never got comfortable with -> ;-)) (def transhashmap (transient {}) (assoc transhashmap "a" 1) (assoc transhashmap "b" 2) etc Then when I did (cou

Re: Transient Data Structures

2009-08-07 Thread tmountain
get a performance boost. > > Travis > > On Aug 3, 5:25 pm, Rich Hickey wrote: > > > I've been doing some work on Transient Data Structures. You can read > > about them here: > > >http://clojure.org/transients > > > Feedback welcome, > > > R

Re: Transient Data Structures

2009-08-07 Thread tmountain
This is awesome. I'm curious if support for maps is planned in addition to vectors? A lot of my code makes heavy use of maps, and it would be great to get a performance boost. Travis On Aug 3, 5:25 pm, Rich Hickey wrote: > I've been doing some work on Transient Data Structures.

Re: Transient Data Structures

2009-08-07 Thread Christophe Grand
Hi Patrick ! Can you post some code. here is what I get: user=> (-> {} transient (assoc! :a 1) (assoc! :b 2) (assoc! :c 3) (assoc! :d 4) (assoc! :e 5) (assoc! :f 6) (assoc! :g 7) (assoc! :h 8) (assoc! :i 9) persistent!) {:a 1, :c 3, :b 2, :f 6, :g 7, :d 4, :e 5, :i 9, :h 8} user=> (persistent! (re

Re: Transient Data Structures

2009-08-07 Thread Patrick Sullivan
Testing Transient w/Hashmaps (Thanks Cristophe!) and it seems like the object won't store more then 8 keys. At first I thought it was my frequency function that was rolling it up, but then I simply tried creating a transient object and manually assoc! ing a bunch of items into it. After the 8th

Re: Transient Data Structures

2009-08-06 Thread Andy Fingerhut
On Aug 6, 4:53 am, Rich Hickey wrote: > On Aug 5, 10:10 pm, Luc Prefontaine > wrote: > > > I like this very much... that's the kind of clever optimizations that > > preserves Clojure principles and > > can yield significant performance increases. This could also help > > dealing with performan

Re: Transient Data Structures

2009-08-06 Thread Rich Hickey
On Aug 5, 10:10 pm, Luc Prefontaine wrote: > I like this very much... that's the kind of clever optimizations that > preserves Clojure principles and > can yield significant performance increases. This could also help > dealing with performance critics > in these small mutable languages "benchm

Re: Transient Data Structures

2009-08-05 Thread Eric
On Aug 3, 8:39 pm, Rich Hickey wrote: > > In short, the O(1) overhead is less than the cost of even a single > edit. So, e.g. into/vec/vector now use transients unconditionally if > possible. > > Rich Are we going to feel a big performance boost once all of the core functions are rewritten using

Re: Transient Data Structures

2009-08-05 Thread Luc Prefontaine
I like this very much... that's the kind of clever optimizations that preserves Clojure principles and can yield significant performance increases. This could also help dealing with performance critics in these small mutable languages "benchmarks" that newbies attempt to clone in Clojure. Thank's

Re: Transient Data Structures

2009-08-05 Thread Rich Hickey
On Aug 5, 7:48 pm, Stu Hood wrote: > I really, really like this feature. My only complaint is that you have to > use different names for the modifying functions. If the function signatures > will be identical to their non-transient variants, then I guess the primary > arguments would be: >  * C

Re: Transient Data Structures

2009-08-05 Thread Stu Hood
I really, really like this feature. My only complaint is that you have to use different names for the modifying functions. If the function signatures will be identical to their non-transient variants, then I guess the primary arguments would be: * Clojure convention for names of functions with sid

Re: Transient Data Structures

2009-08-05 Thread Rich Hickey
On Tue, Aug 4, 2009 at 10:13 PM, John Harrop wrote: > On Tue, Aug 4, 2009 at 5:50 PM, Rich Hickey wrote: >> >> On Aug 4, 4:31 pm, John Harrop wrote: >> > What about things like: >> > >> > (persistent! >> >   (reduce >> >     (fn [x [i v]] (assoc! x i v)) >> >     (transient (vec (repeat 0 (reduc

Re: Transient Data Structures

2009-08-05 Thread John Harrop
On Tue, Aug 4, 2009 at 5:50 PM, Rich Hickey wrote: > On Aug 4, 4:31 pm, John Harrop wrote: > > What about things like: > > > > (persistent! > > (reduce > > (fn [x [i v]] (assoc! x i v)) > > (transient (vec (repeat 0 (reduce max (map first > > coll-of-index-val-pairs) > > coll-o

Re: Transient Data Structures

2009-08-04 Thread Rich Hickey
On Aug 4, 4:31 pm, John Harrop wrote: > On Tue, Aug 4, 2009 at 1:39 PM, Rich Hickey wrote: > > > On Tue, Aug 4, 2009 at 1:32 PM, John Newman wrote: > > > > I'm a noob, so this is probably a dumb question but, how does this work > > with > > > closures?  Can transients be closed over? > > > > L

Re: Transient Data Structures

2009-08-04 Thread John Harrop
On Tue, Aug 4, 2009 at 1:39 PM, Rich Hickey wrote: > > On Tue, Aug 4, 2009 at 1:32 PM, John Newman wrote: > > > > I'm a noob, so this is probably a dumb question but, how does this work > with > > closures? Can transients be closed over? > > > > Like, > > > >> (defn make-transient-counter [init-v

Re: Transient Data Structures

2009-08-04 Thread Rich Hickey
On Tue, Aug 4, 2009 at 1:32 PM, John Newman wrote: > > I'm a noob, so this is probably a dumb question but, how does this work with > closures?  Can transients be closed over? > > Like, > >> (defn make-transient-counter [init-val] >>   (let [acounter (transient [init-val])] >>     (fn [add-val] (a

Re: Transient Data Structures

2009-08-04 Thread John Newman
I'm a noob, so this is probably a dumb question but, how does this work with closures? Can transients be closed over? Like, (defn make-transient-counter [init-val] > (let [acounter (transient [init-val])] > (fn [add-val] (assoc! acounter 0 (+ add-val (nth acounter 0)) > Is that possib

Re: Transient Data Structures

2009-08-04 Thread Rich Hickey
t the originating thread. >>> >>> Does this mean we'll need a reduce! >> >> No, reduce farms out the work to the reducing function and doesn't >> create a composite data structure itself. The reducing fn might use a >> transient but that's an

Re: Transient Data Structures

2009-08-04 Thread Howard Lewis Ship
data structure itself. The reducing fn might use a > transient but that's an implementation detail of it. > >> and map!, etc? > > No, map returns a sequence. > >> Would there be an >> advantage to those? Is there an intersection (or potential danger) >> be

Re: Transient Data Structures

2009-08-04 Thread Rich Hickey
o those? Is there an intersection (or potential danger) > between transient structures and laziness? > Transient data structures should never be part of the public interface of anything. They are just an implementation detail, an optimization, e.g. into/vec/vector now use transie

Re: Transient Data Structures

2009-08-04 Thread Howard Lewis Ship
ial danger) between transient structures and laziness? On Mon, Aug 3, 2009 at 2:25 PM, Rich Hickey wrote: > > I've been doing some work on Transient Data Structures. You can read > about them here: > > http://clojure.org/transients > > Feedback welcome, > > Rich >

Re: Transient Data Structures

2009-08-04 Thread Boris Mizhen - 迷阵
> Clojure so far, and I think there is a chance that you could pull this > off beautifully. > > Good luck. > > On Aug 3, 5:25 pm, Rich Hickey wrote: >> I've been doing some work on Transient Data Structures. You can read >> about them

Re: Transient Data Structures

2009-08-04 Thread harrison clarke
right, I need > assoc! not assoc...). > > With all this being said, I'm looking forward to your final version. > Your speedup is impressive, and I know parts of my code (lists of hash > maps) that could use it.  You've done some pretty bad ass stuff with > Clojure so far,

Re: Transient Data Structures

2009-08-04 Thread Sean Devlin
speedup is impressive, and I know parts of my code (lists of hash maps) that could use it. You've done some pretty bad ass stuff with Clojure so far, and I think there is a chance that you could pull this off beautifully. Good luck. On Aug 3, 5:25 pm, Rich Hickey wrote: > I've bee

Re: Transient Data Structures

2009-08-04 Thread Laurent PETIT
ok... 2009/8/4 Rich Hickey > > On Tue, Aug 4, 2009 at 7:54 AM, Laurent PETIT > wrote: > > Looks very interesting ! > > > > One question: wouldn't seem more natural to have transient named > transient! > > and persistent! named persistent ? > > > > I see a call to transient as "Enter the mutable

Re: Transient Data Structures

2009-08-04 Thread Rich Hickey
On Tue, Aug 4, 2009 at 7:54 AM, Laurent PETIT wrote: > Looks very interesting ! > > One question: wouldn't seem more natural to have transient named transient! > and persistent! named persistent ? > > I see a call to transient as "Enter the mutable world", so it seems to me > (transient! []) conve

Re: Transient Data Structures

2009-08-04 Thread Laurent PETIT
ersistent v)] ...) looks better in a pure function than this: (defn some-fn [v] (let [v (persistent! v)] ...) where the ! catches the eye ... HTH, -- Laurent 2009/8/3 Rich Hickey > > I've been doing some work on Transient Data Structures. You can read > about them here: > &g

Re: Transient Data Structures

2009-08-04 Thread Garth Sheldon-Coulson
tent code)) > > Garth > > > On Mon, Aug 3, 2009 at 5:25 PM, Rich Hickey wrote: > >> >> I've been doing some work on Transient Data Structures. You can read >> about them here: >> >> http://clojure.org/transients >> >> Feedback welco

Re: Transient Data Structures

2009-08-04 Thread Garth Sheldon-Coulson
ble. Is there a more appropriate thing to do than the following? (if (resolve 'transient) (transient code) (persistent code)) Garth On Mon, Aug 3, 2009 at 5:25 PM, Rich Hickey wrote: > > I've been doing some work on Transient Data Structures. You can read > about them here:

Re: Transient Data Structures

2009-08-03 Thread Andy Fingerhut
eone forgets to call "persistent!" on a data structure before using it in another thread. Thanks, Andy On Aug 3, 2:25 pm, Rich Hickey wrote: > I've been doing some work on Transient Data Structures. You can read > about them here: > > http://clojure.org/transients > >

Re: Transient Data Structures

2009-08-03 Thread John Harrop
On Mon, Aug 3, 2009 at 7:27 PM, CuppoJava wrote: > > Hi Rich, > This is a very useful addition thanks. I personally find the O(1) > transformation to and back most useful. > > I have a question about capturing the return values of conj! and > assoc!. > > in this code: > (let [v (transient []) >

Re: Transient Data Structures

2009-08-03 Thread Mark Engelberg
On Mon, Aug 3, 2009 at 5:39 PM, Rich Hickey wrote: > In short, the O(1) overhead is less than the cost of even a single > edit. So, e.g. into/vec/vector now use transients unconditionally if > possible. Excellent. I'm glad to hear that the core functions will use transients so I don't have to ma

Re: Transient Data Structures

2009-08-03 Thread Rich Hickey
On Aug 3, 8:06 pm, Mark Engelberg wrote: > So if you want to make 10 changes to a vector, would it be worthwhile > to turn it into a transient, make the 10 changes, and then turn it > back to persistent?  If no, then 100 changes?  1000? > > In other words, how much overhead is there in the tran

Re: Transient Data Structures

2009-08-03 Thread Rich Hickey
On Aug 3, 7:27 pm, CuppoJava wrote: > Hi Rich, > This is a very useful addition thanks. I personally find the O(1) > transformation to and back most useful. > > I have a question about capturing the return values of conj! and > assoc!. > > in this code: > (let [v (transient []) >        v2 (con

Re: Transient Data Structures

2009-08-03 Thread Mark Engelberg
So if you want to make 10 changes to a vector, would it be worthwhile to turn it into a transient, make the 10 changes, and then turn it back to persistent? If no, then 100 changes? 1000? In other words, how much overhead is there in the transformation back and forth, and therefore, about how m

Re: Transient Data Structures

2009-08-03 Thread CuppoJava
Hi Rich, This is a very useful addition thanks. I personally find the O(1) transformation to and back most useful. I have a question about capturing the return values of conj! and assoc!. in this code: (let [v (transient []) v2 (conj! v 0)]) v2 is the captured return value from conj!, wh

Transient Data Structures

2009-08-03 Thread Rich Hickey
I've been doing some work on Transient Data Structures. You can read about them here: http://clojure.org/transients Feedback welcome, Rich --~--~-~--~~~---~--~~ You received this message because you are subscribed to the Google Groups "Clojure" gr

Re: Transient Data Structures

2009-08-03 Thread Rich Hickey
On Aug 3, 5:52 pm, luke wrote: > Interesting. > > So I take it that these are (or should be) entirely a speed > optimization? i.e, most the time, you'd want to write your code using > the normal persistent data structures, and only go back and implement > this within specific functions if they'

Re: Transient Data Structures

2009-08-03 Thread Vagif Verdi
On Aug 3, 1:52 pm, luke wrote: > So you could easily wrap an entirely functional code block > in a transform-to-transient macro that translates the functions to > their transient counterparts, and gain all the performance benefits? I do not think it would be that easy. Transient mode cannot be u

Re: Transient Data Structures

2009-08-03 Thread luke
d gain all the performance benefits? Thanks, -Luke On Aug 3, 5:25 pm, Rich Hickey wrote: > I've been doing some work on Transient Data Structures. You can read > about them here: > > http://clojure.org/transients > > Feedback welcome, > > Rich --~--~-~--