Thanks so much for your input Alex! It was a very helpful confirmation of
the key conclusions arrived at in this thread, and I appreciate the
additional elaborations you gave, especially the insight you passed on
about the stateful transducers using `ArrayList`. I'm glad that I wasn't
the only
Hey all, just catching up on this thread after the weekend. Rich and I
discussed the thread safety aspects of transducers last fall and the
intention is that transducers are expected to only be used in a single
thread at a time, but that thread can change throughout the life of the
transducing
I do have one last question for you on the topic. I'm thinking of using the
below function within a core.async context — specifically within a `go`
block:
(defn reduce-indexed [f init xs]
(reduce (let [i (mutable-long -1)]
(fn [ret x] (f ret (mutable-swap! i inc) x)))
in
You make a very good point. I had been under the misimpression that you
could make an `r/folder` out of any thread-safe transducer like so, and it
would work out of the box:
(defn broken-reducers-map-indexed [f coll]
(r/folder coll (map-indexed-transducer-concurrently-multi-threaded f)))
and
In your example transducer, the problem is with the `result` parameter. The
specification of transducers is that the result of `(rf result x)` should
be fed into the next call to `rf`. In other words: (-> result (rf x1) (rf
x2) (rf x3))` trying to do that in a parallel context is next to
impossible
That makes sense about them not being designed for that use case. I would
add, though, that transducers could certainly be used in a parallel context
*if* the current transducer implementations were abstracted such that you
could pass internal state generator and modifier functions and use the
Transducers were never designed to work in parallel context. So I'd define
any behavior that arises from using the same transducers in multiple
threads *at the same time*, as undefined behavior.
On Sun, Apr 9, 2017 at 4:39 PM, Alexander Gunnarson <
alexandergunnar...@gmail.com> wrote:
> I should
I should add that, as Timothy pointed out, if multiple threads mutate and
read the value but only one ever does so at a time, as is the case in
`core.async`, then a volatile is sufficient. My preliminary conclusions
above about volatiles apply only to concurrent mutation via e.g. `fold` or
the
It looks that way to me too, Seth, though I'd have to comb over the details
of the locks implemented there to give a reasoned opinion of my own. But
yes, if that's the case, the volatile isn't adding anything.
Anyway, I'm not trying to poke holes in the current implementation of
transducers — o
I'll defer to Timothy on the particulars of core.async but it looks like
[1] the transducer in channel is protected by a lock. If that's the case
volatile isn't adding anything in terms memory barriers.
1:
https://github.com/clojure/core.async/blob/master/src/main/clojure/clojure/core/async/imp
Thanks so much for your well-considered reply, Timothy! That makes sense about
volatiles being used in e.g. core.async or core.reducers contexts where the
reducing function that closes over the mutable value of the stateful transducer
is called in different threads. Why, then, are unsynchronized
My guess is that partition-all and partition use non-volatile references
because none of the built-in stuff will return control back to the caller
at a finer resolution than output value (AFAIK). That's why take needs
volatile but partition-all doesn't (because for take the state persists
betwe
The volatile! is needed for the case where a transducer is only used by one
thread at a time, but the thread executing the transducer may change from
one call to the next. This happens fairly often with core.async. If you
used a non-atomic, non-volatile mutable field, the JVM would be free to
perfo
13 matches
Mail list logo
