Hi Daniel,
On Tuesday, November 4, 2014 11:54:30 AM UTC+1, Daniel Marjenburgh wrote:
>
> Hi Jean,
>
> In this case, we are dealing with 1 key-value pair, so there is only one
> transient reference all the time. It is not my experience that each update
> returns a new transient value (here, each update is identical to the
> original, as per the identical? predicate).
>
I probably worded myself poorly: When I wrote transient value, I meant the
contents of a transient, not the reference to it.
I like to think of transients like this: They work exactly like persistent
data structures, but when any kind of update is performed on the transient,
the runtime should be allowed to deallocate the old transient head if it
wants to. If I use transients correctly, I should never see any segfaults
(or whatever you want to call it) due to transient usage.
Now, the runtime doesn't actually deallocate the transient head, it
(usually) reuses it for performance. Which means that the old transient
references are equal to the new transient references, even though the old
transient contents are overwritten (and therefore "invalidated"). So even
though you can use the old transient reference to refer to the new
transient value, it's not referring to the old transient value.
If it helps you, you can also think of transients as linear types, like the
ones in Rust. Their constraints are very similar, but correct transient
usage have to be verified by us, not the compiler. Now, whether that's a
good idea or not is another matter.
> I know you shouldn't use transients in the way I illustrated, but use it
> as if it always returns a new value. In that scenario, you would not get
> inconsistent results, as multiple threads would have to coordinate to pass
> the reference around (as happens in core.async) and the birth-thread check
> is unnecessary.
>
> Still, the onus is now put on the user to ensure correct use of transients
> in a multi-threaded environment, as no warning or error is given anymore.
> It has always been the case that Clojure makes it hard to write unsafe
> mutable code, you had to go out of your way. With this change, I feel we
> are giving up this safeguard a bit too easily.
>
> Maybe another solution could be to have the birth-thread explicitly
> relinquish ownership of the transient, for example, by calling
> (relinquish-transient m). Then the thread that makes the next transient
> update would become the new owner. If another thread updates a transient
> while another threads owns it, the usual error is thrown. This way we
> maintain the safety as before and it becomes a issue for core.async to deal
> with transients in logical threads, not clojure.core. I guess a go macro
> could recognize transients in go-blocks and auto-relinguish them before
> parking operations so that the user can keep thinking in logical threads.
> I'd like to know how feasible a solution like that would be.
>
> Op dinsdag 4 november 2014 11:12:08 UTC+1 schreef Jean Niklas L'orange:
>>
>> Hi there,
>>
>> On Tuesday, November 4, 2014 8:05:03 AM UTC+1, Daniel Marjenburgh wrote:
>>>
>>> I know transients aren't bash-in-place, as they may return new
>>> references, but that is not the problem I'm having here.
>>> If you bash in place, you would get unexpected results, but still the
>>> same result every time. The problem here is that the 'value' of the
>>> transient is changing underneath you because of another thread.
>>>
>>
>> To ensure consistency, you must ensure that you update a transient value
>> at most once – all calls (including reads) on the same transient value
>> after an update is illegal/undefined behaviour. Transient updates – minus
>> persistent! – will always return a new transient value, even though the
>> transient returned may be the same reference. So the code
>>
>> (let [v (transient {:a 0})
>> f1 (future (reduce #(assoc! % (+ (:a %) %2)) v (range 10)))
>> f2 (future (reduce #(assoc! % (+ (:a %) %2)) v (range 10)))]
>> @f1 @f2 ; wait for futures
>> (persistent! @f1))
>>
>> will result in undefined behaviour, because the original v is updated
>> twice (and is potentially read after an update). The correct way would be
>> to do the following:
>>
>> (let [v {:a 0}
>> f1 (future (reduce #(assoc! % (+ (:a %) %2)) (transient v) (range
>> 10)))
>> f2 (future (reduce #(assoc! % (+ (:a %) %2)) (transient v) (range
>> 10)))]
>> @f1 @f2 ; wait for futures
>> (persistent! @f1))
>>
>> I don't see an explicit rule at http://clojure.org/transients stating
>> this, although I feel it is implicit from the bash in-place rule. Perhaps
>> that's something worth adding to the page to avoid confusion.
>>
>> --
>> Regards,
>> Jean Niklas L'orange
>>
>>
>>
>>> Op dinsdag 4 november 2014 06:13:00 UTC+1 schreef Alex Miller:
>>>>
>>>>
>>>>
>>>> On Monday, November 3, 2014 10:44:19 PM UTC-6, Atamert Ölçgen wrote:
>>>>>
>>>>> Thanks Alex!
>>>>>
>>>>> Now that I took a second look at Daniel's code, it seems assoc! is
>>>>> used like swap!, as if it would modify m in place. So I would expect, if
>>>>> it
>>>>> runs without errors, result to be {:a 0}.
>>>>>
>>>>
>>>> Right, that is an incorrect usage - it will actually modify with
>>>> changes though, but not in expected ways (this is independent of the
>>>> change
>>>> we're discussing - you can get the same behavior in a single thread
>>>> modifying a transient without reusing the return).
>>>>
>>>> Given that transients are values (not reference types like ref or atom)
>>>>> I can't think of a case where they can be modified concurrently.
>>>>>
>>>>
>>>> You can, but not without going pretty far out of normal Clojure code.
>>>>
>>>>
>>>>>
>>>>>
>>>>> On Tue, Nov 4, 2014 at 11:19 AM, Alex Miller <al...@puredanger.com>
>>>>> wrote:
>>>>>
>>>>>>
>>>>>>
>>>>>> On Monday, November 3, 2014 9:00:10 PM UTC-6, Atamert Ölçgen wrote:
>>>>>>>
>>>>>>>
>>>>>>>
>>>>>>> On Mon, Nov 3, 2014 at 5:57 PM, Daniel Marjenburgh <
>>>>>>> dmarje...@gmail.com> wrote:
>>>>>>>
>>>>>>>> Hi,
>>>>>>>>
>>>>>>>> I just want to address this issue (CLJ-1498
>>>>>>>> <http://dev.clojure.org/jira/browse/CLJ-1498>). It was accepted in
>>>>>>>> 1.7-alpha2 and I haven't seen a lot of discussion around it, even
>>>>>>>> though
>>>>>>>> it's quite a big change.
>>>>>>>>
>>>>>>>> With this change the following code is possible:
>>>>>>>>
>>>>>>>
>>>>>>> With persistents the result would be the same, every time. If this
>>>>>>> is now valid Clojure, I didn't run it myself, we are sacrificing
>>>>>>> consistency. I don't understand what we're getting in return.
>>>>>>>
>>>>>>> Even the simple example in the ticket (with one future) doesn't make
>>>>>>> a lot of sense to me.
>>>>>>>
>>>>>>> Am I missing something obvious?
>>>>>>>
>>>>>>
>>>>>> Transients always expect thread isolation. In the past this was
>>>>>> locked to a single thread (the one that made the transient). That
>>>>>> restriction now extends to being used by multiple threads, but isolation
>>>>>> should still be maintained for proper use.
>>>>>>
>>>>>> What we are gaining is the ability to use transient collections in go
>>>>>> blocks with core.async, where the thread being used is just one pool out
>>>>>> of
>>>>>> a thread. For example (just typing this not running it, so excuse any
>>>>>> typos):
>>>>>>
>>>>>> (defn drain [ch coll]
>>>>>> (let [t (transient []]
>>>>>> (go-loop
>>>>>> (if-some [v (<! ch)]
>>>>>> (do (conj! t v) (recur))
>>>>>> (persistent! t)))))
>>>>>>
>>>>>> This doesn't necessarily work in core.async because each time the <!
>>>>>> parks, it may be woken up with a different thread from the pool. The
>>>>>> transient change allows this kind of code to succeed.
>>>>>>
>>>>>>
>>>>>>>
>>>>>>>> (let [m (transient {:a 0})
>>>>>>>> futs (for [n (range 100)]
>>>>>>>> (future (assoc! m :a (inc (:a m)))))]
>>>>>>>> (mapv deref futs) ; wait until futures are done
>>>>>>>> (persistent! m))
>>>>>>>>
>>>>>>>>
>>>>>>>> The results will vary per run, where it used to throw
>>>>>>>> an IllegalAccessError: "Transient used by non-owner thread".
>>>>>>>>
>>>>>>>> I understand the problems of not being able to have 1 go routine
>>>>>>>> access a transient, even though it would be safe, but this solution
>>>>>>>> feels
>>>>>>>> like it's throwing out the baby with the bathwater. Basically, it's
>>>>>>>> doing
>>>>>>>> away with what the following block on clojure.org
>>>>>>>> <http://clojure.org/transients> says:
>>>>>>>>
>>>>>>>> *Transients enforce thread isolation**.* Because each result of a
>>>>>>>>> transient operation shares (mutable) structure with the previous, it
>>>>>>>>> would
>>>>>>>>> be very dangerous if more than one thread were to manipulate a
>>>>>>>>> transient at
>>>>>>>>> once. In order to prevent this, transients will detect any (read or
>>>>>>>>> write)
>>>>>>>>> use from a thread other than the one that created them and throw an
>>>>>>>>> exception.
>>>>>>>>
>>>>>>>>
>>>>>>>>
>>>>>>>>>
>>>>>>>>> This may not sound like a concurrency story, but single-thread
>>>>>>>>> isolation is actually a very useful concurrency semantic. The whole
>>>>>>>>> point
>>>>>>>>> of using transients is that doing so is safe, because their use is an
>>>>>>>>> isolated implementation detail of otherwise functional code. Having
>>>>>>>>> that be
>>>>>>>>> enforced means that some things that would normally be very hard to
>>>>>>>>> make
>>>>>>>>> safe with ordinary mutable data structures become easy.
>>>>>>>>
>>>>>>>>
>>>>>>>> I don't have a good solution for dealing with transients and
>>>>>>>> logical threads, but I would much prefer keeping the semantics of
>>>>>>>> transients as they are and maybe pass an option to transient to
>>>>>>>> disable
>>>>>>>> owner checking.
>>>>>>>>
>>>>>>>> --
>>>>>>>> You received this message because you are subscribed to the Google
>>>>>>>> Groups "Clojure" group.
>>>>>>>> To post to this group, send email to clo...@googlegroups.com
>>>>>>>> Note that posts from new members are moderated - please be patient
>>>>>>>> with your first post.
>>>>>>>> To unsubscribe from this group, send email to
>>>>>>>> clojure+u...@googlegroups.com
>>>>>>>> For more options, visit this group at
>>>>>>>> http://groups.google.com/group/clojure?hl=en
>>>>>>>> ---
>>>>>>>> You received this message because you are subscribed to the Google
>>>>>>>> Groups "Clojure" group.
>>>>>>>> To unsubscribe from this group and stop receiving emails from it,
>>>>>>>> send an email to clojure+u...@googlegroups.com.
>>>>>>>> For more options, visit https://groups.google.com/d/optout.
>>>>>>>>
>>>>>>>
>>>>>>>
>>>>>>>
>>>>>>> --
>>>>>>> Kind Regards,
>>>>>>> Atamert Ölçgen
>>>>>>>
>>>>>>> -+-
>>>>>>> --+
>>>>>>> +++
>>>>>>>
>>>>>>> www.muhuk.com
>>>>>>>
>>>>>> --
>>>>>> You received this message because you are subscribed to the Google
>>>>>> Groups "Clojure" group.
>>>>>> To post to this group, send email to clo...@googlegroups.com
>>>>>> Note that posts from new members are moderated - please be patient
>>>>>> with your first post.
>>>>>> To unsubscribe from this group, send email to
>>>>>> clojure+u...@googlegroups.com
>>>>>> For more options, visit this group at
>>>>>> http://groups.google.com/group/clojure?hl=en
>>>>>> ---
>>>>>> You received this message because you are subscribed to the Google
>>>>>> Groups "Clojure" group.
>>>>>> To unsubscribe from this group and stop receiving emails from it,
>>>>>> send an email to clojure+u...@googlegroups.com.
>>>>>> For more options, visit https://groups.google.com/d/optout.
>>>>>>
>>>>>
>>>>>
>>>>>
>>>>> --
>>>>> Kind Regards,
>>>>> Atamert Ölçgen
>>>>>
>>>>> -+-
>>>>> --+
>>>>> +++
>>>>>
>>>>> www.muhuk.com
>>>>>
>>>>
--
You received this message because you are subscribed to the Google
Groups "Clojure" group.
To post to this group, send email to clojure@googlegroups.com
Note that posts from new members are moderated - please be patient with your
first post.
To unsubscribe from this group, send email to
clojure+unsubscr...@googlegroups.com
For more options, visit this group at
http://groups.google.com/group/clojure?hl=en
---
You received this message because you are subscribed to the Google Groups
"Clojure" group.
To unsubscribe from this group and stop receiving emails from it, send an email
to clojure+unsubscr...@googlegroups.com.
For more options, visit https://groups.google.com/d/optout.
