Re: shared state/structure and concurrency

2010-03-15 Thread Mark Triggs
Hi Lee, I wouldn't expect multiple reader threads to conflict in the absence of mutation or locking. My first impulse when reading your message was to have a look at what the garbage collector is doing as, in my experience, the "initially fast but gradual slowdown" sort of behaviour ends up being

Re: Long-running STM updates will never complete... is there a solution?

2010-03-15 Thread Garth Sheldon-Coulson
Ah, I see! Thank you very much. Yes, it would be helpful to get some more clarity on the transaction-long readlocking of ensure. On 3/15/10, ataggart wrote: > You'd need to use locking in both places since it's holding the > monitor of the Ref instance, not the read/write locks, e.g.: > (future

Re: Long-running STM updates will never complete... is there a solution?

2010-03-15 Thread ataggart
You'd need to use locking in both places since it's holding the monitor of the Ref instance, not the read/write locks, e.g.: (future (loop [] (Thread/sleep 500) (locking r (dosync (alter r inc))) (recur))) (locking r (dosync (alter r f))) The point of ensure, as I see it, is to say only that the v

Re: Ensure

2010-03-15 Thread Raoul Duke
On Mon, Mar 15, 2010 at 1:13 AM, Christophe Grand wrote: > It hasn't blocked in the sense of "wait and continue": it retries the > transaction after growing the history of the ref (refs have an adaptive > history): i'm confused about being able to set :max-history, does that not run the risk of b

Re: Long-running STM updates will never complete... is there a solution?

2010-03-15 Thread Garth Sheldon-Coulson
I would be happy to use locking if it's the more appropriate construct. I tried it before ensure, in fact, because it seemed more intuitive. ataggart, could you take a look at my first message? The code I posted using locking didn't work for some reason, and I imagine I was doing something wrong.

Re: Ensure

2010-03-15 Thread Raoul Duke
hi Mark, On Mon, Mar 15, 2010 at 2:16 AM, Mark Engelberg wrote: > Wow, the STM works far, far differently than I imagined, but I think I get > it now.  The adaptive history clarification helped tremendously.  Thanks, per chance, might you be willing/able to summarize the latest mental model you

Re: Long-running STM updates will never complete... is there a solution?

2010-03-15 Thread ataggart
Yes, I went off the rails because I was thinking of *a lock*, instead of *the locks*, namely the read and write locks. Since readers block writers (but not the other way around), ensure grabbing a read lock on the ref means anyone wanting to get a write lock will get blocked. On Mar 15, 3:17 pm,

Re: Long-running STM updates will never complete... is there a solution?

2010-03-15 Thread Chouser
On Mon, Mar 15, 2010 at 5:17 PM, Christophe Grand wrote: > On Mon, Mar 15, 2010 at 10:41 PM, ataggart wrote: >> >> On Mar 15, 1:43 pm, Michał Marczyk wrote: >> > On 15 March 2010 21:08, Meikel Brandmeyer wrote: >> > >> > > Now I'm confused. Calling ensure on r shouldn't have an effect since >>

Re: Long-running STM updates will never complete... is there a solution?

2010-03-15 Thread Christophe Grand
On Mon, Mar 15, 2010 at 10:41 PM, ataggart wrote: > > On Mar 15, 1:43 pm, Michał Marczyk wrote: > > On 15 March 2010 21:08, Meikel Brandmeyer wrote: > > > > > Now I'm confused. Calling ensure on r shouldn't have an effect since we > > > call alter on r anyway, no? > > > > ensure "protects the r

Re: Long-running STM updates will never complete... is there a solution?

2010-03-15 Thread ataggart
If using ensure solves the problem then something else is going on, because ensure doesn't lock a ref for the life of the transaction any more than ref-set does. As the doc for ensure notes, it allows for *more* concurrency than ref-set, not less. From clojure.org/refs: "All changes made to Refs

Re: Long-running STM updates will never complete... is there a solution?

2010-03-15 Thread Garth Sheldon-Coulson
Well it definitely seems that ensure has the behavior Michal described, because the ensure code I posted works. I'm glad this behavior is available, because I don't think there is any other way to achieve the combination of synchronization and locking I need. (I couldn't get locking to work on a re

Re: Long-running STM updates will never complete... is there a solution?

2010-03-15 Thread ataggart
On Mar 15, 1:43 pm, Michał Marczyk wrote: > On 15 March 2010 21:08, Meikel Brandmeyer wrote: > > > Now I'm confused. Calling ensure on r shouldn't have an effect since we > > call alter on r anyway, no? > > ensure "protects the ref from modification by other transactions" > (from the docs). alt

Re: Long-running STM updates will never complete... is there a solution?

2010-03-15 Thread Michał Marczyk
On 15 March 2010 21:53, Laurent PETIT wrote: > Ok, but maybe the point of Meikel is that doing so may (?) be relying > on undocumented implementation detail ... ? Well, I don't think so -- that passage I quoted from ensure's docs seems to be an attempt at documenting it. In fact, as far as I can

Re: Long-running STM updates will never complete... is there a solution?

2010-03-15 Thread Laurent PETIT
Hi, Ok, but maybe the point of Meikel is that doing so may (?) be relying on undocumented implementation detail ... ? 2010/3/15 Michał Marczyk : > On 15 March 2010 21:08, Meikel Brandmeyer wrote: >> Now I'm confused. Calling ensure on r shouldn't have an effect since we >> call alter on r anyway

Re: Ensure

2010-03-15 Thread Michał Marczyk
On 15 March 2010 08:44, Mark Engelberg wrote: > Thanks Michal, those annotations helped quite a bit. You're welcome! > It also demonstrates > to me that my mental model of how the STM works is quite out of touch with > how it actually does. I rather think I'm in that situation myself. Take the

Re: Ensure

2010-03-15 Thread Michał Marczyk
On 15 March 2010 09:13, Christophe Grand wrote: > It hasn't blocked in the sense of "wait and continue": it retries the > transaction after growing the history of the ref (refs have an adaptive > history): Thanks a bunch for pointing this one out! I sort of recall reading ref's docs and not reall

Re: Long-running STM updates will never complete... is there a solution?

2010-03-15 Thread Michał Marczyk
On 15 March 2010 21:08, Meikel Brandmeyer wrote: > Now I'm confused. Calling ensure on r shouldn't have an effect since we > call alter on r anyway, no? ensure "protects the ref from modification by other transactions" (from the docs). alter does not. Reading into the Java code, ensure puts a lo

Re: Long-running STM updates will never complete... is there a solution?

2010-03-15 Thread Meikel Brandmeyer
Hi, On Mon, Mar 15, 2010 at 01:21:22PM -0400, Garth Sheldon-Coulson wrote: > Ah, maybe I should have read the thread on ensure before I posted this! =) > > (dosync (ensure r) (alter r f)) > > works. Now I'm confused. Calling ensure on r shouldn't have an effect since we call alter on r anyway,

shared state/structure and concurrency

2010-03-15 Thread Lee Spector
I've been running the genetic programming system that I mentioned in an earlier post (http://hampshire.edu/lspector/clojush/) on a dual quad-core mac and seeing some very nice concurrency, with up to 1300% CPU utilization in some configurations. (Apparently there's some magic that lets the 8 co

take-to-first & partition-when

2010-03-15 Thread Greg Fodor
Hi there, I am just learning Clojure and am processing some BER encoded integer values. Basically, the most significant bit of the integer in the stream indicates the split point between integers, and so I was looking into partition-by to see if that would help. Turns out, what I really need are tw

Re: Newbie question: Why does clojure prefer java.lang.Exception?

2010-03-15 Thread Brian Schlining
> > > > > > That's my impression too. Unfortunately it's inconsistent with Java > > convention. Java generally uses (extensions of) RuntimeException for > > such cases. Other extensions of Exception are reserved for cases you > > *are* expected to handle. > > How do you expect to handle an exceptio

Re: A Tour of Enlive

2010-03-15 Thread David Nolen
On Mon, Mar 15, 2010 at 1:10 PM, Chouser wrote: > Did you mean to change "see" to "set"? > Oops fixed. > Also, I'd recommend a slight change to how you load the > namespaces. Specifically, I'd recommend doing the load first: > >(load "scrape1") > > And then using in-ns to change the R

Re: Long-running STM updates will never complete... is there a solution?

2010-03-15 Thread Garth Sheldon-Coulson
Ah, maybe I should have read the thread on ensure before I posted this! =) (dosync (ensure r) (alter r f)) works. On Mon, Mar 15, 2010 at 1:10 PM, Garth Sheldon-Coulson wrote: > Apologies in advance if this question has a simple answer that I'm > overlooking! > > Suppose I have a ref or atom a

Long-running STM updates will never complete... is there a solution?

2010-03-15 Thread Garth Sheldon-Coulson
Apologies in advance if this question has a simple answer that I'm overlooking! Suppose I have a ref or atom and I periodically need to call a long-running function on it: (def r (ref 0)) (defn f [_] (Thread/sleep 1000) -10) Suppose also that the ref is being modified from other threads at short

Re: A Tour of Enlive

2010-03-15 Thread Chouser
Very nice, David! I'll be reading that again in the future, next time I actually need to use enlive. A couple things I noticed... On Mon, Mar 15, 2010 at 12:43 PM, David Nolen wrote: > > Wow thanks for pointing that out. Turns out that section is way off! It can > be expressed like so: > > (htm

Re: A Tour of Enlive

2010-03-15 Thread David Nolen
On Mon, Mar 15, 2010 at 1:30 AM, Brandon Mason wrote: > Awesome David! Just skimmed over this to get a sense of what Enlive > is. > > I'm currently building a demo of a RESTful API using Compojure. My > goal is to define resources using Clojure data structures and have > them converted to XML/HT

Re: A Tour of Enlive

2010-03-15 Thread David Nolen
On Mon, Mar 15, 2010 at 3:56 AM, wilig wrote: > Thanks David, > > As a new clojurian, this is extremely helpful! I have a question. In > this function: > > (html/deftemplate index "tutorial/template1.html" > [ctxt] > [:p#message] #(if-let [msg (or (:message ctxt) >

Re: Newbie question: Why does clojure prefer java.lang.Exception?

2010-03-15 Thread CloudiDust
Thanks very much for your reply! This is to say, those exceptions are expected to be treated like Java's compilation errors. On 3月15日, 下午1时50分, ataggart wrote: > Exceptions are over/mis-used in java, particularly checked exceptions > (runtime exceptions aren't so bad).  My sense is that those pl

Re: A Tour of Enlive

2010-03-15 Thread Brandon Mason
Awesome David! Just skimmed over this to get a sense of what Enlive is. I'm currently building a demo of a RESTful API using Compojure. My goal is to define resources using Clojure data structures and have them converted to XML/HTML/JSON/etc. based on the user agent string. I've currently got a

Re: A Tour of Enlive

2010-03-15 Thread wilig
Thanks David, As a new clojurian, this is extremely helpful! I have a question. In this function: (html/deftemplate index "tutorial/template1.html" [ctxt] [:p#message] #(if-let [msg (or (:message ctxt) "Nothing to see here!")] ((html/conten

Re: Newbie question: Why does clojure prefer java.lang.Exception?

2010-03-15 Thread ataggart
On Mar 15, 4:44 am, B Smith-Mannschott wrote: > On Mon, Mar 15, 2010 at 06:50, ataggart wrote: > > Exceptions are over/mis-used in java, particularly checked exceptions > > (runtime exceptions aren't so bad).  My sense is that those places > > where java.lang.Exception is thrown, the exception is

Re: Newbie question: Why does clojure prefer java.lang.Exception?

2010-03-15 Thread B Smith-Mannschott
On Mon, Mar 15, 2010 at 06:50, ataggart wrote: > Exceptions are over/mis-used in java, particularly checked exceptions > (runtime exceptions aren't so bad).  My sense is that those places > where java.lang.Exception is thrown, the exception is not expected to > be handled by user code, or perhaps

Re: [ANN] clj-sandbox

2010-03-15 Thread Heinz Nikolaus Gies
My brain is a sive, I forgot the github link o.O http://github.com/Licenser/clj-sandbox sorry for the smap. Best regards, Heinz -- You received this message because you are subscribed to the Google Groups "Clojure" group. To post to this group, send email to clojure@googlegroups.com Note that po

[ANN] clj-sandbox

2010-03-15 Thread Heinz Nikolaus Gies
Greetings everyone, after a discussion on #clojure a week ago or so with bsteuber and hiredman about sandboxing there was the consent that a lib that would make sandboxing clojure code into a secure environment very cool. So I took the task and decided I give it a try. So after some experimenting,

Re: Ensure

2010-03-15 Thread Mark Engelberg
Wow, the STM works far, far differently than I imagined, but I think I get it now. The adaptive history clarification helped tremendously. Thanks, Mark -- You received this message because you are subscribed to the Google Groups "Clojure" group. To post to this group, send email to clojure@goo

Re: Ensure

2010-03-15 Thread Christophe Grand
On Mon, Mar 15, 2010 at 8:44 AM, Mark Engelberg wrote: > Thanks Michal, those annotations helped quite a bit. It also demonstrates > to me that my mental model of how the STM works is quite out of touch with > how it actually does. > > I'm especially surprised that derefs work the way your exampl

Re: Ensure

2010-03-15 Thread Mark Engelberg
Thanks Michal, those annotations helped quite a bit. It also demonstrates to me that my mental model of how the STM works is quite out of touch with how it actually does. I'm especially surprised that derefs work the way your example illustrates. >From the STM docs, it really sounded like derefs

Re: Ensure

2010-03-15 Thread Michał Marczyk
2010/3/15 Mark Engelberg : > Michal, aren't those examples backwards?  I would expect the ensure version > to be the one that needs to retry, and the deref version to be the one that > doesn't. Oh, I failed to notice this message, sorry... I've answered your reply to Meikel first. Anyway, here's a

Re: Ensure

2010-03-15 Thread Michał Marczyk
On 15 March 2010 08:07, Mark Engelberg wrote: > So what I'm finally realizing is that although you are guaranteed a > consistent read value throughout the transaction, that value might become > out of touch with "reality" unless you use ensure. That's not true actually. For example: user> (def r

Re: Ensure

2010-03-15 Thread Mark Engelberg
2010/3/14 Meikel Brandmeyer > In short: ensure is used when you require only read access to a Ref, > but need the value to be constant through the transaction. If you > modify the a Ref via alter or ref-set, the ensure is not necessary. > > OK, I think I've finally figured out why I've always fou

Re: Ensure

2010-03-15 Thread Mark Engelberg
Michal, aren't those examples backwards? I would expect the ensure version to be the one that needs to retry, and the deref version to be the one that doesn't. 2010/3/14 Michał Marczyk > On 15 March 2010 06:47, Mark Engelberg wrote: > > What's a minimal example that demonstrates when ensure is