Hi,
Here is the situation. There is a function "f" retrieving some data from
various sources (including reading files, a lot of io, e.g. map-reduce)
expected by design to return the same result for given input. Results of
"f" invocation from parallely running futures are stored in an atom wrapped
map and everything works just fine.
With a small exception when "f" invocations with the same arguments
overlap - the same expensive "io" is kicked off 2 or more time s. Not a
tragedy, but still very unpleasant and wasteful. The expectation would be
that a subsequent "f" call would just bail without doing anything while
initial invocation assoc new data into an atom,
The first intuition was to use memoize, however I do not think its
semantics fit well into this case, since the result is a "side-effect" on
an atom as opposed to return value.
The easiest solution, is to create another atom with a map of the arguments
into a "state". If some other "future" is already there working hard on the
problem, we simply bail. However, that leads to another problem.
"compare-and-set!" just operates on atoms as wholes, which is fine for
unstructured data, but .... falls short for things somewhat more complex,
like here.
While updating an atom in this context is trivial:
user=> (def a (atom {}))
user=> (swap! a assoc ["arg1" "arg2"] :PENDING)
{["arg1" "arg2"] "resultA"}
I would like to link the condition and future together somehow like that:
user=> (when ((complement contains?) @a ["arg1" "arg3"]) (swap! a assoc
["arg1" "arg3"] :PENDING ))
That obviously will not work, unless I wrap it with "locking", which is not
necessarily a nice thing to do. I also tried to use refs here, however they
do not fit well here either and the solution is not as nice as it could be.
After that long introduction, I would like ask for some insight .... Thanks
in advance ...
Best,
Andy
--
You received this message because you are subscribed to the Google
Groups "Clojure" group.
To post to this group, send email to clojure@googlegroups.com
Note that posts from new members are moderated - please be patient with your
first post.
To unsubscribe from this group, send email to
clojure+unsubscr...@googlegroups.com
For more options, visit this group at
http://groups.google.com/group/clojure?hl=en
---
You received this message because you are subscribed to the Google Groups
"Clojure" group.
To unsubscribe from this group and stop receiving emails from it, send an email
to clojure+unsubscr...@googlegroups.com.
For more options, visit https://groups.google.com/d/optout.
