Re: Seeking a function to partially parallelize collection processing

2017-06-20 Thread Tom Connors
Thanks for the suggestion, Didier, but I was unable to find a way to make pmap work for my use case. For those interested, here's what I came up with, then some questions: (defn parallel-per "Handle records from input-chan in parallel, but records with matching `splitter` return values seria

Re: Seeking a function to partially parallelize collection processing

2017-06-20 Thread Justin Smith
Aside from style issues of mixing channel input/output with program logic, and hiding the useful return value of go-loop, the real problem here is doing your work inside a go block. Go blocks are not meant for blocking tasks, whether CPU or IO bound; doing real work inside go blocks risks starving

Re: Seeking a function to partially parallelize collection processing

2017-06-20 Thread Tom Connors
Thanks, Justin. Regarding the mixing of program logic with channel io, I'm don't understand why that's a problem in this case or how it could be improved. Do you mind explaining that a bit more? -- You received this message because you are subscribed to the Google Groups "Clojure" group. To pos

Re: Stubbornly eager results in clojure.java.jdbc

2017-06-20 Thread Lubomir Konstantinov
^{:once true} should deal with this. Try the following simplified test case with and without it: (defn query [result-fn] (let [x (for [n (range 1e6)] (make-array Object 10)) f (^:once fn* [rs] (result-fn rs))] (f x))) (defn testq [] (let [myfn (fn [rs] (doseq [r rs] nil))]

Re: Seeking a function to partially parallelize collection processing

2017-06-20 Thread Justin Smith
channel operations are io, and intermixing them with processing leads to code that is difficult to read and debug. core.async has facilities to help you code more declaratively over channels. I think TImothy Baldridge's talk at the last Clojure/West does a great job of presenting the issue https://

Re: Seeking a function to partially parallelize collection processing

2017-06-20 Thread Tom Connors
Great, I'll watch that video. Thanks again. -- You received this message because you are subscribed to the Google Groups "Clojure" group. To post to this group, send email to clojure@googlegroups.com Note that posts from new members are moderated - please be patient with your first post. To unsu

Why does gen-class executes with *ns* bound to clojure.core?

2017-06-20 Thread Didier
Take this code: (ns dda.main (:gen-class)) (defn -main [] (prn (ns-name *ns*))) If you bootstrap this through clojure.main, such as what lein does, it will print the namespace as "user". But if you bootsrap it through the generated java main class, it will return "clojure.core". My intuiti

Re: Why does gen-class executes with *ns* bound to clojure.core?

2017-06-20 Thread Didier
Especially given this: (ns dda.main (:gen-class)) (def should-exist "Do I exist?") (defn -main [] (prn (str should-exist \space (ns-name *ns* The (def) is evaluated, since should-exist is found when -main is called from the generated class, but why isn't (ns) evaluated? Which should ha

Re: Why does gen-class executes with *ns* bound to clojure.core?

2017-06-20 Thread Justin Smith
*ns* is a dynamic var, so it points to the current namespace when your function is running. Most code doesn't switch into a target ns in order to execute functions from it. On Tue, Jun 20, 2017 at 4:51 PM Didier wrote: > Especially given this: > > (ns dda.main > (:gen-class)) > > (def should-e

Re: Why does gen-class executes with *ns* bound to clojure.core?

2017-06-20 Thread Sean Corfield
The “trick” I’ve adopted to get around this: (def my-ns *ns*) I stick that near the top of any file in which I want to refer to _that_ namespace (as opposed to whatever the _current_ value of *ns* is during evaluation, since it’s dynamic). Sean Corfield -- (970) FOR-SEAN -- (904) 302-

Re: Why does gen-class executes with *ns* bound to clojure.core?

2017-06-20 Thread Didier
> > *ns* is a dynamic var, so it points to the current namespace when your > function is running. Most code doesn't switch into a target ns in order to > execute functions from it. > I understand that, and I understand most code does not switch into the target ns. But this is not most code, th

Re: Seeking a function to partially parallelize collection processing

2017-06-20 Thread Didier
Do you want something like this? (ns dda.test) (def test-infinite-lazy-seq (repeatedly (fn [] {:id (rand-int 2) :val (rand-int 10)}))) (def test-finite-seq [{:id 1 :val 1} {:id 1 :val 2}

Re: Seeking a function to partially parallelize collection processing

2017-06-20 Thread Didier
Here: (ns dda.test) (def test-infinite-lazy-seq (repeatedly (fn [] {:id (rand-int 2) :val (rand-int 10)}))) (def test-finite-seq [{:id 1 :val 1} {:id 1 :val 2} {:id 3 :val 1}]) (defn pa

Re: Why does gen-class executes with *ns* bound to clojure.core?

2017-06-20 Thread Alex Miller
On Tuesday, June 20, 2017 at 7:43:53 PM UTC-5, Didier wrote: > > *ns* is a dynamic var, so it points to the current namespace when your >> function is running. Most code doesn't switch into a target ns in order to >> execute functions from it. >> > > I understand that, and I understand most cod

Re: Why does gen-class executes with *ns* bound to clojure.core?

2017-06-20 Thread Alex Miller
Actually, scratch some of what I said (it's late). It's not that the value gets compiled into the class - the -main function will actually look up the value of *ns*. The difference is only in the ordering of what happens in the two scenarios. If you do "lein run", that will invoke clojure.main