Stubbornly eager results in clojure.java.jdbc

Riddle me this: https://gist.github.com/hagmonk/a75621b143501966c22f53ed1e2bc36e Wherein I synthesize a large table in Postgres, then attempt to lazily load the table, discarding each row as I receive it. I tried *many* permuta

Re: Seeking a function to partially parallelize collection processing

Hello Tom, I think you are talking about distribution, not parallelization. As I see (sorry for not reading enough previously) you want a way to handle different things in a sequential way where each sequence of things (already grouped) are handled in a different thread. You can put the things

Re: Seeking a function to partially parallelize collection processing

Thanks Justin. My mistake. Point 2 stands. On Friday, June 16, 2017 at 3:58:38 PM UTC-4, Justin Smith wrote: > > pmap is rarely actually useful, but point 1 is false, pmap doesn't require > that it's input or output fit in memory > > On Fri, Jun 16, 2017 at 12:52 PM Tom Connors > wrote: > >> Hel

Re: Seeking a function to partially parallelize collection processing

pmap is rarely actually useful, but point 1 is false, pmap doesn't require that it's input or output fit in memory On Fri, Jun 16, 2017 at 12:52 PM Tom Connors wrote: > Hello Jose, > Thank you for the response, but pmap does not address my use case. It's > insufficient for two reasons: 1) the en

Re: Seeking a function to partially parallelize collection processing

Hello Jose, Thank you for the response, but pmap does not address my use case. It's insufficient for two reasons: 1) the entire collection must fit in memory. My use case is handling records from a Kinesis stream. and 2) pmap parallelizes over the whole collection, whereas I want to parallelize

Re: feedback on file parsing with Clojure

The primary suggestion I'd make here is to replace the doseq/reset! construction in your main loop with reduce using a hash-map accumulator representing each value you are updating with a separate key. This isn't just more idiomatic, it also performs better. Instead of: (let [hexagrams (atom (sor

feedback on file parsing with Clojure

hello, i'm looking for some feedback on how i've used Clojure to do some file parsing still getting the hang of Clojure ways of thinking and i'd love to hear any advice on how to improve what i've done for example, i'm guessing the way i've used cond blocks is a bit sketchy - at that point, i w

Re: Seeking a function to partially parallelize collection processing

Hello, there are many videos on how parallelize sequential processing on ClojureTV, but, the most basic way in Clojure I think is *pmap* Saludos. El viernes, 16 de junio de 2017, 9:13:11 (UTC-5), Tom Connors escribió: > > I'm looking for a function that would likely be named something like >

Seeking a function to partially parallelize collection processing

I'm looking for a function that would likely be named something like "sequential-by" or "parallel-per" that takes some data-producing thing like a lazy seq or a core async channel, a function to split records from that input, and a function to handle each item. Each item with an identical retur