On Saturday, March 3, 2018 at 1:39:23 PM UTC+5:30, Aditya Athalye wrote: > > On Friday, March 2, 2018 at 9:58:35 PM UTC+5:30, Rob Nikander wrote: >> >> >> >> On Friday, March 2, 2018 at 12:48:28 AM UTC-5, Daniel wrote: >>> >>> How do you know this code is causing the error? Unless this is all your >>> code does, descriptions of the error suggest the memory leak might be >>> coming from anywhere. This tight loop might trigger too many successive GCs >>> though: have you observed if the error only occurs under load, or under >>> large time intervals? Some combination? >>> >> >> Okay, thanks for looking it over. This is a single-threaded, short lived >> script, so this is the only thing happening. But it's possible there is an >> unexpected large result set. I'll look into that. It may be smarter to use >> `limit N` in my SQL, rather than the time range. >> > > Rob, > > While I could see the intent of the original code, I found it hard to > reason about where the problem might lie, given the loops in loops in loops. > > Assuming the code is causing an error, I feel it will help to separate out > the db query, and factor the rest of the logic into a pipeline. > > This way one could: > - profile the DB query and fix it separately > - isolate and check each part of the records transformation pipeline > - make more obvious what's going on (aid visual audit) > > [snip ...] >
P.S. Oops, replied to the wrong message. I took a slight departure from Gary Johnson's approach and meant to add on top of that. -- You received this message because you are subscribed to the Google Groups "Clojure" group. To post to this group, send email to clojure@googlegroups.com Note that posts from new members are moderated - please be patient with your first post. To unsubscribe from this group, send email to clojure+unsubscr...@googlegroups.com For more options, visit this group at http://groups.google.com/group/clojure?hl=en --- You received this message because you are subscribed to the Google Groups "Clojure" group. To unsubscribe from this group and stop receiving emails from it, send an email to clojure+unsubscr...@googlegroups.com. For more options, visit https://groups.google.com/d/optout.
