I would look for unncessary object creation inside reduction function (use transients) and lazy seq realization (realize only what you really need). Calling reduce inside reduce (inside reduce...) may easily make the complexity of your code exponential. Consider caching of intermediate results if that's the case.
You can also consider switching to reducers/transducers, which may clean your design. In that case, the call to reduce should be as high in hierarchy as possible, ideally in your top level function, with other functions dealing with the transformation of 'step' values. Jozef On Mon, Nov 17, 2014 at 11:28 AM, Alexander L. <alexander.lo...@gmail.com> wrote: > Hi all, > > I understand that the following question is a long shot without any proper > proof/tests from my side but it's a little bit difficult to make a test > case from the specific part of my app so I will just ask anyway in case > anyone knows anything. > > The situation is like this: > > - I have a hashmap with *3386* items that I pass through few functions > in order to append new keys or update existing ones to each hashmap entry. > - Each hashmap item has 20 keys with various data types (mostly > strings) > - All my transformation functions use `reduce`. > > The problem: > > I have a top level function which I inside it I call 7 other functions > (all written by me) and for some reason I haven't discovered is that it > needs around 2 seconds to return a result even though the items aren't many. > Now, i used `time` to benchmark each function and when I found which one > is taking a lot of time to return, after I removed it, I discovered that > the problem still existed but now moved on to a different function. > > I did a bunch of tests with those 3386 on the REPL and reduce but I didn't > notice anything weird/slow so it must be a combination of things. Also, i > doubt that this is a RAM problem, i have allocated 4GB for the JVM. > > So, my question is, has anyone every seen a situation like this with a > bunch of `reduce` calls? Is there anything at all that I should check and > maybe missed it? > > Regards > > Alex > > -- > You received this message because you are subscribed to the Google > Groups "Clojure" group. > To post to this group, send email to clojure@googlegroups.com > Note that posts from new members are moderated - please be patient with > your first post. > To unsubscribe from this group, send email to > clojure+unsubscr...@googlegroups.com > For more options, visit this group at > http://groups.google.com/group/clojure?hl=en > --- > You received this message because you are subscribed to the Google Groups > "Clojure" group. > To unsubscribe from this group and stop receiving emails from it, send an > email to clojure+unsubscr...@googlegroups.com. > For more options, visit https://groups.google.com/d/optout. > -- You received this message because you are subscribed to the Google Groups "Clojure" group. To post to this group, send email to clojure@googlegroups.com Note that posts from new members are moderated - please be patient with your first post. To unsubscribe from this group, send email to clojure+unsubscr...@googlegroups.com For more options, visit this group at http://groups.google.com/group/clojure?hl=en --- You received this message because you are subscribed to the Google Groups "Clojure" group. To unsubscribe from this group and stop receiving emails from it, send an email to clojure+unsubscr...@googlegroups.com. For more options, visit https://groups.google.com/d/optout.
