Ramesh, I think the main problem is you're trying to do a recursion over a tree, so you're blowing a stack if you're not careful, concat is lazy so I think you're just shifting the problem around, file-seq gets around this by using tree-seq, which uses 'walk', which is a linearization of the tree.
On Mon, May 20, 2013 at 4:45 PM, Ramesh <ramesh10dul...@gmail.com> wrote: > So, I think concat is the problem here. I wish there were recommendation > for other options and reason in the Stacktrace to help me code better! > > And file-seq is exactly what I'm looking for :). Thanks all! > > > Thanks, > ramesh > > > On Mon, May 20, 2013 at 1:21 PM, Ben Wolfson <wolf...@gmail.com> wrote: > >> Basically, you can't use "concat" in a loop like that. You could wrap it >> in a (doall ...) to avoid the stack overflow. >> >> >> On Mon, May 20, 2013 at 1:12 PM, Ramesh <ramesh10dul...@gmail.com>wrote: >> >>> Hi all, >>> >>> I have the following function to list all the files recursively under a >>> directory. >>> >>> (defn list-all-files2 [path] >>> (let [basepath (java.io.File. path)] >>> (loop [origlist [basepath] finallist []] >>> (if-let [cur (first origlist)] >>> ;This line is not required (if (.isDirectory cur) >>> (recur (concat (rest origlist) (.listFiles cur)) (conj finallist >>> (.getCanonicalPath cur))) >>> finallist)))) >>> >>> >>> It runs fine, when the number of files are quite less, 10-20. >>> >>> But when I recurse through even moderately large amount of files, it >>> fails with stackoverflow! The total number of files are not that much >>> (50114). So even if each of the filename is 100 chars long, it would >>> roughly take (5M bytes, which is 5MB). But I get a stack overflow error, >>> and I have no clue. I think I am doing something wrong with the lazy >>> functions. >>> >>> Here is the stacktrace: >>> Exception in thread "main" java.lang.RuntimeException: >>> java.lang.StackOverflowError >>> at clojure.lang.Util.runtimeException(Util.java:165) >>> at clojure.lang.Compiler.eval(Compiler.java:6476) >>> at clojure.lang.Compiler.eval(Compiler.java:6455) >>> at clojure.lang.Compiler.eval(Compiler.java:6431) >>> at clojure.core$eval.invoke(core.clj:2795) >>> at clojure.main$eval_opt.invoke(main.clj:296) >>> at clojure.main$initialize.invoke(main.clj:315) >>> at clojure.main$null_opt.invoke(main.clj:348) >>> at clojure.main$main.doInvoke(main.clj:426) >>> at clojure.lang.RestFn.invoke(RestFn.java:421) >>> at clojure.lang.Var.invoke(Var.java:405) >>> at clojure.lang.AFn.applyToHelper(AFn.java:163) >>> at clojure.lang.Var.applyTo(Var.java:518) >>> at clojure.main.main(main.java:37) >>> Caused by: java.lang.StackOverflowError >>> at clojure.core$concat$fn__3528.invoke(core.clj:661) >>> at clojure.lang.LazySeq.sval(LazySeq.java:42) >>> at clojure.lang.LazySeq.seq(LazySeq.java:60) >>> at clojure.lang.RT.seq(RT.java:466) >>> at clojure.core$seq.invoke(core.clj:133) >>> at clojure.core$concat$fn__3528.invoke(core.clj:661) >>> at clojure.lang.LazySeq.sval(LazySeq.java:42) >>> at clojure.lang.LazySeq.seq(LazySeq.java:60) >>> at clojure.lang.RT.seq(RT.java:466) >>> at clojure.core$seq.invoke(core.clj:133) >>> at clojure.core$concat$fn__3528.invoke(core.clj:661) >>> at clojure.lang.LazySeq.sval(LazySeq.java:42) >>> at clojure.lang.LazySeq.seq(LazySeq.java:60) >>> at clojure.lang.RT.seq(RT.java:466) >>> at clojure.core$seq.invoke(core.clj:133) >>> at clojure.core$concat$fn__3528.invoke(core.clj:661) >>> at clojure.lang.LazySeq.sval(LazySeq.java:42) >>> at clojure.lang.LazySeq.seq(LazySeq.java:60) >>> at clojure.lang.RT.seq(RT.java:466) >>> at clojure.core$seq.invoke(core.clj:133) >>> at clojure.core$concat$fn__3528.invoke(core.clj:661) >>> at clojure.lang.LazySeq.sval(LazySeq.java:42) >>> at clojure.lang.LazySeq.seq(LazySeq.java:60) >>> at clojure.lang.RT.seq(RT.java:466) >>> at clojure.core$seq.invoke(core.clj:133) >>> at clojure.core$concat$fn__3528.invoke(core.clj:661) >>> at clojure.lang.LazySeq.sval(LazySeq.java:42) >>> at clojure.lang.LazySeq.seq(LazySeq.java:60) >>> at clojure.lang.RT.seq(RT.java:466) >>> at clojure.core$seq.invoke(core.clj:133) >>> at clojure.core$concat$fn__3528.invoke(core.clj:661) >>> at clojure.lang.LazySeq.sval(LazySeq.java:42) >>> at clojure.lang.LazySeq.seq(LazySeq.java:60) >>> at clojure.lang.RT.seq(RT.java:466) >>> at clojure.core$seq.invoke(core.clj:133) >>> at clojure.core$concat$fn__3528.invoke(core.clj:661) >>> at clojure.lang.LazySeq.sval(LazySeq.java:42) >>> at clojure.lang.LazySeq.seq(LazySeq.java:60) >>> at clojure.lang.RT.seq(RT.java:466) >>> at clojure.core$seq.invoke(core.clj:133) >>> at clojure.core$concat$fn__3528.invoke(core.clj:661) >>> at clojure.lang.LazySeq.sval(LazySeq.java:42) >>> at clojure.lang.LazySeq.seq(LazySeq.java:60) >>> at clojure.lang.RT.seq(RT.java:466) >>> at clojure.core$seq.invoke(core.clj:133) >>> at clojure.core$concat$fn__3528.invoke(core.clj:661) >>> at clojure.lang.LazySeq.sval(LazySeq.java:42) >>> at clojure.lang.LazySeq.seq(LazySeq.java:60) >>> at clojure.lang.RT.seq(RT.java:466) >>> at clojure.core$seq.invoke(core.clj:133) >>> at clojure.core$concat$fn__3528.invoke(core.clj:661) >>> at clojure.lang.LazySeq.sval(LazySeq.java:42) >>> at clojure.lang.LazySeq.seq(LazySeq.java:60) >>> at clojure.lang.RT.seq(RT.java:466) >>> at clojure.core$seq.invoke(core.clj:133) >>> at clojure.core$concat$fn__3528.invoke(core.clj:661) >>> at clojure.lang.LazySeq.sval(LazySeq.java:42) >>> at clojure.lang.LazySeq.seq(LazySeq.java:60) >>> at clojure.lang.RT.seq(RT.java:466) >>> at clojure.core$seq.invoke(core.clj:133) >>> at clojure.core$concat$fn__3528.invoke(core.clj:661) >>> at clojure.lang.LazySeq.sval(LazySeq.java:42) >>> at clojure.lang.LazySeq.seq(LazySeq.java:60) >>> at clojure.lang.RT.seq(RT.java:466) >>> at clojure.core$seq.invoke(core.clj:133) >>> at clojure.core$concat$fn__3528.invoke(core.clj:661) >>> at clojure.lang.LazySeq.sval(LazySeq.java:42) >>> at clojure.lang.LazySeq.seq(LazySeq.java:60) >>> at clojure.lang.RT.seq(RT.java:466) >>> at clojure.core$seq.invoke(core.clj:133) >>> at clojure.core$concat$fn__3528.invoke(core.clj:661) >>> at clojure.lang.LazySeq.sval(LazySeq.java:42) >>> at clojure.lang.LazySeq.seq(LazySeq.java:60) >>> at clojure.lang.RT.seq(RT.java:466) >>> at clojure.core$seq.invoke(core.clj:133) >>> at clojure.core$concat$fn__3528.invoke(core.clj:661) >>> at clojure.lang.LazySeq.sval(LazySeq.java:42) >>> at clojure.lang.LazySeq.seq(LazySeq.java:60) >>> at clojure.lang.RT.seq(RT.java:466) >>> at clojure.core$seq.invoke(core.clj:133) >>> at clojure.core$concat$fn__3528.invoke(core.clj:661) >>> at clojure.lang.LazySeq.sval(LazySeq.java:42) >>> at clojure.lang.LazySeq.seq(LazySeq.java:60) >>> at clojure.lang.RT.seq(RT.java:466) >>> at clojure.core$seq.invoke(core.clj:133) >>> at clojure.core$concat$fn__3528.invoke(core.clj:661) >>> at clojure.lang.LazySeq.sval(LazySeq.java:42) >>> at clojure.lang.LazySeq.seq(LazySeq.java:60) >>> at clojure.lang.RT.seq(RT.java:466) >>> at clojure.core$seq.invoke(core.clj:133) >>> at clojure.core$concat$fn__3528.invoke(core.clj:661) >>> at clojure.lang.LazySeq.sval(LazySeq.java:42) >>> at clojure.lang.LazySeq.seq(LazySeq.java:60) >>> at clojure.lang.RT.seq(RT.java >>> >>> >>> Thanks, >>> ramesh >>> >>> -- >>> -- >>> You received this message because you are subscribed to the Google >>> Groups "Clojure" group. >>> To post to this group, send email to clojure@googlegroups.com >>> Note that posts from new members are moderated - please be patient with >>> your first post. >>> To unsubscribe from this group, send email to >>> clojure+unsubscr...@googlegroups.com >>> For more options, visit this group at >>> http://groups.google.com/group/clojure?hl=en >>> --- >>> You received this message because you are subscribed to the Google >>> Groups "Clojure" group. >>> To unsubscribe from this group and stop receiving emails from it, send >>> an email to clojure+unsubscr...@googlegroups.com. >>> For more options, visit https://groups.google.com/groups/opt_out. >>> >>> >>> >> >> >> >> -- >> Ben Wolfson >> "Human kind has used its intelligence to vary the flavour of drinks, >> which may be sweet, aromatic, fermented or spirit-based. ... Family and >> social life also offer numerous other occasions to consume drinks for >> pleasure." [Larousse, "Drink" entry] >> >> -- >> -- >> You received this message because you are subscribed to the Google >> Groups "Clojure" group. >> To post to this group, send email to clojure@googlegroups.com >> Note that posts from new members are moderated - please be patient with >> your first post. >> To unsubscribe from this group, send email to >> clojure+unsubscr...@googlegroups.com >> For more options, visit this group at >> http://groups.google.com/group/clojure?hl=en >> --- >> You received this message because you are subscribed to the Google Groups >> "Clojure" group. >> To unsubscribe from this group and stop receiving emails from it, send an >> email to clojure+unsubscr...@googlegroups.com. >> For more options, visit https://groups.google.com/groups/opt_out. >> >> >> > > -- > -- > You received this message because you are subscribed to the Google > Groups "Clojure" group. > To post to this group, send email to clojure@googlegroups.com > Note that posts from new members are moderated - please be patient with > your first post. > To unsubscribe from this group, send email to > clojure+unsubscr...@googlegroups.com > For more options, visit this group at > http://groups.google.com/group/clojure?hl=en > --- > You received this message because you are subscribed to the Google Groups > "Clojure" group. > To unsubscribe from this group and stop receiving emails from it, send an > email to clojure+unsubscr...@googlegroups.com. > For more options, visit https://groups.google.com/groups/opt_out. > > > -- -- You received this message because you are subscribed to the Google Groups "Clojure" group. To post to this group, send email to clojure@googlegroups.com Note that posts from new members are moderated - please be patient with your first post. To unsubscribe from this group, send email to clojure+unsubscr...@googlegroups.com For more options, visit this group at http://groups.google.com/group/clojure?hl=en --- You received this message because you are subscribed to the Google Groups "Clojure" group. To unsubscribe from this group and stop receiving emails from it, send an email to clojure+unsubscr...@googlegroups.com. For more options, visit https://groups.google.com/groups/opt_out.
