Trouble Running Chime Application outside for repl
Have a program I want to run every 5 seconds unless the previous run is not finished then I want it to wait on the previous run to finish. To do this I used a library called Chime (https://github.com/jarohen/chime). *The Problem* is the program works perfectly in repl but will not work outside out of it? lein trampoline run - does not work lein run - does not work Java -jar (point to the uberjar file) - does not work lein repl then running main -does work Here is a small example I put together https://gitlab.com/csnyder/chime-test/tree/master -- You received this message because you are subscribed to the Google Groups "Clojure" group. To post to this group, send email to clojure@googlegroups.com Note that posts from new members are moderated - please be patient with your first post. To unsubscribe from this group, send email to clojure+unsubscr...@googlegroups.com For more options, visit this group at http://groups.google.com/group/clojure?hl=en --- You received this message because you are subscribed to the Google Groups "Clojure" group. To unsubscribe from this group and stop receiving emails from it, send an email to clojure+unsubscr...@googlegroups.com. For more options, visit https://groups.google.com/d/optout.
Re: Trouble Running Chime Application outside for repl
Hi Chris, Thanks for posting an example - unfortunately I can't get onto it at the moment because it seems gitlab is down :( FWIW, I've seen a similar issue before where the schedule will run for a minute or so, then the process will exit successfully (even though you'd like it to continue) - is this what you're seeing? If so, this was caused by there being no non-daemon threads left running in the JVM, so the JVM exited - we worked around that by adding `@(promise)` to the end of our `-main` function, which had the effect of indefinitely pausing the main thread, and allowing the schedule to continue. If not, I'll try gitlab again in a bit! James On Monday, November 21, 2016 at 3:03:45 PM UTC, Chris Snyder wrote: > > > Have a program I want to run every 5 seconds unless the previous run is > not finished then I want it to wait on the previous run to finish. > To do this I used a library called Chime (https://github.com/jarohen/chime). > > > *The Problem* is the program works perfectly in repl but will not work > outside out of it? > > lein trampoline run - does not work > lein run - does not work > Java -jar (point to the uberjar file) - does not work > > lein repl then running main -does work > > Here is a small example I put together > https://gitlab.com/csnyder/chime-test/tree/master > -- You received this message because you are subscribed to the Google Groups "Clojure" group. To post to this group, send email to clojure@googlegroups.com Note that posts from new members are moderated - please be patient with your first post. To unsubscribe from this group, send email to clojure+unsubscr...@googlegroups.com For more options, visit this group at http://groups.google.com/group/clojure?hl=en --- You received this message because you are subscribed to the Google Groups "Clojure" group. To unsubscribe from this group and stop receiving emails from it, send an email to clojure+unsubscr...@googlegroups.com. For more options, visit https://groups.google.com/d/optout.
Re: Trouble Running Chime Application outside for repl
That Solved it. Thanks for your help. added `@(promise)` and works like a charm On Monday, November 21, 2016 at 10:14:06 AM UTC-5, James Henderson wrote: > > Hi Chris, > > Thanks for posting an example - unfortunately I can't get onto it at the > moment because it seems gitlab is down :( > > FWIW, I've seen a similar issue before where the schedule will run for a > minute or so, then the process will exit successfully (even though you'd > like it to continue) - is this what you're seeing? If so, this was caused > by there being no non-daemon threads left running in the JVM, so the JVM > exited - we worked around that by adding `@(promise)` to the end of our > `-main` function, which had the effect of indefinitely pausing the main > thread, and allowing the schedule to continue. > > If not, I'll try gitlab again in a bit! > > James > > On Monday, November 21, 2016 at 3:03:45 PM UTC, Chris Snyder wrote: >> >> >> Have a program I want to run every 5 seconds unless the previous run is >> not finished then I want it to wait on the previous run to finish. >> To do this I used a library called Chime ( >> https://github.com/jarohen/chime). >> >> *The Problem* is the program works perfectly in repl but will not work >> outside out of it? >> >> lein trampoline run - does not work >> lein run - does not work >> Java -jar (point to the uberjar file) - does not work >> >> lein repl then running main -does work >> >> Here is a small example I put together >> https://gitlab.com/csnyder/chime-test/tree/master >> > -- You received this message because you are subscribed to the Google Groups "Clojure" group. To post to this group, send email to clojure@googlegroups.com Note that posts from new members are moderated - please be patient with your first post. To unsubscribe from this group, send email to clojure+unsubscr...@googlegroups.com For more options, visit this group at http://groups.google.com/group/clojure?hl=en --- You received this message because you are subscribed to the Google Groups "Clojure" group. To unsubscribe from this group and stop receiving emails from it, send an email to clojure+unsubscr...@googlegroups.com. For more options, visit https://groups.google.com/d/optout.
Re: Help me understand what part of this code is slow, and how to make it faster?
I experimented with this a lot, and took everyone's advice, and this is the
fastest I got, even faster then Michael Gardner's answer.
(deftype point5 [^long i ^long j]
Object
(equals [this that] (and (= (.i this) (.i ^point5 that))
(= (.j this) (.j ^point5 that
(hashCode [this] (+ (.i this) (* 4000 (.j this)
(defn iter-neighbors5 [f ^point5 p]
(let [i ^long (.i p)
j ^long (.j p)]
(f (->point5 (dec i) j))
(f (->point5 (inc i) j))
(f (->point5 i (dec j)))
(f (->point5 i (inc j)
(defn nth5 [n p]
(loop [n ^long n
s1 (HashSet. [p])
s2 (HashSet.)]
(if (zero? n)
s1
(let [s0 (HashSet.)]
(letfn [(add [p]
(when (not (or (.contains s1 p) (.contains s2 p)))
(.add s0 p)))]
(doseq [p s1] (iter-neighbors5 add p))
(recur (dec n) s0 s1))
This is basically the closest I can get to the F#, Rust and OCaml solution.
I create a minimal class where I overload equals and hashCode using the
same logic the F# solution uses. It gives me equal performance to F# on my
machine: 3s.
Here's the run down from slowest to fastest with a size of 100 using
criterium. All of them I did my best to avoid reflection and boxed numeric
and had enabled uncheck math. Granted I did not find that boxing and
unchecked math really provided a drastic improvement in my case.:
1. Using a record to hold the point with a java HashSet - 826ms
2. Using a struct to hold the point with a java HashSet - 246ms
3. Using a volatile with a Clojure HashSet and vectors for point - 50ms
4. Using an atom with a Clojure HashSet and vectors for point - 51ms
5. Using java.awt Point with a java HashSet and add as a closure - 41ms
6. Using vectors with a java HashSet, but with add done inside doseq,
without a closure - 11ms
7. Using vectors with a java HashSet and add as a closure - 9ms
8. Using java.awt Point with a java HashSet, but with add done inside
doseq, without a closure (Michael Gardner's solution) - 7.5ms
9. Using deftype with overloaded equals and hashCode as points with a
java HashSet, but with add done inside doseq, without a closure - 6ms
10. Using deftype with overloaded equals and hashCode as points with a
java HashSet and add as a closure - 4ms
This is what I gathered from my experiments:
- Using java's mutable HashSet instead of Clojure's persistent hashset
gave a significant speed improvement. From 30s to 5s. This was the most
significant speed boost.
- Records are really slow, probably because of the time to instantiate
them. So slow, that even though it was using a mutable HashSet, it was
still slower then Clojure sets with vectors.
- Similarly, structs are also pretty slow, which I'd guess is also
because of instantiating them. Though they were faster then Records.
- Volatile did not noticeably improve performance compared to Atom, but
I learned about them today, did not know Clojure had that.
- Unsafe Math and type hinting numeric to get rid of boxing did not
noticeably improve performance, but I learned about it today too. Speaking
of which, is ^long faster then ^int?
- Vectors were as fast as java.awt.Point. Though for some reason,
putting the java.awt.Point inside a closure slowed the whole thing down
quite a bit. Where as without it, using a closure actually performed
faster. Maybe I did something wrong here, since I find that a little
surprising and confusing.
- Defining my own type optimized for my specific use-case was the
fastest. deftypes are lightweight, and instantiate quickly, and with the
custom equals and hashCode tailored to the problem, they performed best,
matching F# and beating out OCaml and Rust.
On Tuesday, 15 November 2016 19:39:43 UTC-8, Didier wrote:
>
> Hey all,
>
> I came upon a benchmark of F#, Rust and OCaml, where F# performs much
> faster then the other two. I decided for fun to try and port it to Clojure
> to see how Clojure does. Benchmark link:
> https://github.com/c-cube/hashset_benchs
>
> This is my code for it:
> https://gist.github.com/didibus/1fd4c00b69d927745fbce3dcd7ca461a
>
> (ns hash-set-bench
> "A Benchmark I modified to Clojure from:
>https://github.com/c-cube/hashset_benchs";)
>
> (defn iterNeighbors [f [i j]]
> (f [(dec i) j])
> (f [(inc i) j])
> (f [i (dec j)])
> (f [i (inc j)]))
>
> (defn nth* [n p]
> (loop [n n s1 #{p} s2 #{}]
> (if (= n 0)
> s1
> (let [s0 (atom #{})]
> (letfn [(add [p]
> (when (not (or (contains? s1 p) (contains? s2 p)))
>(reset! s0 (conj @s0 p]
>(doseq [p s1] (iterNeighbors add p))
>(recur (dec n) @s0 s1))
>
> #_(printf "result is %d" (count (time (nth* 2000 [0 0]
>
> And here's the F# code:
> https://github.com/c-cube
Re: Help me understand what part of this code is slow, and how to make it faster?
Not sure which version of Clojure you are using, but in all versions up to the latest 'official' release, Clojure 1.8.0, records have their hash value calculated every time they are needed, with no caching of the calculated value. The hash value is needed every time an operation is done in a Clojure set, and probably a java HashSet, too. The latest alpha release, Clojure 1.9.0-alpha14, contains a performance improvement where records cache their hash value after it is first calculated. http://dev.clojure.org/jira/browse/CLJ-1224 I don't know if that change would make your version of the code using records and Java HashSet's competitive with your fastest code or not, but you may wish to try it. Andy On Mon, Nov 21, 2016 at 12:05 PM, Didier wrote: > I experimented with this a lot, and took everyone's advice, and this is > the fastest I got, even faster then Michael Gardner's answer. > > (deftype point5 [^long i ^long j] > Object > (equals [this that] (and (= (.i this) (.i ^point5 that)) >(= (.j this) (.j ^point5 that > (hashCode [this] (+ (.i this) (* 4000 (.j this) > > (defn iter-neighbors5 [f ^point5 p] > (let [i ^long (.i p) > j ^long (.j p)] > (f (->point5 (dec i) j)) > (f (->point5 (inc i) j)) > (f (->point5 i (dec j))) > (f (->point5 i (inc j) > > (defn nth5 [n p] > (loop [n ^long n > s1 (HashSet. [p]) > s2 (HashSet.)] > (if (zero? n) > s1 > (let [s0 (HashSet.)] > (letfn [(add [p] > (when (not (or (.contains s1 p) (.contains s2 p))) >(.add s0 p)))] >(doseq [p s1] (iter-neighbors5 add p)) >(recur (dec n) s0 s1)) > > This is basically the closest I can get to the F#, Rust and OCaml > solution. I create a minimal class where I overload equals and hashCode > using the same logic the F# solution uses. It gives me equal performance to > F# on my machine: 3s. > > Here's the run down from slowest to fastest with a size of 100 using > criterium. All of them I did my best to avoid reflection and boxed numeric > and had enabled uncheck math. Granted I did not find that boxing and > unchecked math really provided a drastic improvement in my case.: > >1. Using a record to hold the point with a java HashSet - 826ms >2. Using a struct to hold the point with a java HashSet - 246ms >3. Using a volatile with a Clojure HashSet and vectors for point - 50ms >4. Using an atom with a Clojure HashSet and vectors for point - 51ms >5. Using java.awt Point with a java HashSet and add as a closure - 41ms >6. Using vectors with a java HashSet, but with add done inside doseq, >without a closure - 11ms >7. Using vectors with a java HashSet and add as a closure - 9ms >8. Using java.awt Point with a java HashSet, but with add done inside >doseq, without a closure (Michael Gardner's solution) - 7.5ms >9. Using deftype with overloaded equals and hashCode as points with a >java HashSet, but with add done inside doseq, without a closure - 6ms >10. Using deftype with overloaded equals and hashCode as points with a >java HashSet and add as a closure - 4ms > > This is what I gathered from my experiments: > >- Using java's mutable HashSet instead of Clojure's persistent hashset >gave a significant speed improvement. From 30s to 5s. This was the most >significant speed boost. >- Records are really slow, probably because of the time to instantiate >them. So slow, that even though it was using a mutable HashSet, it was >still slower then Clojure sets with vectors. >- Similarly, structs are also pretty slow, which I'd guess is also >because of instantiating them. Though they were faster then Records. >- Volatile did not noticeably improve performance compared to Atom, >but I learned about them today, did not know Clojure had that. >- Unsafe Math and type hinting numeric to get rid of boxing did not >noticeably improve performance, but I learned about it today too. Speaking >of which, is ^long faster then ^int? >- Vectors were as fast as java.awt.Point. Though for some reason, >putting the java.awt.Point inside a closure slowed the whole thing down >quite a bit. Where as without it, using a closure actually performed >faster. Maybe I did something wrong here, since I find that a little >surprising and confusing. >- Defining my own type optimized for my specific use-case was the >fastest. deftypes are lightweight, and instantiate quickly, and with the >custom equals and hashCode tailored to the problem, they performed best, >matching F# and beating out OCaml and Rust. > > > On Tuesday, 15 November 2016 19:39:43 UTC-8, Didier wrote: >> >> Hey all, >> >> I came upon a benchmark of F#, Rust and OCaml, where F# performs much >> faster then the other two. I decided for fun to try and port it to Clojure >> to se
[ANN] maxiphobe 0.0.1 – Persistent Meldable Priority Queues
Hi, I am pleased to announce the initial release of maxiphobe, a meldable priority queue library based on Okasaki's maxiphobic heaps (see Okasaki, "Alternatives to Two Classic Data Structures"). Maxiphobic heaps are a strikingly simple purely functional approach to implementing priority queues that offers very reasonable performance given the available operations. https://github.com/michalmarczyk/maxiphobe https://clojars.org/maxiphobe Maxiphobe priority queues are, in effect, persistent lists that maintain a non-decreasing order of elements, as determined either by Clojure's default comparator or a custom comparator passed in by the user. Additionally, instances using the same comparator can be "melded" into a single priority queue containing all the elements of the inputs. (require '[maxiphobe.core :as pq]) (pq/pq [0 -1 3 4 2]) ;= (-1 0 2 3 4) (pq/pqueue 0 -1 3 4 2) ;= (-1 0 2 3 4) (pq/pq-by > [0 -1 3 4 2]) ;= (4 3 2 0 -1) (pq/pqueue-by > 0 -1 3 4 2) ;= (4 3 2 0 -1) (peek (pq/pqueue-by > 0 -1 3 4 2)) ; or first ;= 4 (pop (pq/pqueue-by > 0 -1 3 4 2)); or next ;= (3 2 0 -1) (pq/meld (pq/pqueue -1 3 2 8) (pq/pqueue 0 -3 2 4)) ;= (-3 -1 0 2 2 3 4 8) To include maxiphobe in your project: [maxiphobe "0.0.1"] maxiphobe maxiphobe 0.0.1 compile "maxiphobe:maxiphobe:0.0.1" See below for some benchmark results. Cheers, Michał Benchmarks == 1. Compare basic operations to sorted-set-as-PQ and to regular (FIFO) peristent queues to get a high-level picture of maxiphobe performance. FIFO queue operations are included to provide a point of reference – they are, of course, completely different semantically. (let [pq (pq/pq (range 1000)) s (apply sorted-set (range 1000)) q (into clojure.lang.PersistentQueue/EMPTY (range 1000))] (assert (= (seq s) pq q)) (println "peek") (c/quick-bench (peek pq)) (c/quick-bench (first s)) (c/quick-bench (peek q)) (println "pop") (c/quick-bench (pop pq)) (c/quick-bench (disj s (first s))) (c/quick-bench (pop q)) (let [pq (nth (iterate pop pq) 500) s (apply sorted-set pq) q (nth (iterate pop q) 500)] (assert (= (seq s) pq q)) (println "pop2") (c/quick-bench (pop pq)) (c/quick-bench (disj s (first s))) (c/quick-bench (pop q peek ;;; maxiphobe PQ: ;;; (peek pq) Evaluation count : 32624694 in 6 samples of 5437449 calls. Execution time mean : -2.422742 ns Execution time std-deviation : 0.630313 ns Execution time lower quantile : -2.934682 ns ( 2.5%) Execution time upper quantile : -1.681600 ns (97.5%) Overhead used : 21.406378 ns ;;; sorted set: ;;; (first s) Evaluation count : 1944576 in 6 samples of 324096 calls. Execution time mean : 3.262937 µs Execution time std-deviation : 440.634979 ns Execution time lower quantile : 2.728010 µs ( 2.5%) Execution time upper quantile : 3.753706 µs (97.5%) Overhead used : 21.406378 ns ;;; queue ;;; (peek q) Evaluation count : 22712976 in 6 samples of 3785496 calls. Execution time mean : 7.395679 ns Execution time std-deviation : 0.615558 ns Execution time lower quantile : 6.766551 ns ( 2.5%) Execution time upper quantile : 8.050620 ns (97.5%) Overhead used : 21.406378 ns pop ;;; maxiphobe PQ: ;;; (pop pq) Evaluation count : 1107840 in 6 samples of 184640 calls. Execution time mean : 2.143321 µs Execution time std-deviation : 1.064519 µs Execution time lower quantile : 362.028152 ns ( 2.5%) Execution time upper quantile : 3.175097 µs (97.5%) Overhead used : 21.406378 ns ;;; sorted set: ;;; (disj s (first s)) Evaluation count : 508608 in 6 samples of 84768 calls. Execution time mean : 10.498305 µs Execution time std-deviation : 2.658746 µs Execution time lower quantile : 6.260925 µs ( 2.5%) Execution time upper quantile : 12.859465 µs (97.5%) Overhead used : 21.406378 ns ;;; queue: ;;; (pop q) Evaluation count : 8807040 in 6 samples of 1467840 calls. Execution time mean : 730.736239 ns Execution time std-deviation : 122.924195 ns Execution time lower quantile : 561.778923 ns ( 2.5%) Execution time upper quantile : 839.133234 ns (97.5%) Overhead used : 21.406378 ns pop2 ;;; maxiphobe PQ: ;;; (pop pq) Evaluation count : 1816404 in 6 samples of 302734 calls. Execution time mean : 2.874078 µs Execution time std-deviation : 310.794245 ns Execution time lower quantile : 2.614476 µs ( 2.5%) Execution time upper quantile : 3.233927 µs (97.5%) Overhead used : 21.406378 ns ;;; sorted set: ;;; (disj s (first s)) Evaluation count : 554400 in 6 samples of 92400 calls. Execution time mean : 11.149426 µs Execution time std-deviation : 1.675888 µs Execution time lower qua
Socket server and PuTTY
Hi, When I connect to the socket server with PuTTY, what are the correct connection parameters? Telnet mode gives strange error messages, raw mode seems to work but when doing (find-doc "string") the output has strange blanks and carriage returns. Help is appreciated. With kind regards Thomas -- You received this message because you are subscribed to the Google Groups "Clojure" group. To post to this group, send email to clojure@googlegroups.com Note that posts from new members are moderated - please be patient with your first post. To unsubscribe from this group, send email to clojure+unsubscr...@googlegroups.com For more options, visit this group at http://groups.google.com/group/clojure?hl=en --- You received this message because you are subscribed to the Google Groups "Clojure" group. To unsubscribe from this group and stop receiving emails from it, send an email to clojure+unsubscr...@googlegroups.com. For more options, visit https://groups.google.com/d/optout.
Re: Help me understand what part of this code is slow, and how to make it faster?
> On Nov 21, 2016, at 3:05 PM, Didier wrote: > > I experimented with this a lot, and took everyone's advice, and this is the > fastest I got, even faster then Michael Gardner's answer. > > (deftype point5 [^long i ^long j] > Object > (equals [this that] (and (= (.i this) (.i ^point5 that)) >(= (.j this) (.j ^point5 that > (hashCode [this] (+ (.i this) (* 4000 (.j this) Java equals and hashCode are notoriously tricky to get right. Equals should handle any kind of “that” so you typically need to test instance? (basically, instanceOf in Java). Here are a couple of reference links: http://stackoverflow.com/questions/27581/what-issues-should-be-considered-when-overriding-equals-and-hashcode-in-java http://www.angelikalanger.com/Articles/JavaSolutions/SecretsOfEquals/Equals.html In this case, I suggest you try something like this: (equals [this that] (or (identical? this that) (and (instance? point5 that) (= (.i this) (.i ^point5 that)) (= (.j this) (.j ^point5 that) It might be a little slower but it should be safer. If you’re out for speed, you could also try unchecked-add, unchecked-multiply, unchecked-inc, and unchecked-dec where it's safe to ignore the chance of overflows. I mention these only because you’re trying to compete on benchmarks. Most of the time, you shouldn’t use them. -- You received this message because you are subscribed to the Google Groups "Clojure" group. To post to this group, send email to clojure@googlegroups.com Note that posts from new members are moderated - please be patient with your first post. To unsubscribe from this group, send email to clojure+unsubscr...@googlegroups.com For more options, visit this group at http://groups.google.com/group/clojure?hl=en --- You received this message because you are subscribed to the Google Groups "Clojure" group. To unsubscribe from this group and stop receiving emails from it, send an email to clojure+unsubscr...@googlegroups.com. For more options, visit https://groups.google.com/d/optout.
Re: Help me understand what part of this code is slow, and how to make it faster?
I tried it with the safe equals, and it is slightly slower, but still
faster then all others at 4.5ms. The non safe equals gives me 4s. Though
this is now within my error margin. If ire-run quick-bench, I sometime get
a mean equal for each, so I don't think the instance check adds that much
overhead if any at all.
@miner: Doesn't using the flag (set! *unchecked-math* :warn-on-boxed) gives
me unchecked math automatically? I was under the impression that +, -, /, *
etc. would all now perform in an equal way to unchecked-add, etc. If not,
what is the difference?
@Andy: I tried with the "1.9.0-alpha14" version, and Records were still
just as slow as with "1.8.0". Maybe I'm using them wrong.
On Tuesday, 15 November 2016 19:39:43 UTC-8, Didier wrote:
>
> Hey all,
>
> I came upon a benchmark of F#, Rust and OCaml, where F# performs much
> faster then the other two. I decided for fun to try and port it to Clojure
> to see how Clojure does. Benchmark link:
> https://github.com/c-cube/hashset_benchs
>
> This is my code for it:
> https://gist.github.com/didibus/1fd4c00b69d927745fbce3dcd7ca461a
>
> (ns hash-set-bench
> "A Benchmark I modified to Clojure from:
>https://github.com/c-cube/hashset_benchs";)
>
> (defn iterNeighbors [f [i j]]
> (f [(dec i) j])
> (f [(inc i) j])
> (f [i (dec j)])
> (f [i (inc j)]))
>
> (defn nth* [n p]
> (loop [n n s1 #{p} s2 #{}]
> (if (= n 0)
> s1
> (let [s0 (atom #{})]
> (letfn [(add [p]
> (when (not (or (contains? s1 p) (contains? s2 p)))
>(reset! s0 (conj @s0 p]
>(doseq [p s1] (iterNeighbors add p))
>(recur (dec n) @s0 s1))
>
> #_(printf "result is %d" (count (time (nth* 2000 [0 0]
>
> And here's the F# code:
> https://github.com/c-cube/hashset_benchs/blob/master/neighbors2.fsx
>
> Currently, this takes about 30s in Clojure, while it only takes around 3s
> for OCaml, Rust and F#.
>
> From what I see, the differences between my code and theirs are:
>
>- Lack of a Point struct, I'm just using a vector.
>- They use a mutable set, I don't.
>- They overrode Hashing for their point struct, as well as equality. I
>rely on Clojure's default hashing, and vector equality.
>
> I'm not sure if any of these things should really impact performance that
> much though. And what I could do in Clojure if I wanted to improve it.
>
>
> Any Help?
>
>
> Thanks.
>
--
You received this message because you are subscribed to the Google
Groups "Clojure" group.
To post to this group, send email to clojure@googlegroups.com
Note that posts from new members are moderated - please be patient with your
first post.
To unsubscribe from this group, send email to
clojure+unsubscr...@googlegroups.com
For more options, visit this group at
http://groups.google.com/group/clojure?hl=en
---
You received this message because you are subscribed to the Google Groups
"Clojure" group.
To unsubscribe from this group and stop receiving emails from it, send an email
to clojure+unsubscr...@googlegroups.com.
For more options, visit https://groups.google.com/d/optout.
Re: Help me understand what part of this code is slow, and how to make it faster?
Some further optimizations for a factor of ~2.3 speed-up over nth5 as copy
& pasted from upthread (6.713742 ms → 2.897630 ms) in
(c/quick-bench (nth-shell 100 (point. 0 0)))
(1) java.util.HashSet has a ctor that takes initial capacity of the set as
an int. Passing in (* 4 (.size s1)) when constructing s0 – (HashSet. (* 4
(.size s1)) – gives me a fair speed-up on its own.
(2) Also, (java.util.HashSet. [p]) is a reflective call – replacing that
with (doto (java.util.HashSet.) (.add p)) seems to result in a tiny, but
measurable speed-up as well (to 5.245931 ms).
(3) Using an iterator results in a good speed-up, particularly in a
size-based loop (counting down from (.size current-set) rather than calling
(.hasNext current-iterator)).
(4) Finally, inlining all the visiting and switching to direct ctor calls
also helped.
Final code:
(deftype point [^long i ^long j]
Object
(equals [this that]
(and (= (.i this) (.i ^point that))
(= (.j this) (.j ^point that
(hashCode [this]
(+ (.i this) (* 4000 (.j this)
(defn nth-shell [^long n p]
(loop [n n
s1 (doto (HashSet.) (.add p))
s2 (HashSet.)]
(if (zero? n)
s1
(let [s0 (HashSet. (* 4 (.size s1)))
i1 (.iterator s1)]
(dotimes [_ (.size s1)]
(let [^point p (.next i1)
i ^long (.i p)
j ^long (.j p)
p1 (point. (dec i) j)
p2 (point. (inc i) j)
p3 (point. i (dec j))
p4 (point. i (inc j))]
(if-not (or (.contains s1 p1) (.contains s2 p1))
(.add s0 p1))
(if-not (or (.contains s1 p2) (.contains s2 p2))
(.add s0 p2))
(if-not (or (.contains s1 p3) (.contains s2 p3))
(.add s0 p3))
(if-not (or (.contains s1 p4) (.contains s2 p4))
(.add s0 p4
(recur (dec n) s0 s1)
Also, to check that this still outputs the same neighbours the original
impl (nth*) does:
(defn persistent-shell [shell]
(persistent!
(reduce (fn [out ^point p]
(conj! out [(.-i p) (.-j p)]))
(transient #{})
shell)))
(= (sort (persistent-shell (nth-shell 500 (point. 0 0
(sort (nth* 500 [0 0])))
Cheers,
Michał
On 22 November 2016 at 02:03, Didier wrote:
> I tried it with the safe equals, and it is slightly slower, but still
> faster then all others at 4.5ms. The non safe equals gives me 4s. Though
> this is now within my error margin. If ire-run quick-bench, I sometime get
> a mean equal for each, so I don't think the instance check adds that much
> overhead if any at all.
>
> @miner: Doesn't using the flag (set! *unchecked-math* :warn-on-boxed)
> gives me unchecked math automatically? I was under the impression that +,
> -, /, * etc. would all now perform in an equal way to unchecked-add, etc.
> If not, what is the difference?
>
> @Andy: I tried with the "1.9.0-alpha14" version, and Records were still
> just as slow as with "1.8.0". Maybe I'm using them wrong.
>
> On Tuesday, 15 November 2016 19:39:43 UTC-8, Didier wrote:
>
>> Hey all,
>>
>> I came upon a benchmark of F#, Rust and OCaml, where F# performs much
>> faster then the other two. I decided for fun to try and port it to Clojure
>> to see how Clojure does. Benchmark link: https://github.com/c-cube/hash
>> set_benchs
>>
>> This is my code for it: https://gist.github.com/didibu
>> s/1fd4c00b69d927745fbce3dcd7ca461a
>>
>> (ns hash-set-bench
>> "A Benchmark I modified to Clojure from:
>>https://github.com/c-cube/hashset_benchs";)
>>
>> (defn iterNeighbors [f [i j]]
>> (f [(dec i) j])
>> (f [(inc i) j])
>> (f [i (dec j)])
>> (f [i (inc j)]))
>>
>> (defn nth* [n p]
>> (loop [n n s1 #{p} s2 #{}]
>> (if (= n 0)
>> s1
>> (let [s0 (atom #{})]
>> (letfn [(add [p]
>> (when (not (or (contains? s1 p) (contains? s2 p)))
>>(reset! s0 (conj @s0 p]
>>(doseq [p s1] (iterNeighbors add p))
>>(recur (dec n) @s0 s1))
>>
>> #_(printf "result is %d" (count (time (nth* 2000 [0 0]
>>
>> And here's the F# code: https://github.com/c-cube/hash
>> set_benchs/blob/master/neighbors2.fsx
>>
>> Currently, this takes about 30s in Clojure, while it only takes around 3s
>> for OCaml, Rust and F#.
>>
>> From what I see, the differences between my code and theirs are:
>>
>>- Lack of a Point struct, I'm just using a vector.
>>- They use a mutable set, I don't.
>>- They overrode Hashing for their point struct, as well as equality.
>>I rely on Clojure's default hashing, and vector equality.
>>
>> I'm not sure if any of these things should really impact performance that
>> much though. And what I could do in Clojure if I wanted to improve it.
>>
>>
>> Any Help?
>>
>>
>> Thanks.
>>
> --
> You received this message because you are subscribed to the Google
> Groups "Clojure" group.
> To post to this group, sen
Re: Help me understand what part of this code is slow, and how to make it faster?
PS. Results for the original input on my box. Going by the the timings
posted above, yours is rather beefier, so this is probably faster than the
current F# version.
(c/quick-bench (nth-shell 2000 (point. 0 0)))
Evaluation count : 6 in 6 samples of 1 calls.
Execution time mean : 2.956519 sec
Execution time std-deviation : 22.423970 ms
Execution time lower quantile : 2.930938 sec ( 2.5%)
Execution time upper quantile : 2.978283 sec (97.5%)
Overhead used : 21.423788 ns
On 22 November 2016 at 05:21, Michał Marczyk
wrote:
> Some further optimizations for a factor of ~2.3 speed-up over nth5 as copy
> & pasted from upthread (6.713742 ms → 2.897630 ms) in
>
> (c/quick-bench (nth-shell 100 (point. 0 0)))
>
> (1) java.util.HashSet has a ctor that takes initial capacity of the set as
> an int. Passing in (* 4 (.size s1)) when constructing s0 – (HashSet. (* 4
> (.size s1)) – gives me a fair speed-up on its own.
>
> (2) Also, (java.util.HashSet. [p]) is a reflective call – replacing that
> with (doto (java.util.HashSet.) (.add p)) seems to result in a tiny, but
> measurable speed-up as well (to 5.245931 ms).
>
> (3) Using an iterator results in a good speed-up, particularly in a
> size-based loop (counting down from (.size current-set) rather than calling
> (.hasNext current-iterator)).
>
> (4) Finally, inlining all the visiting and switching to direct ctor calls
> also helped.
>
> Final code:
>
> (deftype point [^long i ^long j]
> Object
> (equals [this that]
> (and (= (.i this) (.i ^point that))
> (= (.j this) (.j ^point that
> (hashCode [this]
> (+ (.i this) (* 4000 (.j this)
>
> (defn nth-shell [^long n p]
> (loop [n n
> s1 (doto (HashSet.) (.add p))
> s2 (HashSet.)]
> (if (zero? n)
> s1
> (let [s0 (HashSet. (* 4 (.size s1)))
> i1 (.iterator s1)]
> (dotimes [_ (.size s1)]
> (let [^point p (.next i1)
> i ^long (.i p)
> j ^long (.j p)
> p1 (point. (dec i) j)
> p2 (point. (inc i) j)
> p3 (point. i (dec j))
> p4 (point. i (inc j))]
> (if-not (or (.contains s1 p1) (.contains s2 p1))
> (.add s0 p1))
> (if-not (or (.contains s1 p2) (.contains s2 p2))
> (.add s0 p2))
> (if-not (or (.contains s1 p3) (.contains s2 p3))
> (.add s0 p3))
> (if-not (or (.contains s1 p4) (.contains s2 p4))
> (.add s0 p4
> (recur (dec n) s0 s1)
>
> Also, to check that this still outputs the same neighbours the original
> impl (nth*) does:
>
> (defn persistent-shell [shell]
> (persistent!
> (reduce (fn [out ^point p]
> (conj! out [(.-i p) (.-j p)]))
> (transient #{})
> shell)))
>
> (= (sort (persistent-shell (nth-shell 500 (point. 0 0
>(sort (nth* 500 [0 0])))
>
> Cheers,
> Michał
>
>
> On 22 November 2016 at 02:03, Didier wrote:
>
>> I tried it with the safe equals, and it is slightly slower, but still
>> faster then all others at 4.5ms. The non safe equals gives me 4s. Though
>> this is now within my error margin. If ire-run quick-bench, I sometime get
>> a mean equal for each, so I don't think the instance check adds that much
>> overhead if any at all.
>>
>> @miner: Doesn't using the flag (set! *unchecked-math* :warn-on-boxed)
>> gives me unchecked math automatically? I was under the impression that +,
>> -, /, * etc. would all now perform in an equal way to unchecked-add, etc.
>> If not, what is the difference?
>>
>> @Andy: I tried with the "1.9.0-alpha14" version, and Records were still
>> just as slow as with "1.8.0". Maybe I'm using them wrong.
>>
>> On Tuesday, 15 November 2016 19:39:43 UTC-8, Didier wrote:
>>
>>> Hey all,
>>>
>>> I came upon a benchmark of F#, Rust and OCaml, where F# performs much
>>> faster then the other two. I decided for fun to try and port it to Clojure
>>> to see how Clojure does. Benchmark link: https://github.com/c-cube/hash
>>> set_benchs
>>>
>>> This is my code for it: https://gist.github.com/didibu
>>> s/1fd4c00b69d927745fbce3dcd7ca461a
>>>
>>> (ns hash-set-bench
>>> "A Benchmark I modified to Clojure from:
>>>https://github.com/c-cube/hashset_benchs";)
>>>
>>> (defn iterNeighbors [f [i j]]
>>> (f [(dec i) j])
>>> (f [(inc i) j])
>>> (f [i (dec j)])
>>> (f [i (inc j)]))
>>>
>>> (defn nth* [n p]
>>> (loop [n n s1 #{p} s2 #{}]
>>> (if (= n 0)
>>> s1
>>> (let [s0 (atom #{})]
>>> (letfn [(add [p]
>>> (when (not (or (contains? s1 p) (contains? s2 p)))
>>>(reset! s0 (conj @s0 p]
>>>(doseq [p s1] (iterNeighbors add p))
>>>(recur (dec n) @s0 s1))
>>>
>>> #_(printf "result is %d" (count (time (nth* 2000 [0 0]
>>>
>>> And here's the F# code: https://github.com/c-cube/hash
>>> set_benchs/blob/mast
