Re: I need help tracking down a performance problem.

2009-04-01 Thread Vincent Foley
No change. I think made a small script that spends a large amount of time in java.lang.reflect.Array.setInt too: (set! *warn-on-reflection* true) (time (dotimes [_ 1] (let [#^ints arr (int-array 200)] (dotimes [i 200] (aset-int arr i i) So maybe I should try to see what

Re: I need help tracking down a performance problem.

2009-03-31 Thread David Nolen
As as a side note, putting type hints into the body of the macro expansion doesn't do anything. These will be discarded. You will need to coerce types via other methods, investigating how ints is implemented in core.clj is a good place to start. On Tue, Mar 31, 2009 at 11:04 PM, David Nolen wrote:

Re: I need help tracking down a performance problem.

2009-03-31 Thread David Nolen
15.1% 0 + 1711java.lang.reflect.Array.setInt Is definitely pointing at the aset-int as being the time gobbler, I think the expression in the macro should be this (aset-int (ints arr#) i# (int (~mask-fn (. buf# (~get-fn) to be extra safe. On Tue, Mar 31, 2009 at 10:00 PM, Vincent F

Re: I need help tracking down a performance problem.

2009-03-31 Thread Vincent Foley
I tried it just now; it made no difference. Nevertheless, thank you for you help and time! On Mar 31, 9:38 pm, David Nolen wrote: > Did you try > (int (mask8 (. buf__2572__auto__ (get > > ? > > Your macro should like this: > > (defmacro make-reader >   [get-fn mask-fn] >   `(fn [#^ByteBuffe

Re: I need help tracking down a performance problem.

2009-03-31 Thread David Nolen
Did you try (int (mask8 (. buf__2572__auto__ (get ? Your macro should like this: (defmacro make-reader [get-fn mask-fn] `(fn [#^ByteBuffer buf# len#] (if (= len# 1) (~mask-fn (. buf# (~get-fn))) (let [#^"[I" arr# (int-array len#)] (dotimes [i# len#]

Re: I need help tracking down a performance problem.

2009-03-31 Thread Vincent Foley
I tried surrounding the call to the (. buf# (get)) method and putting the coercion directly inside the mask8 and mask16 functions. Neither worked. I want to mention at this point that I have *warn-on- reflection* set to true for the little script that uses the library and it doesn't report any c

Re: I need help tracking down a performance problem.

2009-03-31 Thread David Nolen
Thanks to cl-format: (fn [buf__2572__auto__ len__2573__auto__] (if (= len__2573__auto__ 1) (mask8 (. buf__2572__auto__ (get))) (let [arr__2574__auto__ (int-array len__2573__auto__)] (dotimes [i__2575__auto__ len__2573__auto__] (aset-int arr__2574__auto__

Re: I need help tracking down a performance problem.

2009-03-31 Thread Aaron Cohen
On Tue, Mar 31, 2009 at 1:24 PM, Aaron Cohen wrote: > I'm sorry if I missed you mentioning it, but have you tried running > your code with (set! *warn-on-reflection* true) in effect? > Ugh, I should have looked at your code before I sent that. There it is on line 1. ;) -- Aaron --~--~

Re: I need help tracking down a performance problem.

2009-03-31 Thread Aaron Cohen
I'm sorry if I missed you mentioning it, but have you tried running your code with (set! *warn-on-reflection* true) in effect? -- Aaron --~--~-~--~~~---~--~~ You received this message because you are subscribed to the Google Groups "Clojure" group. To post to thi

Re: I need help tracking down a performance problem.

2009-03-31 Thread Vincent Foley
I tried using aset-int and I tried using int to coerce the result of mask-fn, the input argument to mask-fn and few other things, but none of that seems to make a difference so far. Mind you, this is an aspect of Clojure that I find a little confusing, so I'm just putting int calls here and there

Re: I need help tracking down a performance problem.

2009-03-31 Thread Christophe Grand
Did you try to coerce the result of (~mask-fn ...) with int? (or use aset-int as suggested by David) On Tue, Mar 31, 2009 at 4:17 PM, Vincent Foley wrote: > > No, but in my defense I did not know such a function existed :) I'll > give it a whirl and report back! > > On Mar 31, 9:57 am, David N

Re: I need help tracking down a performance problem.

2009-03-31 Thread Vincent Foley
No, but in my defense I did not know such a function existed :) I'll give it a whirl and report back! On Mar 31, 9:57 am, David Nolen wrote: > Did you try using aset-int instead of aset? > > On Tue, Mar 31, 2009 at 8:25 AM, Vincent Foley wrote: > > > For those interested, I managed to improve

Re: I need help tracking down a performance problem.

2009-03-31 Thread David Nolen
Did you try using aset-int instead of aset? On Tue, Mar 31, 2009 at 8:25 AM, Vincent Foley wrote: > > For those interested, I managed to improve the performance of my > original program from 2 minutes 40 seconds to decode 1000+ files down > to 2 minutes. I'm still far from my goal, but it's an

Re: I need help tracking down a performance problem.

2009-03-31 Thread Vincent Foley
For those interested, I managed to improve the performance of my original program from 2 minutes 40 seconds to decode 1000+ files down to 2 minutes. I'm still far from my goal, but it's an improvement, especially since the code is shorter and (IMO) cleaner. You can see it here: http://bitbucket

Bit-syntax (Was: Re: I need help tracking down a performance problem.)

2009-03-24 Thread Kyle Schaffrick
On Mon, 23 Mar 2009 19:41:07 -0700 Mark Engelberg wrote: > > On Mon, Mar 23, 2009 at 7:31 PM, Vincent Foley > wrote: > > More generally, is it possible that I'm just doing this whole thing > > wrong?  That using vectors to represent binary fields and records > > in a declarative is just a bad i

Re: I need help tracking down a performance problem.

2009-03-23 Thread Mark Engelberg
On Mon, Mar 23, 2009 at 7:31 PM, Vincent Foley wrote: > More generally, is it possible that I'm just doing this whole thing > wrong?  That using vectors to represent binary fields and records in a > declarative is just a bad idea and that I should try and explore lower- > level alternatives? >

Re: I need help tracking down a performance problem.

2009-03-23 Thread Vincent Foley
I can try that. More generally, is it possible that I'm just doing this whole thing wrong? That using vectors to represent binary fields and records in a declarative is just a bad idea and that I should try and explore lower- level alternatives? Vincent On Mar 23, 3:35 pm, Christophe Grand wr

Re: I need help tracking down a performance problem.

2009-03-23 Thread Christophe Grand
Hi Vincent! Vincent Foley a écrit : > Using the new versions of null-string and read-field-aux that you gave > me, in my real application, the execution time went from 160 seconds > to 150 seconds. As for using macros, I wrote one for the example > program, but I realized that it wouldn't work i

Re: I need help tracking down a performance problem.

2009-03-23 Thread Vincent Foley
Using the new versions of null-string and read-field-aux that you gave me, in my real application, the execution time went from 160 seconds to 150 seconds. As for using macros, I wrote one for the example program, but I realized that it wouldn't work in my application, because I sometimes use (ap

Re: I need help tracking down a performance problem.

2009-03-22 Thread Vincent Foley
How would I do that? Make a macro that expands into a map literal with the appropriate calls to .get, .getShort and .getInt? On Mar 22, 4:20 pm, Christophe Grand wrote: > Vincent Foley a écrit :> The code is available at this > URL:http://code.google.com/p/bwhf/ > > (look at the hu.becliza.and

Re: I need help tracking down a performance problem.

2009-03-22 Thread Christophe Grand
Vincent Foley a écrit : > The code is available at this URL: http://code.google.com/p/bwhf/ > (look at the hu.becliza.andras.bwhf.control.[BinRepParser, > BinReplayUnpacker] files) > > It is definitely not as dynamic, which helps quite a lot, but I wanted > to have something high level and declara

Re: I need help tracking down a performance problem.

2009-03-21 Thread Vincent Foley
The code is available at this URL: http://code.google.com/p/bwhf/ (look at the hu.becliza.andras.bwhf.control.[BinRepParser, BinReplayUnpacker] files) It is definitely not as dynamic, which helps quite a lot, but I wanted to have something high level and declarative. Thank you for your help, I'l

Re: I need help tracking down a performance problem.

2009-03-21 Thread Christophe Grand
Christophe Grand a écrit : > With these two changes, it's the dispatch fn that now dominates. > Correction: it's the dispatching, not the computation of the dispatch value that dominates. -- Professional: http://cgrand.net/ (fr) On Clojure: http://clj-me.blogspot.com/ (en) --~--~

Re: I need help tracking down a performance problem.

2009-03-21 Thread Christophe Grand
Hello, Is the java code online somewhere to compare? (Since your code is an interpreter for byte-fields specs there's a lot of stuff going to read a single byte, I wonder if the java code you're benchmarking against is that dynamic) Right now, null-string and read-field-aux are the most time

Re: I need help tracking down a performance problem.

2009-03-20 Thread Vincent Foley
Here: http://gist.github.com/82352 I have posted memory and cpu profiling figures. On Mar 20, 6:56 am, Christophe Grand wrote: > Hello Vincent, > > Vincent Foley a écrit : > > > Hello, > > > For the past few days, I've been trying, unsuccessfully, to make an > > application I wrote faster.  A J

Re: I need help tracking down a performance problem.

2009-03-20 Thread Christophe Grand
Hello Vincent, Vincent Foley a écrit : > Hello, > > For the past few days, I've been trying, unsuccessfully, to make an > application I wrote faster. A Java program that performs, more or > less, the same task takes 12 seconds (on my machine) to parse 1000 > files; my Clojure program takes nearl

Re: I need help tracking down a performance problem.

2009-03-19 Thread stephaner
Hi Mr Foley, Running your code on my machine give the following result: (load-file "slow.clj") "Elapsed time: 1736.034068 msecs" nil user> (load-file "slow.clj") "Elapsed time: 678.850779 msecs" nil I'm on Ubuntu 8.10 on a Hp dv6700 with 2gig RAM. Stephane On Mar 19, 8:12 pm, Vincent Foley wr

I need help tracking down a performance problem.

2009-03-19 Thread Vincent Foley
Hello, For the past few days, I've been trying, unsuccessfully, to make an application I wrote faster. A Java program that performs, more or less, the same task takes 12 seconds (on my machine) to parse 1000 files; my Clojure program takes nearly 3 minutes. This more than an order of magnitude