Re: [ANN] Sparkling, a Clojure-API to Apache Spark.

[Dmitriy Morozov]

Hi Tilak!

You are trying to call groupByKey on instance of JavaRDD which doesn't have 
this method. You need JavaPairRDD which you can create with 
spark/map-to-pair.

Good luck!

On Wednesday, January 7, 2015 at 2:23:42 PM UTC+2, chris_betz wrote:
>
> Hi,
>
>
> we just released Sparkling (https://gorillalabs.github.io/sparkling/), 
> our take on an API to Apache Spark.
>
>
>
> With an eye on speed for very large amounts of data we improved clj-spark 
> and flambo to get us the speed we need for our production environment.
>
>
> See https://gorillalabs.github.io/sparkling/articles/getting_started.html 
> for a quickstart or dive directly into our playground project by 
> git clone https://github.com/gorillalabs/sparkling-getting-started.git
>
>
>
> Happy hacking
>
> Chris
> (@gorillalabs_de )
>

-- 
You received this message because you are subscribed to the Google
Groups "Clojure" group.
To post to this group, send email to clojure@googlegroups.com
Note that posts from new members are moderated - please be patient with your 
first post.
To unsubscribe from this group, send email to
clojure+unsubscr...@googlegroups.com
For more options, visit this group at
http://groups.google.com/group/clojure?hl=en
--- 
You received this message because you are subscribed to the Google Groups 
"Clojure" group.
To unsubscribe from this group and stop receiving emails from it, send an email 
to clojure+unsubscr...@googlegroups.com.
For more options, visit https://groups.google.com/d/optout.

Looking for a clojure expert to help fill in a language feature reference

[Dan Cancro]

Hi,

I'm not a Clojure user but I maintain a reference to help people understand 
the tradeoffs and features of technologies.  I was hoping someone among you 
would be interested in helping to contribute some knowledge about it to the 
project.

Click here to see the reference 

.

It should not take more than a few minutes for someone who knows Clojure 
well and will really help people understand quickly what it's about and how 
it's different.

If you would like to help out, I've made a questionnaire that should make 
it a little easier:

Click here to complete a short questionnaire about Clojure 


I'm sure I've left out a lot of benefits that Clojure delivers.  Please let 
me know about the big things I've missed and I'll add them to the list.

Thanks,
Dan

-- 
You received this message because you are subscribed to the Google
Groups "Clojure" group.
To post to this group, send email to clojure@googlegroups.com
Note that posts from new members are moderated - please be patient with your 
first post.
To unsubscribe from this group, send email to
clojure+unsubscr...@googlegroups.com
For more options, visit this group at
http://groups.google.com/group/clojure?hl=en
--- 
You received this message because you are subscribed to the Google Groups 
"Clojure" group.
To unsubscribe from this group and stop receiving emails from it, send an email 
to clojure+unsubscr...@googlegroups.com.
For more options, visit https://groups.google.com/d/optout.

Re: Using :refer 'sparingly'

[Luc Prefontaine]

We systematically use refer all on tools.trace and a few other of our name 
spaces used for production support.

It becomes handy in a live repl in production.

Luc P.


> I agree with the general sentiment expressed here, but would just like to 
> add that `:refer`-ing a few frequently used functions (as Colin Yates 
> stated, particularly when it's assumed there is strong coupling or 
> closeness between the two namespaces involved), is a much more minor 
> nuisance than `:refer :all`. At least with `:refer [some-fn 
> some-other-fn]`, you _can_ figure out where the function came from by going 
> up to the `ns` declaration, and if you're fast with your editor, this is 
> easy to do. Both `:refer :all` and `:use`/ `(use)` should (IMHO) only be 
> used for hacking around at the repl.
> 
> -- 
> You received this message because you are subscribed to the Google
> Groups "Clojure" group.
> To post to this group, send email to clojure@googlegroups.com
> Note that posts from new members are moderated - please be patient with your 
> first post.
> To unsubscribe from this group, send email to
> clojure+unsubscr...@googlegroups.com
> For more options, visit this group at
> http://groups.google.com/group/clojure?hl=en
> --- 
> You received this message because you are subscribed to the Google Groups 
> "Clojure" group.
> To unsubscribe from this group and stop receiving emails from it, send an 
> email to clojure+unsubscr...@googlegroups.com.
> For more options, visit https://groups.google.com/d/optout.
> 
--
Luc Prefontaine sent by ibisMail!

-- 
You received this message because you are subscribed to the Google
Groups "Clojure" group.
To post to this group, send email to clojure@googlegroups.com
Note that posts from new members are moderated - please be patient with your 
first post.
To unsubscribe from this group, send email to
clojure+unsubscr...@googlegroups.com
For more options, visit this group at
http://groups.google.com/group/clojure?hl=en
--- 
You received this message because you are subscribed to the Google Groups 
"Clojure" group.
To unsubscribe from this group and stop receiving emails from it, send an email 
to clojure+unsubscr...@googlegroups.com.
For more options, visit https://groups.google.com/d/optout.

Re: What is a real example of the Observer pattern?

[martin madera]

The best you can do is read the GoF book (yes, the one which is referenced 
on the very beginning of the Mr. Sierra's article).

And please feel free to post your opinion if you have different than I 
have. I just finished writing a diploma thesis on design patterns in Java 
7, Scala and Clojure, so I can be a little biased, plus I have 6 years of 
commercial experience in writing C#, my employment in Java (or Clojure?) is 
yet to come.

As Mr. Sierra stated on the beginning of the article, Norvig somewhat 
criticized the GoF book 1 and half year after it was publicized; that 
presentation is here: http://norvig.com/design-patterns/design-patterns.pdf.

Observer pattern "defines a one-to-many dependency between objects so that 
*when 
one object changes state, all its dependants are notified and *updated" 
(the GoF book, page 293). So imagine that you have a data object, which is 
changed from one screen of your application (user updates data). Another 
screen gets notified about this change and updates itself so it shows 
updated data. But of course it's general, not only tied to UI ... but most 
common usage is updating UI on data change.

In Clojure, you can use function *add-watch* or implement this behaviour 
yourself, much like in the book (GoF has sample code written in C++ and 
Smalltalk, no Java in 1994 :-) but there are many books which basically 
translated the patterns to another object oriented language, plenty of them 
to Java).

I consider this book to be the "basics" for object oriented programmers, 
but Norvig had a good point that more expressive language makes a lot of 
the patterns invisible or they completely disappear. Even C# (basically a 
"Java with renamed namespaces, lower amount of libraries and some nice 
constructs which make the language more complex and often hides what should 
not be hidden" ;-) ) you can see that the Observer pattern is usually 
simplified (= less amount of boilerplate code) thanks to events which Java 
doesn't have. In Clojure I'd advise to try re-think the architecture if you 
think you need to use Observer before really implementing it (which however 
can be done without problems).

Martin


On Thursday, 7 May 2015 04:33:24 UTC+2, larry google groups wrote:
>
> I am looking here: 
>
>
> https://strange-loop-2012-notes.readthedocs.org/en/latest/monday/functional-design-patterns.html
>
> I read: 
>
> Observer Pattern 
> 
>
>- Register an observer with a stateful function
>
> The observer could take the old and new state, along with either the 
> delta, the triggering event, or the container.
>
> *Can anyone point to something on Github that shows an example of this? *
>
>
>
>
>
>

-- 
You received this message because you are subscribed to the Google
Groups "Clojure" group.
To post to this group, send email to clojure@googlegroups.com
Note that posts from new members are moderated - please be patient with your 
first post.
To unsubscribe from this group, send email to
clojure+unsubscr...@googlegroups.com
For more options, visit this group at
http://groups.google.com/group/clojure?hl=en
--- 
You received this message because you are subscribed to the Google Groups 
"Clojure" group.
To unsubscribe from this group and stop receiving emails from it, send an email 
to clojure+unsubscr...@googlegroups.com.
For more options, visit https://groups.google.com/d/optout.

Re: [ANN] Sparkling, a Clojure-API to Apache Spark.

[Tilak Thapa]

Oh yes. Thanks Dmitriy.

On Monday, May 18, 2015 at 4:13:09 AM UTC-5, Dmitriy Morozov wrote:
>
>
> Hi Tilak!
>
> You are trying to call groupByKey on instance of JavaRDD which doesn't 
> have this method. You need JavaPairRDD which you can create with 
> spark/map-to-pair.
>
> Good luck!
>
> On Wednesday, January 7, 2015 at 2:23:42 PM UTC+2, chris_betz wrote:
>>
>> Hi,
>>
>>
>> we just released Sparkling (https://gorillalabs.github.io/sparkling/), 
>> our take on an API to Apache Spark.
>>
>>
>>
>> With an eye on speed for very large amounts of data we improved clj-spark 
>> and flambo to get us the speed we need for our production environment.
>>
>>
>> See https://gorillalabs.github.io/sparkling/articles/getting_started.html 
>> for a quickstart or dive directly into our playground project by 
>> git clone 
>> https://github.com/gorillalabs/sparkling-getting-started.git
>>
>>
>>
>> Happy hacking
>>
>> Chris
>> (@gorillalabs_de )
>>
>

-- 
You received this message because you are subscribed to the Google
Groups "Clojure" group.
To post to this group, send email to clojure@googlegroups.com
Note that posts from new members are moderated - please be patient with your 
first post.
To unsubscribe from this group, send email to
clojure+unsubscr...@googlegroups.com
For more options, visit this group at
http://groups.google.com/group/clojure?hl=en
--- 
You received this message because you are subscribed to the Google Groups 
"Clojure" group.
To unsubscribe from this group and stop receiving emails from it, send an email 
to clojure+unsubscr...@googlegroups.com.
For more options, visit https://groups.google.com/d/optout.

Re: Looking for a clojure expert to help fill in a language feature reference

[Atamert Ölçgen]

Hi Dan,

Who are the people you help understand the tradeoffs and features of
technologies? Is this a public project with a public website?

On Mon, May 18, 2015 at 11:51 AM, Dan Cancro  wrote:

> Hi,
>
> I'm not a Clojure user but I maintain a reference to help people
> understand the tradeoffs and features of technologies.  I was hoping
> someone among you would be interested in helping to contribute some
> knowledge about it to the project.
>
> Click here to see the reference
> 
> .
>
> It should not take more than a few minutes for someone who knows Clojure
> well and will really help people understand quickly what it's about and how
> it's different.
>
> If you would like to help out, I've made a questionnaire that should make
> it a little easier:
>
> Click here to complete a short questionnaire about Clojure
> 
>
> I'm sure I've left out a lot of benefits that Clojure delivers.  Please
> let me know about the big things I've missed and I'll add them to the list.
>
> Thanks,
> Dan
>
> --
> You received this message because you are subscribed to the Google
> Groups "Clojure" group.
> To post to this group, send email to clojure@googlegroups.com
> Note that posts from new members are moderated - please be patient with
> your first post.
> To unsubscribe from this group, send email to
> clojure+unsubscr...@googlegroups.com
> For more options, visit this group at
> http://groups.google.com/group/clojure?hl=en
> ---
> You received this message because you are subscribed to the Google Groups
> "Clojure" group.
> To unsubscribe from this group and stop receiving emails from it, send an
> email to clojure+unsubscr...@googlegroups.com.
> For more options, visit https://groups.google.com/d/optout.
>



-- 
Kind Regards,
Atamert Ölçgen

◻◼◻
◻◻◼
◼◼◼

www.muhuk.com
www.olcgen.com

-- 
You received this message because you are subscribed to the Google
Groups "Clojure" group.
To post to this group, send email to clojure@googlegroups.com
Note that posts from new members are moderated - please be patient with your 
first post.
To unsubscribe from this group, send email to
clojure+unsubscr...@googlegroups.com
For more options, visit this group at
http://groups.google.com/group/clojure?hl=en
--- 
You received this message because you are subscribed to the Google Groups 
"Clojure" group.
To unsubscribe from this group and stop receiving emails from it, send an email 
to clojure+unsubscr...@googlegroups.com.
For more options, visit https://groups.google.com/d/optout.

New Functional Programming Job Opportunities

[Functional Jobs]

Here are some functional programming job opportunities that were posted

recently:



Senior Software Engineer (Functional Programming/Scala) at AdAgility

http://functionaljobs.com/jobs/8826-senior-software-engineer-functional-programming-scala-at-adagility



Cheers,

Sean Murphy

FunctionalJobs.com


-- 
You received this message because you are subscribed to the Google
Groups "Clojure" group.
To post to this group, send email to clojure@googlegroups.com
Note that posts from new members are moderated - please be patient with your 
first post.
To unsubscribe from this group, send email to
clojure+unsubscr...@googlegroups.com
For more options, visit this group at
http://groups.google.com/group/clojure?hl=en
--- 
You received this message because you are subscribed to the Google Groups 
"Clojure" group.
To unsubscribe from this group and stop receiving emails from it, send an email 
to clojure+unsubscr...@googlegroups.com.
For more options, visit https://groups.google.com/d/optout.

Re: [ANN] PigPen 0.3.0 - Now with Cascading!

['Matt Bossenbroek' via Clojure]

No complaints, so PigPen 0.3.0 is now officially released.

Enjoy!

-Matt


On Monday, May 11, 2015 at 8:40 AM, Matt Bossenbroek wrote:

> I'm excited to announce the release of PigPen v0.3.0, which now includes 
> support for Cascading.
> 
> PigPen is Map-Reduce for Clojure - you write idiomatic Clojure code, we 
> compile it into an Apache Pig script or a Cascading flow that runs on Hadoop.
> 
> https://github.com/Netflix/PigPen
> 
> An RC build is currently available in Maven (0.3.0-rc.7). We've been using it 
> at Netflix for a little over a month now with no issues, but we want to get 
> it out in the wild before locking down the release.
> 
> There are a couple of breaking changes in this release, mostly just moving 
> pig-specific functions into another namespace. Check out the release notes 
> for more details.
> 
> Also new since the last major release: semi-joins, anti-joins, load-json, 
> store-json, load-csv (RFC 4180), load-parquet, store-parquet, and load-avro 
> (parquet & avro support not yet available for cascading).
> 
> A huge thanks to Piotr Kozikowski for contributing all of the Cascading work 
> to the project.
> 
> For questions or complaints: pigpen-supp...@googlegroups.com 
> (mailto:pigpen-supp...@googlegroups.com)
> 
> -Matt
> 
> 
> ps. The PigPen name & logo aren't really ideal now that we've added another 
> platform and plan to add more. Apparently we're only clever enough for one 
> name & logo. If you've got an idea for a better name for the project, please 
> let us know. :)
> 
> -- 
> You received this message because you are subscribed to the Google
> Groups "Clojure" group.
> To post to this group, send email to clojure@googlegroups.com 
> (mailto:clojure@googlegroups.com)
> Note that posts from new members are moderated - please be patient with your 
> first post.
> To unsubscribe from this group, send email to
> clojure+unsubscr...@googlegroups.com 
> (mailto:clojure+unsubscr...@googlegroups.com)
> For more options, visit this group at
> http://groups.google.com/group/clojure?hl=en
> --- 
> You received this message because you are subscribed to the Google Groups 
> "Clojure" group.
> To unsubscribe from this group and stop receiving emails from it, send an 
> email to clojure+unsubscr...@googlegroups.com 
> (mailto:clojure+unsubscr...@googlegroups.com).
> For more options, visit https://groups.google.com/d/optout.

-- 
You received this message because you are subscribed to the Google
Groups "Clojure" group.
To post to this group, send email to clojure@googlegroups.com
Note that posts from new members are moderated - please be patient with your 
first post.
To unsubscribe from this group, send email to
clojure+unsubscr...@googlegroups.com
For more options, visit this group at
http://groups.google.com/group/clojure?hl=en
--- 
You received this message because you are subscribed to the Google Groups 
"Clojure" group.
To unsubscribe from this group and stop receiving emails from it, send an email 
to clojure+unsubscr...@googlegroups.com.
For more options, visit https://groups.google.com/d/optout.

Re: ANN: clojure.java.shell2 drop in replacement for clojure.java.shell

[Max Gonzih]

Hello,

I just found about this nice improvement over default java.shell provided 
by clojure stdlib.
What is current status of this project? Is it still useful or maybe there 
are alternatives?
Does it support latest clojure versions (1.6 or even maybe 1.7-beta)?

Thanks!

On Tuesday, June 25, 2013 at 5:57:58 PM UTC+2, mlimotte wrote:
>
> I'm announcing java.shell2 .  It 
> is backward compatible with clojure.java.shell.  This is a Clojure library 
> to facilitate launching of sub-processes and piping (streaming) data.
>
> Features
> - A declarative syntax for defining new processes to specify input, 
> output, encoding, and other behavior
> - Handling for common use-cases (i.e. pass stdout/err of the process to 
> the same destination as the parent, merge stderr of the process to stdout, 
> output directly to a File, etc)
> - The pipe macro handles all the complexity of managing multipe streams 
> and threads for streaming data through multiple processes and clojure 
> functions.
> - Backward compatible with existing code that uses clojure.java.shell 
> (i.e. a drop-in replacement)
>
> Shell has additional predicates like :pass, which will connect STDOUT or 
> STDERR of the process to the STDOUT/ERR of the parent JVM.
>
> (sh "wc" "-l" :in input :err :pass :out (io/file "/tmp/foo"))
>
> So the above form reads input (which can be a file, stream, string, etc), 
> forwards the output to a file and redirects STDERR to STDERR of the JVM.
>
> And here's an example of a pipe:
>
> (pipe
>   (sh "cat" :in input)
>   my-filter-fn;A clojure function -- data is streamed
>   (sh "wc" "-l"))
>
>
> This library was developed at The Climate Corporation 
> , so a big thank you to them for allowing me to open 
> source this code under an EPL license.  Climate Corp has one of the largest 
> Clojure development teams.  We are an established startup with offices in 
> San Francisco and Seattle; and are currently hiring full-time Clojure 
> developers, data scientists, product managers and more 
> .
>
> See the README for more details and many examples in the unit tests.  
>
> I know there are other shell libraries for Clojure.  My main motivation is 
> that I wanted something closer to the clojure.java.shell api.
>
> Marc Limotte
>
>
>

-- 
You received this message because you are subscribed to the Google
Groups "Clojure" group.
To post to this group, send email to clojure@googlegroups.com
Note that posts from new members are moderated - please be patient with your 
first post.
To unsubscribe from this group, send email to
clojure+unsubscr...@googlegroups.com
For more options, visit this group at
http://groups.google.com/group/clojure?hl=en
--- 
You received this message because you are subscribed to the Google Groups 
"Clojure" group.
To unsubscribe from this group and stop receiving emails from it, send an email 
to clojure+unsubscr...@googlegroups.com.
For more options, visit https://groups.google.com/d/optout.

Re: ANN: clojure.java.shell2 drop in replacement for clojure.java.shell

[Marc Limotte]

Hi Max.

I'm not actively doing any work on it.  Mainly because there are no
requests for changes.  It's pretty straight-forward.  I haven't tested it
with later versions of Clojure, but I'm not aware of any breaking changes,
so I would expect it to work.

There are some alternatives, e.g. https://github.com/Raynes/conch

marc


On Mon, May 18, 2015 at 2:55 PM, Max Gonzih  wrote:

> Hello,
>
> I just found about this nice improvement over default java.shell provided
> by clojure stdlib.
> What is current status of this project? Is it still useful or maybe there
> are alternatives?
> Does it support latest clojure versions (1.6 or even maybe 1.7-beta)?
>
> Thanks!
>
> On Tuesday, June 25, 2013 at 5:57:58 PM UTC+2, mlimotte wrote:
>>
>> I'm announcing java.shell2 .
>> It is backward compatible with clojure.java.shell.  This is a Clojure
>> library to facilitate launching of sub-processes and piping (streaming)
>> data.
>>
>> Features
>> - A declarative syntax for defining new processes to specify input,
>> output, encoding, and other behavior
>> - Handling for common use-cases (i.e. pass stdout/err of the process to
>> the same destination as the parent, merge stderr of the process to stdout,
>> output directly to a File, etc)
>> - The pipe macro handles all the complexity of managing multipe streams
>> and threads for streaming data through multiple processes and clojure
>> functions.
>> - Backward compatible with existing code that uses clojure.java.shell
>> (i.e. a drop-in replacement)
>>
>> Shell has additional predicates like :pass, which will connect STDOUT or
>> STDERR of the process to the STDOUT/ERR of the parent JVM.
>>
>> (sh "wc" "-l" :in input :err :pass :out (io/file "/tmp/foo"))
>>
>> So the above form reads input (which can be a file, stream, string, etc),
>> forwards the output to a file and redirects STDERR to STDERR of the JVM.
>>
>> And here's an example of a pipe:
>>
>> (pipe
>>   (sh "cat" :in input)
>>   my-filter-fn;A clojure function -- data is streamed
>>   (sh "wc" "-l"))
>>
>>
>> This library was developed at The Climate Corporation
>> , so a big thank you to them for allowing me to
>> open source this code under an EPL license.  Climate Corp has one of the
>> largest Clojure development teams.  We are an established startup with
>> offices in San Francisco and Seattle; and are currently hiring full-time
>> Clojure developers, data scientists, product managers and more
>> .
>>
>> See the README for more details and many examples in the unit tests.
>>
>> I know there are other shell libraries for Clojure.  My main motivation
>> is that I wanted something closer to the clojure.java.shell api.
>>
>> Marc Limotte
>>
>>
>>  --
> You received this message because you are subscribed to the Google
> Groups "Clojure" group.
> To post to this group, send email to clojure@googlegroups.com
> Note that posts from new members are moderated - please be patient with
> your first post.
> To unsubscribe from this group, send email to
> clojure+unsubscr...@googlegroups.com
> For more options, visit this group at
> http://groups.google.com/group/clojure?hl=en
> ---
> You received this message because you are subscribed to the Google Groups
> "Clojure" group.
> To unsubscribe from this group and stop receiving emails from it, send an
> email to clojure+unsubscr...@googlegroups.com.
> For more options, visit https://groups.google.com/d/optout.
>

-- 
You received this message because you are subscribed to the Google
Groups "Clojure" group.
To post to this group, send email to clojure@googlegroups.com
Note that posts from new members are moderated - please be patient with your 
first post.
To unsubscribe from this group, send email to
clojure+unsubscr...@googlegroups.com
For more options, visit this group at
http://groups.google.com/group/clojure?hl=en
--- 
You received this message because you are subscribed to the Google Groups 
"Clojure" group.
To unsubscribe from this group and stop receiving emails from it, send an email 
to clojure+unsubscr...@googlegroups.com.
For more options, visit https://groups.google.com/d/optout.

Re: Why does the following Clojure code take 10x the time the C# version does? How to improve the Clojure version?

[Leif]

Summary:  With a new algorithm + parallelism, I reduced it from 528s to 11s.

This sounded fun, so I took a crack at it, starting with your solution.  
Description and patch files here: 
https://gist.github.com/leifp/a864bca941ecdacb5840

Starting with your solution, I:

   1. Divided the canvas into 100 blocks, created an index of {blockindex 
   -> papers that intersect that block}. The reasoning is that if we calculate 
   which block a pixel is in, we only need to check the papers that intersect 
   that block.  In the extreme case, certain blocks only intersected one paper 
   (the background).  In the original code we would have had to check all 100 
   papers for each pixel in that block; now we just check one. (5x average 
   speedup)
   2. Changed the code to calculate color areas for a block at a time; 
   after that, it was a simple 2-line change to parallelize the work using 
pmap. 
   (8x speedup on 12-core machine) 
   3. Make paper a record; use direct field access (this resulted in a 
   modest ~10% improvement, but maybe not worth it).

So clojure was helpful in trying out algorithm ideas and parallelizing the 
code.  The final program would no doubt also be faster in C#, but my goal 
is "fast enough."

Further idea (which I don't think I'll implement):  Index the papers using 
an data structure built for geometrical indexing, like an R-tree or 
variation, to get a near-optimal index without tuning.

I hope my solution is interesting to you.  Questions / comments welcome.
Leif

P.S.  I apologize for the messy, repetitive, stream-of-consciousness code.


On Thursday, May 14, 2015 at 4:02:42 AM UTC-4, Amith George wrote:
>
> I wrote the following code to solve this challenge - 
> https://www.reddit.com/r/dailyprogrammer/comments/35s2ds/20150513_challenge_214_intermediate_pile_of_paper/
> .
>
> Code - 
> https://github.com/amithgeorge/reddit-dailyprogrammer-clojure/blob/56ce1dbb6a08e96150dc85934caecfeb68108a53/src/rdp/214_intermediate.clj
>
> I executed the -main function using `lein run 1`. 
>
> Output
>
> ;; lein run 1
>
> 0 12605919
> 1 3578145
> 2 15356894
> 3 19134293
> 4 2394558
> 5 15030409
> 6 6424953
> 7 14893444
> 8 1592254
> 9 1914025
> 10 7075106
> "Elapsed time: 501168.972435 msecs"
>
> The code originally used an immutable hashmap, but I lost patience waiting 
> for the computation to end. With mutable hashmap, it still takes around 8 
> mins.
>
> I wrote a C# version of the above code - 
> https://gist.github.com/amithgeorge/766b8220f39d48221e58. It finishes 
> under 40secs. The C# exe was built under Release mode and executed directly 
> from the commandline. I expected the Clojure version to perform similarly.
>
> Any tips on what I am doing wrong?
>
> -
> Explanation of the code - Create a vector of all paper sheets, such that 
> the sheet placed last is the first element of the vector and the last 
> element is the canvas. To compute the frequency of each visible color - for 
> each point in the canvas find the first sheet in the vector that covers the 
> point. Store/increment its count in the hashmap. I understand there might 
> be better more efficient ways to solve this, but currently I am interested 
> in why the Clojure versions is so slow vis-a-vis the C# version.
>
>

-- 
You received this message because you are subscribed to the Google
Groups "Clojure" group.
To post to this group, send email to clojure@googlegroups.com
Note that posts from new members are moderated - please be patient with your 
first post.
To unsubscribe from this group, send email to
clojure+unsubscr...@googlegroups.com
For more options, visit this group at
http://groups.google.com/group/clojure?hl=en
--- 
You received this message because you are subscribed to the Google Groups 
"Clojure" group.
To unsubscribe from this group and stop receiving emails from it, send an email 
to clojure+unsubscr...@googlegroups.com.
For more options, visit https://groups.google.com/d/optout.

One more argument for cyclic dependencies

[Mars0i]

Sorry for the length of this post--I feel I have to spell out the details 
in order to head off irrelevant responses.  I'm saving you the trouble of 
reading a long chain posts just focused on misunderstandings.

For anything below, I'd be happy to find out that I'm wrong, confused, 
misguided, etc.

I've started using MASON, a library for agent-based models (ABMs) written 
in Java.  Using gen-class to follow the MASON model works pretty well, even 
if the result is somewhat "Java-esque" Clojure.

Using MASON in the normal way involves cyclic dependencies.  This is no 
problem if I let Clojure figure out the types at runtime--but I can get a 
speed improvement of 11 times the original speed if I add type hints, after 
a bit of code reorganization to avoid almost all of the cyclic type 
dependencies.  And by adding one additional type hint--more on this 
below--I get a further 6.5X speed improvement, for a total improvement due 
to type hints of roughly 70X.  Note that speed often matters for ABMs that 
involve stochastic effects; it's typically necessary to run the same 
simulation many times, and it's useful to be able to do the needed runs in 
minutes or hours rather than days.

However, the last type hint involves a cyclic type dependency.  It doesn't 
generate a cyclic dependency error, but gives class not found errors no 
matter which of two interdependent classes is compiled first. My solution 
is to delete the type hint, compile all of the Clojure files, add the type 
hint back, and recompile that one modified file. That's how I get the full 
85X speed improvement.  (*That* is an ugly solution.)

The cyclic dependencies are due to decisions by MASON's designers that seem 
entirely reasonable for an agent-based modeling framework--even if I would 
do things differently--and *have* done things differently (in a 
narrowly-focused, application-specific ABM framework that's purely 
functional except for reporting).

**

So: In order to use a well-designed Java library, I have to choose between 
slow code or awkward workarounds for a cyclic dependency.

**

Let me emphasize that I *do not* think that Clojure should incorporate 
every feature that someone thinks useful.  I certainly don't think that 
Clojure should provide every feature offered by Java.  Ugh.  I love Clojure 
because it doesn't include everything plus the kitchen sink, but instead 
provides a small set of flexible, simple, well-thought out functions.

However, the lack of cyclic dependencies is not the absence of a 
substantial language feature.  It's an arbitrary limitation placed on 
compile time that's not present at run time.  Allowing cyclic dependencies 
wouldn't make Clojure less simple or elegant, because it wouldn't really 
add anything to the language.

(Yes, if cyclic dependencies are allowed, then people can abuse them. 
People can also abuse type hints, interwoven side effects and laziness, 
macros, and non-idiomatic coding styles.  The Clojure community teaches 
routinely teaches novices how to avoid these mistakes.)

(If cyclic dependencies are really considered so harmful, one could hide 
them behind a special compiler option that emits a warning when used.)

---

Optional reading: Why there's a cyclic dependency in MASON simulations.

MASON simulations typically consist of 

(a) One or more "agents", which implement an interface (Steppable) with a 
single method, step().

(b) An instance of a class that extends SimState, which contains a 
scheduler that repeatedly calls step() in every agent that's registered 
with it.

The agents have to interact with each other, and the way that they find out 
about each others' states is typically through the the SimState object 
that's automatically passed to step().  The point is that the only way for 
teach Student to know about other instances of Students is by knowing about 
an object that also has to know about Students. There's the cycle.  (An 
alternative might be to have each Student maintain a collection of all 
other Students, but that's ugly and unnecessary.)

I use gen-class in order to implement Steppable and to extend SimState. The 
type hint that I delete and restore is for the second argument to the 
step() function required by Steppable:

(defn -step [^students.Student this ^students.StudentsSimState sim-state]

-- 
You received this message because you are subscribed to the Google
Groups "Clojure" group.
To post to this group, send email to clojure@googlegroups.com
Note that posts from new members are moderated - please be patient with your 
first post.
To unsubscribe from this group, send email to
clojure+unsubscr...@googlegroups.com
For more options, visit this group at
http://groups.google.com/group/clojure?hl=en
--- 
You received this message because you are subscribed to the Google Groups 
"Clojure" group.
To unsubscribe from this group and stop receiving emails from it, send an email 
to clojure+unsubscr...@googlegroups.com.
For m

Re: One more argument for cyclic dependencies

[Mars0i]

Sorry, the reference to an 85X speed difference was a typo.  Should be 70X.

-- 
You received this message because you are subscribed to the Google
Groups "Clojure" group.
To post to this group, send email to clojure@googlegroups.com
Note that posts from new members are moderated - please be patient with your 
first post.
To unsubscribe from this group, send email to
clojure+unsubscr...@googlegroups.com
For more options, visit this group at
http://groups.google.com/group/clojure?hl=en
--- 
You received this message because you are subscribed to the Google Groups 
"Clojure" group.
To unsubscribe from this group and stop receiving emails from it, send an email 
to clojure+unsubscr...@googlegroups.com.
For more options, visit https://groups.google.com/d/optout.