which GC optimizations work better with Clojure?

[Niels van Klaveren]

GC parameters used by a lot of projects are handed down like traditions for 
years, even spreading to others. They're almost never re-evaluated, which means 
they might actually lose out on new performance improvements, or worse, might 
even cause performance regressions.

Most parameters in the Overtone lib for instance have been the default for 
quite some time (ParNew GC, TLAB). Where restricting New Gen might be a good 
idea to reduce new gen GC latency, it can cause unwanted promotions, longer 
tenured GC's and in the end heap fragmentation resulting in a full defrag gc 
that can take 10s of seconds.

Defaults are there for a reason, to take care of most situations. Any settings 
other than the three default Garbage Collectors (Default for shortest time to 
finish, CMS for shortest pause , G1 for short pauses but long running times) 
and at most a pause hint for their heuristics means you're entering 
case-specific  territory and should be treating it so.

This means realistic benchmarking, profiling and comparisons between different 
settings. I'd treat any JVM product or library with more than 2 GC parameters 
with care, because if they need so much tweaking as their default, you can be 
sure to need both a good understanding of their inner workings and JVM memory 
managent as well to keep them performing in your specific case.

-- 
You received this message because you are subscribed to the Google
Groups "Clojure" group.
To post to this group, send email to clojure@googlegroups.com
Note that posts from new members are moderated - please be patient with your 
first post.
To unsubscribe from this group, send email to
clojure+unsubscr...@googlegroups.com
For more options, visit this group at
http://groups.google.com/group/clojure?hl=en
--- 
You received this message because you are subscribed to the Google Groups 
"Clojure" group.
To unsubscribe from this group and stop receiving emails from it, send an email 
to clojure+unsubscr...@googlegroups.com.
For more options, visit https://groups.google.com/d/optout.

Re: Schemas for DSLs on top of Clojure

[Olek]

Ok. Thank you. Not quite what I was expecting but nice to see that tool too.


On Friday, 29 April 2016 20:45:27 UTC+2, Michael Willis wrote:
>
> The convention that I've seen among the Clojure community is to represent 
> these kinds of things as data structures, and define your contraints using 
> something like https://github.com/plumatic/schema
>
> On Friday, April 29, 2016 at 1:10:29 PM UTC-5, Olek wrote:
>>
>> Hi!
>>
>> Clojure data structures can express tree data structures, same as in XML 
>> what is ideal for DSL. Especially that thanks to macros you can also change 
>> the evaluation order and hide execution complexity and treat DSL in terms 
>> of declarations and not statements nor functions. What is more the grammar 
>> of such structure is simplified to what you can see in XML.
>> For example let take the XML example:
>>
>> 
>> 
>> 
>> 
>>
>> Now lets look at Clojure:
>>
>> (books
>>   (book { :iban "31232" :title "some title" :author "some author" })
>>  (book { :ban "43232" :title "another title2" :author "another author 2" 
>> }) )
>>
>> You can notice that when you build macros & functions for that you have 
>> simple structure of input arguments like:
>>
>> (defmacro/defn books[ attrs & books-col ]
>>(do some stuff))
>>
>> (defn book[ attrs ]
>>   (do some stuff))
>>
>> Now all you need is to slurp the specification made in terms of created 
>> DSL, call eval on it and as the result of execution you will spit the 
>> resulting data/mutate the state/do side effect.
>> Finally for XML you can create XSD to constraint possibilities of tags 
>> which can be put into DSL.
>> But as you can see, such possibility is missing for Clojure. Do you have 
>> any ready tool for that or do I have to create one?
>>
>> Thanks in advance,
>> Olek
>>
>>
>>
>>

-- 
You received this message because you are subscribed to the Google
Groups "Clojure" group.
To post to this group, send email to clojure@googlegroups.com
Note that posts from new members are moderated - please be patient with your 
first post.
To unsubscribe from this group, send email to
clojure+unsubscr...@googlegroups.com
For more options, visit this group at
http://groups.google.com/group/clojure?hl=en
--- 
You received this message because you are subscribed to the Google Groups 
"Clojure" group.
To unsubscribe from this group and stop receiving emails from it, send an email 
to clojure+unsubscr...@googlegroups.com.
For more options, visit https://groups.google.com/d/optout.

Re: [ANN] components.md

[Derek Troy-West]

I've quite enjoyed reading the various approaches to using Component (or 
not as the case may be), so much so that I've added my own two pence:

http://derek.troywest.com/articles/how-i-use-component

On Tuesday, April 5, 2016 at 6:28:48 PM UTC+10, Renzo Borgatti wrote:
>
> Hello clj, 
>
> kinda late into that discussion of a while ago about “how I use 
> components”, I finally wrote-up about it: 
>
> https://github.com/reborg/scccw/blob/master/COMPONENTS.md 
>
> The result is a markdown document that is meant to be used by 
> copy-pasting. So no libraries, no frameworks. The reason is that I’m mainly 
> introducing an architectural convention to handle stateful parts of a 
> Clojure application and I believe your app should bootstrap from these 
> general principles and extend on them, evolutionary. It’s not different 
> from saying that you should isolate impure functions in your code: you 
> wouldn’t create a library out of it. 
>
> Let me know what you think. Between this organization for components and 
> Fluorine for config (https://github.com/reborg/fluorine/), I’m a happier 
> Clojure dev these days :) 
>
> Renzo 
> @reborg

-- 
You received this message because you are subscribed to the Google
Groups "Clojure" group.
To post to this group, send email to clojure@googlegroups.com
Note that posts from new members are moderated - please be patient with your 
first post.
To unsubscribe from this group, send email to
clojure+unsubscr...@googlegroups.com
For more options, visit this group at
http://groups.google.com/group/clojure?hl=en
--- 
You received this message because you are subscribed to the Google Groups 
"Clojure" group.
To unsubscribe from this group and stop receiving emails from it, send an email 
to clojure+unsubscr...@googlegroups.com.
For more options, visit https://groups.google.com/d/optout.

Re: is reduce/reduced faster than loop/recur?

[Michał Marczyk]

And just to add a pointer in case anybody's interested in opting in to this
behaviour in a custom type, it's possible to do so by implementing
clojure.core.protocols/IKVReduce (data.avl does this) or
clojure.lang.IKVReduce (the iface implemented by built-in maps;
clojure.core provides implementation of clojure.core.protocols/IKVReduce
targeting clojure.lang.IKVReduce). Of course then it is the responsibility
of that custom implementation to implement reduce-kv's semantics correctly.

Cheers,
Michał


On 30 April 2016 at 02:18, Timothy Baldridge  wrote:

> Yes, and it happens for most collections. Vectors, maps, etc. There's even
> a fast path for reduce-kv on maps that doesn't create key value entry
> objects.
>
>
> Timothy
>
> On Fri, Apr 29, 2016 at 6:06 PM, Mark Engelberg 
> wrote:
>
>> So you're saying that this is an optimization that is automatically
>> called when you invoke Clojure's standard reduce function on something like
>> a vector?
>>
>>
>> On Fri, Apr 29, 2016 at 1:14 PM, Alex Miller  wrote:
>>
>>> The main internal protocol is really CollReduce for collections that can
>>> reduce themselves.  InternalReduce is for concrete seq implementations that
>>> can reduce themselves.
>>>
>>> For cases where you are creating new things, you can also plug in a
>>> little more easily by implementing the IReduceInit (reduce with an init
>>> value) or IReduce (extends IReduceInit for the case where an init value is
>>> not supplied) Java interfaces. I would generally prefer these if you are
>>> creating a new thing.
>>>
>>>
>>> On Friday, April 29, 2016 at 2:41:10 PM UTC-5, Camilo Roca wrote:

 puzzler,
 No, Clojure actually has quite a lot of protocols for reducing
 "things". But they are so many that I got lost in which does what and how,
 so I wanted a clarification on the subject.

 Alex miller, excellent answer already gave me some overview of the
 topic.

 Here is a link to Clojure's protocols for reduce:
 https://github.com/clojure/clojure/blob/master/src/clj/clojure/core/protocols.clj
 

 El viernes, 29 de abril de 2016, 21:17:42 (UTC+2), puzzler escribió:
>
> By "internal reduce", are you all talking about the Clojure reducers
> library, or something else?
>
> --
>>> You received this message because you are subscribed to the Google
>>> Groups "Clojure" group.
>>> To post to this group, send email to clojure@googlegroups.com
>>> Note that posts from new members are moderated - please be patient with
>>> your first post.
>>> To unsubscribe from this group, send email to
>>> clojure+unsubscr...@googlegroups.com
>>> For more options, visit this group at
>>> http://groups.google.com/group/clojure?hl=en
>>> ---
>>> You received this message because you are subscribed to the Google
>>> Groups "Clojure" group.
>>> To unsubscribe from this group and stop receiving emails from it, send
>>> an email to clojure+unsubscr...@googlegroups.com.
>>> For more options, visit https://groups.google.com/d/optout.
>>>
>>
>> --
>> You received this message because you are subscribed to the Google
>> Groups "Clojure" group.
>> To post to this group, send email to clojure@googlegroups.com
>> Note that posts from new members are moderated - please be patient with
>> your first post.
>> To unsubscribe from this group, send email to
>> clojure+unsubscr...@googlegroups.com
>> For more options, visit this group at
>> http://groups.google.com/group/clojure?hl=en
>> ---
>> You received this message because you are subscribed to the Google Groups
>> "Clojure" group.
>> To unsubscribe from this group and stop receiving emails from it, send an
>> email to clojure+unsubscr...@googlegroups.com.
>> For more options, visit https://groups.google.com/d/optout.
>>
>
>
>
> --
> “One of the main causes of the fall of the Roman Empire was that–lacking
> zero–they had no way to indicate successful termination of their C
> programs.”
> (Robert Firth)
>
> --
> You received this message because you are subscribed to the Google
> Groups "Clojure" group.
> To post to this group, send email to clojure@googlegroups.com
> Note that posts from new members are moderated - please be patient with
> your first post.
> To unsubscribe from this group, send email to
> clojure+unsubscr...@googlegroups.com
> For more options, visit this group at
> http://groups.google.com/group/clojure?hl=en
> ---
> You received this message because you are subscribed to the Google Groups
> "Clojure" group.
> To unsubscribe from this group and stop receiving emails from it, send an
> email to clojure+unsubscr...@googlegroups.com.
> For more options, visit https://groups.google.com/d/optout.
>

-- 
You received this message because you are subscribed to the Google
Groups "Clojure" group.
To post to this group, send em

Re: which GC optimizations work better with Clojure?

[Nando Breiter]

I've found Censum by jClarity to be an excellent tool to tune JVM GC
parameters to your specific application running on a particular server. You
add a few GC parameters to enable logging that the tool needs, run your app
under load for enough time to get sufficient data, and then feed the log
directly into Censum, which analyzes it and then suggests improvements that
can be made to your GC parameters. There is also a list where you can ask
questions about the analysis and its suggestions and get in depth replies
from people who have extensive experience tuning JVMs.

https://www.jclarity.com/censum/



Aria Media Sagl
+41 (0)76 303 4477 cell
skype: ariamedia

On Sat, Apr 30, 2016 at 11:50 AM, Niels van Klaveren <
niels.vanklave...@gmail.com> wrote:

> GC parameters used by a lot of projects are handed down like traditions
> for years, even spreading to others. They're almost never re-evaluated,
> which means they might actually lose out on new performance improvements,
> or worse, might even cause performance regressions.
>
> Most parameters in the Overtone lib for instance have been the default for
> quite some time (ParNew GC, TLAB). Where restricting New Gen might be a
> good idea to reduce new gen GC latency, it can cause unwanted promotions,
> longer tenured GC's and in the end heap fragmentation resulting in a full
> defrag gc that can take 10s of seconds.
>
> Defaults are there for a reason, to take care of most situations. Any
> settings other than the three default Garbage Collectors (Default for
> shortest time to finish, CMS for shortest pause , G1 for short pauses but
> long running times) and at most a pause hint for their heuristics means
> you're entering case-specific  territory and should be treating it so.
>
> This means realistic benchmarking, profiling and comparisons between
> different settings. I'd treat any JVM product or library with more than 2
> GC parameters with care, because if they need so much tweaking as their
> default, you can be sure to need both a good understanding of their inner
> workings and JVM memory managent as well to keep them performing in your
> specific case.
>
> --
> You received this message because you are subscribed to the Google
> Groups "Clojure" group.
> To post to this group, send email to clojure@googlegroups.com
> Note that posts from new members are moderated - please be patient with
> your first post.
> To unsubscribe from this group, send email to
> clojure+unsubscr...@googlegroups.com
> For more options, visit this group at
> http://groups.google.com/group/clojure?hl=en
> ---
> You received this message because you are subscribed to the Google Groups
> "Clojure" group.
> To unsubscribe from this group and stop receiving emails from it, send an
> email to clojure+unsubscr...@googlegroups.com.
> For more options, visit https://groups.google.com/d/optout.
>

-- 
You received this message because you are subscribed to the Google
Groups "Clojure" group.
To post to this group, send email to clojure@googlegroups.com
Note that posts from new members are moderated - please be patient with your 
first post.
To unsubscribe from this group, send email to
clojure+unsubscr...@googlegroups.com
For more options, visit this group at
http://groups.google.com/group/clojure?hl=en
--- 
You received this message because you are subscribed to the Google Groups 
"Clojure" group.
To unsubscribe from this group and stop receiving emails from it, send an email 
to clojure+unsubscr...@googlegroups.com.
For more options, visit https://groups.google.com/d/optout.

Re: which GC optimizations work better with Clojure?

[Timothy Baldridge]

I should mention here that if you are tuning the GC anywhere except at the
end of your development cycle you're probably doing it wrong. 99% of the
time you'll get better performance by reviewing your algorithms, running a
CPU profiler, generating less garbage (via transients or something like
that),  or adding some missing type hints. You can spend a ton of time in
GC optimization, but I've never seen yield huge results. Maybe 10% here or
there, but I normally get orders of magnitude performance increases
spending just a few hours with a CPU profiler.

So it's one of those things where "if you don't know if you should change a
GC setting form the default, you probably don't".

On Sat, Apr 30, 2016 at 12:44 PM, Nando Breiter 
wrote:

> I've found Censum by jClarity to be an excellent tool to tune JVM GC
> parameters to your specific application running on a particular server. You
> add a few GC parameters to enable logging that the tool needs, run your app
> under load for enough time to get sufficient data, and then feed the log
> directly into Censum, which analyzes it and then suggests improvements that
> can be made to your GC parameters. There is also a list where you can ask
> questions about the analysis and its suggestions and get in depth replies
> from people who have extensive experience tuning JVMs.
>
> https://www.jclarity.com/censum/
>
>
>
> Aria Media Sagl
> +41 (0)76 303 4477 cell
> skype: ariamedia
>
> On Sat, Apr 30, 2016 at 11:50 AM, Niels van Klaveren <
> niels.vanklave...@gmail.com> wrote:
>
>> GC parameters used by a lot of projects are handed down like traditions
>> for years, even spreading to others. They're almost never re-evaluated,
>> which means they might actually lose out on new performance improvements,
>> or worse, might even cause performance regressions.
>>
>> Most parameters in the Overtone lib for instance have been the default
>> for quite some time (ParNew GC, TLAB). Where restricting New Gen might be a
>> good idea to reduce new gen GC latency, it can cause unwanted promotions,
>> longer tenured GC's and in the end heap fragmentation resulting in a full
>> defrag gc that can take 10s of seconds.
>>
>> Defaults are there for a reason, to take care of most situations. Any
>> settings other than the three default Garbage Collectors (Default for
>> shortest time to finish, CMS for shortest pause , G1 for short pauses but
>> long running times) and at most a pause hint for their heuristics means
>> you're entering case-specific  territory and should be treating it so.
>>
>> This means realistic benchmarking, profiling and comparisons between
>> different settings. I'd treat any JVM product or library with more than 2
>> GC parameters with care, because if they need so much tweaking as their
>> default, you can be sure to need both a good understanding of their inner
>> workings and JVM memory managent as well to keep them performing in your
>> specific case.
>>
>> --
>> You received this message because you are subscribed to the Google
>> Groups "Clojure" group.
>> To post to this group, send email to clojure@googlegroups.com
>> Note that posts from new members are moderated - please be patient with
>> your first post.
>> To unsubscribe from this group, send email to
>> clojure+unsubscr...@googlegroups.com
>> For more options, visit this group at
>> http://groups.google.com/group/clojure?hl=en
>> ---
>> You received this message because you are subscribed to the Google Groups
>> "Clojure" group.
>> To unsubscribe from this group and stop receiving emails from it, send an
>> email to clojure+unsubscr...@googlegroups.com.
>> For more options, visit https://groups.google.com/d/optout.
>>
>
> --
> You received this message because you are subscribed to the Google
> Groups "Clojure" group.
> To post to this group, send email to clojure@googlegroups.com
> Note that posts from new members are moderated - please be patient with
> your first post.
> To unsubscribe from this group, send email to
> clojure+unsubscr...@googlegroups.com
> For more options, visit this group at
> http://groups.google.com/group/clojure?hl=en
> ---
> You received this message because you are subscribed to the Google Groups
> "Clojure" group.
> To unsubscribe from this group and stop receiving emails from it, send an
> email to clojure+unsubscr...@googlegroups.com.
> For more options, visit https://groups.google.com/d/optout.
>



-- 
“One of the main causes of the fall of the Roman Empire was that–lacking
zero–they had no way to indicate successful termination of their C
programs.”
(Robert Firth)

-- 
You received this message because you are subscribed to the Google
Groups "Clojure" group.
To post to this group, send email to clojure@googlegroups.com
Note that posts from new members are moderated - please be patient with your 
first post.
To unsubscribe from this group, send email to
clojure+unsubscr...@googlegroups.com
For more options, visit this group at
http://groups.google.com/group/clojure?hl=en
--

[ANNs] growmonster and datomic-booties

[Daniel Higginbotham]

growmonster helps you write database fixtures
https://github.com/flyingmachine/growmonster 


datomic-booties helps you stand up a datomic database in boot (create the db, 
migrate schema and seed data, delete and recreate the db)
https://github.com/flyingmachine/datomic-booties 


-- 
You received this message because you are subscribed to the Google
Groups "Clojure" group.
To post to this group, send email to clojure@googlegroups.com
Note that posts from new members are moderated - please be patient with your 
first post.
To unsubscribe from this group, send email to
clojure+unsubscr...@googlegroups.com
For more options, visit this group at
http://groups.google.com/group/clojure?hl=en
--- 
You received this message because you are subscribed to the Google Groups 
"Clojure" group.
To unsubscribe from this group and stop receiving emails from it, send an email 
to clojure+unsubscr...@googlegroups.com.
For more options, visit https://groups.google.com/d/optout.