Re: Past and future of data.generators

[Linus Ericsson]

I do agree that the name data.generators is not where to look for a
controllable random source. A more specific name for these functions should
be considered.

The java.util.Random has been an issue for me in stress-testing random read
and writes to a huge memory-area by several threads. If I was to do it
again I would use the java.util.concurrent.ThreadLocalRandom to generate
random numbers in parallel. (j.u.c.TLR is only availiable in jdk >= 1.7.0,
clojure.core aims for 1.6.0 as well. The core.async library
do use ThreadLocalRandom. The reducers functionality has a conditional
import, which I think is only to be used as a very last resort in
clojure.core.

A surprise and caveat is that the performance was really bad when
live-generating random memory addresses - likely because of cachetrashing.
The performance was indeed much higher when using a prerealized (very
long) random sequence of random data.

A functionality for generating random memory addresses would likely benefit
from having a buffer for helping the hardware pre-fetch memory (which is
often a realistic scenario in stream processing).

summary:
- better namespace for random object/number generation
- ThreadLocalRandom is only avail in jdk 1.7.0
- stresstests do benefit from buffering incoming random data, which is more
realistic as well.

I will dig deeper in criterium to see if this is already implemented there.

/Linus

On Thursday, June 5, 2014, Mars0i  wrote:

> clojure.core provides a minimal set of functions for random effects: rand,
> rand-int, and rand-nth, currently with no simple ability to base these on a
> resettable random number generator or on different RNGs in different
> threads.  (But see this ticket
>  pointed out by Andy
> Fingerhut in another thread.)
>
> data.generators includes additional useful general-purpose functions
> involving random numbers and random choices, but this is entirely not
> obvious when you read the docstrings.  (Some of the docstrings are pretty
> mysterious.)  It's also not necessarily what one would guess from the name
> of the library.  (None of this is a criticism of anyone or anything about
> the project.  Data.generators is at an 0.n.m release stage.  I'm very
> grateful for the work that people have put in on it.)
>
> As I understand it, data.generators was split off from test.generative,
> which sounds like a good idea.So data.generators was intended to provide
> functions that generate random data for testing.  (I imagine that the
> existing documentation makes more sense in the context of test.generative,
> too.)
>
> However, what's in data.generator has more general applications, for
> people who want random numbers, samples, etc. outside of software testing.
> (In my case, that would be for random effects in scientific simulations.)
> Off the top of my head, it seems to me that these other applications might
> have slightly different needs from the use of data.generators by
> test.generative.
>
> For one thing, efficiency might matter a lot in some simulations, but not
> in software testing.  (At least, *I* wouldn't care if my test functions
> were slow.)  I'm not saying that functions in data.generator are slow, but
> I don't think there's a good reason to worry about making them efficient if
> they're only intended for software testing.
>
> Further, there are other needs than are currently provided by
> test.generators.  See the sampling functions in bigml/sampling
>  or Incanter ,
> for example, and lots of other random functions that Incanter provides.
> Some of those should remain in Incanter, of course, but I wonder whether
> Clojure would benefit from a contributed library that satisfied a set of
> core needs for random effects.  (Incanter partly builds on clojure.core's
> rand at this point.)
>
> Maybe data.generators is/will be that library.  Or maybe parts of
> data.generators would make more sense as part of a separate library
> (math.random? data.random? math.probability?) that could be split out of
> data.generators.  (If it doesn't make sense to split data.generators, then
> would a new name for the library be more appropriate?)
>
> Just some things I was wondering about.  Curious to see what others say.
>
> (Fun tip: Check out data.generators' anything function, which is like
> Emacs' Zippy the Pinhead functions for people who prefer industrial atonal
> music composed by randomly filtered Jackson Pollock paintings, to speech.
> Or: People who want to thoroughly test their functions by throwing random
> randomly-typed data at them.)
>
> --
> You received this message because you are subscribed to the Google
> Groups "Clojure" group.
> To post to this group, send email to clojure@googlegroups.com
> 
> Note that posts from new members are moderated - please be patient with
> your first post.
> To unsubscribe from this group, send email to
> clojure+unsubscr...@googlegroups

Re: Is it the right Clojure group for a newbie

[Darren Syzling]

There are also a few other lisp/scheme books (including the little schemer)
that people have worked through and posted their Clojure code which you
might find instructive.


On 5 June 2014 05:10, Mars0i  wrote:

>
>
> On Tuesday, June 3, 2014 8:41:33 AM UTC-5, Gregg Reynolds wrote:
>>
>> I know you've asked for online resources, but I can't resist a plug for The
>> Little Schemer .  It's
>> short, very clear, starts at the very beginning, and you don't even need a
>> computer to read it.  (It's also annoyingly cute, but you can overlook that
>> with a little effort.)  It's far and away the best introduction I've ever
>> come across to the kind of thinking one needs to master in order to program
>> in lispy languages like Clojure.  It uses scheme, but everything translates
>> more or less directly to Clojure.
>>
>
> I'll second Gregg's recommendation, especially given your comment that
> you're coming from a non-technical background.  This book will get you into
> a good mindset for Clojure programming.
>
> --
> You received this message because you are subscribed to the Google
> Groups "Clojure" group.
> To post to this group, send email to clojure@googlegroups.com
> Note that posts from new members are moderated - please be patient with
> your first post.
> To unsubscribe from this group, send email to
> clojure+unsubscr...@googlegroups.com
> For more options, visit this group at
> http://groups.google.com/group/clojure?hl=en
> ---
> You received this message because you are subscribed to the Google Groups
> "Clojure" group.
> To unsubscribe from this group and stop receiving emails from it, send an
> email to clojure+unsubscr...@googlegroups.com.
> For more options, visit https://groups.google.com/d/optout.
>



-- 
Regards
Darren

-- 
You received this message because you are subscribed to the Google
Groups "Clojure" group.
To post to this group, send email to clojure@googlegroups.com
Note that posts from new members are moderated - please be patient with your 
first post.
To unsubscribe from this group, send email to
clojure+unsubscr...@googlegroups.com
For more options, visit this group at
http://groups.google.com/group/clojure?hl=en
--- 
You received this message because you are subscribed to the Google Groups 
"Clojure" group.
To unsubscribe from this group and stop receiving emails from it, send an email 
to clojure+unsubscr...@googlegroups.com.
For more options, visit https://groups.google.com/d/optout.

Re: Are zippers the right choice for "sequence-like" trees?

[Paul Butcher]

Thanks Jason - that’s helpful.

--
paul.butcher->msgCount++

Silverstone, Brands Hatch, Donington Park...
Who says I have a one track mind?

http://www.paulbutcher.com/
LinkedIn: http://www.linkedin.com/in/paulbutcher
Skype: paulrabutcher

Author of Seven Concurrency Models in Seven Weeks: When Threads Unravel
http://pragprog.com/book/pb7con

On 4 June 2014 at 14:33:27, Jason Felice (jason.m.fel...@gmail.com) wrote:

In general, I've found that zippers make complicated edits super-easy, while a 
recursive phrasing of the same algorithm - if it exists and isn't super 
complicated to write - performs better and gives more control over structural 
sharing.  I might prove out an algorithm with zippers, but when the pedal hits 
the road for performance, I'll try to rephrase it.

If you can phrase things in terms of (catvec (subvec ...) (subvec ...)), then 
I'll bet (but you should check) that rrb-vector will perform better and do more 
structural sharing.



On Wed, Jun 4, 2014 at 8:29 AM, Paul Butcher  wrote:
I am working with “sequence like” trees - by which I mean that they’re very 
broad (typically the root node will have several thousand children) and shallow 
(no more than 2 levels deep). I’m often dealing with a degenerate tree that’s 
really just a sequence (all the nodes in the tree are children of the root).

A typical edit is either changing the value of a single element, inserting a 
single element, or deleting a small set of contiguous elements.

I’m currently using clojure.zip to edit the tree, and it’s working just fine. 
But I’m concerned that it won’t be giving me much value in terms of structure 
sharing for this type of tree and this type of edit.

So I have two questions:

1) What would be a good way for me to instrument my code to determine what 
level of structure sharing I am achieving?

2) Should I consider switching to something based on (say?) rrb-vector?

--
paul.butcher->msgCount++

Silverstone, Brands Hatch, Donington Park...
Who says I have a one track mind?

http://www.paulbutcher.com/
LinkedIn: http://www.linkedin.com/in/paulbutcher
Skype: paulrabutcher

Author of Seven Concurrency Models in Seven Weeks: When Threads Unravel
http://pragprog.com/book/pb7con
--
You received this message because you are subscribed to the Google
Groups "Clojure" group.
To post to this group, send email to clojure@googlegroups.com
Note that posts from new members are moderated - please be patient with your 
first post.
To unsubscribe from this group, send email to
clojure+unsubscr...@googlegroups.com
For more options, visit this group at
http://groups.google.com/group/clojure?hl=en
---
You received this message because you are subscribed to the Google Groups 
"Clojure" group.
To unsubscribe from this group and stop receiving emails from it, send an email 
to clojure+unsubscr...@googlegroups.com.
For more options, visit https://groups.google.com/d/optout.

--
You received this message because you are subscribed to the Google
Groups "Clojure" group.
To post to this group, send email to clojure@googlegroups.com
Note that posts from new members are moderated - please be patient with your 
first post.
To unsubscribe from this group, send email to
clojure+unsubscr...@googlegroups.com
For more options, visit this group at
http://groups.google.com/group/clojure?hl=en
---
You received this message because you are subscribed to the Google Groups 
"Clojure" group.
To unsubscribe from this group and stop receiving emails from it, send an email 
to clojure+unsubscr...@googlegroups.com.
For more options, visit https://groups.google.com/d/optout.

-- 
You received this message because you are subscribed to the Google
Groups "Clojure" group.
To post to this group, send email to clojure@googlegroups.com
Note that posts from new members are moderated - please be patient with your 
first post.
To unsubscribe from this group, send email to
clojure+unsubscr...@googlegroups.com
For more options, visit this group at
http://groups.google.com/group/clojure?hl=en
--- 
You received this message because you are subscribed to the Google Groups 
"Clojure" group.
To unsubscribe from this group and stop receiving emails from it, send an email 
to clojure+unsubscr...@googlegroups.com.
For more options, visit https://groups.google.com/d/optout.

Re: Past and future of data.generators

[Mikera]

One of the challenges with random number generation is that there are quite 
a few specialised requirements. I don't believe a generic approach can meet 
all needs. I think we actually need a few things:

1. Better implementation for clojure.core/rand etc. I think conditional 
usage of j.u.c.ThreadLocalRandom for Java > 1.7 would be great if we can 
make it work - there are plenty of concurrent workloads where a shared 
regular java.util.Random isn't a good solution.
2. A library generic random number generation tools (e.g. "data.random" - 
should be general purpose, able to generate a wide range of useful 
districutions, allow arbitrary java.util.Random instances to be passed as 
seeds etc.)
3. More specialised solutions can live in specific libraries (e.g. 
core.matrix will be getting support for generation of random matrices 
etc.). Often specialised implementations will offer much better performance 
for specific use cases, so we need to keep this option open. An example 
would be generating large random boolean matrices - generating and storing 
individual bits in bulk is *much* more efficient than going via generic 
random number functions for each bit.

I think we should clearly separate random number generation from sample 
data construction. The latter certainly depends upon the former, but random 
numbers have a lot of other independent use cases. Hence I'm in favour of 
something like "data.random" being separate from "data.generators"

On Thursday, 5 June 2014 05:53:10 UTC+1, Mars0i wrote:
>
> clojure.core provides a minimal set of functions for random effects: rand, 
> rand-int, and rand-nth, currently with no simple ability to base these on a 
> resettable random number generator or on different RNGs in different 
> threads.  (But see this ticket 
>  pointed out by Andy 
> Fingerhut in another thread.)
>
> data.generators includes additional useful general-purpose functions 
> involving random numbers and random choices, but this is entirely not 
> obvious when you read the docstrings.  (Some of the docstrings are pretty 
> mysterious.)  It's also not necessarily what one would guess from the name 
> of the library.  (None of this is a criticism of anyone or anything about 
> the project.  Data.generators is at an 0.n.m release stage.  I'm very 
> grateful for the work that people have put in on it.)
>
> As I understand it, data.generators was split off from test.generative, 
> which sounds like a good idea.So data.generators was intended to provide 
> functions that generate random data for testing.  (I imagine that the 
> existing documentation makes more sense in the context of test.generative, 
> too.)
>
> However, what's in data.generator has more general applications, for 
> people who want random numbers, samples, etc. outside of software testing.  
> (In my case, that would be for random effects in scientific simulations.)  
> Off the top of my head, it seems to me that these other applications might 
> have slightly different needs from the use of data.generators by 
> test.generative.  
>
> For one thing, efficiency might matter a lot in some simulations, but not 
> in software testing.  (At least, *I* wouldn't care if my test functions 
> were slow.)  I'm not saying that functions in data.generator are slow, but 
> I don't think there's a good reason to worry about making them efficient if 
> they're only intended for software testing.
>
> Further, there are other needs than are currently provided by 
> test.generators.  See the sampling functions in bigml/sampling 
>  or Incanter , 
> for example, and lots of other random functions that Incanter provides.  
> Some of those should remain in Incanter, of course, but I wonder whether 
> Clojure would benefit from a contributed library that satisfied a set of 
> core needs for random effects.  (Incanter partly builds on clojure.core's 
> rand at this point.)
>
> Maybe data.generators is/will be that library.  Or maybe parts of 
> data.generators would make more sense as part of a separate library 
> (math.random? data.random? math.probability?) that could be split out of 
> data.generators.  (If it doesn't make sense to split data.generators, then 
> would a new name for the library be more appropriate?)
>
> Just some things I was wondering about.  Curious to see what others say.
>
> (Fun tip: Check out data.generators' anything function, which is like 
> Emacs' Zippy the Pinhead functions for people who prefer industrial atonal 
> music composed by randomly filtered Jackson Pollock paintings, to speech.  
> Or: People who want to thoroughly test their functions by throwing random 
> randomly-typed data at them.)
>

-- 
You received this message because you are subscribed to the Google
Groups "Clojure" group.
To post to this group, send email to clojure@googlegroups.com
Note that posts from new members are moderated - please be pat

Re: What's clojure killer app? I don't see any.

[douglas smith]

found Atl-g in snap-shop -thankyou again -doug

On Thursday, June 5, 2014 12:03:59 AM UTC-4, douglas smith wrote:
>
> Jony, Hey thanks 
>
> Just finish watching and reading about Gorilla-Repl (will be digging 
> deeper) and WOW really clean and simple UI -Nice. 
> Much Much easier to install than IPython notebook was and its Clojure!
> Couldn't follow much of your coding as I said I am just starting out.
> I like the keystrokes and no buttons - I do need to remap chrome-browser 
> ctrl-g to something else -don't use it anyway.
> I like how you can show the underling 'value' of things.
>
> Couldn't this type of application run remotely and potentially provide a 
> 'live work space' for dislocated groups of students?
> Maybe a keystroke that toggles screen control.  
> - a 'kitchen table coders' thing but virtual.
>  -just a thought.
>
> I really am convinced that clojure is a great first language to learn- 
>  and I'm determined to learn it.
>
> Thanks -Doug
>
>  
>  
>
>
>
>  
>
>
>  
>
> On Tuesday, June 3, 2014 6:54:31 AM UTC-4, Jony Hudson wrote:
>>
>>
>>
>> On Monday, 2 June 2014 18:42:32 UTC+1, douglas smith wrote:
>>  
>>
>>> A killer app for me and I think MANY others like me would be very 
>>> similar to a Kovas' 'Session' 'pretty like Light Table'  and beefy like 
>>> IPythons Notebooks. 
>>>
>>
>> Hope you'll excuse a shameless plug, but have you seen Gorilla REPL (
>> http://gorilla-repl.org) ? It may or may not have some of the things 
>> you're looking for :-)
>>
>>
>> Jony 
>>
>

-- 
You received this message because you are subscribed to the Google
Groups "Clojure" group.
To post to this group, send email to clojure@googlegroups.com
Note that posts from new members are moderated - please be patient with your 
first post.
To unsubscribe from this group, send email to
clojure+unsubscr...@googlegroups.com
For more options, visit this group at
http://groups.google.com/group/clojure?hl=en
--- 
You received this message because you are subscribed to the Google Groups 
"Clojure" group.
To unsubscribe from this group and stop receiving emails from it, send an email 
to clojure+unsubscr...@googlegroups.com.
For more options, visit https://groups.google.com/d/optout.

Re: Past and future of data.generators

[Thomas]

Hi,

I have used http://maths.uncommons.org/ in a few of my projects, so that 
could be used in data.random. I have also played with the random.org API in 
the past as a source of random numbers.

Thomas

ps. in one of my use cases I also care about the performance of the random 
generator as I potentially need to create loads (millions) and then 
performance can have in impact.

-- 
You received this message because you are subscribed to the Google
Groups "Clojure" group.
To post to this group, send email to clojure@googlegroups.com
Note that posts from new members are moderated - please be patient with your 
first post.
To unsubscribe from this group, send email to
clojure+unsubscr...@googlegroups.com
For more options, visit this group at
http://groups.google.com/group/clojure?hl=en
--- 
You received this message because you are subscribed to the Google Groups 
"Clojure" group.
To unsubscribe from this group and stop receiving emails from it, send an email 
to clojure+unsubscr...@googlegroups.com.
For more options, visit https://groups.google.com/d/optout.

Integrating clojurescript and clojure in one library

[Sven Richter]

Hi,

While working on the admin interface of friend-ui 
(https://groups.google.com/forum/#!searchin/clojure/friend-ui/clojure/QoJHPFXjDuc/IbBsHmOLNdIJ)
 
I got the idea to add some cljs (om / kioo) to my library and then provide 
both, the compiled js and the compiled java code in one jar.

Now, it works partially, I can generate both of them and pack them with 
lein jar into one jar file. But whenever I want to use the generated js 
(provided by the friendui jar) files in a different project the ring server 
cannot find them.

Now I wonder if my approach to pack to different things into one library 
does make sense or even works somehow. I got the feeling that it's wrong, 
but I would like to hear some opinions about it.

Best Regards,
Sven

-- 
You received this message because you are subscribed to the Google
Groups "Clojure" group.
To post to this group, send email to clojure@googlegroups.com
Note that posts from new members are moderated - please be patient with your 
first post.
To unsubscribe from this group, send email to
clojure+unsubscr...@googlegroups.com
For more options, visit this group at
http://groups.google.com/group/clojure?hl=en
--- 
You received this message because you are subscribed to the Google Groups 
"Clojure" group.
To unsubscribe from this group and stop receiving emails from it, send an email 
to clojure+unsubscr...@googlegroups.com.
For more options, visit https://groups.google.com/d/optout.

Re: Eval-ing a map expression in Emacs Live doesn't seem to work

[gvim]

On 05/06/2014 00:57, J Irving wrote:

Your cursor was probably on the closing paren at the end - you eval'd
the previous expression, which was the vector.

Check out the key bindings here:

https://github.com/clojure-emacs/cider#cider-mode

If your cursor is anywhere on that expression, you probably want C-c C-c.




On the next line C-x C-e works as expected. Don't know where I read to 
place the cursor on the closing paren with C-x C-e.


If you could help with this other problem I'd appreciate it. In Emacs 
Live, after cider-jack-in, I go to the repl window, change namespace to 
same as my open Lein project file and attempt to call a function (foo 
"name") defined in the open file. When I try to eval it, with C-x C-e 
from the following line, I get the error: "nREPL server not connected".


gvim

--
You received this message because you are subscribed to the Google
Groups "Clojure" group.
To post to this group, send email to clojure@googlegroups.com
Note that posts from new members are moderated - please be patient with your 
first post.
To unsubscribe from this group, send email to
clojure+unsubscr...@googlegroups.com
For more options, visit this group at
http://groups.google.com/group/clojure?hl=en
--- 
You received this message because you are subscribed to the Google Groups "Clojure" group.

To unsubscribe from this group and stop receiving emails from it, send an email 
to clojure+unsubscr...@googlegroups.com.
For more options, visit https://groups.google.com/d/optout.

Re: Effective Clojure book?

[Charlie Griefer]

On Jun 4, 2014, at 6:30 PM, Mike Fikes  wrote:

> Are there any books yet that prescribe best practices for Clojure, à la 
> Meyers or Bloch?


Not a book, but there is the Clojure Style Guide at 
https://github.com/bbatsov/clojure-style-guide

--
Charlie Griefer
http://charlie.griefer.com

"Give light, and the darkness will disappear of itself." 
-- Desiderius Erasmus

-- 
You received this message because you are subscribed to the Google
Groups "Clojure" group.
To post to this group, send email to clojure@googlegroups.com
Note that posts from new members are moderated - please be patient with your 
first post.
To unsubscribe from this group, send email to
clojure+unsubscr...@googlegroups.com
For more options, visit this group at
http://groups.google.com/group/clojure?hl=en
--- 
You received this message because you are subscribed to the Google Groups 
"Clojure" group.
To unsubscribe from this group and stop receiving emails from it, send an email 
to clojure+unsubscr...@googlegroups.com.
For more options, visit https://groups.google.com/d/optout.

[ANN] Avi 0.1.4

[Jason Felice]

This is the "it lives!" version.  It's still not pleasant to use, but you
can use it to code!

Since last we met: i,o,O,a,A,:w,:wq,:q!
avi

A lively vi.
Vision

We love vim. We want more! Test coverage. Flexibilty. Live REPLs!
Guiding
Principles

   - Test driven. All functionality covered by tests.
   - Don't defeat vim muscle memory.
   - Code is for people. Be expressive as hell.
   - Be friendly. Especially to noobs.


Contributing

We track upcoming work on a Trello board .
This board has many small things that are easy to pick up, and we'd love to
see you.

I (Jason Felice) would love to walk through the code with you pretty much
any time during the US Eastern work day. Ping me any time on Twitter - I'm
@eraserhd  - or gchat.
Status

Our intention is for the unit tests to provide friendly documention of
what's been implemented so far. To run the unit tests with verbose output,
use

$ lein midje :print-facts

Installing

   - Works with Leiningen 2.3.4
   - On Linux, ncurses-dev or curses-dev or equivalent must be installed.

$ git clone https://github.com/maitria/avi.git
$ cd avi
$ lein install

On Linux, you'll likely not have access to /usr/local. In that case you'll
need to use sudo lein install instead.
License

Copyright 2014 Maitria

You have permission to use this in any way you like (modify it, sell it,
republish it), provided you agree to all the following conditions:

   - you don't mislead anyone about it
   - you don't interfere with our ability to use it
   - you release us from any claims of liability if it causes problems for
   you

-- 
You received this message because you are subscribed to the Google
Groups "Clojure" group.
To post to this group, send email to clojure@googlegroups.com
Note that posts from new members are moderated - please be patient with your 
first post.
To unsubscribe from this group, send email to
clojure+unsubscr...@googlegroups.com
For more options, visit this group at
http://groups.google.com/group/clojure?hl=en
--- 
You received this message because you are subscribed to the Google Groups 
"Clojure" group.
To unsubscribe from this group and stop receiving emails from it, send an email 
to clojure+unsubscr...@googlegroups.com.
For more options, visit https://groups.google.com/d/optout.

Re: non-lazy clojure?

[Lee Spector]

A followup on these issues, after experimenting a bit with reducers, just for 
anyone who may be interested:

- Naive use of reducers in place of seq functions, throughout my program, got 
messy and led to some new problems with memory management, I think (or at least 
to jerkier execution over time). I realized that reducers probably shouldn't be 
used so indiscriminately.

- With more moderate use of reducers, and using fold, I do indeed see 
performance improvements relative to sequential code and also relative to the 
previous version of my code that used agents for concurrency. I don't actually 
seem to be getting much benefit from the way that reducers avoid building 
intermediate results -- all of the benefit appears to be due to using a 
combination of r/fold and one r/map to process collection elements 
concurrently. I've only done small numbers of tests and they all involve 
randomness, but I do see improvements on the order of 1/3.

- The best behaving/looking approach for me seems to be to stick with lazy 
sequence operations and aggressive de-lazifying for the bulk of my code 
(possibly cleaning some things up by defining my own forv, repeatv, etc., or 
possibly rewriting some of the core lazy sequence functions to not be lazy in 
the first place), but to use r/fold and r/map for concurrency for my top-level 
tasks. I'm getting the best results by using reducers only in one place, in my 
definition of a concurrent mapping function, which I then use only for 
top-level functions (never nested):

(defn pmapall
   [f coll]
   (if single-thread-mode
 (doall (map f coll))
 (r/fold 1 r/cat r/append! (r/map f coll

This replaces my previous agent-based version:

(defn pmapall
   [f coll]
   (if single-thread-mode
 (doall (map f coll))
 (let [agents (map #(agent % :error-handler 
   (fn [agnt except] (clojure.repl/pst except 1000) 
(System/exit 0))) 
   coll)]
   (dorun (map #(send % f) agents))
   (apply await agents)
   (doall (map deref agents)

The one that uses reducers is neater and seems to make my program run faster, 
although I'm not really sure why.

In any event, I'm using reducers now only as an alternative concurrency 
mechanism and solving my original problems by de-lazification.

 -Lee



On Jun 4, 2014, at 7:21 PM, Lee Spector  wrote:

> 
> On Jun 4, 2014, at 1:29 PM, Timothy Baldridge  wrote:
> 
>> Although your original complaint was about clojure seqs being lazy. It 
>> should be noted that reducers are also lazy down to the point of a fold or 
>> reduce, so I'm not sure what you're really getting there. It wouldn't be 
>> hard at all to write map, filter, remove, etc. in terms of list operations. 
>> Perhaps that's what you're looking for? If I understand your original 
>> complaint, you want map, filter, etc, to be eager.
>> 
> 
> True, my original concern was (and my main concern still is) to avoid crashes 
> (in this case, stack overflow errors) that stem from unexpected (to me) 
> consequences of laziness.
> 
> Those problems could indeed be solved by rewriting map and filter etc, or 
> (easier) by "wringing out the laziness" wherever it arises, e.g. by forcing 
> everything to be a vector (or a list, as you suggest). It's a little 
> cumbersome to do this everywhere, though, and I was wondering if there was an 
> existing library or approach for this issue.
> 
> Then, however, it was suggested that I could have my cake and another kind of 
> cake too, by using reducers. The suggestion was that this could banish the 
> laziness-related issues while also providing significant performance 
> improvements. Sounds good to me! I'm still working on getting my current 
> project to behave well using reducers, but it seems promising.
> 
>> 
>> On the other hand, time spent learning how lazy seqs work, and the caveats 
>> involved will make it easier for you to understand the code produced by the 
>> rest of the community. So perhaps that's the better option. 
> 
> I agree that it's important to understand lazy sequences, and I do think I 
> understand the core concepts reasonably well (and I've implemented lazy 
> evaluation in other languages, etc.). But I've also traced some really 
> annoying and hard to find bugs to unexpected (and sometimes never fully 
> explained) consequences of laziness, so I'd like to find the best ways to 
> avoid it when I don't really want it. If that best way turns out to make 
> things run faster too then that'd be fantastic.
> 
> -Lee

-- 
You received this message because you are subscribed to the Google
Groups "Clojure" group.
To post to this group, send email to clojure@googlegroups.com
Note that posts from new members are moderated - please be patient with your 
first post.
To unsubscribe from this group, send email to
clojure+unsubscr...@googlegroups.com
For more options, visit this group at
http://groups.google.com/group/clojure?hl=en
--- 
You received this messag

Re: non-lazy clojure?

[Lee Spector]

Oops -- something was wrong with my benchmarks, and my "improvements on the 
order of 1/3" was wrong. I still see improvements with r/fold as compared to my 
agent-based approach, but the difference now appears to be only something like 
1/20.

 -Lee

On Jun 5, 2014, at 7:19 PM, Lee Spector  wrote:

> 
> A followup on these issues, after experimenting a bit with reducers, just for 
> anyone who may be interested:
> 
> - Naive use of reducers in place of seq functions, throughout my program, got 
> messy and led to some new problems with memory management, I think (or at 
> least to jerkier execution over time). I realized that reducers probably 
> shouldn't be used so indiscriminately.
> 
> - With more moderate use of reducers, and using fold, I do indeed see 
> performance improvements relative to sequential code and also relative to the 
> previous version of my code that used agents for concurrency. I don't 
> actually seem to be getting much benefit from the way that reducers avoid 
> building intermediate results -- all of the benefit appears to be due to 
> using a combination of r/fold and one r/map to process collection elements 
> concurrently. I've only done small numbers of tests and they all involve 
> randomness, but I do see improvements on the order of 1/3.
> 
> - The best behaving/looking approach for me seems to be to stick with lazy 
> sequence operations and aggressive de-lazifying for the bulk of my code 
> (possibly cleaning some things up by defining my own forv, repeatv, etc., or 
> possibly rewriting some of the core lazy sequence functions to not be lazy in 
> the first place), but to use r/fold and r/map for concurrency for my 
> top-level tasks. I'm getting the best results by using reducers only in one 
> place, in my definition of a concurrent mapping function, which I then use 
> only for top-level functions (never nested):
> 
> (defn pmapall
>   [f coll]
>   (if single-thread-mode
> (doall (map f coll))
> (r/fold 1 r/cat r/append! (r/map f coll
> 
> This replaces my previous agent-based version:
> 
> (defn pmapall
>   [f coll]
>   (if single-thread-mode
> (doall (map f coll))
> (let [agents (map #(agent % :error-handler 
>   (fn [agnt except] (clojure.repl/pst except 
> 1000) (System/exit 0))) 
>   coll)]
>   (dorun (map #(send % f) agents))
>   (apply await agents)
>   (doall (map deref agents)
> 
> The one that uses reducers is neater and seems to make my program run faster, 
> although I'm not really sure why.
> 
> In any event, I'm using reducers now only as an alternative concurrency 
> mechanism and solving my original problems by de-lazification.
> 
> -Lee
> 
> 
> 
> On Jun 4, 2014, at 7:21 PM, Lee Spector  wrote:
> 
>> 
>> On Jun 4, 2014, at 1:29 PM, Timothy Baldridge  wrote:
>> 
>>> Although your original complaint was about clojure seqs being lazy. It 
>>> should be noted that reducers are also lazy down to the point of a fold or 
>>> reduce, so I'm not sure what you're really getting there. It wouldn't be 
>>> hard at all to write map, filter, remove, etc. in terms of list operations. 
>>> Perhaps that's what you're looking for? If I understand your original 
>>> complaint, you want map, filter, etc, to be eager.
>>> 
>> 
>> True, my original concern was (and my main concern still is) to avoid 
>> crashes (in this case, stack overflow errors) that stem from unexpected (to 
>> me) consequences of laziness.
>> 
>> Those problems could indeed be solved by rewriting map and filter etc, or 
>> (easier) by "wringing out the laziness" wherever it arises, e.g. by forcing 
>> everything to be a vector (or a list, as you suggest). It's a little 
>> cumbersome to do this everywhere, though, and I was wondering if there was 
>> an existing library or approach for this issue.
>> 
>> Then, however, it was suggested that I could have my cake and another kind 
>> of cake too, by using reducers. The suggestion was that this could banish 
>> the laziness-related issues while also providing significant performance 
>> improvements. Sounds good to me! I'm still working on getting my current 
>> project to behave well using reducers, but it seems promising.
>> 
>>> 
>>> On the other hand, time spent learning how lazy seqs work, and the caveats 
>>> involved will make it easier for you to understand the code produced by the 
>>> rest of the community. So perhaps that's the better option. 
>> 
>> I agree that it's important to understand lazy sequences, and I do think I 
>> understand the core concepts reasonably well (and I've implemented lazy 
>> evaluation in other languages, etc.). But I've also traced some really 
>> annoying and hard to find bugs to unexpected (and sometimes never fully 
>> explained) consequences of laziness, so I'd like to find the best ways to 
>> avoid it when I don't really want it. If that best way turns out to make 
>> things run faster too then that'd be fantastic.
>> 
>> -

arguments passed to a macro

[Christopher Howard]

Is there a trick for "pre-expanding" some of the arguments passed to a
macro? Say you wanted to do this

code:

(def swing-imports '(javax.swing JFrame JButton ImageIcon JPanel))

(ns example.myprogram
  (:import swing-imports))
  ...


ns will complain that it can't find the "swing-imports" class.

I've been trying various things I thought might work (like backtick
evaluation and evals) but haven't made progress.

-- 
You received this message because you are subscribed to the Google
Groups "Clojure" group.
To post to this group, send email to clojure@googlegroups.com
Note that posts from new members are moderated - please be patient with your 
first post.
To unsubscribe from this group, send email to
clojure+unsubscr...@googlegroups.com
For more options, visit this group at
http://groups.google.com/group/clojure?hl=en
--- 
You received this message because you are subscribed to the Google Groups 
"Clojure" group.
To unsubscribe from this group and stop receiving emails from it, send an email 
to clojure+unsubscr...@googlegroups.com.
For more options, visit https://groups.google.com/d/optout.

Re: [ANN] core.async (and more) video tutorials

[Dylan Butman]

Thanks for doing these Tim! Very good stuff.

Would like to echo the above results for code. Can you put what you're 
working from on github for all the videos? I found myself wanting to use 
some of your pipeline stuff today and I'd rather not watch 20 minutes of 
video again to type it out

-- 
You received this message because you are subscribed to the Google
Groups "Clojure" group.
To post to this group, send email to clojure@googlegroups.com
Note that posts from new members are moderated - please be patient with your 
first post.
To unsubscribe from this group, send email to
clojure+unsubscr...@googlegroups.com
For more options, visit this group at
http://groups.google.com/group/clojure?hl=en
--- 
You received this message because you are subscribed to the Google Groups 
"Clojure" group.
To unsubscribe from this group and stop receiving emails from it, send an email 
to clojure+unsubscr...@googlegroups.com.
For more options, visit https://groups.google.com/d/optout.

Re: [ANN] core.async (and more) video tutorials

[Dylan Butman]

>
> Thanks for doing these Tim! Very good stuff.
>
> Would like to echo the above requests for source code. Can you put what 
> you're working from on github for all the videos? I found myself wanting to 
> use some of your pipeline stuff today and I'd rather not watch 20 minutes 
> of video again to type it out
>

-- 
You received this message because you are subscribed to the Google
Groups "Clojure" group.
To post to this group, send email to clojure@googlegroups.com
Note that posts from new members are moderated - please be patient with your 
first post.
To unsubscribe from this group, send email to
clojure+unsubscr...@googlegroups.com
For more options, visit this group at
http://groups.google.com/group/clojure?hl=en
--- 
You received this message because you are subscribed to the Google Groups 
"Clojure" group.
To unsubscribe from this group and stop receiving emails from it, send an email 
to clojure+unsubscr...@googlegroups.com.
For more options, visit https://groups.google.com/d/optout.

Re: non-lazy clojure?

[Gary Johnson]

Fair enough. Fortunately, Clojure provides so many different tools to 
select from in creating you perfect recipe. ;-)

I'm glad to hear that reducers ultimately provided you with some benefits 
over your previous concurrency approach.
The one thing that seems rather odd to me though is that your group-size is 
1. I'm presuming that the function you're r/mapping must take a substantial 
amount of time and resources for that to be efficient. Have you 
experimented with larger group sizes to avoid too much thread swapping?

  ~Gary

-- 
You received this message because you are subscribed to the Google
Groups "Clojure" group.
To post to this group, send email to clojure@googlegroups.com
Note that posts from new members are moderated - please be patient with your 
first post.
To unsubscribe from this group, send email to
clojure+unsubscr...@googlegroups.com
For more options, visit this group at
http://groups.google.com/group/clojure?hl=en
--- 
You received this message because you are subscribed to the Google Groups 
"Clojure" group.
To unsubscribe from this group and stop receiving emails from it, send an email 
to clojure+unsubscr...@googlegroups.com.
For more options, visit https://groups.google.com/d/optout.

Re: arguments passed to a macro

[James Reeves]

Well, the easiest option is to just use the import function:

(ns example.myprogram)
(import swing-imports)

There is a reader macro, #=, that might work, but I wouldn't recommend it:

(ns example.myprogram
  (:import #=swing-imports))

- James


On 6 June 2014 01:12, Christopher Howard  wrote:

> Is there a trick for "pre-expanding" some of the arguments passed to a
> macro? Say you wanted to do this
>
> code:
> 
> (def swing-imports '(javax.swing JFrame JButton ImageIcon JPanel))
>
> (ns example.myprogram
>   (:import swing-imports))
>   ...
> 
>
> ns will complain that it can't find the "swing-imports" class.
>
> I've been trying various things I thought might work (like backtick
> evaluation and evals) but haven't made progress.
>
> --
> You received this message because you are subscribed to the Google
> Groups "Clojure" group.
> To post to this group, send email to clojure@googlegroups.com
> Note that posts from new members are moderated - please be patient with
> your first post.
> To unsubscribe from this group, send email to
> clojure+unsubscr...@googlegroups.com
> For more options, visit this group at
> http://groups.google.com/group/clojure?hl=en
> ---
> You received this message because you are subscribed to the Google Groups
> "Clojure" group.
> To unsubscribe from this group and stop receiving emails from it, send an
> email to clojure+unsubscr...@googlegroups.com.
> For more options, visit https://groups.google.com/d/optout.
>

-- 
You received this message because you are subscribed to the Google
Groups "Clojure" group.
To post to this group, send email to clojure@googlegroups.com
Note that posts from new members are moderated - please be patient with your 
first post.
To unsubscribe from this group, send email to
clojure+unsubscr...@googlegroups.com
For more options, visit this group at
http://groups.google.com/group/clojure?hl=en
--- 
You received this message because you are subscribed to the Google Groups 
"Clojure" group.
To unsubscribe from this group and stop receiving emails from it, send an email 
to clojure+unsubscr...@googlegroups.com.
For more options, visit https://groups.google.com/d/optout.

Re: [ANN] core.async (and more) video tutorials

[Timothy Baldridge]

Code is here: https://github.com/halgari/clojure-tutorial-source

I'll try to get a forum setup soon, or at least some sort of mailing list


Timothy Baldridge


On Thu, Jun 5, 2014 at 6:15 PM, Dylan Butman  wrote:

> Thanks for doing these Tim! Very good stuff.
>>
>> Would like to echo the above requests for source code. Can you put what
>> you're working from on github for all the videos? I found myself wanting to
>> use some of your pipeline stuff today and I'd rather not watch 20 minutes
>> of video again to type it out
>>
>  --
> You received this message because you are subscribed to the Google
> Groups "Clojure" group.
> To post to this group, send email to clojure@googlegroups.com
> Note that posts from new members are moderated - please be patient with
> your first post.
> To unsubscribe from this group, send email to
> clojure+unsubscr...@googlegroups.com
> For more options, visit this group at
> http://groups.google.com/group/clojure?hl=en
> ---
> You received this message because you are subscribed to the Google Groups
> "Clojure" group.
> To unsubscribe from this group and stop receiving emails from it, send an
> email to clojure+unsubscr...@googlegroups.com.
> For more options, visit https://groups.google.com/d/optout.
>



-- 
“One of the main causes of the fall of the Roman Empire was that–lacking
zero–they had no way to indicate successful termination of their C
programs.”
(Robert Firth)

-- 
You received this message because you are subscribed to the Google
Groups "Clojure" group.
To post to this group, send email to clojure@googlegroups.com
Note that posts from new members are moderated - please be patient with your 
first post.
To unsubscribe from this group, send email to
clojure+unsubscr...@googlegroups.com
For more options, visit this group at
http://groups.google.com/group/clojure?hl=en
--- 
You received this message because you are subscribed to the Google Groups 
"Clojure" group.
To unsubscribe from this group and stop receiving emails from it, send an email 
to clojure+unsubscr...@googlegroups.com.
For more options, visit https://groups.google.com/d/optout.

Re: non-lazy clojure?

[Lee Spector]

On Jun 5, 2014, at 8:51 PM, Gary Johnson  wrote:

> Fair enough. Fortunately, Clojure provides so many different tools to select 
> from in creating you perfect recipe. ;-)
> 
> I'm glad to hear that reducers ultimately provided you with some benefits 
> over your previous concurrency approach.
> The one thing that seems rather odd to me though is that your group-size is 
> 1. I'm presuming that the function you're r/mapping must take a substantial 
> amount of time and resources for that to be efficient. Have you experimented 
> with larger group sizes to avoid too much thread swapping?

Yes, I've tried other group sizes and I get nearly no speedup over 
single-threaded code with the default group size (512). Group size 1 seems to 
be about as good as any other value i've tried.

 -Lee

-- 
You received this message because you are subscribed to the Google
Groups "Clojure" group.
To post to this group, send email to clojure@googlegroups.com
Note that posts from new members are moderated - please be patient with your 
first post.
To unsubscribe from this group, send email to
clojure+unsubscr...@googlegroups.com
For more options, visit this group at
http://groups.google.com/group/clojure?hl=en
--- 
You received this message because you are subscribed to the Google Groups 
"Clojure" group.
To unsubscribe from this group and stop receiving emails from it, send an email 
to clojure+unsubscr...@googlegroups.com.
For more options, visit https://groups.google.com/d/optout.