Scheduling state change?

[Thomas]

Hi,

I would like to schedule updates to a state machine (eg. move each second 
to a new state and do something) and I will have potentially 10.000+ of 
these state machines. Ideally I would like to do something like this:

(defn my-func [state]
  (let [new-state (do-something state)]
(schedule-in-1sec my-func new-state)))

I can't really use (Thread/sleep 1000) as that would mean I would have a as 
many threads as I have state machines, which would only work for a small 
number of state machines/threads.

Any ides how best to achieve this in Clojure? I already had a look at the 
various scheduling libraries (at-at, cronj and Quartzite), but from what I 
understand they don't support this behaviour, but please correct me if I am 
wrong.

Thomas

-- 
You received this message because you are subscribed to the Google
Groups "Clojure" group.
To post to this group, send email to clojure@googlegroups.com
Note that posts from new members are moderated - please be patient with your 
first post.
To unsubscribe from this group, send email to
clojure+unsubscr...@googlegroups.com
For more options, visit this group at
http://groups.google.com/group/clojure?hl=en
--- 
You received this message because you are subscribed to the Google Groups 
"Clojure" group.
To unsubscribe from this group and stop receiving emails from it, send an email 
to clojure+unsubscr...@googlegroups.com.
For more options, visit https://groups.google.com/d/optout.

Re: Scheduling state change?

[Michael Klishin]

On 17 July 2014 at 14:40:57, Thomas (th.vanderv...@gmail.com) wrote:
> > Any ides how best to achieve this in Clojure? I already had a look  
> at the various scheduling libraries (at-at, cronj and Quartzite),  
> but from what I understand they don't support this behaviour,  
> but please correct me if I am wrong.

JDK executors. Quartzite can do what you want but you don't need a lot of its
functionality so try JDK executors first.
--  
MK 

-- 
You received this message because you are subscribed to the Google
Groups "Clojure" group.
To post to this group, send email to clojure@googlegroups.com
Note that posts from new members are moderated - please be patient with your 
first post.
To unsubscribe from this group, send email to
clojure+unsubscr...@googlegroups.com
For more options, visit this group at
http://groups.google.com/group/clojure?hl=en
--- 
You received this message because you are subscribed to the Google Groups 
"Clojure" group.
To unsubscribe from this group and stop receiving emails from it, send an email 
to clojure+unsubscr...@googlegroups.com.
For more options, visit https://groups.google.com/d/optout.

Re: Scheduling state change?

[Colin Fleming]

I agree with Michael. Specifically, the ScheduledExecutorService

is probably what you want. Note that Clojure functions implement Runnable,
so you can pass one directly to the executor.

Cheers,
Colin


On 17 July 2014 12:51, Michael Klishin  wrote:

> On 17 July 2014 at 14:40:57, Thomas (th.vanderv...@gmail.com) wrote:
> > > Any ides how best to achieve this in Clojure? I already had a look
> > at the various scheduling libraries (at-at, cronj and Quartzite),
> > but from what I understand they don't support this behaviour,
> > but please correct me if I am wrong.
>
> JDK executors. Quartzite can do what you want but you don't need a lot of
> its
> functionality so try JDK executors first.
> --
> MK
>
> --
> You received this message because you are subscribed to the Google
> Groups "Clojure" group.
> To post to this group, send email to clojure@googlegroups.com
> Note that posts from new members are moderated - please be patient with
> your first post.
> To unsubscribe from this group, send email to
> clojure+unsubscr...@googlegroups.com
> For more options, visit this group at
> http://groups.google.com/group/clojure?hl=en
> ---
> You received this message because you are subscribed to the Google Groups
> "Clojure" group.
> To unsubscribe from this group and stop receiving emails from it, send an
> email to clojure+unsubscr...@googlegroups.com.
> For more options, visit https://groups.google.com/d/optout.
>

-- 
You received this message because you are subscribed to the Google
Groups "Clojure" group.
To post to this group, send email to clojure@googlegroups.com
Note that posts from new members are moderated - please be patient with your 
first post.
To unsubscribe from this group, send email to
clojure+unsubscr...@googlegroups.com
For more options, visit this group at
http://groups.google.com/group/clojure?hl=en
--- 
You received this message because you are subscribed to the Google Groups 
"Clojure" group.
To unsubscribe from this group and stop receiving emails from it, send an email 
to clojure+unsubscr...@googlegroups.com.
For more options, visit https://groups.google.com/d/optout.

Calculating the number of timestamps logged within a specific time period

[emptya45]

Hi, 

I have a list of epoch times which map to HTTP requests. 

'(1405060202611
1405060201157
1405060201361
1405060201261
1405060200391
1405060201458
1405060201705
1405060201058
1405060205062
1405060201558
1405060201761
)

I am trying to find out how many HTTP requests I have in a specified time 
period, so I can graph requests per second.

I was thinking of using reverse and subtracting the first from the list 
from the second and recurring, which would give me the time period between 
every two requests - which is a start.

Any help appreciated here. 

Many Thanks

Aidy

-- 
You received this message because you are subscribed to the Google
Groups "Clojure" group.
To post to this group, send email to clojure@googlegroups.com
Note that posts from new members are moderated - please be patient with your 
first post.
To unsubscribe from this group, send email to
clojure+unsubscr...@googlegroups.com
For more options, visit this group at
http://groups.google.com/group/clojure?hl=en
--- 
You received this message because you are subscribed to the Google Groups 
"Clojure" group.
To unsubscribe from this group and stop receiving emails from it, send an email 
to clojure+unsubscr...@googlegroups.com.
For more options, visit https://groups.google.com/d/optout.

Re: Calculating the number of timestamps logged within a specific time period

[Mike Fikes]

As a start, you could use group-by with a function that squashes together 
items that fall in the same second and then count the size of each value.


(reduce-kv (fn [c k v] (assoc c k (count v))) {} (group-by #(quot % 1000) 
epochs))
;=> {1405060205 1, 1405060200 1, 1405060201 8, 1405060202 1} 

-- 
You received this message because you are subscribed to the Google
Groups "Clojure" group.
To post to this group, send email to clojure@googlegroups.com
Note that posts from new members are moderated - please be patient with your 
first post.
To unsubscribe from this group, send email to
clojure+unsubscr...@googlegroups.com
For more options, visit this group at
http://groups.google.com/group/clojure?hl=en
--- 
You received this message because you are subscribed to the Google Groups 
"Clojure" group.
To unsubscribe from this group and stop receiving emails from it, send an email 
to clojure+unsubscr...@googlegroups.com.
For more options, visit https://groups.google.com/d/optout.

Re: Calculating the number of timestamps logged within a specific time period

[Ashish Negi]

You need not reverse the list for counting requests per second.
If the data is in sorted order, you can start with any direction.
Take the first as the base and keep separating untill you get 1 second or X 
units like 1000 milliseconds if your data is in milliseconds.
The count would be an item of the list and now your base changes to 
timestamp where you ended.

But looking at the little data you provided it seems it is not sorted.

If it is not sorted, one quick solution you can try is like binary search 
tree, where each node counts the no-requests-from-T-to-T+1
BST would be useful if the data is sparse like a lot of time server is not 
used and then used continuously.
Otherwise instead of bst, you can store the info in array too, faster 
access time.

On Thursday, 17 July 2014 18:19:12 UTC+5:30, empt...@gmail.com wrote:
>
> Hi, 
>
> I have a list of epoch times which map to HTTP requests. 
>
> '(1405060202611
> 1405060201157
> 1405060201361
> 1405060201261
> 1405060200391
> 1405060201458
> 1405060201705
> 1405060201058
> 1405060205062
> 1405060201558
> 1405060201761
> )
>
> I am trying to find out how many HTTP requests I have in a specified time 
> period, so I can graph requests per second.
>
> I was thinking of using reverse and subtracting the first from the list 
> from the second and recurring, which would give me the time period between 
> every two requests - which is a start.
>
> Any help appreciated here. 
>
> Many Thanks
>
> Aidy
>
>

-- 
You received this message because you are subscribed to the Google
Groups "Clojure" group.
To post to this group, send email to clojure@googlegroups.com
Note that posts from new members are moderated - please be patient with your 
first post.
To unsubscribe from this group, send email to
clojure+unsubscr...@googlegroups.com
For more options, visit this group at
http://groups.google.com/group/clojure?hl=en
--- 
You received this message because you are subscribed to the Google Groups 
"Clojure" group.
To unsubscribe from this group and stop receiving emails from it, send an email 
to clojure+unsubscr...@googlegroups.com.
For more options, visit https://groups.google.com/d/optout.

Unexpected ClassCastException when passing generics

[Vladimir Bokov]

I'm trying to communicate with DBus via https://github.com/cathive/fx-dbus 
java lib, but
passing an object which extends java abstract generic inner class 
complains, that it cannot be cast
to parent class:

ClassCastException Cannot cast 
org.freedesktop.dbus.Message$StringMethodArgument to 
[Lorg.freedesktop.dbus.Message$MethodArgument; java.lang.Class.cast 
(Class.java:3258)

Note, that Message$StringMethodArgument inherits from 
Message$MethodArgument.
Full code sample to reproduce here: 
https://gist.github.com/razum2um/432181a0689e5c447b09

P.S. Sorry, if it seems newby, but I'm relatively new to Clojure/JVM world


-- 
You received this message because you are subscribed to the Google
Groups "Clojure" group.
To post to this group, send email to clojure@googlegroups.com
Note that posts from new members are moderated - please be patient with your 
first post.
To unsubscribe from this group, send email to
clojure+unsubscr...@googlegroups.com
For more options, visit this group at
http://groups.google.com/group/clojure?hl=en
--- 
You received this message because you are subscribed to the Google Groups 
"Clojure" group.
To unsubscribe from this group and stop receiving emails from it, send an email 
to clojure+unsubscr...@googlegroups.com.
For more options, visit https://groups.google.com/d/optout.

Re: Unexpected ClassCastException when passing generics

[Sean Corfield]

On Jul 17, 2014, at 11:18 AM, Vladimir Bokov  wrote:
> I'm trying to communicate with DBus via https://github.com/cathive/fx-dbus 
> java lib, but
> passing an object which extends java abstract generic inner class complains, 
> that it cannot be cast
> to parent class:
> 
> ClassCastException Cannot cast 
> org.freedesktop.dbus.Message$StringMethodArgument to 
> [Lorg.freedesktop.dbus.Message$MethodArgument;  java.lang.Class.cast 
> (Class.java:3258)

This says it is expecting an array (that's what the 'L' signifies). Java 
variadic functions like this:

public void addArguments(final MethodArgument... arguments) {

need an array passed in.

You'll need something like (into-array 
org.freedesktop.dbus.Message$MethodArgument [app-name]) instead of just 
app-name in your .addArgument call.

Sean Corfield -- (904) 302-SEAN
An Architect's View -- http://corfield.org/

"Perfection is the enemy of the good."
-- Gustave Flaubert, French realist novelist (1821-1880)





signature.asc
Description: Message signed with OpenPGP using GPGMail

Re: Unexpected ClassCastException when passing generics

[Vladimir Bokov]

Thanks a lot! I really didn't get the sense of [L

But still, I think it's very counterintuitive, I'd expect 
cast org.freedesktop.dbus.Message$StringMethodArgument to 
org.freedesktop.dbus.Message$MethodArgument[]
error message (at least) or something more verbose than chars "[L"

пятница, 18 июля 2014 г., 2:02:43 UTC+7 пользователь Sean Corfield написал:
>
> On Jul 17, 2014, at 11:18 AM, Vladimir Bokov  > wrote:
>
> I'm trying to communicate with DBus via https://github.com/cathive/fx-dbus 
> java lib, but
> passing an object which extends java abstract generic inner class 
> complains, that it cannot be cast
> to parent class:
>
> ClassCastException Cannot cast 
> org.freedesktop.dbus.Message$StringMethodArgument to 
> [Lorg.freedesktop.dbus.Message$MethodArgument; java.lang.Class.cast 
> (Class.java:3258)
>
>
> This says it is expecting an array (that's what the 'L' signifies). Java 
> variadic functions like this:
>
> public void addArguments(final MethodArgument... arguments) {
>
> need an array passed in.
>
> You'll need something like (into-array 
> org.freedesktop.dbus.Message$MethodArgument [app-name]) instead of just 
> app-name in your .addArgument call.
>
> Sean Corfield -- (904) 302-SEAN
> An Architect's View -- http://corfield.org/
>
> "Perfection is the enemy of the good."
> -- Gustave Flaubert, French realist novelist (1821-1880)
>
>
>  
>

-- 
You received this message because you are subscribed to the Google
Groups "Clojure" group.
To post to this group, send email to clojure@googlegroups.com
Note that posts from new members are moderated - please be patient with your 
first post.
To unsubscribe from this group, send email to
clojure+unsubscr...@googlegroups.com
For more options, visit this group at
http://groups.google.com/group/clojure?hl=en
--- 
You received this message because you are subscribed to the Google Groups 
"Clojure" group.
To unsubscribe from this group and stop receiving emails from it, send an email 
to clojure+unsubscr...@googlegroups.com.
For more options, visit https://groups.google.com/d/optout.

Re: Critiques of "my-flatten" which uses CPS

[Mark Engelberg]

Right.  Overall lessons:

In recursion, the call stack automatically keeps track of what still needs
to be done.
In Clojure, due to Java, the stack is significantly more limited than heap,
so this can be a real limitation.
To break free of that limitation, you must somehow keep track of what still
needs to be done in a heap-allocated data structure.
CPS is one way to do that, effectively using closures wrapping other
closures to create a "stack" of things that need to be done.
Unfortunately, CPS is usually not very useful in Clojure -- when the
closures are executed you are again using Clojure's severely-limited stack
to perform that execution, so you'll blow the stack then.
So the better solution is to think explicitly about what kind of "remains
to be done" information needs to be tracked, and just track that
information in a heap-allocated data structure.

So CPS is a good starting point for reasoning about a recursion problem,
but ultimately, you want to convert it into some sort of accumulator.

I'm not sure I see the utility of the CPS/accumulator hybrid model you
proposed.  I think the goal is to get away from CPS once you understand
what is going on behind the scenes.

I saw an interesting analysis once of the factorial function, starting from
the recursive formulation of the function, then doing CPS, then doing an
analysis of exactly what values need to be tracked by the continuations,
and then eliminating the actual continuations.  What you are left with is a
version of the function that begins by building a heap-allocated structure
of all the numbers from 1 to n, and then just reduces them with
multiplication, which is basically what you'd expect the concise version of
the function to be.  I think that's a great example to keep in mind.



On Wed, Jul 16, 2014 at 8:39 PM, Mark P  wrote:

> Interesting - thank you!
>
> Good to know about "into" as a way to achieve a non-lazy concat - in
> conjunction with a vector.
>
> I like your accumulator version - morphing the structure of the input as
> you go to achieve the desired result.
>
> I've come up with another version, that passes through both a "result so
> far" accumulator and a "still to be computed" stack.  It is very much
> non-lazy, but that's okay in some applications (a performance feature
> even!).  Here it is...
>
> (defn my-accandstackbased-flatten
>   [xs]
>   (letfn [(flatten-accandstack [xs accvec stackvec]
> (if (empty? xs)
>   (if (empty? stackvec)
> accvec
> (recur (peek stackvec) accvec (pop stackvec)))
>   (let [x (first xs), ys (next xs)]
> (if (sequential? x)
>   (recur x accvec (conj stackvec ys))
>   (recur ys (conj accvec x) stackvec)]
> (seq (flatten-accandstack xs [] []
>
> This has nice recur behaviour and doesn't mess too much with the structure
> of the input nested lists.  In the case of flatten, retaining structure is
> not important (as you nicely illustrated), but there are generalizations of
> this kind of function where retaining structure becomes more important.
>
> Now here's the thing...  In the above, my stackvec is sort of like a
> continuation.  It contains a stack of computation to be performed later on.
>  That sounds very much like a continuation to me (though I am still in the
> process of getting my head around this stuff).  So I've thought, surely I
> could write an "acc and cps" version of my-flatten?  Here is what I came up
> with...
>
> (defn my-accandcpsbased-flatten
>   [xs]
>   (letfn [(flatten-accandcps [xs accvec k]
> (if (empty? xs)
>   (k accvec)
>   (let [x (first xs), ys (next xs)]
> (if (sequential? x)
>   (recur x accvec (fn [v] (flatten-accandcps ys [] (fn [w]
> (k (into v w))
>   (recur ys (conj accvec x) k)]
> (seq (flatten-accandcps xs [] identity
>
> And I could do the trick you showed me with thunks and a trampoline, if I
> wanted to make it completely stack-avoiding.
>
> What does this show?  I'm not sure.
>
> Presumably the acc and stack version is the most efficient as it uses
> recur everywhere?
>
> The structure of the continuation I pass in with the "acc and cps" version
> is very similar in complexity to the continuation I passed as part of my
> original cps implementation.  So where's the gain?  I guess the inclusion
> of an accvec means that continuations are generated less frequently so that
> the overall size and execution time of continuations is smaller?  And it is
> an improvement over my earlier aps version that used only an acc vector -
> because this version has everything in tail-call position and is amenable
> to trampolining.
>
> Is this attempt at analysis reasonable?  Is there anything else worth
> observing etc?
>
> Presumably I could make the "acc and cps" above more lazy using concat,
> cons and lazy-seq.  I'd need to think about how

Re: Critiques of "my-flatten" which uses CPS

[Steve Miner]

Slightly off-topic from original poster's question, but if you're interested in 
another  implementation of flatten, I suggest you take a look at the reducers 
library.  clojure.core/flatten is elegant but a bit slow.  The reducers version 
is very fast as part of a reduce/fold operation.  The whole reducers library is 
worth studying.

http://clojure.org/reducers

-- 
You received this message because you are subscribed to the Google
Groups "Clojure" group.
To post to this group, send email to clojure@googlegroups.com
Note that posts from new members are moderated - please be patient with your 
first post.
To unsubscribe from this group, send email to
clojure+unsubscr...@googlegroups.com
For more options, visit this group at
http://groups.google.com/group/clojure?hl=en
--- 
You received this message because you are subscribed to the Google Groups 
"Clojure" group.
To unsubscribe from this group and stop receiving emails from it, send an email 
to clojure+unsubscr...@googlegroups.com.
For more options, visit https://groups.google.com/d/optout.

Re: Critiques of "my-flatten" which uses CPS

[Raoul Duke]

> http://clojure.org/reducers

i dare say the "When to use" part should not be at the bottom but come
right after the otherwise laughably specious "yielding code that will
get faster automatically as machines get more cores".

-- 
You received this message because you are subscribed to the Google
Groups "Clojure" group.
To post to this group, send email to clojure@googlegroups.com
Note that posts from new members are moderated - please be patient with your 
first post.
To unsubscribe from this group, send email to
clojure+unsubscr...@googlegroups.com
For more options, visit this group at
http://groups.google.com/group/clojure?hl=en
--- 
You received this message because you are subscribed to the Google Groups 
"Clojure" group.
To unsubscribe from this group and stop receiving emails from it, send an email 
to clojure+unsubscr...@googlegroups.com.
For more options, visit https://groups.google.com/d/optout.

ANN Pantomime 2.3.0 is released

[Michael Klishin]

Pantomime [1] is a tiny Clojure library for working with
MIME types.

Release notes:
http://blog.clojurewerkz.org/blog/2014/07/18/pantomime-2-dot-3-0-is-released/

1. https://github.com/michaelklishin/pantomime
--  
 

-- 
You received this message because you are subscribed to the Google
Groups "Clojure" group.
To post to this group, send email to clojure@googlegroups.com
Note that posts from new members are moderated - please be patient with your 
first post.
To unsubscribe from this group, send email to
clojure+unsubscr...@googlegroups.com
For more options, visit this group at
http://groups.google.com/group/clojure?hl=en
--- 
You received this message because you are subscribed to the Google Groups 
"Clojure" group.
To unsubscribe from this group and stop receiving emails from it, send an email 
to clojure+unsubscr...@googlegroups.com.
For more options, visit https://groups.google.com/d/optout.

Re: Critiques of "my-flatten" which uses CPS

[Mark P]

Coming back to your helpful comments about relationship between acc-style 
and lazy...

It's worth understanding how to go back and forth between accumulator-style 
> and a lazy construction.  You can convert the above non-lazy accumulator 
> version into a similar version that is lazy but has no risk of stack 
> overflow in the realization of that lazy flattened list:
>

I notice in your lazy version (below), you preface the last cons with a 
lazy-seq call, but do not do the same with the other cons calls (within the 
first recur form).  I know it was only the last line that previously had a 
conj call and so participated in this transformation... but I'm wondering 
why you wouldn't also use lazy-seq with the other cons-involving line?

To answer my own wondering, I am thinking it is for two reasons:

   1. The main reason for using lazy-seq in the last line is to defer the 
   recursive call to my-flatten until something wants to use it.  In contrast, 
   the other cons-involving line only conses up things which are relatively 
   harmless to evaluate.
   2. The "cons (first x) ..." will be "undone" almost immediately after 
   the recur is executed, via the "(first xs)".  So inserting laziness here is 
   counter productive.

Another question...

I notice that you bound ys to (rest xs), in contrast to my choice of (next 
xs) in some of my implementations.  I realize this is probably a minor 
point, but just wondering whether (rest xs) was a deliberate choice of 
yours, or just habit.  I realize that rest maximizes laziness in contrast 
to next, but do we want maximum laziness here?

To again attempt to answer my own question...  It probably depends on what 
xs is being passed into my-flatten.  It xs is a fully realized sequence, 
then (next x) would probably do.  But if xs is itself lazy (and possibly 
expensive to compute), then using (rest x) maximizes our laziness 
opportunity.

Maybe the thing to do would be to use (next x) here for this 
implementation, which is angling to be lazy...  But in your earlier 
acc-based my-flatten, to use (next x) instead, because this implementation 
is eager and we know we're going to realize all of xs at some point anyway. 
 It this logic sound?  (I realize I am being pedantic, but I'm trying to 
understand the principles involved.)

Thanks again for showing me the relationship between acc-based and lazy - 
helpful!


> (defn my-flatten
>   [xs]
>   (if (empty? xs) ()
> (let [x (first xs), ys (rest xs)]
>   (if (sequential? x)
> (if (seq x)
>   (recur (cons (first x) (cons (rest x) ys)))
>   (recur ys))
> (lazy-seq (cons x (my-flatten ys)))
>
> Basically, you just get rid of the accumulator, and in the place where you 
> would have conj'd in the next atomic element, you just build the lazy 
> sequence.
>
>

-- 
You received this message because you are subscribed to the Google
Groups "Clojure" group.
To post to this group, send email to clojure@googlegroups.com
Note that posts from new members are moderated - please be patient with your 
first post.
To unsubscribe from this group, send email to
clojure+unsubscr...@googlegroups.com
For more options, visit this group at
http://groups.google.com/group/clojure?hl=en
--- 
You received this message because you are subscribed to the Google Groups 
"Clojure" group.
To unsubscribe from this group and stop receiving emails from it, send an email 
to clojure+unsubscr...@googlegroups.com.
For more options, visit https://groups.google.com/d/optout.

Re: Critiques of "my-flatten" which uses CPS

[Mark P]

Woopse, typo towards the end of my last post...

Maybe the thing to do would be to use (next x) here for this 
> implementation, which is angling to be lazy...  But in your earlier 
> acc-based my-flatten, to use (next x) instead
>

Should be...  Maybe the thing to do would be to use (rest x) ...

-- 
You received this message because you are subscribed to the Google
Groups "Clojure" group.
To post to this group, send email to clojure@googlegroups.com
Note that posts from new members are moderated - please be patient with your 
first post.
To unsubscribe from this group, send email to
clojure+unsubscr...@googlegroups.com
For more options, visit this group at
http://groups.google.com/group/clojure?hl=en
--- 
You received this message because you are subscribed to the Google Groups 
"Clojure" group.
To unsubscribe from this group and stop receiving emails from it, send an email 
to clojure+unsubscr...@googlegroups.com.
For more options, visit https://groups.google.com/d/optout.

Re: Critiques of "my-flatten" which uses CPS

[Mark P]

One other question occurs to me re your comment...

It's worth understanding how to go back and forth between accumulator-style 
> and a lazy construction.  You can convert the above non-lazy accumulator 
> version into a similar version that is lazy but has no risk of stack 
> overflow in the realization of that lazy flattened list:
>

Why does going lazy avoid a stack overflow risk?

Again, to answer my own question :-) ...  The resultant flat list from 
my-flatten will be realized one element at a time.  The call stack will be 
used during this process, but lazy-seq prevents it getting very deep.  I 
think lazy-seq is a bit like a thunk (invoked via the seq call), except 
that it also caches the result.  So its like the thunks in our 
trampolining, except that the "bouncing" doesn't occur eagerly as with 
trampolining, but only on-demand as a consumer traverses the flat list.

Is this a good way of thinking about it?
 

>
> (defn my-flatten
>   [xs]
>   (if (empty? xs) ()
> (let [x (first xs), ys (rest xs)]
>   (if (sequential? x)
> (if (seq x)
>   (recur (cons (first x) (cons (rest x) ys)))
>   (recur ys))
> (lazy-seq (cons x (my-flatten ys)))
>
> Basically, you just get rid of the accumulator, and in the place where you 
> would have conj'd in the next atomic element, you just build the lazy 
> sequence.
>
>

-- 
You received this message because you are subscribed to the Google
Groups "Clojure" group.
To post to this group, send email to clojure@googlegroups.com
Note that posts from new members are moderated - please be patient with your 
first post.
To unsubscribe from this group, send email to
clojure+unsubscr...@googlegroups.com
For more options, visit this group at
http://groups.google.com/group/clojure?hl=en
--- 
You received this message because you are subscribed to the Google Groups 
"Clojure" group.
To unsubscribe from this group and stop receiving emails from it, send an email 
to clojure+unsubscr...@googlegroups.com.
For more options, visit https://groups.google.com/d/optout.

Re: Critiques of "my-flatten" which uses CPS

[Mark Engelberg]

On Thu, Jul 17, 2014 at 7:54 PM, Mark P  wrote:

>
> I notice in your lazy version (below), you preface the last cons with a
> lazy-seq call, but do not do the same with the other cons calls (within the
> first recur form).  I know it was only the last line that previously had a
> conj call and so participated in this transformation... but I'm wondering
> why you wouldn't also use lazy-seq with the other cons-involving line?
>

The lazy-seq is necessary because of the recursive call that isn't in
tail-call position (so you can't use recur).  One way to think about it is
that at that point, we know what the first element of the sequence is going
to be, so the lazy-seq lets us return immediately with the computed first
element and a description of the further computation that can compute the
rest on demand.

lazy-seq is the secret sauce that makes it so non-tail recursive calls
don't overflow the stack.  That's because the sequence is prompted for one
element at a time from an outside consumer -- as you noted in one of your
messages, it's a lot like what trampolining does.  So to make sure you
don't have stack overflow, all you have to do is make sure that you use
bounded stack space in the process of computing the next element.

The other calls to cons are just part of the process of coming up with the
next element, basically sticking "stuff to be computed" on the front of the
list as a trick to avoid a separate accumulator.  Because they are inside a
recur, no additional stack is consumed (the list grows, but that's a
heap-allocated data structure, so not a problem).





>
> Another question...
>
> I notice that you bound ys to (rest xs), in contrast to my choice of (next
> xs) in some of my implementations.  I realize this is probably a minor
> point, but just wondering whether (rest xs) was a deliberate choice of
> yours, or just habit.  I realize that rest maximizes laziness in contrast
> to next, but do we want maximum laziness here?
>

One idiom is to use to use (next xs), and then you can test against it
being nil to determine whether it is empty or not.  But to use that idiom,
you have to be very careful to make sure you call seq on the sequence that
is passed as an input to the function (which you did in your version).  If
you do that and are careful, the performance of next/nil? is slightly
better than rest/empty?.  But I've gotten burned enough times, I simply
prefer to use empty? as my test, and then you might as well use rest rather
than next.  It's my own preference.  I think many people in the Clojure
community prefer to seq the input and then next/nil?.  Either is fine.

-- 
You received this message because you are subscribed to the Google
Groups "Clojure" group.
To post to this group, send email to clojure@googlegroups.com
Note that posts from new members are moderated - please be patient with your 
first post.
To unsubscribe from this group, send email to
clojure+unsubscr...@googlegroups.com
For more options, visit this group at
http://groups.google.com/group/clojure?hl=en
--- 
You received this message because you are subscribed to the Google Groups 
"Clojure" group.
To unsubscribe from this group and stop receiving emails from it, send an email 
to clojure+unsubscr...@googlegroups.com.
For more options, visit https://groups.google.com/d/optout.

Re: Critiques of "my-flatten" which uses CPS

[Mark Phillips]

Thanks again - that all makes sense.

One (hopefully) tiny question... an efficiency one... (and feel free not to 
answer it if I've already taken up enough of your time)

If you do that and are careful, the performance of next/nil? is slightly 
> better than rest/empty?.
>

If I use the next/nil? idiom, is there a performance difference between 
doing:

   1. (if (nil? xs) a b)
   2. (if xs b a)

And a related question...  Clojure has two falsey values: nil and false.  I 
assume that low-level somewhere, when checking for falsey, an (if x a b) 
call will do something like "if x is false, short circuit to b, if x is 
nil, short circuit to b, otherwise a".  Except, I don't know which gets 
priority of place with the short circuits - is it false (as I've just 
written), or is it nil?

Actually, if I'm thinking about it correctly...

If false takes priority then...  A branch to "a" will take three tests for 
1. and two tests for 2  And a branch to "b" will take two tests for 1. 
and two tests for 2..

If nil takes priority then...  A branch to "a" will take three tests for 1. 
and one test for 2  And a branch to "b" will take three tests for 1. 
and two tests for 2..

So 2. is always as good or better than 1. I think - performance wise.

Of course, these things are very much minor factors in overall performance 
(especially when things like immutable data structures are involved), but I 
wouldn't mind knowing.

Cheers,

Mark.

-- 
You received this message because you are subscribed to the Google
Groups "Clojure" group.
To post to this group, send email to clojure@googlegroups.com
Note that posts from new members are moderated - please be patient with your 
first post.
To unsubscribe from this group, send email to
clojure+unsubscr...@googlegroups.com
For more options, visit this group at
http://groups.google.com/group/clojure?hl=en
--- 
You received this message because you are subscribed to the Google Groups 
"Clojure" group.
To unsubscribe from this group and stop receiving emails from it, send an email 
to clojure+unsubscr...@googlegroups.com.
For more options, visit https://groups.google.com/d/optout.

Re: Critiques of "my-flatten" which uses CPS

[Mark Phillips]

I *think* I've found the answer to my own question...

In this post...

  https://groups.google.com/forum/#!topic/clojure/Cuk_bJrIq-Y

I found this link (I changed the line number)...

  
https://github.com/clojure/clojure/blob/master/src/jvm/clojure/lang/Compiler.java#L2589

And if the implementation of eval() within the IfExpr class is anything to 
go by...

public Object eval() {
Object t = testExpr.eval();
if(t != null && t != Boolean.FALSE)
return thenExpr.eval();
return elseExpr.eval();
}


Then I think Rich has implemented it so that nil takes priority over false.

...Which strengthens the case for choosing (if xs b a) over (if (nil? xs) a 
b).

Of course, in practice it probably makes no difference, as either will be 
very quick compared to other things going on.

Cheers,

Mark.

On Friday, 18 July 2014 14:39:53 UTC+9:30, Mark Phillips wrote:
>
> Thanks again - that all makes sense.
>
> One (hopefully) tiny question... an efficiency one... (and feel free not 
> to answer it if I've already taken up enough of your time)
>
> If you do that and are careful, the performance of next/nil? is slightly 
>> better than rest/empty?.
>>
>
> If I use the next/nil? idiom, is there a performance difference between 
> doing:
>
>1. (if (nil? xs) a b)
>2. (if xs b a)
>
> And a related question...  Clojure has two falsey values: nil and false. 
>  I assume that low-level somewhere, when checking for falsey, an (if x a b) 
> call will do something like "if x is false, short circuit to b, if x is 
> nil, short circuit to b, otherwise a".  Except, I don't know which gets 
> priority of place with the short circuits - is it false (as I've just 
> written), or is it nil?
>
> Actually, if I'm thinking about it correctly...
>
> If false takes priority then...  A branch to "a" will take three tests for 
> 1. and two tests for 2  And a branch to "b" will take two tests for 1. 
> and two tests for 2..
>
> If nil takes priority then...  A branch to "a" will take three tests for 
> 1. and one test for 2  And a branch to "b" will take three tests for 1. 
> and two tests for 2..
>
> So 2. is always as good or better than 1. I think - performance wise.
>
> Of course, these things are very much minor factors in overall performance 
> (especially when things like immutable data structures are involved), but I 
> wouldn't mind knowing.
>
> Cheers,
>
> Mark.
>
>

-- 
You received this message because you are subscribed to the Google
Groups "Clojure" group.
To post to this group, send email to clojure@googlegroups.com
Note that posts from new members are moderated - please be patient with your 
first post.
To unsubscribe from this group, send email to
clojure+unsubscr...@googlegroups.com
For more options, visit this group at
http://groups.google.com/group/clojure?hl=en
--- 
You received this message because you are subscribed to the Google Groups 
"Clojure" group.
To unsubscribe from this group and stop receiving emails from it, send an email 
to clojure+unsubscr...@googlegroups.com.
For more options, visit https://groups.google.com/d/optout.

Re: Future of performant software: cores or memory?

[Mark Phillips]

Hi Gary,

I wrote my initial post in January, but I just wanted to say...  Thanks for 
taking the time to write your reply - I very much appreciated it.

I suspect I will be writing algorithms in C++ for a while to come, but at 
some point I hope to do comparisons with Clojure versions.

Regards,

Mark.

On Thursday, 2 January 2014 13:03:41 UTC+10:30, Gary Trakhman wrote:
>
> It depends on your workload and use-cases, and you could fill many 
> textbooks with a discussion of the tradeoffs, but I can think of some rules 
> of thumb that point in clojure's favor.
>
> The most essential metric for me to see has been Amdahl's Law: 
> http://en.wikipedia.org/wiki/File:AmdahlsLaw.svg
>
> To summarize it, "The speedup of a program using multiple processors in 
> parallel computing is limited by the time needed for the sequential 
> fraction of the program."
>
> My observation is that 'the program' can be replaced by 'the system'. 
>  Memory and cpu are sort of like space/time duality, but not 100% either 
> way, since you can see with caches and latency that memory usage has time 
> implications, and cpu performance relies on registers and things.
>
> For example, in a shared memory system, the parallelization of the cache 
> coherence protocol of the hardware can dominate performance aspects of your 
> software once your software's 'good enough', thus I'd argue the protocol 
> itself becomes part of the 'sequential fraction' of the system and program 
> and subject to Amdahl's law.
>
> When this sort of thing happens often enough, programmers write code that 
> takes away control from this aspect of the system and does things manually, 
> since the programmers have better knowledge of the system than the CPU or 
> compiler ever will.  Switching to a message-passing algorithm might be the 
> way to do it in this case, manually overriding data sent between cores, 
> accepting the overhead of data copies and such along the way.
>
> Consider core.async, I think it's a great example of a thread scheduler 
> and set of abstractions to overcome the limitations of OS-provided threads, 
> schedulers, the control offered by them, and the backlash against code 
> written to them.  It's useful because the scenarios it addresses are common 
> enough now that we need to give ourselves back the control that was taken 
> away by OS's in the first place.
>
> Core.async is implemented by thread-pools, which I can consider a 
> subsection and delineation of a common set of resources for a specific 
> usage, similarly it's common in game-dev and I imagine other fields to use 
> memory-pools to offer predictable or better performance and things like 
> instrumentation.
>
> But, I would argue that the greatest hindrance to writing the systems we 
> want is actually programmer productivity.  Taken to an extreme, why 
> wouldn't we write our own OS's and kernels on bare-metal, in ASM or C every 
> time instead of re-using anything?  It's because the tradeoffs that are 
> offered by reusing different subsystems are well-understood and convenient, 
> or anything else is impractical/impossible.
>
> By picking C++ over Clojure/Java for these reasons you're effectively 
> saying (which might be true) that the difficulty of optimization and the 
> need for manual control is more important than all of the benefits(minus 
> drawbacks) afforded by a higher-level system.
>
> So, it's possible to improve memory-locality in clojure and java, people 
> do it all the time.  The java memory model is not a giant leap away from 
> x86/64 and there are ways to avoid being subject to the garbage collector 
> if you want it.
>
> This blog is a great resource: http://mechanical-sympathy.blogspot.com/
>
> I think clojure's value-add in the big-picture of performance and 
> parallelism is effective high-level coding, at-least-as-good-as-java 
> low-level coding, and the ease of traversing those abstraction levels in a 
> unified way.
>
> By default, you're getting immutable and functional code and the 
> implications of clojure's relative ease/difficulty tradeoffs.
>
>
> On Wed, Jan 1, 2014 at 8:59 PM, Mark P > 
> wrote:
>
>> I have watched a number of clojure talks where the concurrent programming 
>> benefits of Clojure are emphasized.  People have suggested that the number 
>> of cores in computers is growing with an exponential trend.  Software 
>> developers will need programming techniques (eg immutable functional 
>> programing) which allow full harnessing of this.  Software that doesn't 
>> utilize the cores will not perform well and will be left behind.
>>
>> But I have heard counter arguments to this view of the future...
>>
>>1. The number of cores isn't growing very fast.
>>   - There isn't much demand for more cores, so this growth will be 
>>   very slow for most computer devices.
>>   2. Memory performance is the key at this time.
>>   - Whereas raw CPU speed previously was growing dramatically for 
>>   many years, thi

Re: Critiques of "my-flatten" which uses CPS

[Mark Engelberg]

Yeah, you've answered your own question.  In practice, I doubt the
difference is measurable.

Another common idiom you see in Clojure code is:
(defn f [xs]
  (if-let [s (seq xs)]
...do something with (first s) and (f (rest s))...
...base case...))

This ensures that you seq-ify the input (rather than assuming it has been
seq'ed before passed in), gives you the fast test against nil, and uses
rest rather than next because next would have the effect of causing an
extra unnecessary call to seq.

In a loop-recur situation, it is more common to do the seq once in the
initialization of the loop and then use next which calls seq:

(defn f [xs]
  (loop [s (seq xs)]
(if s
   ... (recur (next s))...
   ... base case ...)))


Out of habit, I prefer to see the base case first so I don't usually do
either of these, but these two patterns are a very popular style, and very
fast execution.  If you don't have a pre-existing preference, these would
be good choices.

-- 
You received this message because you are subscribed to the Google
Groups "Clojure" group.
To post to this group, send email to clojure@googlegroups.com
Note that posts from new members are moderated - please be patient with your 
first post.
To unsubscribe from this group, send email to
clojure+unsubscr...@googlegroups.com
For more options, visit this group at
http://groups.google.com/group/clojure?hl=en
--- 
You received this message because you are subscribed to the Google Groups 
"Clojure" group.
To unsubscribe from this group and stop receiving emails from it, send an email 
to clojure+unsubscr...@googlegroups.com.
For more options, visit https://groups.google.com/d/optout.