Re: Understanding GraalVM and Clojure

2018-05-01 Thread Khalid Jebbari 
The ClojureScript unit tests pass in the JavaScript engine that comes with
GraalVM 1.0.0-rc1

https://t.co/WiNHWhv04v (via @Mike Fikes on Twitter)

Le mar. 1 mai 2018 à 06:33, Nathan Fisher  a écrit :

> Is there a runnable language test suite that is used to verify CLJ, CLJS,
> and CLR compatibility? Thinking something similar to RubySpec that could be
> used to validate the completeness/gaps in a particular implementation (eg
> GraalVM).
> On Sun, Apr 29, 2018 at 7:16 AM, Khalid Jebbari 
> wrote:
>
>> I've read it, it's really interesting. Alex Miller mentioned on Twitter
>> that they'll work on removing some limitations over time. @Alex, can you
>> confirm and expand a bit more ?
>>
>> Le sam. 28 avr. 2018 à 23:14, Nathan Fisher  a
>> écrit :
>>
>>> Another interesting post on Clojure and Graal
>>>
>>> https://www.innoq.com/en/blog/native-clojure-and-graalvm/
>>> On Sat, Apr 28, 2018 at 10:01 AM, Khalid Jebbari <
>>> khalid.jebb...@gmail.com> wrote:
>>>
 Thank you for the link.

 Le sam. 28 avr. 2018 à 00:35, Egg Syntax  a
 écrit :

> Karin Meier has done some experimentation using Clojure on GraalVM to
> call R and Python, and has a blog post
> 
>  and repo
> 
>  that
> you may find interesting.
>
>
> On Thursday, April 19, 2018 at 6:00:14 AM UTC-4, Khalid Jebbari wrote:
>>
>> Hello,
>>
>> Oracle has just announced GraalVM 1.0 release candidate:
>> https://blogs.oracle.com/developers/announcing-graalvm
>>
>> It mentions a few JVM-based language but not Clojure (maybe just
>> because of popularity).
>> - Does it mean Clojure is not "compatible" with GraalVM ?
>> - Does it mean Clojure needs to be reimplemented in terms of
>> GraalVM's Truffle framework ?
>> - Does it mean Clojure can be run as a native binary with fast
>> startup time and minimized memory footprint ?
>>
>> If someone with some knowledge could explain to me the relationships
>> between Clojure and GraalVM ? Bonus points if Alex Miller answers and 
>> share
>> the plans, if any, about their integration/interaction.
>>
>> Thanks a lot in advance. really curious to understand more.
>>
> --
> You received this message because you are subscribed to the Google
> Groups "Clojure" group.
> To post to this group, send email to clojure@googlegroups.com
> Note that posts from new members are moderated - please be patient
> with your first post.
> To unsubscribe from this group, send email to
> clojure+unsubscr...@googlegroups.com
> For more options, visit this group at
> http://groups.google.com/group/clojure?hl=en
> ---
>
 You received this message because you are subscribed to a topic in the
> Google Groups "Clojure" group.
> To unsubscribe from this topic, visit
> https://groups.google.com/d/topic/clojure/iBY6hwqqp5c/unsubscribe.
> To unsubscribe from this group and all its topics, send an email to
> clojure+unsubscr...@googlegroups.com.


> For more options, visit https://groups.google.com/d/optout.
>
 --
 You received this message because you are subscribed to the Google
 Groups "Clojure" group.
 To post to this group, send email to clojure@googlegroups.com
 Note that posts from new members are moderated - please be patient with
 your first post.
 To unsubscribe from this group, send email to
 clojure+unsubscr...@googlegroups.com
 For more options, visit this group at
 http://groups.google.com/group/clojure?hl=en
 ---
 You received this message because you are subscribed to the Google
 Groups "Clojure" group.
 To unsubscribe from this group and stop receiving emails from it, send
 an email to clojure+unsubscr...@googlegroups.com.
 For more options, visit https://groups.google.com/d/optout.

>>> --
>>> - sent from my mobile
>>>
>>> --
>>> You received this message because you are subscribed to the Google
>>> Groups "Clojure" group.
>>> To post to this group, send email to clojure@googlegroups.com
>>> Note that posts from new members are moderated - please be patient with
>>> your first post.
>>> To unsubscribe from this group, send email to
>>> clojure+unsubscr...@googlegroups.com
>>> For more options, visit this group at
>>> http://groups.google.com/group/clojure?hl=en
>>> ---
>>> You received this message because you are subscribed to a topic in the
>>> Google Groups "Clojure" group.
>>> To unsubscribe from this topic, visit
>>> https://groups.google.com/d/topic/clojure/iBY6hwqqp5c/unsubscribe.
>>> To unsubscribe from this group and all its topics, send an email to
>>> clojure+unsubscr...@googlegroups.com.
>>> For more options, visit https://groups.google.com/d/optout.

Custom core.async go threadpools? Using go parking for heavy calculation parallelism throughput?

2018-05-01 Thread Leon Grapenthin 
Recently I worked on an algorithm where a distributed tree is (sort of) 
flattened in a way that each node runs a commutative aggregation over all 
of its child nodes calculations.

1 A node can obviously not pmap over all the child nodes (would spawn 
exponential amount of threads).

2 If I want to limit the threads using a threadpool, child nodes will 
quickly run out of threads and we can't reuse their parents threads because 
they are blocking, waiting for the child nodes to complete, calculating 
nothing.

It sure could be possible to implement this with an unlimited threadpool 
and some machinery that ensures that no more than N computations are 
happening in parallel. Nonetheless an insane amount of threads would be 
spawned.

(2) Made me wonder why I couldn't use the go machinery for this. Parent 
nodes that would "wait" for their child nodes to complete would park, 
making their thread available to their child nodes.

So of course I could just do it, but the per node computations are 
presumably too heavy for the lightweight go threadpool.

How realistic/useful would it be to have sth. like a (go-with threadpool 
[]) in this scenario? 

-- 
You received this message because you are subscribed to the Google
Groups "Clojure" group.
To post to this group, send email to clojure@googlegroups.com
Note that posts from new members are moderated - please be patient with your 
first post.
To unsubscribe from this group, send email to
clojure+unsubscr...@googlegroups.com
For more options, visit this group at
http://groups.google.com/group/clojure?hl=en
--- 
You received this message because you are subscribed to the Google Groups 
"Clojure" group.
To unsubscribe from this group and stop receiving emails from it, send an email 
to clojure+unsubscr...@googlegroups.com.
For more options, visit https://groups.google.com/d/optout.

Re: Custom core.async go threadpools? Using go parking for heavy calculation parallelism throughput?

2018-05-01 Thread Justin Smith 
Just a couple of small points (and not yet a full answer):

> A node can obviously not pmap over all the child nodes (would spawn
exponential amount of threads)

pmap is not that naive, it uses a pool sized with the assumption that its
work is CPU bound

> (2) Made me wonder why I couldn't use the go machinery for this. Parent
nodes that would "wait" for their child nodes to complete would park,
making their thread available to their child nodes.

This is why async/thread exists, it returns a channel you can park in, and
you can use channels coming into the blocks starting threads to control the
parallelism by hand.

On Tue, May 1, 2018, 03:41 Leon Grapenthin  wrote:

> Recently I worked on an algorithm where a distributed tree is (sort of)
> flattened in a way that each node runs a commutative aggregation over all
> of its child nodes calculations.
>
> 1 A node can obviously not pmap over all the child nodes (would spawn
> exponential amount of threads).
>
> 2 If I want to limit the threads using a threadpool, child nodes will
> quickly run out of threads and we can't reuse their parents threads because
> they are blocking, waiting for the child nodes to complete, calculating
> nothing.
>
> It sure could be possible to implement this with an unlimited threadpool
> and some machinery that ensures that no more than N computations are
> happening in parallel. Nonetheless an insane amount of threads would be
> spawned.
>
> (2) Made me wonder why I couldn't use the go machinery for this. Parent
> nodes that would "wait" for their child nodes to complete would park,
> making their thread available to their child nodes.
>
> So of course I could just do it, but the per node computations are
> presumably too heavy for the lightweight go threadpool.
>
> How realistic/useful would it be to have sth. like a (go-with threadpool
> []) in this scenario?
>
> --
> You received this message because you are subscribed to the Google
> Groups "Clojure" group.
> To post to this group, send email to clojure@googlegroups.com
> Note that posts from new members are moderated - please be patient with
> your first post.
> To unsubscribe from this group, send email to
> clojure+unsubscr...@googlegroups.com
> For more options, visit this group at
> http://groups.google.com/group/clojure?hl=en
> ---
> You received this message because you are subscribed to the Google Groups
> "Clojure" group.
> To unsubscribe from this group and stop receiving emails from it, send an
> email to clojure+unsubscr...@googlegroups.com.
> For more options, visit https://groups.google.com/d/optout.
>

-- 
You received this message because you are subscribed to the Google
Groups "Clojure" group.
To post to this group, send email to clojure@googlegroups.com
Note that posts from new members are moderated - please be patient with your 
first post.
To unsubscribe from this group, send email to
clojure+unsubscr...@googlegroups.com
For more options, visit this group at
http://groups.google.com/group/clojure?hl=en
--- 
You received this message because you are subscribed to the Google Groups 
"Clojure" group.
To unsubscribe from this group and stop receiving emails from it, send an email 
to clojure+unsubscr...@googlegroups.com.
For more options, visit https://groups.google.com/d/optout.

Re: Custom core.async go threadpools? Using go parking for heavy calculation parallelism throughput?

2018-05-01 Thread Leon Grapenthin 
Yeah, that goes in a direction I thought about after my post. I'm going to 
implement sth. like this and we will see how much code overhead is 
necessary. 

At the time it appears to me that being able to locally dedicate threads to 
the go mechanism would make this much easier (than a hand controlled 
implementation) in the same way that Erlangs VM can park everything and 
utilizes only as many threads as cores exist for everything...

Thanks for info regarding pmap, I will check it out.

On Tuesday, May 1, 2018 at 3:45:49 PM UTC+2, Justin Smith wrote:
>
> Just a couple of small points (and not yet a full answer):
>
> > A node can obviously not pmap over all the child nodes (would spawn 
> exponential amount of threads)
>
> pmap is not that naive, it uses a pool sized with the assumption that its 
> work is CPU bound
>
> > (2) Made me wonder why I couldn't use the go machinery for this. Parent 
> nodes that would "wait" for their child nodes to complete would park, 
> making their thread available to their child nodes.
>
> This is why async/thread exists, it returns a channel you can park in, and 
> you can use channels coming into the blocks starting threads to control the 
> parallelism by hand.
>
> On Tue, May 1, 2018, 03:41 Leon Grapenthin  > wrote:
>
>> Recently I worked on an algorithm where a distributed tree is (sort of) 
>> flattened in a way that each node runs a commutative aggregation over all 
>> of its child nodes calculations.
>>
>> 1 A node can obviously not pmap over all the child nodes (would spawn 
>> exponential amount of threads).
>>
>> 2 If I want to limit the threads using a threadpool, child nodes will 
>> quickly run out of threads and we can't reuse their parents threads because 
>> they are blocking, waiting for the child nodes to complete, calculating 
>> nothing.
>>
>> It sure could be possible to implement this with an unlimited threadpool 
>> and some machinery that ensures that no more than N computations are 
>> happening in parallel. Nonetheless an insane amount of threads would be 
>> spawned.
>>
>> (2) Made me wonder why I couldn't use the go machinery for this. Parent 
>> nodes that would "wait" for their child nodes to complete would park, 
>> making their thread available to their child nodes.
>>
>> So of course I could just do it, but the per node computations are 
>> presumably too heavy for the lightweight go threadpool.
>>
>> How realistic/useful would it be to have sth. like a (go-with threadpool 
>> []) in this scenario? 
>>
>> -- 
>> You received this message because you are subscribed to the Google
>> Groups "Clojure" group.
>> To post to this group, send email to clo...@googlegroups.com 
>> 
>> Note that posts from new members are moderated - please be patient with 
>> your first post.
>> To unsubscribe from this group, send email to
>> clojure+u...@googlegroups.com 
>> For more options, visit this group at
>> http://groups.google.com/group/clojure?hl=en
>> --- 
>> You received this message because you are subscribed to the Google Groups 
>> "Clojure" group.
>> To unsubscribe from this group and stop receiving emails from it, send an 
>> email to clojure+u...@googlegroups.com .
>> For more options, visit https://groups.google.com/d/optout.
>>
>

-- 
You received this message because you are subscribed to the Google
Groups "Clojure" group.
To post to this group, send email to clojure@googlegroups.com
Note that posts from new members are moderated - please be patient with your 
first post.
To unsubscribe from this group, send email to
clojure+unsubscr...@googlegroups.com
For more options, visit this group at
http://groups.google.com/group/clojure?hl=en
--- 
You received this message because you are subscribed to the Google Groups 
"Clojure" group.
To unsubscribe from this group and stop receiving emails from it, send an email 
to clojure+unsubscr...@googlegroups.com.
For more options, visit https://groups.google.com/d/optout.