threads?
Has there been any decision yet over what model(s) of threads perl6 will support? Will they be POSIX-like? ithread-like? green-thread-like? It is my hope that more than one model will be supported... something that would allow the most lightweight threads possible to be used where possible, and ithread-like behavior for backwards compatibility, and perhaps something in-between where the lightest threads won't work, but ithreads are too slow. If perl6 can statically (at compile time) analyse subroutines and methods and determine if they're reentrant, then it could automatically use the lightest weight threads when it knows that the entry sub won't have side effects or alter global data. If an otherwise-reentrant subroutine calls other subs which have been labelled by their authors as thread-safe, then that top subroutine can also be assumed to be thread-safe. This would be when the intermediate weight threads might be used. If thread-unsafe subroutines are called, then something like ithreads might be used. To allow the programmer to force perl6 to use lighter threads than it would choose by static analysis, he should be able to declare methods, subs, and blocks to be reentrant or threads safe, even if they don't look that way to the compiler. Of course, he would be doing so at his own risk, but he should be allowed to do it (maybe with a warning).
Re: cb8c84: [S06] s/tail-recursion/recursion/
On Oct 6, 1:28 pm, nore...@github.com wrote: > Branch: refs/heads/master > Home: http://github.com/perl6/specs > > Commit: cb8c8487fa0ab7156fecffdc8a52bf75d4290c1b > http://github.com/perl6/specs/commit/cb8c8487fa0ab7156fecffdc8a52bf75... > Author: Carl Masak > Date: 2010-10-06 (Wed, 06 Oct 2010) > > Changed paths: > M S06-routines.pod > > Log Message: > --- > [S06] s/tail-recursion/recursion/ > > Because you can use &?ROUTINE calls in a non-tail position (and execution > still carries on afterwards), it's really just normal recursion. Shouldn't that also apply to &?BLOCK, too?
Re: threads?
Ben (>): > If perl6 can statically (at compile time) analyse subroutines and > methods and determine if they're reentrant, then it could > automatically use the lightest weight threads when it knows that the > entry sub won't have side effects or alter global data. I'm often at the receiving end of this kind of reply, but... ...to a first approximation, I don't believe such analysis to be possible in Perl 6. Finding out whether something "won't have side effects" is tricky at best, squeezed in as we are between &eval, exuberant dynamism, and the Halting Problem. // Carl
[perl6/specs] 00572d: [S06] s/tail-recursion/recursion/
Branch: refs/heads/master Home: http://github.com/perl6/specs Commit: 00572dd42250299c2d35d1735691cf5f2601c666 http://github.com/perl6/specs/commit/00572dd42250299c2d35d1735691cf5f2601c666 Author: Carl Masak Date: 2010-10-12 (Tue, 12 Oct 2010) Changed paths: M S06-routines.pod Log Message: --- [S06] s/tail-recursion/recursion/ Because you can use &?BLOCK calls in a non-tail position (and execution still carries on afterwards), it's really just normal recursion. Omission found by (Ben Goldberg)++.
Re: threads?
On Mon, Oct 11, 2010 at 12:32 AM, Ben Goldberg wrote: > If thread-unsafe subroutines are called, then something like ithreads > might be used. For the love of $DEITY, let's please not repeat ithreads!
Re: threads?
Leon Timmermans wrote: > For the love of $DEITY, let's please not repeat ithreads! $AMEN! Backwards compatibility is not the major design criterion for Perl 6, so there's no need to recapitulate our own phylogeny here. The problem is: while most people can agree on what have proved to be unsatisfactory threading models, not many people can seem to agree on what would constititute a satisfactory threading model (or, possibly, models). What we really need is some anecdotal evidence from folks who are actually using threading in real-world situations (in *any* languages). What has worked in practice? What has worked well? What was painful? What was error-prone? And for which kinds of tasks? And we also need to stand back a little further and ask: is "threading" the right approach at all? Do threads work in *any* language? Are there better metaphors? Perhaps we need to think more Perlishly and reframe the entire question. Not: "What threading model do we need?", but: "What kinds of non-sequential programming tasks do we want to make easy...and how would we like to be able to specify those tasks?" As someone who doesn't (need to) use threading to solve the kinds of problems I work on, I'm well aware that I'm not the right person to help in this design work. We need those poor souls who already suffer under threads to share their tales of constant misery (and their occasional moments of triumph) so we can identify successful patterns of use and steal^Wborg^Wborrow the very best available solutions. Damian
RE: threads?
Although anecdotal, I've heard good things about Go's "channel" mechanism as a simple lightweight concurrency model and a good alternative to typical threading. Channels are first-class in the language and leverage simple "goroutine" semantics to invoke concurrency. --- Phil -Original Message- From: thoughtstr...@gmail.com [mailto:thoughtstr...@gmail.com] On Behalf Of Damian Conway Sent: October 12, 2010 10:23 AM To: perl6-language@perl.org Subject: Re: threads? Leon Timmermans wrote: > For the love of $DEITY, let's please not repeat ithreads! $AMEN! Backwards compatibility is not the major design criterion for Perl 6, so there's no need to recapitulate our own phylogeny here. The problem is: while most people can agree on what have proved to be unsatisfactory threading models, not many people can seem to agree on what would constititute a satisfactory threading model (or, possibly, models). What we really need is some anecdotal evidence from folks who are actually using threading in real-world situations (in *any* languages). What has worked in practice? What has worked well? What was painful? What was error-prone? And for which kinds of tasks? And we also need to stand back a little further and ask: is "threading" the right approach at all? Do threads work in *any* language? Are there better metaphors? Perhaps we need to think more Perlishly and reframe the entire question. Not: "What threading model do we need?", but: "What kinds of non-sequential programming tasks do we want to make easy...and how would we like to be able to specify those tasks?" As someone who doesn't (need to) use threading to solve the kinds of problems I work on, I'm well aware that I'm not the right person to help in this design work. We need those poor souls who already suffer under threads to share their tales of constant misery (and their occasional moments of triumph) so we can identify successful patterns of use and steal^Wborg^Wborrow the very best available solutions. Damian
Re: threads?
Damian, I use threads in C++ a lot in my day to day job. We use an in-house library which isn't much more than a thread class which you inherit from and give a Run method to, and a load of locks of various (sometimes ill-defined) kinds. Let me say: it's not good. Threads with semaphores and mutexes and all that are just horrible, horrible things. It's probably not helped at all by how C++ itself has no awareness at all of the threading, so there are no hints in the code that something runs in a particular thread, you can't put lock preconditions on functions or data structures or anything like that... I'm not sure what a better model is, but what I'd like to see is something which: - can enforce that certain bits of data are only accessed if you have certain locks, at compile time - can enforce that certain bits of code can only be run when you have certain locks, at compile time - can know that you shouldn't take lock B before lock A if you want to avoid a deadlock - uses a completely different model that nobody's probably thought of yet where none of this matters because all those three things are utterly foul I always liked Software Transactional Memory, which works very nicely in Haskell - but not for all solutions. Whatever concurrency model Perl 6 might support, it's probably going to need more than one of them. Since the language is so extensible, it may be that the core should only implement the very basic primitives, and then there are libraries which provide the rest - some of which might ship alongside the compiler. I don't know, but I do not want people to end up having to count semaphores and verify locking integrity by eye because it's really, truly horrible. I did read a bit about Go's mechanism, and it did look interesting. Some systems are very well-modelled as completely independent processes (which might be threads) throwing messages at each other... Actually something that's very nice as a mental model for server-type systems is a core routine which responds to a trigger (say, a new connection) by spawning a new thread to handle it, which is the only thing which handles it, and maybe uses something like channels to interact with any global data store that's required. For that though you need cheap thread creation or easy thread pool stuff, and you need to have a global data model which isn't going to completely bottleneck your performance. I'm totally rambling now, but I do get the distinct impression from all my experience that safe concurrency is very difficult to do quickly in the general case. Of course, the safest concurrency boils down to sequencing everything and running it all on one core... On 12 October 2010 16:25, wrote: > Although anecdotal, I've heard good things about Go's "channel" mechanism as > a simple lightweight concurrency model and a good alternative to typical > threading. Channels are first-class in the language and leverage simple > "goroutine" semantics to invoke concurrency. > > > --- Phil > > > > -Original Message- > From: thoughtstr...@gmail.com [mailto:thoughtstr...@gmail.com] On Behalf Of > Damian Conway > Sent: October 12, 2010 10:23 AM > To: perl6-language@perl.org > Subject: Re: threads? > > Leon Timmermans wrote: > >> For the love of $DEITY, let's please not repeat ithreads! > > $AMEN! > > Backwards compatibility is not the major design criterion for Perl 6, > so there's no need to recapitulate our own phylogeny here. > > The problem is: while most people can agree on what have proved to be > unsatisfactory threading models, not many people can seem to agree on > what would constititute a satisfactory threading model (or, possibly, models). > > What we really need is some anecdotal evidence from folks who are actually > using threading in real-world situations (in *any* languages). What has worked > in practice? What has worked well? What was painful? What was error-prone? > And for which kinds of tasks? > > And we also need to stand back a little further and ask: is "threading" > the right approach at all? Do threads work in *any* language? Are there > better metaphors? > > Perhaps we need to think more Perlishly and reframe the entire question. > Not: "What threading model do we need?", but: "What kinds of non-sequential > programming tasks do we want to make easy...and how would we like to be > able to specify those tasks?" > > As someone who doesn't (need to) use threading to solve the kinds of > problems I work on, I'm well aware that I'm not the right person to help > in this design work. We need those poor souls who already suffer under > threads to share their tales of constant misery (and their occasional > moments of triumph) so we can identify successful patterns of use > and steal^Wborg^Wborrow the very best available solutions. > > Damian > On 12 October 2010 16:25, wrote: > Although anecdotal, I've heard good things about Go's "channel" mechanism as > a simple lightweight concurrency model and a good alternativ
Re: threads?
On Tue, Oct 12, 2010 at 07:22:33AM -0700, Damian Conway wrote: > What we really need is some anecdotal evidence from folks who are actually > using threading in real-world situations (in *any* languages). What has worked > in practice? What has worked well? What was painful? What was error-prone? > And for which kinds of tasks? > > And we also need to stand back a little further and ask: is "threading" > the right approach at all? Do threads work in *any* language? Are there > better metaphors? > [...] I agree fully with Damian. At YAPC::EU in Pisa, I proposed that since questions of threading and concurrency in Perl 6 (and Parrot) seem to be eternally asked and unanswered questions, that we probably need to start pushing on the answer at least a little bit, if only to have some implementation-level issues to frame discussion and help illustrate the overall outlines of an answer a bit better. My guess is that the overall question of threading and concurrency may be too large/vague to be attacked at once, so let's break it down into smaller, real-world use cases and see if we can find some patterns from those. In Rakudo's case, my plan is to focus on developing some sort of concurrent implementation of the Perl 6 hyperoperators (which ATM is one of Perl 6's most natural expressions of concurrency). Once we have a basic concurrent hyperoperator implementation -- even if it's not a particularly good one, we'll have a lot better understanding of the issues. We can better understand how concurrency might be expressed in other parts of the language, and we can provide Parrot and other underlying runtimes with real examples of the kind of concurrency primitives we'd like to have available. Then, with a few more examples in hand, I think we can start to get at some answers as to what a "Perl concurrency" solution will look like. [N.B.: This is very similar to what has happened in other parts of Perl 6, such as regular expressions and grammars. We didn't get a good feel for what was needed until we actually went through the exercise of building a few Perl 6 standard grammars/parsers in Perl 6.] Pm
Re: [perl6/specs] 58fe2d: [S12] spec setting and getting values of attribute...
Logbot (>):
> [S12] spec setting and getting values of attributes by means of introspection
>
> After lengthy IRC discussion, we concluded that it's a good idea to provide
> some form of introspection that doesn't bother about perceived privacy
> borders, provided that the implementation makes it feasible.
Below I summarize this email thread, the discussion on IRC, and my
thoughts so far.
I make no claims whatsoever to be objective; my aim is to help
understand enough about the issues to help build a good Perl 6.
== About introspection of attributes
I fully understand Damian's (and others') objections, especially since
I used to represent them.
Let me offer a metaphor. There's "normal OO" and "meta-level OO".
Normal OO can be likened to a labyrinth with walls higher than the
humans walking around in them. The walls are restrictions such as
"this attribute is private", offering reminders not to break the rules
of normal OO programming. They are good.
Meta-level OO is like going up in one of the towers conveniently
placed by a corner of the labyrinth. Going up one level eliminates the
walls. (Introspection does this, and serialization to a stream of
bytes does this. Probably other things as well.) From above, you can
see into objects, not through the walls but between them. The
normal-OO restrictions doesn't apply to you any more, because you've
made a *conscious choice* no longer to be restrained by them.
>From this point of view, we already have a word corresponding to
C but for getting through the restrictions put up by
the OO system. It's called C<.HOW>.
I thus find myself in favor of C<.get_value> and C<.set_value>. My
nagging doubts are gone; these two methods don't break any rules,
because they are located at a level where those rules don't apply.
== About C<.perl>
The choice here was between letting C<.perl> display only public
attributes, or keeping it the way it is, displaying public as well as
private attributes. The former is potentially a great help for
debugging; the latter respects the normal-OO level of privacy.
I don't have a horse in this race, and I see sensible arguments on both sides.
== About C<.new>
This method (along with .clone) is problematic because it allows
outsiders to set private attributes in objects. This may sound
innocent, but consider an object that works very hard to maintain an
invariant or other between its private attributes. The default C<.new>
and C<.clone> methods allow any outsider to completely rupture the
invariant in the new object.
I suppose the author of the class could override the constructor
methods to provide "safe" ones, but why should she have to?
The whole current situation seems problematic to me. But I also see
problems looming if we decide to "tighten up" C<.new> and C<.clone>
too much; for one thing because C<.new> and C<.clone> are currently
the only non-C<.HOW> way to set a readonly private attribute:
$ perl6 -e 'class A { has $!x is readonly; method x { $!x } }; say
A.new(x => 42).x'
42
In the end, I suppose it comes down to deciding how much we dislike
that C<.new> and C<.clone> break encapsulation. We like to champion
them as "just normal methods", but as long as they need to use C<.HOW>
introspection to do magic behind the scenes, they aren't really very
normal or innocent, IMO.
// Carl
Re: [perl6/specs] 58fe2d: [S12] spec setting and getting values of attribute...
Carl (>): > == About C<.perl> > > The choice here was between letting C<.perl> display only public > attributes, or keeping it the way it is, displaying public as well as > private attributes. The former is potentially a great help for > debugging; the latter respects the normal-OO level of privacy. Oops, wires crossed. Should have been "The *latter* is potentially a great help for debugging; the *former* respects the normal-OO level of privacy." Of course. // Carl
Re: threads?
When Larry decided that Perl 6 would incorporate concepts from prototype-based objects, he did so at least in part because it's more intuitive for people to work with, e.g., "a cow" than it is to try to work with "the concept of a cow" as a thing unto itself. In a similar way, I think that Perl's dominant concurrency system ought to be of a type that people who aren't computer scientists can grok, at least well enough to do something useful without first having to delve into the arcane depths of computing theory. As such, I'm wondering if an Actor-based concurrency model[1] might be a better way to go than the current threads-based mindset. Certainly, it's often easier to think of actors who talk to each other to get things done than it is to think of processes (or threads) as things unto themselves. [1] http://en.wikipedia.org/wiki/Actor_model -- Jonathan "Dataweaver" Lang
Re: threads?
On Tue, Oct 12, 2010 at 4:22 PM, Damian Conway wrote: > The problem is: while most people can agree on what have proved to be > unsatisfactory threading models, not many people can seem to agree on > what would constititute a satisfactory threading model (or, possibly, models). > > What we really need is some anecdotal evidence from folks who are actually > using threading in real-world situations (in *any* languages). What has worked > in practice? What has worked well? What was painful? What was error-prone? > And for which kinds of tasks? Most languages either implement concurrency in a way that's not very useful (CPython, CRuby) or implement it in a way that's slightly (Java/C/C++) to totally (perl 5) insane. Erlang is the only language I've worked with whose threads I really like, but sadly it's rather weak at a lot of other things. In general, I don't feel that a shared memory model is a good fit for a high level language. I'm very much a proponent of message passing. Unlike shared memory, it's actually easier to do the right thing than not. Implementing it correctly and efficiently is not easier than doing a shared memory system though in my experience (I'm busy implementing it on top of ithreads; yeah I'm masochist like that). > And we also need to stand back a little further and ask: is "threading" > the right approach at all? Do threads work in *any* language? Are there > better metaphors? > > Perhaps we need to think more Perlishly and reframe the entire question. > Not: "What threading model do we need?", but: "What kinds of non-sequential > programming tasks do we want to make easy...and how would we like to be > able to specify those tasks?" I agree. I would prefer implicit over explicit concurrency wherever possible.
Re: threads?
On Tue, Oct 12, 2010 at 10:28 PM, B. Estrade wrote: >> I agree. I would prefer implicit over explicit concurrency wherever possible. > > I know you're speaking about the Perl interface to concurrency, but > you seem to contradict yourself because message passing is explicit > whereas shared memory is implicit - two different models, both of > which could be used together to implement a pretty flexible system. With implicit I mean stuff like concurrent hyperoperators and junctions. Shared memory systems are explicitly concurrent to me because you have to ether explicitly lock or explicitly do a transaction. > It'd be a shame to not provide a way to both use threads directly or > to fallback to some implicitly concurrent constructs. I agree
Re: threads?
Damian Conway wrote: Perhaps we need to think more Perlishly and reframe the entire question. Not: "What threading model do we need?", but: "What kinds of non-sequential programming tasks do we want to make easy...and how would we like to be able to specify those tasks?" The mindset that I use goes something like "most tasks are potentially concurrent: sequentialization is an optimization that most people perform without even thinking". Generally, I would split concurrency into producer-consumer (i.e. message passing) and stream-processing (for hyper and reduction operators -- possibly also for feeds, with a kernel per step). When dealing with compute-tasks, you're basically just choosing how to map a dependency graph to the available compute resources. When dealing with external resources (e.g. sockets, GUI) then explicit parallelism (via message passing) becomes useful. P6 already specifies a whole bunch of non-sequential tasks (hypers, reductions, feeds, background-lazy lists), so no need to reframe the entire question just yet. Implementing the existing concurrency will flush out plenty of flaws in the specs.
Re: threads?
I agree that threads are generelly a difficult issue to cope. What is worse, there are a lot of Java-developers who tell us, that it is not difficult for them, but in the end the software fails on the productive system, for example because the load is different then on the test system, causing different threads to be slowed down to a different extent etc. So people who are having difficulties with multithreading still use them a lot and don't admit the difficulties and they might not even appear during testing... Even though I did see software that heavily uses multithreading and works well. On the other hand I think that there are certain tasks that need to use some kind of parallelism, either for making use of parallel CPU infrastructure or for implementing patterns that can more easily be expressed using something like multithreading. I think that the approach of running several processes instead of several threads is something that can be considered in some cases, but I think it does come with a performance price tag that might not be justified in all situations. Maybe the actor model from Scala is worth looking at, at least the Scala-guys claim that that solves the issue, but I don't know if that concept can easily be adapted for Perl 6. Best regards, Karl
Re: Lessons to learn from ithreads (was: threads?)
On Wed, Oct 13, 2010 at 12:46 AM, Tim Bunce wrote: > So I'd like to use this sub-thread to try to identify when lessons we > can learn from ithreads. My initial thoughts are: > > - Don't clone a live interpreter. > Start a new thread with a fresh interpreter. > > - Don't try to share mutable data or data structures. > Use message passing and serialization. Actually, that sounds *exactly* like what I have been trying to implementing for perl 5 based on ithreads (threads::lite, it's still in a fairly early state though). My experience with it so far taught me that: * Serialization must be cheap for this to scale. For threads::lite this turns out to be the main performance bottleneck. Erlang gets away with this because it's purely functional and thus doesn't need to serialize between local threads, maybe we could do something similar with immutable objects. Here micro-optimizations are going to pay off. * Code sharing is actually quite nice. Loading Moose separately in a hundred threads is not. This is not trivial though, Perl being so dynamic. I suspect this is not possible without running into the same issues as ithreads does. * Creating a thread (/interpreter) should be as cheap as possible, both in CPU-time as in memory. Creating an ithread is relatively expensive, specially memorywise. You can't realistically create a very large number of them the way you can in Erlang. Leon (well actually I learned a lot more; like about non-deterministic unit tests and profilers that don't like threads, but that's an entirely different story)
