Lew wrote: > Jon Harrop wrote: >> I agree entirely but my statements were about parallelism and not >> concurrency. Parallel and concurrent programming have wildly different >> characteristics and solutions. I don't believe shared mutable state is >> overly problematic in the context of parallelism. Indeed, I think it is >> usually the best solution in that context. > > Interesting distinction. Would it be fair to compare concurrent > programming to the bricks used to build the parallel program's edifice?
Concurrent programming certainly underpins the foundations of almost all parallel programs. Not least at the level of the OS scheduling the threads than run the parallel programs. However, that knowledge is probably more confusing than helpful here. In essence, concurrent programming is concerned with reducing latency (e.g. evading blocking) by interleaving computations whereas parallel programming is concerned with maximizing throughput by performing computations at the same time. Historically, concurrency has been of general interest on single core machines in the context of operating systems and IO and has become more important recently due to the ubiquity of web programming. Parallelism was once only important to computational scientists programming shared-memory supercomputers and enterprise developers programming distributed-memory clusters but the advent of multicore machines on the desktop and in the games console has pushed parallelism into the lime light for ordinary developers when performance is important. Solutions for concurrent and parallel programming are also wildly different. Concurrent programming typically schedules work items that are expected to block on a shared queue for a pool of dozens or even hundreds of threads. Parallel programming typically schedules work items that are expected to not block on wait-free work-stealing queues for a pool of "n" threads where "n" is the number of cores. Solutions for concurrent programming (such as the .NET thread pool and asynchronous workflows in F#) can be used as a poor man's solution for parallel programming but the overheads are huge because they were not designed for this purpose so performance is much worse than necessary. Solutions for parallel programming (e.g. Cilk, the Task Parallel Library) are virtually useless for concurrent programming because you quickly end up with all "n" threads blocked and the whole program stalls. -- Dr Jon D Harrop, Flying Frog Consultancy Ltd. http://www.ffconsultancy.com/?u -- http://mail.python.org/mailman/listinfo/python-list
