Sandra-24 wrote: > > You seem to be confused about the nature of multiple-process > > programming. > > > > If you're on a modern Unix/Linux platform and you have static read-only > > data, you can just read it in before forking and it'll be shared > > between the processes.. > > Not familiar with *nix programming, but I'll take your word on it.
You can do the same on Windows if you use CreateProcessEx to create the new processes and pass a NULL SectionHandle. I don't think this helps in your case, but I was correcting your impression that "you'd have to physically double the computer's memory for a dual core, or quadruple it for a quadcore". That's just not even near true. > > Threads are way overused in modern multiexecution programming. The > > <snip> > > > It used to run on windows with multiple processes. If it really won't > > now, use an older version or contribute a fix. > > First of all I'm not in control of spawning processes or threads. > Apache does that, and apache has no MPM for windows that uses more than > 1 process. As I said, Apache used to run on Windows with multiple processes; using a version that supports that is one option. There are good reasons not to do that, though, so you could be stuck with threads. > Secondly "Superior" is definately a matter of opinion. Let's > see how you would define superior. Having memory protection is superior to not having it--OS designers spent years implementing it, why would you toss out a fair chunk of it? Being explicit about what you're sharing is generally better than not. But as I said, threads are a better solution if you're sharing the vast majority of your memory and have complex data structures to share. When you're starting a new project, really think about whether they're worth the considerable tradeoffs, though, and consider the merits of a multiprocess solution. > 3) Rewrite my codebase to use some form of shared memory. This would be > a terrible nightmare that would take at least a month of development > time and a lot of heavy rewriting. It would be very difficult, but I'll > grant that it may work if done properly with only small performance > losses. It's almost certainly not worth rewriting a large established codebasen. > I would find an easier time, I think, porting mod_python to .net and > leaving that GIL behind forever. Thankfully, I'm not considering such > drastic measures - yet. The threads vs. processes thing isn't strongly related to the implementation language (though a few languages like Java basically take the decision out of your hands). Moving to .NET leaves you with the same questions to consider before making the decision--just working in C# doesn't somehow make threads the right choice all the time. > Why on earth would I want to do all of that work? Just because you want > to keep this evil thing called a GIL? No, I agreed that the GIL is a bad thing for some applications. > My suggestion is in python 3 > ditch the ref counting, use a real garbage collector I disagree with this, though. The benefits of deterministic GC are huge and I'd like to see ref-counting semantics as part of the language definition. That's a debate I just had in another thread, though, and don't want to repeat. > > Now, the GIL is independent of this; if you really need threading in > > your situation (you share almost everything and have hugely complex > > data structures that are difficult to maintain in shm) then you're > > still going to run into GIL serialization. If you're doing a lot of > > work in native code extensions this may not actually be a big > > performance hit, if not it can be pretty bad. > > Actually, I'm not sure I understand you correctly. You're saying that > in an environment like apache (with 250 threads or so) and my hugely > complex shared data structures, that the GIL is going to cause a huge > performance hit? I didn't say that. It can be a big hit or it can be unnoticeable. It depends on your application. You have to benchmark to know for sure. But if you're trying to make a guess: if you're doing a lot of heavy lifting in native modules then the GIL may be released during those calls, and you might get good multithreading performance. If you're doing lots of I/O requests the GIL is generally released during those and things will be fine. If you're doing lots of heavy crunching in Python, the GIL is probably held and can be a big performance issue. Since your app sounds like it's basically written, there's not much cause to guess; benchmark it and see if it's fast enough or not. If so, don't spend time and effort optimizing. -- http://mail.python.org/mailman/listinfo/python-list
