Re: [Python-Dev] Yet another "A better story for multi-core Python" comment
Hi Gary, On Tue, Sep 8, 2015 at 4:12 PM, Gary Robinson wrote: > 1) More the reference counts away from data structures, so copy-on-write > isn’t an issue. A general note about PyPy --- sorry, it probably doesn't help your use case because SciPy is not supported right now... Right now, PyPy hits the same problem as CPython, despite not being based on reference counting, because every major collection needs to write flag bits inside the header of every object. However, fixing this issue is much more straightforward here: there are well-documented ways that other virtual machines (for other languages) already do. Mostly, instead of writing one bit in the GC header, we'd write one bit in some compact out-of-line array of bits. Moreover, it is an issue that we hear about every now and again, so we may eventually just do it. A bientôt, Armin. ___ Python-Dev mailing list [email protected] https://mail.python.org/mailman/listinfo/python-dev Unsubscribe: https://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com
Re: [Python-Dev] Choosing an official stance towards module deprecation in Python 3
On Tue, Sep 8, 2015 at 7:59 PM, Brett Cannon wrote: > There are two discussions going on in the issue tracker about deprecating > some modules and it has led to the inevitable discussion of Python 2/3 > compatibility (I'm not even going to bother mentioning the issue #s as this > thread is not about the modules specifically but module deprecation in > general). Because I'm tired of rehashing the same discussion every time a > module deprecation comes up I would like to make an official decision that > we can follow any time we decide to deprecate a module. > > The approaches to module deprecation I have seen are: > 1. Nothing changes to the deprecation process; you deprecate a module and > remove it in one to two releases > 2. Deprecate the module but with no plans for removal until Python 2.7 > reaches its EOL (I have been calling this Python 4) > 3. Document the deprecation but no actual code deprecation > > I'm personally in the #2 camp. I want users to be fully aware that the > module in question is not being updated and possibly not even getting > non-critical bugfixes, but it's still there in Python 3 in order to make > sure that you can have code which straddles Python 2 & 3 more easily. +1 --Berker ___ Python-Dev mailing list [email protected] https://mail.python.org/mailman/listinfo/python-dev Unsubscribe: https://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com
[Python-Dev] [RELEASED] Python 3.5.0rc4 is now available!
On behalf of the Python development community and the Python 3.5 release team, I'm surprised to announce the availability of Python 3.5.0rc4, also known as Python 3.5.0 Release Candidate 4. Python 3.5.0 Release Candidate 3 was only released about a day ago. However: during testing, a major regression was discovered, reported on the issue tracker as #25027: http://bugs.python.org/issue25027 Python 3.5 includes some big changes on how Python is built on Windows. One of those changes resulted in Python processes on Windows exceeding the maximum number of loadable shared libraries. As a result Windows builds of Python could no longer run major Python software stacks like Pandas and Jupyter. Fixing this required non-trivial last-minute changes to the Windows build--and those changes need testing. We therefore elected to insert an extra release candidate before the final release, to get these changes into your hands as soon as possible, so that Windows users could test these changes. As with all other Python 3.5 releases to date, this is a preview release, and its use is not recommended for production settings. However, users are encouraged to test with Python 3.5.0rc4, /especially/ Windows users who build or make use of numerous external packages. (Non-Windows users will find little difference between Python 3.5.0rc3 and Python 3.5.0rc4.) You can find Python 3.5.0rc4 here: https://www.python.org/downloads/release/python-350rc4/ Windows and Mac users: please read the important platform-specific "Notes on this release" section near the end of that page. Please kick the tires, //arry/ p.s. Special thanks from the Python 3.5 release team to Christoph Gohlke for the bug report! ___ Python-Dev mailing list [email protected] https://mail.python.org/mailman/listinfo/python-dev Unsubscribe: https://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com
[Python-Dev] Embedding Python 3.5
First of all, an apology. I know this is probably going to be of limited interest for many on python-dev, but I'm honestly not sure where there's a better audience for the question. As it's related to how people should use the new "embeddable" distribution of Python 3.5, I'm hoping it's sufficiently relevant - but if not, feel free to direct me to a better forum. I've been trying to use the new embeddable distribution to build a Windows installation of vim that includes a Python interpreter, rather than relying on the user having a system install of Python on PATH. This strikes me as precisely the sort of usage that the embeddable distribution would be great for. And indeed, it works fine for that. But when it comes to distribution, I have an issue. The problem is that for vim to find the Python DLL, it needs to be alongside vim.exe (there's a small complication here I'll come to later, but let's ignore that for now). So I unzip the embeddable distribution into the directory containing vim.exe. However, I want that directory on PATH (so I can run vim from the command line). Now, as the embeddable distribution includes python.exe, this pollutes my PATH with an extra copy of python, which may or may not be picked up in preference to my "real" python, depending on how I install python. That's bad - I can delete python.exe and pythonw.exe from my vim installation, but I still have a load of Python DLLs on PATH that could potentially mess up other Python installations on my machine (I honestly don't know how likely that is, particularly if I distribute my vim build to other users with different setups). What I'd like to do is to put the Python stuff in a subdirectory of the Vim installation, so it's safely isolated. In the case of vim, I can do this, as vim has an option to dynamically load the Python DLL via LoadLibrary, and I can patch vim to look in \python before searching PATH. But the LoadLibrary stuff is horribly fragile, and I doubt anyone else embedding Pthon in their application is likely to do that - they will probably just link to python35.dll at load-time. I'm not aware of any way to adjust the loader's search path for DLLs to include a subdirectory that's not on PATH, so it seems to me that there's no good way to avoid having the embeddable distribution visible via the user's PATH. For vim, I'll probably go for the dynamic loading approach with a patch to look in a subdirectory. But have I missed any other approaches that make an embedded Python more isolated from the user's system? Paul ___ Python-Dev mailing list [email protected] https://mail.python.org/mailman/listinfo/python-dev Unsubscribe: https://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com
Re: [Python-Dev] Embedding Python 3.5
A good overview on this topic is given on https://github.com/numpy/numpy/wiki/windows-dll-notes. As a side note the PATH is usually the latest place in the search order of DLLs. Pre-loading python35.dll into the process space from within vim could be a possible solution. 2015-09-09 16:30 GMT+02:00 Paul Moore : > First of all, an apology. I know this is probably going to be of > limited interest for many on python-dev, but I'm honestly not sure > where there's a better audience for the question. As it's related to > how people should use the new "embeddable" distribution of Python 3.5, > I'm hoping it's sufficiently relevant - but if not, feel free to > direct me to a better forum. > > I've been trying to use the new embeddable distribution to build a > Windows installation of vim that includes a Python interpreter, rather > than relying on the user having a system install of Python on PATH. > This strikes me as precisely the sort of usage that the embeddable > distribution would be great for. And indeed, it works fine for that. > But when it comes to distribution, I have an issue. > > The problem is that for vim to find the Python DLL, it needs to be > alongside vim.exe (there's a small complication here I'll come to > later, but let's ignore that for now). So I unzip the embeddable > distribution into the directory containing vim.exe. > > However, I want that directory on PATH (so I can run vim from the > command line). Now, as the embeddable distribution includes > python.exe, this pollutes my PATH with an extra copy of python, which > may or may not be picked up in preference to my "real" python, > depending on how I install python. That's bad - I can delete > python.exe and pythonw.exe from my vim installation, but I still have > a load of Python DLLs on PATH that could potentially mess up other > Python installations on my machine (I honestly don't know how likely > that is, particularly if I distribute my vim build to other users with > different setups). > > What I'd like to do is to put the Python stuff in a subdirectory of > the Vim installation, so it's safely isolated. In the case of vim, I > can do this, as vim has an option to dynamically load the Python DLL > via LoadLibrary, and I can patch vim to look in \python > before searching PATH. But the LoadLibrary stuff is horribly fragile, > and I doubt anyone else embedding Pthon in their application is likely > to do that - they will probably just link to python35.dll at > load-time. > > I'm not aware of any way to adjust the loader's search path for DLLs > to include a subdirectory that's not on PATH, so it seems to me that > there's no good way to avoid having the embeddable distribution > visible via the user's PATH. > > For vim, I'll probably go for the dynamic loading approach with a > patch to look in a subdirectory. But have I missed any other > approaches that make an embedded Python more isolated from the user's > system? > > Paul > ___ > Python-Dev mailing list > [email protected] > https://mail.python.org/mailman/listinfo/python-dev > Unsubscribe: > https://mail.python.org/mailman/options/python-dev/cmkleffner%40gmail.com > ___ Python-Dev mailing list [email protected] https://mail.python.org/mailman/listinfo/python-dev Unsubscribe: https://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com
Re: [Python-Dev] Embedding Python 3.5
On 9 September 2015 at 16:11, Carl Kleffner wrote: > A good overview on this topic is given on > https://github.com/numpy/numpy/wiki/windows-dll-notes. As a side note the > PATH is usually the latest place in the search order of DLLs. Pre-loading > python35.dll into the process space from within vim could be a possible > solution. Thank you for this. From a very quick read, it looks like I could put the embedded Python files in a directory ...\gvim.exe.local. I'll give that a try (there's a downside, in that I'd need a duplicate copy in ...\vim.exe.local if I wanted the console version to work too). It looks like SxS assemblies might help too, but that'll need a bit more reading before I understand it :-) Paul ___ Python-Dev mailing list [email protected] https://mail.python.org/mailman/listinfo/python-dev Unsubscribe: https://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com
[Python-Dev] Network simulator 3 ("python-dev" missing) on macports
Dear I was trying to install NS3 on my mac os, it seems missed the python-dev installed. However, the 'sudo port install python-dev'is not working because the port can not be found. please help for install the 'python-dev' thanks!! best ___ Python-Dev mailing list [email protected] https://mail.python.org/mailman/listinfo/python-dev Unsubscribe: https://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com
Re: [Python-Dev] Network simulator 3 ("python-dev" missing) on macports
This mailing list is for the development *of* Python and not *with* it. Your best bet is to ask NS3 or macports for help as we have nothing to do with macports. On Wed, 9 Sep 2015 at 08:41 Hansong Xu wrote: > Dear > I was trying to install NS3 on my mac os, it seems missed the python-dev > installed. > However, the 'sudo port install python-dev'is not working because the > port can not be found. please help for install the 'python-dev' > thanks!! > best > ___ > Python-Dev mailing list > [email protected] > https://mail.python.org/mailman/listinfo/python-dev > Unsubscribe: > https://mail.python.org/mailman/options/python-dev/brett%40python.org > ___ Python-Dev mailing list [email protected] https://mail.python.org/mailman/listinfo/python-dev Unsubscribe: https://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com
Re: [Python-Dev] Embedding Python 3.5
On 09Sep2015 0819, Paul Moore wrote: On 9 September 2015 at 16:11, Carl Kleffner wrote: A good overview on this topic is given on https://github.com/numpy/numpy/wiki/windows-dll-notes. As a side note the PATH is usually the latest place in the search order of DLLs. Pre-loading python35.dll into the process space from within vim could be a possible solution. Thank you for this. From a very quick read, it looks like I could put the embedded Python files in a directory ...\gvim.exe.local. I'll give that a try (there's a downside, in that I'd need a duplicate copy in ...\vim.exe.local if I wanted the console version to work too). It looks like SxS assemblies might help too, but that'll need a bit more reading before I understand it :-) Don't bother reading into SxS assemblies. It may work, but it will destroy more brain cells than are worth wasting on it. :) Certainly putting the distro into a subdirectory is what I had expected would be common. In general, putting application directories in PATH is considered poor form, but unfortunately we don't have a better approach (there are App Paths, but those only work via the shell). Another approach you could use is making a separate directory to put on PATH that contains stub executables (or symlinks?) to launch the actual ones from a separate directory. That way you can control exactly what is available on PATH while still launching from a directory that is not on PATH. Cheers, Steve Paul ___ Python-Dev mailing list [email protected] https://mail.python.org/mailman/listinfo/python-dev Unsubscribe: https://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com
Re: [Python-Dev] devguide: Drop myself from issue assignment list
On 28.06.15 09:14, nick.coghlan wrote: https://hg.python.org/devguide/rev/452f840bac9f changeset: 749:452f840bac9f user:Nick Coghlan date:Sun Jun 28 16:13:54 2015 +1000 summary: Drop myself from issue assignment list files: experts.rst | 6 +++--- 1 files changed, 3 insertions(+), 3 deletions(-) diff --git a/experts.rst b/experts.rst --- a/experts.rst +++ b/experts.rst @@ -1,4 +1,4 @@ -.. _experts: +.. _experts: Was this change made intentionally? May be it broke the completion in Nosy List on the tracker. ___ Python-Dev mailing list [email protected] https://mail.python.org/mailman/listinfo/python-dev Unsubscribe: https://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com
Re: [Python-Dev] devguide: Drop myself from issue assignment list
Sorry for the nose. I sent this message to the list I accidentally. ___ Python-Dev mailing list [email protected] https://mail.python.org/mailman/listinfo/python-dev Unsubscribe: https://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com
Re: [Python-Dev] [RELEASED] Python 3.5.0rc4 is now available!
On 09Sep2015 0642, Larry Hastings wrote: On behalf of the Python development community and the Python 3.5 release team, I'm surprised to announce the availability of Python 3.5.0rc4, also known as Python 3.5.0 Release Candidate 4. Python 3.5.0 Release Candidate 3 was only released about a day ago. However: during testing, a major regression was discovered, reported on the issue tracker as #25027: http://bugs.python.org/issue25027 Python 3.5 includes some big changes on how Python is built on Windows. One of those changes resulted in Python processes on Windows exceeding the maximum number of loadable shared libraries. As a result Windows builds of Python could no longer run major Python software stacks like Pandas and Jupyter. Fixing this required non-trivial last-minute changes to the Windows build--and those changes need testing. We therefore elected to insert an extra release candidate before the final release, to get these changes into your hands as soon as possible, so that Windows users could test these changes. For more background and info about the change on Windows, I've written a follow-up post to my previous one. http://stevedower.id.au/blog/building-for-python-3-5-part-two/ The short version is that we now include a shared dependency in the installer (vcruntime140.dll), and extensions built with future versions of the compiler but targeting Python 3.5 will include their own private copy of this dependency (which will be named vcruntime1#0.dll where # != 4). Cheers, Steve ___ Python-Dev mailing list [email protected] https://mail.python.org/mailman/listinfo/python-dev Unsubscribe: https://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com
[Python-Dev] what is wrong with hg.python.org
Hi, https://hg.python.org/ returns 503 Service Unavailable for an hour or so. Is it a maintenance? When it is expected to end? Best regards, Ivan ___ Python-Dev mailing list [email protected] https://mail.python.org/mailman/listinfo/python-dev Unsubscribe: https://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com
Re: [Python-Dev] what is wrong with hg.python.org
On Wed, 09 Sep 2015 20:02:38 +0200, Ivan Levkivskyi wrote: > https://hg.python.org/ returns 503 Service Unavailable for an hour or so. > Is it a maintenance? When it is expected to end? It was an attempt at maintenance (upgrade) that went bad. No ETA yet, I'm afraid. The repo is still ssh accessible, but not via https or hgweb. --David ___ Python-Dev mailing list [email protected] https://mail.python.org/mailman/listinfo/python-dev Unsubscribe: https://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com
Re: [Python-Dev] devguide: Drop myself from issue assignment list
On 09.09.15 20:33, Serhiy Storchaka wrote: Sorry for the nose. I sent this message to the list I accidentally. s/nose/noise/ ___ Python-Dev mailing list [email protected] https://mail.python.org/mailman/listinfo/python-dev Unsubscribe: https://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com
Re: [Python-Dev] Embedding Python 3.5
On 9 September 2015 at 17:16, Steve Dower wrote: > Don't bother reading into SxS assemblies. It may work, but it will destroy > more brain cells than are worth wasting on it. :) :-) Yeah, I looked at SxS once before and sprained my brain. But the summary on the numpy wiki looked like a digestible summary, so I may just take another look for curiosity. (That's probably what the guys who summoned Cthulhu thought... ;-)) > Certainly putting the distro into a subdirectory is what I had expected > would be common. In general, putting application directories in PATH is > considered poor form, but unfortunately we don't have a better approach > (there are App Paths, but those only work via the shell). Yep, that's sort of the norm, and I don't *really* object to it. But I'd delete python(w).exe so they didn't hide a full distribution. > Another approach you could use is making a separate directory to put on PATH > that contains stub executables (or symlinks?) to launch the actual ones from > a separate directory. That way you can control exactly what is available on > PATH while still launching from a directory that is not on PATH. That's nice. The only problems with that are that I've never been comfortable with symlinks on Windows (because they need admin privs, mainly) and I prefer to avoid stubs because they mean there's an extra process - that latter's a bit of a silly objection (it works fine for py.exe and pip's executable wrappers), though. Thanks for the ideas. As I thought, it's more of a general Windows development question in the end, rather than a Python one. So sorry for the off-topic question (I blame the Python community for being so helpful ;-)) Paul ___ Python-Dev mailing list [email protected] https://mail.python.org/mailman/listinfo/python-dev Unsubscribe: https://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com
Re: [Python-Dev] Yet another "A better story for multi-core Python" comment
On Tue, Sep 08, 2015 at 10:12:37AM -0400, Gary Robinson wrote: > There was a huge data structure that all the analysis needed to > access. Using a database would have slowed things down too much. > Ideally, I needed to access this same structure from many cores at > once. On a Power8 system, for example, with its larger number of > cores, performance may well have been good enough for production. In > any case, my experimentation and prototyping would have gone more > quickly with more cores. > > But this data structure was simply too big. Replicating it in > different processes used memory far too quickly and was the limiting > factor on the number of cores I could use. (I could fork with the big > data structure already in memory, but copy-on-write issues due to > reference counting caused multiple copies to exist anyway.) This problem is *exactly* the type of thing that PyParallel excels at, just FYI. PyParallel can load large, complex data structures now, and then access them freely from within multiple threads. I'd recommended taking a look at the "instantaneous Wikipedia search server" example as a start: https://github.com/pyparallel/pyparallel/blob/branches/3.3-px/examples/wiki/wiki.py That loads trie with 27 million entries, creates ~27.1 million PyObjects, loads a huge NumPy array, and has a WSS of ~11GB. I've actually got a new version in development that loads 6 tries of the most frequent terms for character lengths 1-6. Once everything is loaded, the data structures can be accessed for free in parallel threads. There are more details regarding how this is achieved on the landing page: https://github.com/pyparallel/pyparallel I've done a couple of consultancy projects now that were very data science oriented (with huge data sets), so I really gained an appreciation for how common the situation you describe is. It is probably the best demonstration of PyParallel's strengths. > Gary Robinson [email protected] http://www.garyrobinson.net Trent. ___ Python-Dev mailing list [email protected] https://mail.python.org/mailman/listinfo/python-dev Unsubscribe: https://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com
Re: [Python-Dev] Yet another "A better story for multi-core Python" comment
On 09/09/2015 01:33 PM, Trent Nelson wrote: This problem is *exactly* the type of thing that PyParallel excels at [...] Sorry if I missed it, but is PyParallel still Windows only? -- ~Ethan~ ___ Python-Dev mailing list [email protected] https://mail.python.org/mailman/listinfo/python-dev Unsubscribe: https://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com
Re: [Python-Dev] Yet another "A better story for multi-core Python" comment
On Wed, Sep 09, 2015 at 01:43:19PM -0700, Ethan Furman wrote: > On 09/09/2015 01:33 PM, Trent Nelson wrote: > > >This problem is *exactly* the type of thing that PyParallel excels at [...] > > Sorry if I missed it, but is PyParallel still Windows only? Yeah, still Windows only. Still based off 3.3.5. I'm hoping to rebase off 3.5 after its tagged and get it into a state where it can at least build on POSIX (i.e. stub enough functions such that it'll compile). That's going to be a lot of work though, would love to get some help with it. Trent. ___ Python-Dev mailing list [email protected] https://mail.python.org/mailman/listinfo/python-dev Unsubscribe: https://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com
Re: [Python-Dev] Yet another "A better story for multi-core Python" comment
I’m going to seriously consider installing Windows or using a dedicated hosted windows box next time I have this problem so that I can try your solution. It does seem pretty ideal, although the STM branch of PyPy (using http://codespeak.net/execnet/ to access SciPy) might also work at this point. Thanks! I still hope CPython has a solution at some point… maybe PyParallelel functionality will be integrated into Python 4 circa 2023… :) -- Gary Robinson [email protected] http://www.garyrobinson.net > On Sep 9, 2015, at 4:33 PM, Trent Nelson wrote: > > On Tue, Sep 08, 2015 at 10:12:37AM -0400, Gary Robinson wrote: >> There was a huge data structure that all the analysis needed to >> access. Using a database would have slowed things down too much. >> Ideally, I needed to access this same structure from many cores at >> once. On a Power8 system, for example, with its larger number of >> cores, performance may well have been good enough for production. In >> any case, my experimentation and prototyping would have gone more >> quickly with more cores. >> >> But this data structure was simply too big. Replicating it in >> different processes used memory far too quickly and was the limiting >> factor on the number of cores I could use. (I could fork with the big >> data structure already in memory, but copy-on-write issues due to >> reference counting caused multiple copies to exist anyway.) > > This problem is *exactly* the type of thing that PyParallel excels at, > just FYI. PyParallel can load large, complex data structures now, and > then access them freely from within multiple threads. I'd recommended > taking a look at the "instantaneous Wikipedia search server" example as > a start: > > https://github.com/pyparallel/pyparallel/blob/branches/3.3-px/examples/wiki/wiki.py > > That loads trie with 27 million entries, creates ~27.1 million > PyObjects, loads a huge NumPy array, and has a WSS of ~11GB. I've > actually got a new version in development that loads 6 tries of the > most frequent terms for character lengths 1-6. Once everything is > loaded, the data structures can be accessed for free in parallel > threads. > > There are more details regarding how this is achieved on the landing > page: > > https://github.com/pyparallel/pyparallel > > I've done a couple of consultancy projects now that were very data > science oriented (with huge data sets), so I really gained an > appreciation for how common the situation you describe is. It is > probably the best demonstration of PyParallel's strengths. > >> Gary Robinson [email protected] http://www.garyrobinson.net > >Trent. ___ Python-Dev mailing list [email protected] https://mail.python.org/mailman/listinfo/python-dev Unsubscribe: https://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com
Re: [Python-Dev] Yet another "A better story for multi-core Python" comment
On Wed, Sep 09, 2015 at 04:52:39PM -0400, Gary Robinson wrote: > I’m going to seriously consider installing Windows or using a > dedicated hosted windows box next time I have this problem so that I > can try your solution. It does seem pretty ideal, although the STM > branch of PyPy (using http://codespeak.net/execnet/ to access SciPy) > might also work at this point. I'm not sure how up-to-date this is: http://pypy.readthedocs.org/en/latest/stm.html But it sounds like there's a 1.5GB memory limit (or maybe 2.5GB now, I just peaked at core.h linked in that page) and a 4-core segment limit. PyParallel has no memory limit (although it actually does have support for throttling back memory pressure by not accepting new connections when the system hits 90% physical memory used) and no core limit, and it scales linearly with cores+concurrency. PyPy-STM and PyParallel are both pretty bleeding edge and experimental though so I'm sure we both crash as much as each other when exercised outside of our comfort zones :-) I haven't tried getting the SciPy stack running with PyParallel yet. Trent. ___ Python-Dev mailing list [email protected] https://mail.python.org/mailman/listinfo/python-dev Unsubscribe: https://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com
Re: [Python-Dev] Yet another "A better story for multi-core Python" comment
> > I haven't tried getting the SciPy stack running with PyParallel yet. That would be essential for my use. I would assume a lot of potential PyParallel users are in the same boat. Thanks for the info about PyPy limits. You have a really interesting project. -- Gary Robinson [email protected] http://www.garyrobinson.net > On Sep 9, 2015, at 7:02 PM, Trent Nelson wrote: > > On Wed, Sep 09, 2015 at 04:52:39PM -0400, Gary Robinson wrote: >> I’m going to seriously consider installing Windows or using a >> dedicated hosted windows box next time I have this problem so that I >> can try your solution. It does seem pretty ideal, although the STM >> branch of PyPy (using http://codespeak.net/execnet/ to access SciPy) >> might also work at this point. > > I'm not sure how up-to-date this is: > > http://pypy.readthedocs.org/en/latest/stm.html > > But it sounds like there's a 1.5GB memory limit (or maybe 2.5GB now, I > just peaked at core.h linked in that page) and a 4-core segment limit. > > PyParallel has no memory limit (although it actually does have support > for throttling back memory pressure by not accepting new connections > when the system hits 90% physical memory used) and no core limit, and it > scales linearly with cores+concurrency. > > PyPy-STM and PyParallel are both pretty bleeding edge and experimental > though so I'm sure we both crash as much as each other when exercised > outside of our comfort zones :-) > > I haven't tried getting the SciPy stack running with PyParallel yet. > >Trent. ___ Python-Dev mailing list [email protected] https://mail.python.org/mailman/listinfo/python-dev Unsubscribe: https://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com
