from:"Ray"



New submission from Ray Donnelly :

See attached reproducer

--
components: Interpreter Core, Windows
files: ppe.py
messages: 366258
nosy: Ray Donnelly, paul.moore, steve.dower, tim.golden, zach.ware
priority: normal
severity: normal
status: open
title: Follow on bug from https://bugs.python.org/issue26903 (ValueError 
exception on _winapi.WaitForMultipleObjects)
versions: Python 3.7, Python 3.8
Added file: https://bugs.python.org/file49055/ppe.py

___
Python tracker 
<https://bugs.python.org/issue40263>
___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue40263] Follow on bug from https://bugs.python.org/issue26903 (ValueError exception on _winapi.WaitForMultipleObjects)



Ray Donnelly  added the comment:

See my proposed patch. I am happy to make a PR on github for this too if people 
agree it's the right fix.

--
Added file: 
https://bugs.python.org/file49056/-bpo-26903-Limit-ProcessPoolExecutor-to-61-workers-on-Windows.patch.ref

___
Python tracker 
<https://bugs.python.org/issue40263>
___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue40263] Follow on bug from https://bugs.python.org/issue26903 (ValueError exception on _winapi.WaitForMultipleObjects)



Change by Ray Donnelly :


Removed file: 
https://bugs.python.org/file49056/-bpo-26903-Limit-ProcessPoolExecutor-to-61-workers-on-Windows.patch.ref

___
Python tracker 
<https://bugs.python.org/issue40263>
___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue40263] Follow on bug from https://bugs.python.org/issue26903 (ValueError exception on _winapi.WaitForMultipleObjects)



Change by Ray Donnelly :


--
keywords: +patch
Added file: 
https://bugs.python.org/file49057/-Fix-off-by-one-error-in-_winapi_WaitForMultipleObjec.patch

___
Python tracker 
<https://bugs.python.org/issue40263>
___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue40263] ValueError exception on _winapi.WaitForMultipleObjects

2020-04-13 Thread Ray Donnelly



Change by Ray Donnelly :


--
nosy: +Ray.Donnelly
nosy_count: 5.0 -> 6.0
pull_requests: +18852
stage:  -> patch review
pull_request: https://github.com/python/cpython/pull/19501

___
Python tracker 
<https://bugs.python.org/issue40263>
___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue40263] ValueError exception on _winapi.WaitForMultipleObjects

2020-04-13 Thread Ray Donnelly



Ray Donnelly  added the comment:

https://github.com/python/cpython/pull/19501

--

___
Python tracker 
<https://bugs.python.org/issue40263>
___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue26903] ProcessPoolExecutor(max_workers=64) crashes on Windows

2020-04-13 Thread Ray Donnelly



Ray Donnelly  added the comment:

I took the liberty of filing this: https://bugs.python.org/issue40263

Cheers.

--
nosy: +Ray Donnelly

___
Python tracker 
<https://bugs.python.org/issue26903>
___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue35644] venv doesn't do what it claims to do (apears not to work at all?)


New submission from Ray Donnelly :

Happy New Year!

I'm not sure if this is a misunderstanding on my part, a docs bug or a code bug.

At https://docs.python.org/3/library/venv.html we see:

"The solution for this problem is to create a virtual environment, a 
self-contained directory tree that contains a Python installation for a 
particular version of Python, plus a number of additional packages"

and

"This will create the tutorial-env directory if it doesn’t exist, and also 
create directories inside it containing a copy of the Python interpreter, the 
standard library, and various supporting files."

However, when testing with 
https://www.python.org/ftp/python/3.7.2/python-3.7.2-amd64.exe I see no Python 
interpreter (nor DLL) in my venv directory:

```
python.exe -m venv %TEMP%\venv
%TEMP%\venv\Scripts\activate.bat
dir %TEMP%\venv
```

gives:

```
 Directory of C:\Users\RDONNE~1\AppData\Local\Temp\venv

02/01/2019  19:38  .
02/01/2019  19:38  ..
02/01/2019  19:38  Include
02/01/2019  19:38  Lib
02/01/2019  19:38   121 pyvenv.cfg
02/01/2019  19:38  Scripts
   1 File(s)121 bytes
   5 Dir(s)  912,281,780,224 bytes free
```

pyvenv.cfg contains:

```
home = C:\Users\rdonnelly\AppData\Local\Programs\Python\Python37
include-system-site-packages = false
version = 3.7.2
```

Further to this, after activating, I do not see the `venv` directory in 
`sys.path`:

```
python -c "import sys; print(sys.path)"
['',
 
'C:\\Users\\rdonnelly\\AppData\\Local\\Programs\\Python\\Python37\\python37.zip',
 'C:\\Users\\rdonnelly\\AppData\\Local\\Programs\\Python\\Python37\\DLLs',
 'C:\\Users\\rdonnelly\\AppData\\Local\\Programs\\Python\\Python37\\lib',
 'C:\\Users\\rdonnelly\\AppData\\Local\\Programs\\Python\\Python37',
 
'C:\\Users\\rdonnelly\\AppData\\Local\\Programs\\Python\\Python37\\lib\\site-packages']
```

>From past experience, the old `virtualenv` project would copy the interpreter 
>and DLL across.

Any help here would be appreciated!

--
components: Windows
messages: 332892
nosy: Ray Donnelly, paul.moore, steve.dower, tim.golden, zach.ware
priority: normal
severity: normal
status: open
title: venv doesn't do what it claims to do (apears not to work at all?)
type: behavior
versions: Python 3.7

___
Python tracker 
<https://bugs.python.org/issue35644>
___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue35644] venv doesn't do what it claims to do (apears not to work at all?)



Ray Donnelly  added the comment:

I found the executable is in the `Scripts` directory, closing. The real issue 
I'm facing is on Anaconda Distribution's build of Python 3 which I'm updating 
to 3.7.2.

Closing,

Cheers!

--
stage:  -> resolved
status: open -> closed

___
Python tracker 
<https://bugs.python.org/issue35644>
___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue35644] venv doesn't do what it claims to do (apears not to work at all?)



Ray Donnelly  added the comment:

Bit of an update to this, I'm re-opening it as there appears to be a regression 
from Python 3.7.1 to 3.7.2 for the case when there is no venvlauncher.exe 
present (i.e. when there are no python{w,}.exes in Lib\venv\scripts\nt). The 
old code of copying `sys.executable` is no longer run (on Windows only).

Currently Anaconda Distribution's venv is done this way. Should we change it to 
use the same method as official CPython Windows releases?

Here is a diff I think we need to apply for now, if you feel it is reasonable 
to make a PR then I'm happy to do so.

```
$ diff -urN Lib/venv/__init__.py.orig Lib/venv/__init__.py
--- Lib/venv/__init__.py.orig   2019-01-02 20:56:45.015131800 +
+++ Lib/venv/__init__.py2019-01-02 20:56:55.330130800 +
@@ -188,9 +188,9 @@
 binpath = context.bin_path
 path = context.env_exe
 copier = self.symlink_or_copy
+copier(context.executable, path)
 dirname = context.python_dir
 if os.name != 'nt':
-copier(context.executable, path)
 if not os.path.islink(path):
 os.chmod(path, 0o755)
 for suffix in ('python', 'python3'):
```

--
status: closed -> open

___
Python tracker 
<https://bugs.python.org/issue35644>
___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue35644] venv doesn't do what it claims to do (apears not to work at all?)



Ray Donnelly  added the comment:

The commit that my patch modifies is: 
https://github.com/python/cpython/commit/1c3de541e64f75046b20cdd27bada1557e550bcd

Cheers.

--

___
Python tracker 
<https://bugs.python.org/issue35644>
___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue35644] venv doesn't work on Windows when no venvlauncher executable present



Change by Ray Donnelly :


--
title: venv doesn't do what it claims to do (apears not to work at all?) -> 
venv doesn't work on Windows when no venvlauncher executable present

___
Python tracker 
<https://bugs.python.org/issue35644>
___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue35644] venv doesn't work on Windows when no venvlauncher executable present

2019-01-03 Thread Ray Donnelly



Ray Donnelly  added the comment:

Thanks Steve, the sys.path value from the first comment can be discarded, it 
was running the wrong Python!

The 'old' mechanism (which my patch reverts to) does copy all the necessary 
DLLs already. I released builds with this patch now and venv works fine (tested 
with pyperformance which uses venv).

However, we are more than happy to switch to the venvlauncher method as not 
deviating from upstream unnecessarily is always a good thing!

Do you have any pointers about how to build venvlauncher? I'll try to schedule 
some time for that.

--

___
Python tracker 
<https://bugs.python.org/issue35644>
___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue36354] Use CreateProcessW for Python 2.7 on Windows.

2019-03-18 Thread Ray Donnelly



New submission from Ray Donnelly :

Hi all,

I'd like to entertain some discussion around the idea of calling CreateProcessW 
instead of CreateProcess on Windows.

I've written a patch as a proof of concept and I would love to get some 
feedback. I guess I've broken the normal ACP/getfilesystemencoding() 
expectation for byte strings here. My idea to fix this was to use 
CreateProcessW only when all arguments (program name, arguments, cwd, 
environment) are unicode already.

The reason we'd like to use it on Anaconda Distribution is that we would like 
for conda to be able to handle Unicode as well as possible in as many 
situations as possible, including running a Python2 conda and creating conda 
envs with all sorts of glyphs in it.

We run into bug reports quite frequently from people who try to install 
Miniconda2 or Anaconda2 in their home directories due to their username 
containing certain codepoints.

--
files: 0017-Use-CreateProcessW-to-support-Unicode.patch
keywords: patch
messages: 338270
nosy: Ray Donnelly, giampaolo.rodola, paul.moore, steve.dower, tim.golden, 
zach.ware
priority: normal
severity: normal
status: open
title: Use CreateProcessW for Python 2.7 on Windows.
Added file: 
https://bugs.python.org/file48216/0017-Use-CreateProcessW-to-support-Unicode.patch

___
Python tracker 
<https://bugs.python.org/issue36354>
___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue36354] Use CreateProcessW for Python 2.7 on Windows.

2019-03-18 Thread Ray Donnelly



Ray Donnelly  added the comment:

.. and alternative to my ACP idea would be to use `GetACP()` or 
`getfilesystemencoding()` .. or? Suggestions welcome!

--

___
Python tracker 
<https://bugs.python.org/issue36354>
___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue33232] Segmentation fault in operator.attrgetter

2018-04-05 Thread Ray Donnelly


Change by Ray Donnelly :


--
nosy: +Ray Donnelly

___
Python tracker 
<https://bugs.python.org/issue33232>
___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue32457] Windows Python cannot handle an early PATH entry containing ".." and python.exe

2017-12-30 Thread Ray Donnelly


New submission from Ray Donnelly :

Over on the Anaconda Distribution we received a (private) bug report about a 
crash when trying to use scons. I thought initially it was due to one of our 
patches but I tested it out with official CPython and also with WinPython and 
ran into the same crash.

To reproduce this, from cmd.exe on CPython (here I installed CPython as part of 
Visual Studio 2017, then updated it to the latest 3.6.4 release):

```
set "PATH=C:\Program Files (x86)\Microsoft Visual 
Studio\Shared\Python36_64\Scripts\..;%PATH%"

python.exe

..

python
Fatal Python error: Py_Initialize: unable to load the file system codec
ModuleNotFoundError: No module named 'encodings'

Current thread 0x0328 (most recent call first):
```

The trigger for this bug is the following batch code in scons.bat:

https://bitbucket.org/scons/scons/src/c0172db149b1a151eeb76910d55c81746bfede05/src/script/scons.bat?at=default&fileviewer=file-view-default#scons.bat-19

My current thinking is that the best fix here is to modify 
get_progpath()/get_program_full_path() so that it uses PathCchCanonicalizeEx() 
at 
https://github.com/python/cpython/blob/9bee329130aae5a13050c08dab9d349b76e66835/PC/getpathp.c#L558-L559

--
components: Windows
messages: 309242
nosy: Ray Donnelly, paul.moore, steve.dower, tim.golden, zach.ware
priority: normal
severity: normal
status: open
title: Windows Python cannot handle an early PATH entry containing ".." and 
python.exe
type: crash
versions: Python 3.6

___
Python tracker 
<https://bugs.python.org/issue32457>
___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue32457] Windows Python cannot handle an early PATH entry containing ".." and python.exe

2017-12-30 Thread Ray Donnelly


Ray Donnelly  added the comment:

.. though I will also ask the scons people to change this to use pushd and %CD% 
instead. Even if you were to make Python capable of handling such bad input, 
who knows what other programs will fail, and build systems should be extra 
careful not to mess the environment up like this.

--

___
Python tracker 
<https://bugs.python.org/issue32457>
___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue3871] cross and native build of python for mingw32 with packaging


Changes by Ray Donnelly :


Added file: 
http://bugs.python.org/file25761/python-py3k-20120318-MINGW-330a2.patch

___
Python tracker 
<http://bugs.python.org/issue3871>
___
___
Python-bugs-list mailing list
Unsubscribe: 
http://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue3754] cross-compilation support for python build


Changes by Ray Donnelly :


Added file: 
http://bugs.python.org/file25762/python-py3k-20120318-CROSS-330a2.patch

___
Python tracker 
<http://bugs.python.org/issue3754>
___
___
Python-bugs-list mailing list
Unsubscribe: 
http://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue3754] cross-compilation support for python build


Ray Donnelly  added the comment:

Hi Roumen,

Many thanks for your patches, I've been using a 2.7.1 version of your patches 
for Python integration with GDB (pretty-printing) of my own version of the 
Android NDK for ages now (part of the Necessitas Qt project) and I really 
appreciate your efforts in this area. I've recently started looking into cross 
compilation of Python as I want to build everything on Linux and hopefully I 
can be useful in this effort.

What you said about wchar_t on Android is true on API level <= 8, but fixed for 
>8. See:

https://groups.google.com/group/android-ndk/browse_thread/thread/e40e3fd241e72d3f?pli=1

To test this:

test.c:
#include 
char bytes[(signed)sizeof(wchar_t)-4];

compile failure:
arm-linux-androideabi-gcc -I$NDK/platforms/android-8/arch-arm/usr/include/ -S 
test.c
test.c:2: error: size of array 'bytes' is negative

compile success:
arm-linux-androideabi-gcc -I$NDK/platforms/android-9/arch-arm/usr/include/ -S 
test.c

I've attached a version of your latest patch re-made against released 3.3.0a2. 
I've also done the same thing for your MinGW patch at 
http://bugs.python.org/issue3871

I'll follow up shortly with a patch that addresses some of the remaining issues 
with cross building for MinGW on Linux (and also the beginnings of Darwin on 
Linux). It'll be attached to http://bugs.python.org/issue3871 though as it's 
more specific to MinGW than general cross.

--
nosy: +Ray.Donnelly

___
Python tracker 
<http://bugs.python.org/issue3754>
___
___
Python-bugs-list mailing list
Unsubscribe: 
http://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue3871] cross and native build of python for mingw32 with packaging


Changes by Ray Donnelly :


Added file: 
http://bugs.python.org/file25764/python-py3k-20120318-MINGW-330a2.patch

___
Python tracker 
<http://bugs.python.org/issue3871>
___
___
Python-bugs-list mailing list
Unsubscribe: 
http://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue3871] cross and native build of python for mingw32 with packaging


Changes by Ray Donnelly :


Removed file: 
http://bugs.python.org/file25764/python-py3k-20120318-MINGW-330a2.patch

___
Python tracker 
<http://bugs.python.org/issue3871>
___
___
Python-bugs-list mailing list
Unsubscribe: 
http://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue3871] cross and native build of python for mingw32 with packaging


Ray Donnelly  added the comment:

Although mainly focussed on MinGW, I've begun adding Darwin cross support. I've 
also built Darwin cross compilers. The latest Linux version can be found at:

http://mingw-and-ndk.googlecode.com/files/multiarch-darwin11-cctools127.2-gcc42-5666.3-llvmgcc42-2336.1-Linux-120531.tar.xz

Here's a run-down of what's in the patch:

configure.ac:
 Set MACHDEP=darwin when using darwin-cross compilers.
 Set DELIM to ; for MACHDEP=win.
 Switch from PC/getpathp.c to Modules/getpath.c for MinGW build.
  This gives us a posix like layout as needed by autotools projects
  such as GDB.
 Add MSYSVPATH as an env. var to configure (used in getpath.c):
  MSYS make uses a 'virtual' VPATH, but getpath.c uses
  GetModuleFileNameW (replacing \ with /). This allows the user to
  define the 'actual 'real' value. Note, it should contain / not \,
  (i.e. use ecactly what is printed by "pwd -W")
 Fixes to build with MinGW64 for REPARSE_DATA_BUFFER (needs defining)

Include/fileutils.h:
 Define Py_wstat for MinGW/MinGW64.

Include/osdefs.h:
 Define SEP to / and ALTSEP to \ for MinGW.

cygwinccompiler.py:
 Fix RE_VERSION in for handling MinGW64 compiler versioning.
 Ask gcc where ld is instead of requiring it to be passed in os.environ
 
plat-generic/regen:
 Allow passing in the CC so sysroots can be determined from -print-search-dirs
 If this CC passing is detected, use $CCINSTALL/include/stddef.h as 
netinet/in.h doesn't come with MinGW.

Moules/faulthandler.c:
 Avoid _set_abort_behavior on MinGW64 as it's broken.

Modules/getpath.c:
 Add support for MinGW.

Modules/socketmodule.c:
 Change case (to lower) of #include  for MinGW/MinGW64 cross.

Parser/metagrammar.c:
 Rename _PyParser_Grammar to _PyParser_MetaGrammar to avoid multiply defined 
symbols.

Python/pythonrun.c:
 Translate ALTSEP (\) to SEP (/) in Py_SetProgramName

setup.py:
 Add 'msi','cabinet','rpcrt4' as libs to link against for _msi module.
 Fix an error with self.get_platform() no longer being defined (changed to 
host_platform)

Best regards,

Ray Donnelly.

--
nosy: +Ray.Donnelly
Added file: 
http://bugs.python.org/file25765/python-py3k-20120318-MINGW-FIXES-USE-POSIX-GETPATH-330a2.patch

___
Python tracker 
<http://bugs.python.org/issue3871>
___
___
Python-bugs-list mailing list
Unsubscribe: 
http://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue3871] cross and native build of python for mingw32 with packaging


Ray Donnelly  added the comment:

Hi Éric,

Do you mean this bit?:

diff -urN a/Lib/distutils/cygwinccompiler.py b/Lib/distutils/cygwinccompiler.py
--- a/Lib/distutils/cygwinccompiler.py  2012-05-30 07:33:00.234438631 +0100
+++ b/Lib/distutils/cygwinccompiler.py  2012-05-30 07:33:03.320855811 +0100
@@ -392,7 +392,7 @@
 return (CONFIG_H_UNCERTAIN,
 "couldn't read '%s': %s" % (fn, exc.strerror))
 
-RE_VERSION = re.compile(b'(\d+\.\d+(\.\d+)*)')
+RE_VERSION = re.compile(b'[\D\s]*(\d+\.\d+(\.\d+)*)[\D\s]*$')

It's hardly touching distutils and only the cygwinccompiler.py part (which I 
doubt is used very much anyway). It there not some extensive testsuite we can 
run patches against that would allow distutils changes?

I guess the very limited amount of changes I've made to distutils is academic 
though as it depends on two patches from Roumen Petrov which touch distutils a 
lot more.

I thought that the distutils-is-frozen rule was only for Python 2.x and that 
they would be allowed for 3.x? Is this not true?

Obviously making Python cross compile properly is a very desired feature (in 
this day and age I'd say cross-compilation for any major software is expected 
to be honest with you). How can we progress this task in a way that is 
acceptable? Please advise.

Best regards,

Ray.

--

___
Python tracker 
<http://bugs.python.org/issue3871>
___
___
Python-bugs-list mailing list
Unsubscribe: 
http://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue3754] cross-compilation support for python build

2012-06-07 Thread Ray Donnelly


Ray Donnelly  added the comment:

Re Android

Ah of course. In that case, there's always the NDK built by Dmitry (his latest 
is v7 AFAIK):

http://www.crystax.net/en/android/ndk

--

___
Python tracker 
<http://bugs.python.org/issue3754>
___
___
Python-bugs-list mailing list
Unsubscribe: 
http://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue3871] cross and native build of python for mingw32 with packaging

2012-06-07 Thread Ray Donnelly


Ray Donnelly  added the comment:

Thanks for the details. I've got more fixes but I want to get the whole thing 
building (at least) before I post them.

--

___
Python tracker 
<http://bugs.python.org/issue3871>
___
___
Python-bugs-list mailing list
Unsubscribe: 
http://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue3754] cross-compilation support for python build

2012-06-30 Thread Ray Donnelly


Ray Donnelly  added the comment:

NCURSES_INTERNALS stuff appears to be redundant:

Mac OS X curses.h, Linux curses.h and Windows PDCurses.h don't reference it, 
nor does the Python 3.3.0b1 source code. Of course, I haven't checked any other 
systems.

However: see http://bugs.python.org/issue14598, so maybe for older Linux 
ncurses and Cygwin it is still needed.

NCURSES_OPAQUE investigation:

Linux ncurses.h:
#ifndef NCURSES_OPAQUE
#define NCURSES_OPAQUE 0
#endif

Mac OS X ncurses.h:
#ifndef NCURSES_OPAQUE
#define NCURSES_OPAQUE 1
#endif

So for OSX we need to be defining it to 0...

but IMHO, these parts are not directly related to cross compilation support so 
shouldn't be part of this patch.

--

___
Python tracker 
<http://bugs.python.org/issue3754>
___
___
Python-bugs-list mailing list
Unsubscribe: 
http://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue3871] cross and native build of python for mingw32 with packaging

2012-07-01 Thread Ray Donnelly


Ray Donnelly  added the comment:

For me, it's all about being able to build useful software. I like Python a 
lot, but my goal is and has always been cross compilation of gdb with Python 
support. To that end, if I have to maintain some out of tree patches for 2.7.3 
then so be it. For 3.x things are looking better and I'm hopeful that we'll get 
everything merged at some point.

For anyone who's interested in my almost-final set of patches against the 
released tarballs for both 2.7.3 and 3.3.0b1 they're at:

http://mingw-and-ndk.googlecode.com/files/python-273-and-330b1-patches-WIP.7z

AFAIK, the only real change I'll be making is to the build script.

Once Matthias has finished merging Roumen's cross compilation work to 3, I'll 
rebase my 3.3.0b1 patches against that and open an issue for each patch. If 
mingw cross touches too much of distutils for people's liking then hopefully we 
can get mac osx cross merged and that'll at least mean that the infrastructure 
is cross-compilation friendly.

--

___
Python tracker 
<http://bugs.python.org/issue3871>
___
___
Python-bugs-list mailing list
Unsubscribe: 
http://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue3871] cross and native build of python for mingw32 with packaging

2012-07-01 Thread Ray Donnelly


Ray Donnelly  added the comment:

I'm not personally interested in Cygwin (it's too slow for my liking) and I've 
spent a lot of time patching and building software to avoid forcing people to 
use it (no offence meant to Cygwin people, I can see the value of it, but IMHO 
native is always best) .

All of my patches target mingw-w64 (and Mac OS X cross); I'll do my best to 
make sure that the Cygwin port still works though (and also mingw32).

--

___
Python tracker 
<http://bugs.python.org/issue3871>
___
___
Python-bugs-list mailing list
Unsubscribe: 
http://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue3871] cross and native build of python for mingw32 with packaging

2012-07-01 Thread Ray Donnelly


Ray Donnelly  added the comment:

Thanks Matthias, I might wait until b2 (or rc1) before I rebase my patches, 
just because I'm so unfamiliar with Mercurial.

--

___
Python tracker 
<http://bugs.python.org/issue3871>
___
___
Python-bugs-list mailing list
Unsubscribe: 
http://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue16235] Add python-config.sh for use during cross compilation.

2012-10-14 Thread Ray Donnelly


New submission from Ray Donnelly:

Creates python-config.sh which aims to behave exactly the same as python-config 
except it doesn't depend on a Python interpreter on the build machine (only 
depends on a posixy shell, tested with bash, dash, MSYS-bash).

I use this for cross compiling gdb with Python pretty printer support.

--
components: Cross-Build
files: -add-python-config-sh.patch
keywords: patch
messages: 172925
nosy: Ray.Donnelly
priority: normal
severity: normal
status: open
title: Add python-config.sh for use during cross compilation.
type: enhancement
versions: Python 3.4
Added file: http://bugs.python.org/file27572/-add-python-config-sh.patch

___
Python tracker 
<http://bugs.python.org/issue16235>
___
___
Python-bugs-list mailing list
Unsubscribe: 
http://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue16291] Fix some general cross compile issues and some darwin specific ones. Tested with build=linux, host=darwin


New submission from Ray Donnelly:

I've fixed up the very few missing configure bits and one issue with 
PYTHON_FOR_BUILD using srcdir instead of builddir. I also rolled in a simple 
warning fix that I ran into (the #ifdef HAVE_CHROOT added to posixmodule.c)

configured with:
$srcdir/configure --prefix=$PWD/install/darwin-x86_64/python-3.4.x 
--build=x86_64-linux-gnu --host=x86_64-apple-darwin11

using these compilers:
http://mingw-and-ndk.googlecode.com/files/multiarch-darwin11-cctools127.2-gcc42-5666.3-llvmgcc42-2336.1-Linux-120724.tar.xz

and MacOSX10.7.sdk,

I can successfully build Python for Mac OS X on Linux.

--
components: Cross-Build
files: 0010-DARWIN-CROSS.patch
keywords: patch
messages: 173420
nosy: Ray.Donnelly, doko
priority: normal
severity: normal
status: open
title: Fix some general cross compile issues and some darwin specific ones. 
Tested with build=linux, host=darwin
type: enhancement
versions: Python 3.4
Added file: http://bugs.python.org/file27641/0010-DARWIN-CROSS.patch

___
Python tracker 
<http://bugs.python.org/issue16291>
___
___
Python-bugs-list mailing list
Unsubscribe: 
http://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue16292] Cross compilation fixes (general)


New submission from Ray Donnelly:

This patch fixes 2 cross compilation bugs and improves a comment.

--
components: Cross-Build
files: 0005-cross-fixes.patch
keywords: patch
messages: 173423
nosy: Ray.Donnelly, doko
priority: normal
severity: normal
status: open
title: Cross compilation fixes (general)
type: compile error
versions: Python 3.4
Added file: http://bugs.python.org/file27643/0005-cross-fixes.patch

___
Python tracker 
<http://bugs.python.org/issue16292>
___
___
Python-bugs-list mailing list
Unsubscribe: 
http://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue16291] Fix some general cross compile issues and some darwin specific ones. Tested with build=linux, host=darwin


Ray Donnelly added the comment:

> It seems to me that such an animal would be fragile and incomplete at best.

There's no reason for it to be either fragile or incomplete. My cross builds 
have a goal of being comprehensive.

> Python builds on OS X normally require certain OS X frameworks outside of 
> those included with the compiler.

If the user supplies the Apple Mac OS X SDK then the frameworks are correctly 
detected and used. To enable this, I use a set of wrapper scripts to enable 
this, here's the contents of x86_64-apple-darwin11-gcc:

#!/bin/sh
# Auto-generated, do not edit
/home/ray/darwin-cross/apple-osx/bin/i686-apple-darwin11-gcc -isysroot 
/home/ray/darwin-cross/MacOSX10.7.sdk -mmacosx-version-min=10.5 
-DMAXOSX_DEPLOYEMENT_TARGET=10.5 -m64 "$@"

> Why would we want to officially support this particular variant of cross 
> building on another platform?

There's 3 separate issues merged into this patch, general cross compilation 
fixes, the new darwin cross compilation feature and a few darwin warning fixes 
(that apply equally to cross and native). I shouldn't have done that, and am in 
the process of splitting this issue up so that each part can be judged in 
isolation. Please see the new issue I've made for the general cross compilation 
issues: http://bugs.python.org/issue16292
..later I'll work on a new patch for this issue.

--

___
Python tracker 
<http://bugs.python.org/issue16291>
___
___
Python-bugs-list mailing list
Unsubscribe: 
http://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue16292] Cross compilation fixes (general)


Changes by Ray Donnelly :


--
nosy: +ned.deily

___
Python tracker 
<http://bugs.python.org/issue16292>
___
___
Python-bugs-list mailing list
Unsubscribe: 
http://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue16291] Fix some general cross compile issues and some darwin specific ones. Tested with build=linux, host=darwin

2012-10-21 Thread Ray Donnelly


Changes by Ray Donnelly :


Removed file: http://bugs.python.org/file27641/0010-DARWIN-CROSS.patch

___
Python tracker 
<http://bugs.python.org/issue16291>
___
___
Python-bugs-list mailing list
Unsubscribe: 
http://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue16291] Enable darwin-host cross compilation. Tested with build=linux, host=darwin

2012-10-21 Thread Ray Donnelly


Ray Donnelly added the comment:

Here's the darwin cross enabling portion. I've also removed the warnings fixes 
from it as they're not relevant.

--
title: Fix some general cross compile issues and some darwin specific ones. 
Tested with build=linux, host=darwin -> Enable darwin-host cross compilation. 
Tested with build=linux, host=darwin
Added file: http://bugs.python.org/file27647/0010-cross-darwin-feature.patch

___
Python tracker 
<http://bugs.python.org/issue16291>
___
___
Python-bugs-list mailing list
Unsubscribe: 
http://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue16291] Enable darwin-host cross compilation. Tested with build=linux, host=darwin

2012-10-22 Thread Ray Donnelly


Ray Donnelly added the comment:

> A small question about the patch, why this case in the cross_arch function:

>From a real Mac Book Pro:

$ uname -a
Darwin MACBOOKPRO.local 11.4.2 Darwin Kernel Version 11.4.2: Thu Aug 23 
16:25:48 PDT 2012; root:xnu-1699.32.7~1/RELEASE_X86_64 x86_64
$ /usr/bin/arch
i386
$ uname -m
x86_64

It would seem arch on OS X can only be used to determine PPC vs Intel (which is 
all that configure uses it for anyway).

> Why is cross-compiling to OSX useful at all?

Cross compiling is always useful for build engineers who have to target 
multiple hosts. 

> You still have to test if the output of the compilation works, and for that 
> you need an OSX system.

Ideally yes, or a Virtual Machine with OSX on it or confidence based on the 
results of a previous test built with the same sources or maloader (...though 
maybe not, I've never personally tried maloader).

> The patch will only help with cross-compiling C code, it won't compile 
> resource files (like the NIB file in Python Launcher)

My personal interest is for having Python embedded in other things (GDB) and 
using it from the command-line. Python Launcher is not part of the scope.

> This will make support harder, I already get support questions where the 
> answer depends on the way Python was build and that will likely get worse 
> with cross-compiling.

I'd be happy to do whatever I can to help out with this. If we could identify 
in the build that it was built via cross compilation somehow or have my email 
address in the help would that make it more palatable?

> * Where would users get the SDK? I'm not a lawyer (or even familiar with US 
> law), but the Xcode license seems to indicate that it can only be used on 
> Apple systems.

The users would get the SDK from being registered as a developer with Apple. As 
for the license, while also not being a lawyer, my interpretation is this:

In the Xcode License.rtf file (and the Command Line tools license), the 
relevant part is "you are granted a limited, non-exclusive license to use the 
Developer Software on Apple-branded computers to develop and test application 
and other software". This doesn't say anything about the OS that's running on 
said Apple-branded computer, and my interpretation is that you could install a 
Linux distro on an Apple computer or on a VM on an Apple computer and still be 
compliant with that license.

--

___
Python tracker 
<http://bugs.python.org/issue16291>
___
___
Python-bugs-list mailing list
Unsubscribe: 
http://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue16235] Add python-config.sh for use during cross compilation.

2012-11-04 Thread Ray Donnelly


Ray Donnelly added the comment:

I agree in principle about not having to maintain two implementations but I 
would worry about breaking external packages that depend on the python 
implementation.

In the meantime, please find an updated version of this patch that adds support 
for --configdir

--
Added file: http://bugs.python.org/file27880/-add-python-config-sh.patch

___
Python tracker 
<http://bugs.python.org/issue16235>
___
___
Python-bugs-list mailing list
Unsubscribe: 
http://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue16235] Add python-config.sh for use during cross compilation.

2012-11-04 Thread Ray Donnelly


Ray Donnelly added the comment:

I also checked the Windows releases and python-config isn't included. However, 
I don't think there's a good reason for that. python-config would in theory 
work fine if people wanted to link their programs to Python on Windows, as 
would python-config.sh given a posix-y shell that e.g. Cygwin or MSYS provide.

--

___
Python tracker 
<http://bugs.python.org/issue16235>
___
___
Python-bugs-list mailing list
Unsubscribe: 
http://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue16235] Add python-config.sh for use during cross compilation.

2012-11-07 Thread Ray Donnelly


Ray Donnelly added the comment:

> is there a need for the built vs. installed prefix?
>this is logic not found in the python implementation.
>what is this supposed to do?

You are right, it is not found in the original python implementation, but I 
feel that it's useful. Without it, you couldn't give someone a -devel style 
precompiled Python archive that they can extract where-ever they want and embed 
in other software without requiring them to install it to the same prefix used 
during the build process.

> is `local' available in all shells? just avoid it.

I can't be sure about all shells, but local is *very* well supported (bash, 
dash, csh); I'm happy to remove it though as it's not necessary.

> LDLIBRARY and LDSHARED are expressed in terms of Makefile
>   macros, leading to syntax errors.

This shouldn't be the case, see:

# Replace makefile compat. variable references with shell script compat. ones; 
$(VAR) -> ${VAR}
+   sed -e "s,\$$(\([A-Za-z0-9_]*\)),\$$\{\1\},g" < Misc/python-config.sh 
>python-config.sh

> - --includes doesn't include the plat specific include dir

You are right, I will correct this oversight.

> - abi flags are missing from the include and lib dirs.

ABIFLAGS propagates through other variables, namely LDLIBRARY and LDVERSION, 
but again, there are some issues for me to fix with this.

I will write a new part on my build script that compares the output from the 
shell script with that from the python script and post a new patch when they 
agree 100% on all my test platforms.

--

___
Python tracker 
<http://bugs.python.org/issue16235>
___
___
Python-bugs-list mailing list
Unsubscribe: 
http://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue16235] Add python-config.sh for use during cross compilation.

2012-11-07 Thread Ray Donnelly


Ray Donnelly added the comment:

> hmm, but python.pc, _sysconfigdata.py and Makefile still have this 
> information.

In my experience, as long as PYTHONHOME is set correctly before the Python .so, 
.dll or .dylib is loaded then everything works correctly.

>From what I've seen, .pc files are also useless for the purposes of a 
>relocatable -devel package as there's no capability for relative directories 
>in pkg-config.

> maybe should only be included, if it's different.

Ok.

--

___
Python tracker 
<http://bugs.python.org/issue16235>
___
___
Python-bugs-list mailing list
Unsubscribe: 
http://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue16235] Add python-config.sh for use during cross compilation.

2012-11-08 Thread Ray Donnelly


Ray Donnelly added the comment:

I've addressed everything (local, ABIFLAGS) except the platform includes.

AFAICT, platform includes only differs from includes during the Python build 
phase. This script can only be run post-install.

I also added --extension-suffix, tidied it up and allowed it to return multiple 
values in a single invocation, as per the Python script.

--
Added file: http://bugs.python.org/file27922/-add-python-config-sh.patch

___
Python tracker 
<http://bugs.python.org/issue16235>
___
___
Python-bugs-list mailing list
Unsubscribe: 
http://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue16454] Mostly for discussion: _winapi as builtin for bootstrapping distutils.

2012-11-11 Thread Ray Donnelly


New submission from Ray Donnelly:

I'm submitting this patch as it was suggested I do so by Antoine Pitrou on 
#python-dev. As such, it's more a point of discussion than a patch I'm trying 
to get merged (for one thing, my build is a MinGW GCC compiled and I've not 
considered the MSVC Windows build).

I ran into a problem with a customisation I made to cygwinccompiler.py where I 
wanted to use Popen to ask gcc where it's ld is:

out = Popen(gcc+' --print-prog-name ld', shell=True, stdout=PIPE).stdout

This has worked fine before on 2.7.3, but with 3.3.0, subprocess can't be 
imported as _winapi isn't builtin, instead, it's built by setup.py as a module.

So is this the right thing to do? Or is distutils not supposed to use 
subprocess during bootstrapping?

--
files: _winapi_as_builtin_for_distutils_bootstrap.patch
keywords: patch
messages: 175393
nosy: Ray.Donnelly, pitrou, sbt
priority: normal
severity: normal
status: open
title: Mostly for discussion: _winapi as builtin for bootstrapping distutils.
type: behavior
versions: Python 3.4
Added file: 
http://bugs.python.org/file27962/_winapi_as_builtin_for_distutils_bootstrap.patch

___
Python tracker 
<http://bugs.python.org/issue16454>
___
___
Python-bugs-list mailing list
Unsubscribe: 
http://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue16454] Mostly for discussion: _winapi as builtin for bootstrapping distutils.

2012-11-11 Thread Ray Donnelly


Ray Donnelly added the comment:

I should have checked the situation better, it is indeed towards the back end 
of a series of patches to enable MinGW compilation, so the issue is irrelevant 
without the preceding patches.

Sorry for the noise.

--
resolution:  -> invalid

___
Python tracker 
<http://bugs.python.org/issue16454>
___
___
Python-bugs-list mailing list
Unsubscribe: 
http://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue16235] Add python-config.sh for use during cross compilation.

2012-11-30 Thread Ray Donnelly


Ray Donnelly added the comment:

New patch attached:

LIBPL changed to a more direct AC_SUBST and PLATINCDIR used for --includes and 
--cflags.

--
Added file: http://bugs.python.org/file28164/-add-python-config-sh.patch

___
Python tracker 
<http://bugs.python.org/issue16235>
___
___
Python-bugs-list mailing list
Unsubscribe: 
http://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue3871] cross and native build of python for mingw* hosts

2012-12-28 Thread Ray Donnelly


Ray Donnelly added the comment:

Good work Jason,

I think it would be great if everyone could collaborate better with this stuff; 
I'm as guilty as the next person on that score though. I might have been able 
to save you some effort by pointing you at either:

https://github.com/mingwandroid/crucifixion-freedom/tree/master/patches/python/3.3.0
 [Python cross and MinGW compilation]

or:

https://github.com/niXman/mingw-builds/tree/master/patches/Python-3.3.0 [MinGW 
toolchains including Python pretty-printing GDB]

..there might be some patches in there that are useful for you. Both these 
projects use basically the same patches, the 'source' of them is 
crucifixion-freedom.

Our patching strategies are more sane (many patches as opposed to a monolithic 
patch - in fact, just after Roumen's huge patch we apply some patches to remove 
large parts of it so we can re-introduce those independent fixes via separate 
patches).

Both crucifixion-freedom and mingw-builds also provide patches for 2.7.3 and 
support building tkinter and IDLE. The unique feature of crucifixion-freedom is 
fairly comprehensive cross compilation capability 
({Linux,Windows,Mac}*{32bit,64bit}).

Given how intrusive these patches are, getting them merged into CPython is a 
bit of a pipe dream (I'd love to be wrong about that), so making it as easy as 
possible for the interested parties to maintain (forward port), build, test and 
package Python releases is probably the best we can hope for. If we can make 
the patches small and independently reviewable then maybe some parts could be 
merged.

--

___
Python tracker 
<http://bugs.python.org/issue3871>
___
___
Python-bugs-list mailing list
Unsubscribe: 
http://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue16235] Add python-config.sh for use during cross compilation.

2013-01-25 Thread Ray Donnelly


Ray Donnelly added the comment:

Great. My only query is whether the changes to sed will work on non-GNU sed.

It used to be like:
sed < in >out

you've now got:
sed in >out

I'm not saying it won't work, I'm not in a position to check, but I think it's 
safer to keep it as it was.

--

___
Python tracker 
<http://bugs.python.org/issue16235>
___
___
Python-bugs-list mailing list
Unsubscribe: 
http://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue16235] Add python-config.sh for use during cross compilation.


Ray Donnelly added the comment:

Thank you Matthias!

--

___
Python tracker 
<http://bugs.python.org/issue16235>
___
___
Python-bugs-list mailing list
Unsubscribe: 
http://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue3871] cross and native build of python for mingw* hosts

Ray Donnelly added the comment:

When I say “our patches” I mean mine and Alexey Pavlov’s jointly maintained
patch-set.

I hope you don’t mind that I find you saying:

"I tried some of these patches, but they aren't very organinzed. I really need
some docemntaiton to better understand what each patch does and which pairs go
together and which don't."

...and then continuing to present a single uber-patch to be mildly
funny/ironic? You've already accepted that you should split them up and remove
the needless formatting changes to make them more reviewable so I won't labour
on this point.

"ctypes
bz2
multiprocessing
sqlite3
ssl
pyexpat
zlib"

Likewise for ours, but also curses, curses_panel, readline, TKinter, IDLE (and
probably more besides) for both Win32, Win64, Mac OS X 32bit and 64bit.

The main idea behind our patches is (mostly in common with all patches) that
they:
1. Are as atomic as possible in their operation (which as few interdependencies
as necessary).
2. Are applied in an obvious ordering.
3. Are named sensibly according to what they do.
4. Build upon - and can reuse updates to - Roumen's uber-patch (and also clean
it up a bit; let’s face it Roumen did a lot of the hard work and everyone else
is trying to finish it off so we can build releases from it).
5. Enable comprehensive cross-compilation.

Given the name of each patch and the fact that they do one thing (the novel
ones at least) and that by-and-large they fit on a few screens, I had hoped
that they were usefully descriptive.

To explain which ones go together:
The numbering skips 5 each time for each independent bug-fix or feature - to
allow space for later, related modifications. Therefore, where the numbering
changes monotonically, the patches are related. This only happens twice, once
to address the problem caused by Roumen providing an uber-patch and once due to
a block of fixes all related to curses (ncurses and PDCurses are both
supported). I assume my addressing of Roumen's uber-patch is what led you to
write:

"However, I'm noticing a ton of descrepancies with these patches. Some are
removing pthreads, others are adding them back."

There is logic to what you describe as a ton of discrepancies:
Roumen's patch is included in it's entirety as 0030-py3k-20121004-MINGW.patch,
then I remove one feature of that patch that needed fixing (threading changes)
with 0031-py3k-20121004-MINGW-removal-of-pthread-patch.patch, then I add a
patch that provides my newer fix, 0032-py3k-20121004-MINGW-ntthreads.patch. The
exact same thing happens with the libffi changes. The pthreads stuff needs
fixing due to libwinpthread in MinGW-w64 and libffi is broken when targetting a
64bit Windows host.

My reason for doing this is so that I can directly drop updated versions from
Roumen in over 0030 and it will (in theory at least) still work. He can
continue to use a uber-patch if he wishes (though I don't think he does this
for "new" patches). Also by taking this approach, my patch for the threading
issues has no dependencies at all on Roumen’s patch, since a previous patch
“handled” the dependency by removing that part of it.

Everything you said about build environment and the test-suite I agree with. My
crucifixion-freedom project includes a .vbs to setup the most basic MSYS and
MinGW-w64 environment:
https://raw.github.com/mingwandroid/crucifixion-freedom/master/scripts/windows/BootstrapMinGW64.vbs
...this is essential to prevent spending time chasing down issues due to
discrepancies between build environments.

However, we must go further and add that the patches *cannot* break any other
native or cross-compilation, which - as I think Matthias is alluding to - is
probably not the case with your patch. This issue is called "cross and native
build of python for mingw* hosts", so that your patch possibly breaks builds on
build-machine is unfortunate (especially so as a build-machine Python is
required to run setup.py as part of the cross-build procedure). Even if it
doesn't break building for the build-machine it breaks building the
cross-MinGW-w64. Each occurrence of "/mingw" in your patch is probably a part
of this problem (you can’t expect cross build systems to be be able to be
rooted to /mingw, often you’ll find ~/mingw or /tmp/mingw)

You said, "It seems some are also compiling and using the mingw version outside
of the msys shell, which I'm not certain of the advantages there when u can
just use the mscv version", I’m not entirely sure what you mean here; if
compiling MinGW Python on a Windows system with my patches, MSYS is 100% a
requirement (cmd.exe will never run the configure script). I can only guess
this relates to the mingw_posix2win_path stuff in your patch. Forgive me if I’m
wrong, but it seems that you are conflating MSYS and MinGW when they are not

[issue3871] cross and native build of python for mingw* hosts


Ray Donnelly added the comment:

Re: basing the patches against the latest master branch or targeting released 
versions, I wasn't clear enough about my thinking.

For sure, when trying to get any patches merged, the submitted patch must be 
re-based (forward ported) and tested against the master branch. 

But from the perspective of having a project that people can reliably expect to 
build a working MinGW-w64 Python from (until that elusive fully-merged-day 
comes), having patches based on the latest released versions seems to me to be 
the best thing to do (for that goal of course)

I'll keep forward porting these patches to each new release, and at those times 
I'll also submit some of them as issues to bugs.python.org as that's the best 
way for me to manage my limited spare time.

Do you know what the situation is with packaging/distutils2? I think our next 
good merge opportunity for the bulk of these patches will be when we can target 
that instead of distutils.

> Maybe you could even separate the patches into a patch supporting a native 
> mingw build and a cmingw ross build?

I'm already doing that:
https://github.com/mingwandroid/crucifixion-freedom/tree/master/patches/python/3.3.0

 and 0005 are now merged with upstream.  you merged today, 0005 was 
merged also (mostly via Roumen's patches which 2/3 of my patch duplicated). At 
this stage, the framework for cross compiling Python is mostly merged; other 
than 0065-cross-dont-add-multiarch-paths-if-cross-compiling.patch

The stumbling block for me is that there's no working example of 
cross-compilation in CPython so it will surely rot. This was what I tried to 
fix with my next patch, 0010-cross-darwin-feature.patch. I went for darwin 
instead of MinGW as the amount of changes needed is tiny but it was met with 
some resistance and hasn't seen any activity for 3 months:

http://bugs.python.org/issue16291

I should also clarify, by cross compilation I'm specifically excluding 
GNU-Linux/GNU-Linux combinations such as x86-GNU-Linux/arm-GNU-Linux as they're 
not as exhaustive a test-case as cross-compiling between different OSes.

> Maybe you could even separate the patches into a patch supporting a native 
> mingw build and a cmingw ross build?

Not sure if that was directed at me as my set of patches are logically split up 
I think. Ones with 'mingw' in the name relate to mingw (either cross or 
native), ones with 'msys' in the name relate to building mingw natively and 
ones with 'cross' in the name relate to general cross compilation matters; ones 
with 'hack' in the name generally need improving, rewriting or removing.

--

___
Python tracker 
<http://bugs.python.org/issue3871>
___
___
Python-bugs-list mailing list
Unsubscribe: 
http://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue3871] cross and native build of python for mingw* hosts


Ray Donnelly added the comment:

Roumen, I think it would be really great if you could split 
py3k-20121004-MINGW.patch up into separate bits.

The pthread stuff and libffi stuff being two obvious candidates for atomic 
parts.

--

___
Python tracker 
<http://bugs.python.org/issue3871>
___
___
Python-bugs-list mailing list
Unsubscribe: 
http://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue3871] cross and native build of python for mingw* hosts

2013-02-02 Thread Ray Donnelly


Ray Donnelly added the comment:

Great Roumen, please split into new issues then make a new overall tracking 
issue, then this one can be left for historians and archaeologists.

For Posix installation scheme I've got:

https://github.com/niXman/mingw-builds/blob/master/patches/Python-3.3.0/0050-mingw-sysconfig-like-posix.patch

It needs some changes to allow for normal NT scheme when not using MinGW though.

For system libffi there's:

https://github.com/niXman/mingw-builds/blob/master/patches/Python-3.3.0/0035-mingw-system-libffi.patch

When you make your new patches I will base a new set of patches on them.

--

___
Python tracker 
<http://bugs.python.org/issue3871>
___
___
Python-bugs-list mailing list
Unsubscribe: 
http://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue3754] cross-compilation support for python build

2013-02-05 Thread Ray Donnelly


Ray Donnelly added the comment:

Yes, patches are still required.

Mainly Roumen's big patch [1] and then a load more too. Matthias Klose has 
merged a few cross compilation patches.

Here my project with patches for 3.3.0 and an emphasis on cross:
https://github.com/mingwandroid/crucifixion-freedom

...and here's mingwbuilds with the exact same patches (but likely tidier build 
scripts) without the cross focus:
https://github.com/niXman/mingw-builds/tree/master/patches/Python-3.3.0

Next time there's a release of Python 3, I'll rebase my patches against that.

[1] http://bugs.python.org/issue3871

--

___
Python tracker 
<http://bugs.python.org/issue3754>
___
___
Python-bugs-list mailing list
Unsubscribe: 
http://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue28938] match_hostname treats SAN IP address as DNS name and fails to check CN then

2016-12-11 Thread Ray Satiro


Changes by Ray Satiro :


--
nosy: +raysatiro

___
Python tracker 
<http://bugs.python.org/issue28938>
___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue13517] readdir() in os.listdir not threadsafe on OSX 10.6.8


New submission from Thouis (Ray) Jones :

On my system (OSX 10.6.8) using the python.org 32/64-bit build of 2.7.2, I see 
incorrect results from os.listdir() in a threaded program.  The error is that 
the result of os.listdir() is missing a few files from its list.

First, my use case.  I work with large image-based datasets, often with 
hundreds of thousands of images.  The first step in processing is to locate all 
of these images and extract some basic information (size, channels, etc.).  To 
do this more efficiently on network filesystems, where listing directories and 
stat()ing files is often slow, I wrote a multithreaded analog to os.walk().  
While validating its results against unix 'find', I saw discrepancies in the 
number of files found.

My guess is that OSX's readdir() is not reentrant when dealing with SMB shares, 
even on different DIR pointers.  It's also possible that readdir() is not 
reentrant with lstat(), as some of my tests seemed to indicate this, but I need 
to run some more tests to be sure that's what I was actually seeing.

In any case, there are three possible ways to fix this, I think.

- Remove the Py_BEGIN_ALLOW_THREADS/Py_END_ALLOW_THREADS around readdir() in 
posixmodule.c

- Put a mutex on readdir()

- Use readdir_r().  I've attached a potential patch for 2.7.2 for this solution.

I would prefer the second or last approach, as they preserve the ability to do 
other work while listing large directories.

By my reading of the python 3.0 to 3.4 sources, this problem exists in those 
versions, as well.

--
components: Library (Lib)
files: py272_readdir_r.patch
keywords: patch
messages: 148737
nosy: thouis
priority: normal
severity: normal
status: open
title: readdir() in os.listdir not threadsafe on OSX 10.6.8
type: behavior
versions: Python 2.7, Python 3.1, Python 3.2, Python 3.3, Python 3.4
Added file: http://bugs.python.org/file23832/py272_readdir_r.patch

___
Python tracker 
<http://bugs.python.org/issue13517>
___
___
Python-bugs-list mailing list
Unsubscribe: 
http://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue13517] readdir() in os.listdir not threadsafe on OSX 10.6.8


Thouis (Ray) Jones  added the comment:

Here is the script I use to detect the failure.
% python filefinder.py /PATH/TO/LARGE/DIRECTORY/TREE

(note that I was working over samba with an 8ish-level deep directory with 
around 25 files).

Compare its final output in the FOUND column with 
% find /PATH/TO/LARGE/DIRECTORY/TREE | wc -l

--
Added file: http://bugs.python.org/file23833/filefinder.py

___
Python tracker 
<http://bugs.python.org/issue13517>
___
___
Python-bugs-list mailing list
Unsubscribe: 
http://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue13517] readdir() in os.listdir not threadsafe on OSX 10.6.8


Thouis (Ray) Jones  added the comment:

I should add the caveat that I am not completely confident that I have 
stress-tested the patch enough to be sure that it actually addresses the 
problem.  It is still possible that this is an error in OSX or the remote 
fileserver in which a large amount of concurrent traffic is causing it to 
actually return invalid data.  This is somewhat belied by the fact that I was 
running 'find' at the same time, and did not see it give variable answers, 
ever. I will continue testing.

--

___
Python tracker 
<http://bugs.python.org/issue13517>
___
___
Python-bugs-list mailing list
Unsubscribe: 
http://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue13517] readdir() in os.listdir not threadsafe on OSX 10.6.8


Changes by Thouis (Ray) Jones :


Added file: http://bugs.python.org/file23834/py272_readdir_r.v2.patch

___
Python tracker 
<http://bugs.python.org/issue13517>
___
___
Python-bugs-list mailing list
Unsubscribe: 
http://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue13517] readdir() in os.listdir not threadsafe on OSX 10.6.8


Thouis (Ray) Jones  added the comment:

Reading through many pages discussing readdir vs. readdir_r (many on security 
mailing lists, a large number referring to the page linked in the patch), I get 
the impression that most implementations are thread-safe as long as separate 
threads do not call readdir() using the same DIR pointer.

I believe there is some ambiguity in the POSIX specification as to whether this 
is the only way in which readdir() might be thread-unsafe.

--

___
Python tracker 
<http://bugs.python.org/issue13517>
___
___
Python-bugs-list mailing list
Unsubscribe: 
http://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue13517] readdir() in os.listdir not threadsafe on OSX 10.6.8