Re: [Python-Dev] x86_64 Interix - Advise needed on size of long

[Jerker Bäck]

Hello Martin,
Thanks very much for answering.

> As for the static vs. shared libpython: On Unix, Python is typically
> built as a single executable (only linked shared with the system
> libraries). The challenge is then with extension modules, which are
> shared libraries. In particular, it is a challenge that those want
> to find symbols defined in the executable, without being linked with
> it. So you have three options:
Aha, now it lightens a bit. As I understand, I will need a x86_64 PE GNU ld
to get this to work as intended - however, there is no such thing at this
moment.
> 2. Don't use extension modules. Edit Modules/Setup to statically link
>all extension modules into the interpreter binary.
This is the way. But how to do that?

Shell output:
../configure --disable-shared
...
ar cr libpython2.5.a Objects/
ar cr libpython2.5.a Python/
ar cr libpython2.5.a Modules/
ar cr libpython2.5.a Modules/
cc -o python \
Modules/python.o \
libpython2.5.a -lsocket -lm
CC='cc' LDSHARED='ld' OPT='-DNDEBUG -O' ./python -E ../setup.py build;;
Memory fault (core dumped)
make: *** [sharedmods] Error 139

I assume the "Modules/" are the extension modules. To get
them statically linked, the functions must be called somewhere. Statically
linked = "Builtin modules"? You mean I should list all of these in
"Modules/Setup"? FYI I got the "dynload_stub.c" compiled in. BTW, shouldn't
"--disable-shared" take care of this?

> OK in what sense? You making these changes locally? You can make
> whatever changes you please; this is free software. I can't
> see *why* you want to make all these changes, but if you so
> desire...
It's really very simple - I got LP64 libraries (Interix SDK). To get them to
work with a LLP64 compiler I need explicit sized types in case of long.

FYI: cc is a shell script wrapper of a x64 PE compiler, which in this case
is the MS x64 compiler v 14.00.50727.762 - POSIX mode. It automatically
translates all longs to long long in case of a 64bit compile. Thus, cc
cannot easily be used in e.g. Visual Studio.

> This becoming part of Python? No way. It is intentional that
> PyInt_AsLong returns long (why else would the function be called
> this way?), and it is also intentional that the int type has
> its internal representation as a long.
Oh, it was never my intention to propose a change to the LLP64 model. And
your right: All exports should be according to the LP64 model in case of a
POSIX compile. One must follow some rules! But you must admit it's tempting
with all these:
#if SIZEOF_LONG > 4
< get rid of the 64bit crap >
#endif
In my case the different paradigms are a real pain. I must take it into
account all the time when porting. I can only hope people stop using 
in favour of explicit sized types or types like size_t, intptr_t etc. I
would love to see the damn thing obsolete.

Cheers,
Erik

___
Python-Dev mailing list
Python-Dev@python.org
http://mail.python.org/mailman/listinfo/python-dev
Unsubscribe: 
http://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com

Re: [Python-Dev] x86_64 Interix - Advise needed on size of long

[Martin v. Löwis]

>> 2. Don't use extension modules. Edit Modules/Setup to statically link
>>all extension modules into the interpreter binary.
> This is the way. But how to do that?
> 
> Shell output:
> ../configure --disable-shared

--disable-shared should be the default, so you don't need to specify
it explicitly. It works fine for me on Linux, so if it crashes for
you, it is likely an Interix problem. Can you debug this?

Dynamic loading of extension modules is automatically detected;
it is active if the system is AIX, BeOS, HPUX, Darwin, atheos,
or if dlopen(3) has been found. If you explicitly want to disable
it, you can set DYNLOADFILE to dynload_stub.o in configure.
Not sure whether Interix has dlopen, but even if does, it should
do no harm to use it provided there aren't any modules to load.

> I assume the "Modules/" are the extension modules. To get
> them statically linked, the functions must be called somewhere. Statically
> linked = "Builtin modules"? You mean I should list all of these in
> "Modules/Setup"?

Exactly so. They are already listed - just uncomment them all
(with proper command line flags and libraries where necessary).

> FYI I got the "dynload_stub.c" compiled in.

Ok, so you don't have dynamic loading.

> BTW, shouldn't "--disable-shared" take care of this?

No, this talks only about libpythonxy.so.

> It's really very simple - I got LP64 libraries (Interix SDK). To get them to
> work with a LLP64 compiler I need explicit sized types in case of long.

I still don't understand. Are you *certain* that these are LP64
libraries? Can you kindly refer to some official document that says
Interix uses LP64 on AMD64?

And if so, how did Microsoft manage to build them, if their compiler
does not support LP64? (I see you kind of answer that below - although
I'm unsure what "translate all longs to long long means - you mean
literal text replacement?)

Methinks you should just activate the LP64 mode of VC 2005, and be
done (and no, I don't know how to do that :-)

> FYI: cc is a shell script wrapper of a x64 PE compiler, which in this case
> is the MS x64 compiler v 14.00.50727.762 - POSIX mode. It automatically
> translates all longs to long long in case of a 64bit compile. Thus, cc
> cannot easily be used in e.g. Visual Studio.

So don't use Visual Studio, then. What's wrong with the Makefile?

> Oh, it was never my intention to propose a change to the LLP64 model. And
> your right: All exports should be according to the LP64 model in case of a
> POSIX compile. One must follow some rules! But you must admit it's tempting
> with all these:
> #if SIZEOF_LONG > 4
> < get rid of the 64bit crap >
> #endif

My view would be different - I find it tempting not to use Interix,
let alone on AMD64...

> In my case the different paradigms are a real pain. I must take it into
> account all the time when porting. I can only hope people stop using 
> in favour of explicit sized types or types like size_t, intptr_t etc. I
> would love to see the damn thing obsolete.

You mean the long type? I see nothing wrong with it. The real fault here
is with Microsoft, who managed to provide a system for which they don't
provide a C compiler, but just a hack that looks like a C compiler from
remote.

Regards,
Martin
___
Python-Dev mailing list
Python-Dev@python.org
http://mail.python.org/mailman/listinfo/python-dev
Unsubscribe: 
http://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com

Re: [Python-Dev] [Python-3000] test_asyncore fails intermittently on Darwin

[Alan McIntyre]

On 8/4/07, Jeffrey Yasskin <[EMAIL PROTECTED]> wrote:
> Well, regardless of the brokenness of the patch, I do get two
> different failures from this test on OSX. The first is caused by
> trying to socket.bind() a port that's already been bound recently:

> That looks pretty easy to fix.

It was fixed in the trunk on July 28 as part of rev 56604, by letting
the OS assign the port (binding to port 0). I apologize if everybody
was expecting me to fix this in Python 3000; I thought the initial
complaint was in reference to 2.6. I'm working on test improvements
for 2.6, so I'm sort of fixated on the trunk at the moment. :)  I
wouldn't mind trying to roll my changes forward into Py3k after GSoC
is done if I have the time, though.

Alan
___
Python-Dev mailing list
Python-Dev@python.org
http://mail.python.org/mailman/listinfo/python-dev
Unsubscribe: 
http://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com

[Python-Dev] cStringIO.StringIO() buffer behavior

[Georg Brandl]

See bugs #1548891 and #1730114.

In the former, it was reported that cStringIO works differently from StringIO
when handling unicode strings; it used GetReadBuffer which returned the raw
internal UCS-2 or UCS-4 encoded string.

I changed it to use GetCharBuffer, which converts to a string using the
default encoding first. This fix was also in 2.5.1.

The latter bug now complains that this excludes things like array.array()s
from being used as an argument to cStringIO.StringIO(), which worked before
with GetReadBuffer.

What's the preferred solution here?

Georg

-- 
Thus spake the Lord: Thou shalt indent with four spaces. No more, no less.
Four shall be the number of spaces thou shalt indent, and the number of thy
indenting shall be four. Eight shalt thou not indent, nor either indent thou
two, excepting that thou then proceed to four. Tabs are right out.

___
Python-Dev mailing list
Python-Dev@python.org
http://mail.python.org/mailman/listinfo/python-dev
Unsubscribe: 
http://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com

Re: [Python-Dev] cStringIO.StringIO() buffer behavior

[Guido van Rossum]

Methinks that this was a fundamental limitation of cStringIO, not  a
bug. Certainly not something to be "fixed" in a bugfix release.

On 8/5/07, Georg Brandl <[EMAIL PROTECTED]> wrote:
> See bugs #1548891 and #1730114.
>
> In the former, it was reported that cStringIO works differently from StringIO
> when handling unicode strings; it used GetReadBuffer which returned the raw
> internal UCS-2 or UCS-4 encoded string.
>
> I changed it to use GetCharBuffer, which converts to a string using the
> default encoding first. This fix was also in 2.5.1.
>
> The latter bug now complains that this excludes things like array.array()s
> from being used as an argument to cStringIO.StringIO(), which worked before
> with GetReadBuffer.
>
> What's the preferred solution here?
>
> Georg
>
> --
> Thus spake the Lord: Thou shalt indent with four spaces. No more, no less.
> Four shall be the number of spaces thou shalt indent, and the number of thy
> indenting shall be four. Eight shalt thou not indent, nor either indent thou
> two, excepting that thou then proceed to four. Tabs are right out.
>
> ___
> Python-Dev mailing list
> Python-Dev@python.org
> http://mail.python.org/mailman/listinfo/python-dev
> Unsubscribe: 
> http://mail.python.org/mailman/options/python-dev/guido%40python.org
>


-- 
--Guido van Rossum (home page: http://www.python.org/~guido/)
___
Python-Dev mailing list
Python-Dev@python.org
http://mail.python.org/mailman/listinfo/python-dev
Unsubscribe: 
http://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com

Re: [Python-Dev] x86_64 Interix - Advise needed on size of long

[Jerker Bäck]

Hello Martin,

> > You mean I should list all of these in "Modules/Setup"?
> Exactly so. They are already listed - just uncomment them all
> (with proper command line flags and libraries where necessary).
OK, I will try to get it compiled and tested.

Meanwhile, you asked so:
> I still don't understand. Are you *certain* that these are LP64
> libraries? Can you kindly refer to some official document that says
> Interix uses LP64 on AMD64?
MS is surprisingly very quiet of the POSIX subsystem and the Interix BSD
implementation, so it's hard to find any official info on the net. But here
is one the developers:

In Interix SDK releasenotes.htm (SDK download):
"64-bit compilation supports the LP64 data model."
Interix general:



To find details on how it all really works, one will have to look in the
headers and try different features oneself. (Which actually is pretty fun
because it's really fast and usually works well). The SDK comes with support
for x86, x86_64 (EM64T or AMD64) and IA64.

> And if so, how did Microsoft manage to build them, if their compiler
> does not support LP64? (I see you kind of answer that below - although
> I'm unsure what "translate all longs to long long means - you mean
> literal text replacement?)
Sure, cc precompiles the source file to a temporary file, flip it, runs a
conversion tool - "l2ll" => all  are converted to  and
finally compiles the converted file. The compile is done via a call from
POSIX to the Windows subsystem and the compiler found in the POSIX path
environment. To understand the details one has to know that the POSIX
environment runs directly on top of the NT kernel and know nothing of
Windows, Windows paths etc. This is kind of a compile on the fly. The
libraries are also of two kinds: 
1 The core POSIX libraries - part of the OS, uses DDK tools
2 Interix SDK - BSD libc and utils, uses cc and Interix gcc (x86 only)

The DDK tools is turned to LP64 support via special defines in the headers.
But here is some unclear issues with functions directly exported from the OS
native LLP64 libraries (ntdll.dll) - don't know how this is solved.
 
Somewhere here lies the reason why cc is hard to use with Visual Studio and
why the long type is such a nuisance.

I also tried the Intel x64 PE compiler (for better C99 support), but it
produces applications which relies on Windows API functions (e.g.
VirtualAlloc, LoadLibrary) and thus cannot be used in POSIX.
 
Cheers,
Erik

___
Python-Dev mailing list
Python-Dev@python.org
http://mail.python.org/mailman/listinfo/python-dev
Unsubscribe: 
http://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com

Re: [Python-Dev] cStringIO.StringIO() buffer behavior

[Martin v. Löwis]

> See bugs #1548891 and #1730114.
> 
> In the former, it was reported that cStringIO works differently from StringIO
> when handling unicode strings; it used GetReadBuffer which returned the raw
> internal UCS-2 or UCS-4 encoded string.
> 
> I changed it to use GetCharBuffer, which converts to a string using the
> default encoding first. This fix was also in 2.5.1.
> 
> The latter bug now complains that this excludes things like array.array()s
> from being used as an argument to cStringIO.StringIO(), which worked before
> with GetReadBuffer.
> 
> What's the preferred solution here?

I think the 2.5.0 behavior to accept array.array should be restored (and
a test case be added). What to do about Unicode strings, I don't know.
I agree with Guido that they are officially not supported in cStringIO,
so it would be best to reject them. OTOH, since 2.5.1 already supports
them, another choice would be continue supporting them, in the same way
as they are supported in 2.5.1. Either solution would special-case
Unicode strings.

Regards,
Martin

___
Python-Dev mailing list
Python-Dev@python.org
http://mail.python.org/mailman/listinfo/python-dev
Unsubscribe: 
http://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com

Re: [Python-Dev] cStringIO.StringIO() buffer behavior

[Alexandre Vassalotti]

On 8/5/07, Georg Brandl <[EMAIL PROTECTED]> wrote:
> See bugs #1548891 and #1730114.
>
> In the former, it was reported that cStringIO works differently from StringIO
> when handling unicode strings; it used GetReadBuffer which returned the raw
> internal UCS-2 or UCS-4 encoded string.
>
> I changed it to use GetCharBuffer, which converts to a string using the
> default encoding first. This fix was also in 2.5.1.
>
> The latter bug now complains that this excludes things like array.array()s
> from being used as an argument to cStringIO.StringIO(), which worked before
> with GetReadBuffer.
>
> What's the preferred solution here?
>

The best thing would be add a special case for ascii-only unicode
objects, and keep the old behavior. However, I believe this will be
ugly, especially in O_write. So, it would perhaps be better to simply
stop supporting unicode objects.

-- Alexandre
___
Python-Dev mailing list
Python-Dev@python.org
http://mail.python.org/mailman/listinfo/python-dev
Unsubscribe: 
http://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com