[issue17237] m68k aligns on 16bit boundaries.

Change by Serhiy Storchaka : -- pull_requests: -7408 ___ Python tracker ___ ___ Python-bugs-list mailing list Unsubscribe: https:/

[issue17237] m68k aligns on 16bit boundaries.

Change by INADA Naoki : -- pull_requests: +7408 ___ Python tracker ___ ___ Python-bugs-list mailing list Unsubscribe: https://mail.

[issue17237] m68k aligns on 16bit boundaries.

Antoine Pitrou added the comment: > Oh well, then it’ll just have to wait. Do you have a preferred > place where I can submit the test results, as it’s getting > very off-topic here? Well, if everything works fine, you don't have to submit them! If you get test failures, you can open issues for

[issue17237] m68k aligns on 16bit boundaries.

mirabilos added the comment: Antoine Pitrou dixit: >(note, on 2.7, it's "python -m test.regrtest") That indeed does things. So I had mistaken them for the same problem. >Ah, that's because the system Python install doesn't include the test >suite. Perhaps you have to install an additional pack

[issue17237] m68k aligns on 16bit boundaries.

Antoine Pitrou added the comment: > >"python -m test" (with any options you might like), but we don't > > No, I tried that (as it was the only thing I could find on the > ’net as well) on an i386 system and only get: Ah, that's because the system Python install doesn't include the test suite. P

[issue17237] m68k aligns on 16bit boundaries.

mirabilos added the comment: Antoine Pitrou dixit: >"python -m test" (with any options you might like), but we don't No, I tried that (as it was the only thing I could find on the ’net as well) on an i386 system and only get: tglase@tglase:~ $ python2.7 -m test /usr/bin/python2.7: No module na

[issue17237] m68k aligns on 16bit boundaries.

Antoine Pitrou added the comment: > Now, for “finishing up” this… to follow up on Stefan’s comment… is > there any way I can run the testsuite from an installed Python (from > the Debian packages)? "python -m test" (with any options you might like), but we don't guarantee that all tests pass on

[issue17237] m68k aligns on 16bit boundaries.

mirabilos added the comment: Thanks Antoine! Now, for “finishing up” this… to follow up on Stefan’s comment… is there any way I can run the testsuite from an installed Python (from the Debian packages)? (I build the packages with disabled testsuite, to get the rest of the system running again

[issue17237] m68k aligns on 16bit boundaries.

Antoine Pitrou added the comment: Ok, I hope I got the fix right :) Thanks mirabilos for the comment suggestion, I used a modified version. -- resolution: -> fixed stage: needs patch -> committed/rejected status: open -> closed ___ Python tracker <

[issue17237] m68k aligns on 16bit boundaries.

Roundup Robot added the comment: New changeset 0f8022ac88ad by Antoine Pitrou in branch '3.3': Issue #17237: Fix crash in the ASCII decoder on m68k. http://hg.python.org/cpython/rev/0f8022ac88ad New changeset 201ae2d02328 by Antoine Pitrou in branch 'default': Issue #17237: Fix crash in the ASCI

[issue17237] m68k aligns on 16bit boundaries.

Serhiy Storchaka added the comment: Well, then already too much bikeshedding for such simple fix. Antoine, do you want to commit a fix? -- ___ Python tracker ___ ___

[issue17237] m68k aligns on 16bit boundaries.

mirabilos added the comment: Right, keeping it simple helps in preventing accidents, and the code block looks full of magic enough as-is. Maybe add a comment block that says: /* * m68k is a bit different from most architectures in that objects * do not use "natural alignment" - for example,

[issue17237] m68k aligns on 16bit boundaries.

Antoine Pitrou added the comment: > PyASCIIObject is allocated on heap and should have a maximal alignment > enough for every type. If sizeof(PyASCIIObject) % SIZEOF_LONG == 0 > then dest is at least long-aligned. Currently sizeof(PyASCIIObject) is > 22 on m68k and the optimization is switched of

[issue17237] m68k aligns on 16bit boundaries.

Serhiy Storchaka added the comment: PyASCIIObject is allocated on heap and should have a maximal alignment enough for every type. If sizeof(PyASCIIObject) % SIZEOF_LONG == 0 then dest is at least long-aligned. Currently sizeof(PyASCIIObject) is 22 on m68k and the optimization is switched off a

[issue17237] m68k aligns on 16bit boundaries.

mirabilos added the comment: + dest is always aligned on common platforms + (if sizeof(PyASCIIObject) is divisible by SIZEOF_LONG). Actually, that’s the part that is not

[issue17237] m68k aligns on 16bit boundaries.

mirabilos added the comment: I’m currently thinking this patch. (Will need another day or so for compiles to finish, though.) -- Added file: http://bugs.python.org/file30210/python3.3_3.3.1-1+m68k.1.debdiff ___ Python tracker

[issue17237] m68k aligns on 16bit boundaries.

mirabilos added the comment: Currently 22; it will increase to 24 if a few more bits are added. That’s why I said it’s “fragile” magic. -- ___ Python tracker ___ ___

[issue17237] m68k aligns on 16bit boundaries.

Serhiy Storchaka added the comment: What is sizeof(PyASCIIObject) on m68k? -- ___ Python tracker ___ ___ Python-bugs-list mailing list

[issue17237] m68k aligns on 16bit boundaries.

Antoine Pitrou added the comment: > I think we can be bold and put #if !defined(__m68k__) and #endif > around the entire block and, should there ever be another architecture > with similar issues, whitelist them there. Agreed. -- ___ Python tracker

[issue17237] m68k aligns on 16bit boundaries.

mirabilos added the comment: file30203/ascii_decode_nonaligned.patch is potentially a nop (the struct being a multiple of, in the m68k case 4, bytes is not an indicator of whether to skip it). I think we can be bold and put #if !defined(__m68k__) and #endif around the entire block and, should

[issue17237] m68k aligns on 16bit boundaries.

Serhiy Storchaka added the comment: Well, it will be safer to skip this block on such platforms. -- keywords: +patch Added file: http://bugs.python.org/file30203/ascii_decode_nonaligned.patch ___ Python tracker ___

[issue17237] m68k aligns on 16bit boundaries.

mirabilos added the comment: Antoine: precisely. Serhiy: sure. The times are now in msec per loop. I did three subsequent runs, so the second and third tuple are cache-warm. Without assert: (89,88), (87,87), (89,86) Without block : (79,78), (78,78), (79,78) And in a second run: Without assert

[issue17237] m68k aligns on 16bit boundaries.

Serhiy Storchaka added the comment: Could you please repeat the tests with a larger data (1MB or more)? -- ___ Python tracker ___ ___

[issue17237] m68k aligns on 16bit boundaries.

Antoine Pitrou added the comment: > Which supports your suspicion that the optimised case may not be needed > at all. So we can just skip the assert when __m68k__ is defined? -- ___ Python tracker ___

[issue17237] m68k aligns on 16bit boundaries.

mirabilos added the comment: Hi again, sorry for being a bit late in following up to this. Here are the results for the two runs you requested; these were done on the same machine, intermixed (the first one cache-clean, the second and third run subsequently (so that the third run is cache-fil

[issue17237] m68k aligns on 16bit boundaries.

Serhiy Storchaka added the comment: > Erm, still, which one do I build? Not 3.2 because it obviously > works, at least as packaged in Debian. Any >= 3.3.0. -- ___ Python tracker ___

[issue17237] m68k aligns on 16bit boundaries.

mirabilos added the comment: Serhiy Storchaka dixit: >> Which tree should I build? A release (if so, which)? Or >> some CVS branch? > >It doesn't matter. Erm, still, which one do I build? Not 3.2 because it obviously works, at least as packaged in Debian. bye, //mirabilos -- Yay for having to

[issue17237] m68k aligns on 16bit boundaries.

Serhiy Storchaka added the comment: And of course run the tests on non-debug builds. -- ___ Python tracker ___ ___ Python-bugs-list ma

[issue17237] m68k aligns on 16bit boundaries.

Serhiy Storchaka added the comment: > Which tree should I build? A release (if so, which)? Or > some CVS branch? It doesn't matter. > Do note we clock at roughly 1000 pystones for the fastest > machines… yes 1000 not 1. It doesn't matter. Only relative values have a meaning. What is faster

[issue17237] m68k aligns on 16bit boundaries.

mirabilos added the comment: Serhiy Storchaka dixit: >mirabilos, if you are motivated enough, do the following. Compile two >Python executables - one with deleted assert, and second with deleted >a block between "#if SIZEOF_LONG <= SIZEOF_VOID_P" and "#endif". Run >following microbenchmarks for

[issue17237] m68k aligns on 16bit boundaries.

Serhiy Storchaka added the comment: mirabilos, if you are motivated enough, do the following. Compile two Python executables - one with deleted assert, and second with deleted a block between "#if SIZEOF_LONG <= SIZEOF_VOID_P" and "#endif". Run following microbenchmarks for both executables:

[issue17237] m68k aligns on 16bit boundaries.

mirabilos added the comment: @pitrou: As for performance, 2-byte and 4-byte are the same on m68k, given that they usually have RAM which doesn’t benefit from that kind of alignment, and systems that are structured accordingly. The “best” cpp define I don’t know, but my system defines __m68k__

[issue17237] m68k aligns on 16bit boundaries.

Stefan Krah added the comment: mirabilos wrote: > Please dont break what works. We have almost complete (about three quarters > of roughly 10'000 source packages) Debian unstable working on m68k, with > several versions of Python in use. Thanks! Are you saying that the complete test suite works

[issue17237] m68k aligns on 16bit boundaries.

Antoine Pitrou added the comment: > We had a similar issue in the Linux kernel, where it used the lower > two bits of an address for flags (urgh…) which could only be solved by > using GCC’s __attribute__((__aligned__(4))) on the quantities in > question, but that may or may not be the required c

[issue17237] m68k aligns on 16bit boundaries.

mirabilos added the comment: @skrah: “I don't think we should support this platform officially.” Please don’t break what works. We have almost complete (about three quarters of roughly 10'000 source packages) Debian unstable working on m68k, with several versions of Python in use. Thanks! @pi

[issue17237] m68k aligns on 16bit boundaries.

Serhiy Storchaka added the comment: Perhaps we should disable not only assert, but all optimized code inside #if SIZEOF_LONG <= SIZEOF_VOID_P / #endif block. Benchmarks needed to measure either unaligned writing speedup or slowdown decoding. -- ___

[issue17237] m68k aligns on 16bit boundaries.

Antoine Pitrou added the comment: Ok, so we could simply disable the assert on m68k, if you can confirm it works. Do you want to provide a patch? I don't know what the preprocessor conditional should look like. -- components: +Interpreter Core -Build stage: -> needs patch type: -> cr

[issue17237] m68k aligns on 16bit boundaries.

Alan Hourihane added the comment: Oh, and as for pointer alignment, I probably just wasn't clear in the initial report. But we agree on m68k C ABI. -- ___ Python tracker ___ ___

[issue17237] m68k aligns on 16bit boundaries.

Alan Hourihane added the comment: It's GCC 4.6.3. GCC has the -malign-int but mentions this isn't in best interest of m68k ABI compatibility if it's used. -- components: +Build -Unicode type: crash -> versions: -Python 3.4 ___ Python tracker

[issue17237] m68k aligns on 16bit boundaries.

Antoine Pitrou added the comment: > It must be about pointer alignment, because that's the whole point of > the ASSERT. Indeed, it's about pointer alignment, but it's not about the CPU. It's about the compiler (or the platform's C ABI). Apparently the compiler doesn't align struct fields to nat

[issue17237] m68k aligns on 16bit boundaries.

Changes by Serhiy Storchaka : -- components: +Unicode -Build nosy: +ezio.melotti versions: +Python 3.4 ___ Python tracker ___ ___ Pyth

[issue17237] m68k aligns on 16bit boundaries.

Serhiy Storchaka added the comment: What will happened if you just remove this line? -- nosy: +serhiy.storchaka type: -> crash ___ Python tracker ___ ___

[issue17237] m68k aligns on 16bit boundaries.

Alan Hourihane added the comment: It must be about pointer alignment, because that's the whole point of the ASSERT. As for the backtrace, the gdb support on the platform isn't great yet, but here it is Breakpoint 1, ascii_decode (start=0x30c5b04 "__len__", end=0x30c5b0b "", dest=0x1a

[issue17237] m68k aligns on 16bit boundaries.

Antoine Pitrou added the comment: Ok, could you post the whole C stack trace? (using e.g. gdb) By the way, this is not about the alignment of m68k architectures: x86 can align on a byte boundary and doesn't trigger the assert. -- ___ Python tracker

[issue17237] m68k aligns on 16bit boundaries.

Alan Hourihane added the comment: As mention in the versions. It is Python 3.3.0. -- ___ Python tracker ___ ___ Python-bugs-list maili

[issue17237] m68k aligns on 16bit boundaries.

Antoine Pitrou added the comment: Which branch or version of Python is that? -- nosy: +pitrou ___ Python tracker ___ ___ Python-bugs-l

[issue17237] m68k aligns on 16bit boundaries.

Jesús Cea Avión added the comment: Freescale (ex-Motorola) ColdFire architecture is alive and doing well. And I know people still running Motorola 68040 Apple machines. The problem is not a m68k emulator, but to build all the needed environment on it: OS, compilers, network, etc. If the origi

[issue17237] m68k aligns on 16bit boundaries.

Ramchandra Apte added the comment: Wikipedia says "derivative processors are still widely used in embedded applications." in m68k article. -- nosy: +Ramchandra Apte ___ Python tracker _

[issue17237] m68k aligns on 16bit boundaries.

Stefan Krah added the comment: TBH, I don't think we should support this platform officially. Is that processor still in use (e.g. in embedded systems)? -- nosy: +skrah ___ Python tracker __

[issue17237] m68k aligns on 16bit boundaries.

Alan Hourihane added the comment: I'm willing to help fix, but there are m68k emulators out there which I guess suffice for a test box. -- ___ Python tracker ___ ___

[issue17237] m68k aligns on 16bit boundaries.

Christian Heimes added the comment: We don't have a m86k test box and I don't think we support this platform either. -- nosy: +christian.heimes, trent ___ Python tracker ___

[issue17237] m68k aligns on 16bit boundaries.

New submission from Alan Hourihane: On m68k this assert triggers in Python/unicodeobject.c:4658 because m68k architectures can align on 16bit boundaries. assert(!((size_t) dest & LONG_PTR_MASK)); I'm not sure of the wider implications in Python as to how to rectify. -- components: Bui