[issue11277] test_zlib.test_big_buffer crashes under BSD (Mac OS X and FreeBSD)

Steffen Daode Nurpmeso added the comment: On Tue, Apr 19, 2011 at 10:34:11AM +, STINNER Victor wrote: > 11277.3.diff: this patch looks correct Unbelievable - you really fought yourself through this immense bunch of code in such a short time! :) -- title: Crash with mmap and sparse

[issue11277] test_zlib.test_big_buffer crashes under BSD (Mac OS X and FreeBSD)

STINNER Victor added the comment: 11277.3.diff: this patch looks correct (I'm unable to test it), but can you add a sentence in mmap doc to explain that mmap.mmap() does flush the file on Mac OS X and VMS? -- ___ Python tracker

[issue11277] test_zlib.test_big_buffer crashes under BSD (Mac OS X and FreeBSD)

Steffen Daode Nurpmeso added the comment: (Dropped the tests, too.) -- ___ Python tracker ___ ___ Python-bugs-list mailing list Unsub

[issue11277] test_zlib.test_big_buffer crashes under BSD (Mac OS X and FreeBSD)

Changes by Steffen Daode Nurpmeso : Removed file: http://bugs.python.org/file21684/11277.mmap-2.py ___ Python tracker ___ ___ Python-bugs-list

[issue11277] test_zlib.test_big_buffer crashes under BSD (Mac OS X and FreeBSD)

Changes by Steffen Daode Nurpmeso : Removed file: http://bugs.python.org/file21683/11277.mmap-2.c ___ Python tracker ___ ___ Python-bugs-list

[issue11277] test_zlib.test_big_buffer crashes under BSD (Mac OS X and FreeBSD)

Changes by Steffen Daode Nurpmeso : Removed file: http://bugs.python.org/file21676/11277.mmap-1.c ___ Python tracker ___ ___ Python-bugs-list

[issue11277] test_zlib.test_big_buffer crashes under BSD (Mac OS X and FreeBSD)

Changes by Steffen Daode Nurpmeso : Removed file: http://bugs.python.org/file21672/11277.mmap.c ___ Python tracker ___ ___ Python-bugs-list ma

[issue11277] test_zlib.test_big_buffer crashes under BSD (Mac OS X and FreeBSD)

Steffen Daode Nurpmeso added the comment: (The working patch is http://bugs.python.org/file21715/11277.3.diff.) -- ___ Python tracker ___ ___

[issue11277] test_zlib.test_big_buffer crashes under BSD (Mac OS X and FreeBSD)

Changes by Steffen Daode Nurpmeso : Removed file: http://bugs.python.org/file21675/11277.2.diff ___ Python tracker ___ ___ Python-bugs-list ma

[issue11277] test_zlib.test_big_buffer crashes under BSD (Mac OS X and FreeBSD)

Changes by Steffen Daode Nurpmeso : Removed file: http://bugs.python.org/file21671/11277.1.diff ___ Python tracker ___ ___ Python-bugs-list ma

[issue11277] test_zlib.test_big_buffer crashes under BSD (Mac OS X and FreeBSD)

Changes by Steffen Daode Nurpmeso : Removed file: http://bugs.python.org/file20861/doc_lib_mmap.patch ___ Python tracker ___ ___ Python-bugs-l

[issue11277] test_zlib.test_big_buffer crashes under BSD (Mac OS X and FreeBSD)

STINNER Victor added the comment: @sdaoden: This issue has a lot of patches, can you remove old patches? -- ___ Python tracker ___ __

[issue11277] test_zlib.test_big_buffer crashes under BSD (Mac OS X and FreeBSD)

Steffen Daode Nurpmeso added the comment: Took some time, but here is a patch that makes mmap(2) work on Mac OS X. This also applies to #11779. Background: on OS X, fsync(2) seems to behave as fdatasync(2). To give people the possibility to do some kind of fync(2) nonetheless, a new fcntl(2) ha

[issue11277] test_zlib.test_big_buffer crashes under BSD (Mac OS X and FreeBSD)

Steffen Daode Nurpmeso added the comment: On Sat, Apr 16, 2011 at 02:55:29PM +, Nadeem Vawda wrote: > I don't think it is necessary to further investigate the behaviour of > Snow Leopard's mmap() - we know that it's broken, and we have a fix. > > At the moment, we need someone to actually w

[issue11277] test_zlib.test_big_buffer crashes under BSD (Mac OS X and FreeBSD)

Steffen Daode Nurpmeso added the comment: Yet another bug of Mac OS X: it sometimes creates messed up sparse regions: 14:00 ~/tmp/test $ ~/src/cpython/python.exe test_mmap.py .. 14:01 ~/tmp/test $ zsum32 py-mmap-testfile Adler-32 CRC-32 <78ebae7a> -- py-mmap-testfile 14:03 ~/tmp/test $ ./tes

[issue11277] test_zlib.test_big_buffer crashes under BSD (Mac OS X and FreeBSD)

Nadeem Vawda added the comment: > So i finally did some tests using Nadeem's code snippet > from msg133677. The largest top(1) i ever got was >30477 python3 2.7 00:09.77 101877912M+ 240K > but the system is unusable then. The code I posted was only intended to

[issue11277] test_zlib.test_big_buffer crashes under BSD (Mac OS X and FreeBSD)

Steffen Daode Nurpmeso added the comment: My last idea for today was to split the writes. This also works for the C version, but it does not for test_zlib.py. I attach the updated files. And for completeness: Adler-32 <7a54018b> CRC-32 <7f1be672> -- @test_13713_tmp Adler-32 <7a54018b>

[issue11277] test_zlib.test_big_buffer crashes under BSD (Mac OS X and FreeBSD)

Changes by Steffen Daode Nurpmeso : Removed file: http://bugs.python.org/file20838/issue11277.2.patch ___ Python tracker ___ ___ Python-bugs-l

[issue11277] test_zlib.test_big_buffer crashes under BSD (Mac OS X and FreeBSD)

Steffen Daode Nurpmeso added the comment: I was able to spend more time on that this afternoon. 'Got an unkillable diff(1) along the way which required me to force a cold reboot. Well. I attach a C version (11277.mmap.c) which i've used for testing. The file 11277.zsum32.c is a quick-and-dirty

[issue11277] test_zlib.test_big_buffer crashes under BSD (Mac OS X and FreeBSD)

Steffen Daode Nurpmeso added the comment: And about mmap(2): import os,sys,time,mmap MP0 = ((2**30)-1) MP1 = ((2**31)-1) MP2 = ((2**32)-1) MPV = (2**20) * 100 SIZES = (MP0-MPV, MP0, MP0+MPV, MP1-MPV, MP1, MP1+MPV, MP2-MPV, MP2, MP2+MPV) FILE = 'test.dat' print('Start:', tim

[issue11277] test_zlib.test_big_buffer crashes under BSD (Mac OS X and FreeBSD)

Steffen Daode Nurpmeso added the comment: Pfff. Now i really spent some time on Mac OS X memory management. I did it for Wanda. :) (First: i don't know why you want to drop that nice mmap(2), it's wonderful to test harddisk performance! And if you want to support Apple, then you have to spend

[issue11277] test_zlib.test_big_buffer crashes under BSD (Mac OS X and FreeBSD)

Nadeem Vawda added the comment: > I think there's basically noone and nothing (even among the buildbots) > that runs bigmem tests on a regular basis, so I'd much rather keep the > mmap() solution, even if that means it must be skipped on OS X. Fair enough. (As an aside, if it is preferable to

[issue11277] test_zlib.test_big_buffer crashes under BSD (Mac OS X and FreeBSD)

Antoine Pitrou added the comment: > Another solution would be to rewrite the test to not use mmap() at all: > > @precisionbigmemtest(size=_4G + 4, memuse=1) > def test_big_buffer(self, size): > if size < _4G + 4: > self.skipTest("not enough free memory, need at least

[issue11277] test_zlib.test_big_buffer crashes under BSD (Mac OS X and FreeBSD)

Stefan Krah added the comment: Just to give another data point: A couple of days ago I reduced the memory on the AMD64 FreeBSD bot to (375MB RAM, 2GB swap) and the zlib tests still pass. -- ___ Python tracker

[issue11277] test_zlib.test_big_buffer crashes under BSD (Mac OS X and FreeBSD)

Nadeem Vawda added the comment: > By the way, at this point I think we could simply skip the test on BSDs > and OS X. The tested functionality is cross-platform, so testing under > a limited set of systems should be ok. Another solution would be to rewrite the test to not use mmap() at all:

[issue11277] test_zlib.test_big_buffer crashes under BSD (Mac OS X and FreeBSD)

Changes by Nadeem Vawda : -- nosy: +nadeem.vawda ___ Python tracker ___ ___ Python-bugs-list mailing list Unsubscribe: http://mail.py

[issue11277] test_zlib.test_big_buffer crashes under BSD (Mac OS X and FreeBSD)

Steffen Daode Nurpmeso added the comment: I can't confirm this for my MacBook: 20:39 ~ $ time python3 -E -Wd -m test -r -w -uall test_zlib Using random seed 1960084 [1/1] test_zlib 1 test OK. [91618 refs] real4m1.051s user0m15.031s sys 0m26.908s ... 20:40 ~ $ ll tmp/test_python_

[issue11277] test_zlib.test_big_buffer crashes under BSD (Mac OS X and FreeBSD)

Changes by STINNER Victor : -- title: test_zlib crashes under Snow Leopard buildbot -> test_zlib.test_big_buffer crashes under BSD (Mac OS X and FreeBSD) ___ Python tracker ___