Stefan Krah added the comment:
I agree that this isn't a blocker due to the limited use of
memoryview hashing. Tracking this in #15814.
--
status: open -> closed
___
Python tracker
<http://bugs.python.org
Stefan Krah added the comment:
Martin v. L??wis wrote:
> In general, since memoryview(obj)==obj, it would be necessary that
> hash(memoryview(obj))==hash(obj). However, since memoryview cannot
> know what hashing algorithm obj uses, it cannot compute the hash
> value with the sa
Stefan Krah added the comment:
And since memory_richcompare() and any potentially compatible hash function
are quite tricky to implement by now, perhaps the generally useful parts
could be exposed as PyBuffer_RichCompareBool() and PyBuffer_Hash
Stefan Krah added the comment:
Martin v. L??wis wrote:
> > hash(x) == hash(x.tobytes())
> In the light of this requirement, it's even more difficult to ask
> people that they change their hashing, since some exporters may already
> comply with that original reques
Changes by Stefan Krah :
--
nosy: +scoder
___
Python tracker
<http://bugs.python.org/issue15814>
___
___
Python-bugs-list mailing list
Unsubscribe:
New submission from Stefan Krah:
People are using PyMemoryView_FromBuffer() to create and return
permanent memoryviews from buffers that are allocated on the stack.
See:
http://stackoverflow.com/questions/8123121/how-to-get-back-a-valid-object-from-python-in-c-while-this-object-has-been-con
Changes by Stefan Krah :
--
resolution: -> fixed
status: open -> closed
___
Python tracker
<http://bugs.python.org/issue15724>
___
___
Python-bugs-list
Stefan Krah added the comment:
Here's a doc patch restricting the hash to formats 'B', 'b' and 'c'.
I think non-contiguous views are fine: both __eq__() and tobytes()
handle these, so the equality-hash invariant is preserved.
--
___
Changes by Stefan Krah :
--
keywords: +patch
Added file: http://bugs.python.org/file27058/issue15814-doc.diff
___
Python tracker
<http://bugs.python.org/issue15
Stefan Krah added the comment:
This may be a bigger problem (grep for image_surface_get_data):
http://lists.cairographics.org/archives/cairo/2011-December/022563.html
The previous semantics for PyMemoryView_FromBuffer(view) were:
1) If non-NULL, steal the view.obj reference with automatic
Stefan Krah added the comment:
PyMemoryViewObject already is a PyVarObject with its own shape, strides
and suboffsets. It is the PyManagedBuffer object that directly communicates
with exporters.
The decision to store *exactly* the buffer that is obtained from the
exporter was made in #10181
Stefan Krah added the comment:
On second thought, it's impossible to crash a memoryview generated
by PyMemoryView_FromBuffer(info) even when info->shape etc. point
to stack addresses.
The reason is this: All new memoryviews are created through the mbuf_add*
API. In the first
Changes by Stefan Krah :
--
priority: high -> normal
___
Python tracker
<http://bugs.python.org/issue15821>
___
___
Python-bugs-list mailing list
Unsubscri
Stefan Behnel added the comment:
To add on Dag's comments, this essentially means that any caching of the hash
value is dangerous, unless it can be assured that the underlying buffer
definitely has not changed in the meantime. There is no way for users to
explicitly tell a memoryvi
New submission from Stefan Behnel:
Formatting support for "lld"/"llu" was added (I think) in issue 7228, but the
definition of PY_FORMAT_SIZE_T wasn't adapted. We are getting test output
failures in Cython on Win64 when formatting error messages using the "Id&qu
Stefan Behnel added the comment:
I just saw that we were misusing PY_FORMAT_SIZE_T according to the
documentation (reading that sometimes helps...). It is not supposed to be used
with the Python formatting functions, only with low-level functions.
Closing, sorry for the noise
Stefan Krah added the comment:
Dag Sverre Seljebotn wrote:
> It is perfectly possible for an object to export memory in a read-only
> way that may still change. Another object may have a writeable view:
>
> x = obj.readonly_view
> y = obj.writable_view
> obj.move_to_next
Stefan Krah added the comment:
Dag Sverre Seljebotn wrote:
> OK, I can understand the desire to make memoryviews be bytes-like objects
> (though to my mind, bytes is "frozen" in a very different way...)
We have two desires that sometimes conflict: People who want to use
memoryv
Stefan Krah added the comment:
Alexander, your test case is brilliant and could eventually go into
_testbuffer.c, but I'd prefer a simpler test case for now. :)
Here's a patch that restores the info.obj cleanup facility.
I intentionally did not add copying the format in order to kee
Stefan Krah added the comment:
Alexander Belopolsky wrote:
> I am still getting up to speed with all the changes that went in since
> 3.2. I'll review your patch over the weekend. Meanwhile, I think the
> goal should be that after PyMemoryview_FromBuffer(info) is called, it
>
Stefan Krah added the comment:
In general, the number of calls to PyObject_GetBuffer() must be equal to
the number of calls to PyBuffer_Release(), otherwise bad things will happen
in the same way as with malloc()/free().
Now, in my test case I omitted the call to PyObject_GetBuffer() *with
Stefan Krah added the comment:
Here's a patch that enforces byte formats.
Does everyone agree on (or tolerate at least) allowing 'B', 'b' and 'c'?
I think we should at least commit the doc patch for 3.3.0. Otherwise
implementors of exporting objects m
Stefan Krah added the comment:
The FreeBSD machine decided to hang for 1h:
http://buildbot.python.org/all/builders/AMD64%20FreeBSD%209.0%203.x/builds/3340/steps/test/logs/stdio
If I'm running the tests manually, they take over half an hour and
Stefan Krah added the comment:
Martin v. L??wis wrote:
> Why be more permissive than necessary? -0 on the committed version;
> it should IMO further restrict it to 1D contiguous byte arrays.
Does "byte arrays" include 'b' and 'c' or just 'B'?
Stefan Krah added the comment:
I see that in the following build test_threading also fails, so I
wouldn't spend too much time on debugging test_threaded_import:
==
FAIL: test_waitfor_timeout (test.test_threading.Condition
Stefan Krah added the comment:
Nothing changed on the buildbot. -- It appears that FreeBSD is unable to
handle the low switchinterval. With the patch test_threaded_import runs in
10s.
Since Linux has no problems whatsoever, I'm inclined to think that's
a FreeBSD issue.
I've
Stefan Krah added the comment:
tobytes() is the same as the flattened multi-dimensional list representation
with all elements converted to bytes. If I'm not mistaken, that's how NumPy's
tostring() behaves.
--
___
Python
Stefan Krah added the comment:
> why is it desirable to have deliberate hash collisions between views with
> different shapes?
Since we're now restricting everything to bytes, the multi-dimensional case
is probably not useful at all. As I said above, I would leave it in because
Stefan Krah added the comment:
Disallowing non-contiguous arrays leads to very strange situations though.
I'm positive that there will be a bug report about this:
>>> x = memoryview(b'abc')[::-1]
>>> b = b'cba'
>>> d = {b'cba':
Stefan Krah added the comment:
> py> x = memoryview(array.array('B',b'cba'))
I find the array example is different. The user has to remember one thing:
memoryviews based on arrays don't hash.
For memoryviews based on bytes one would have to remember:
- &#
Stefan Krah added the comment:
The extreme slowness with a low switch interval is already present in
3430d7329a3b, so it's not related to the finer-grained import lock.
The other FreeBSD 9.0 bots don't seem to have that problem, so I'm not
sure any more if the patch is a good id
Stefan Krah added the comment:
My review is done. The Karatsuba function is basically a small stack machine
and very hard to prove formally as far as I can see. The algorithm is cited
in TAOCP and the subdivision is brute force tested for all combinations of
coefficient lengths of the two input
Changes by Stefan Krah :
--
status: open -> closed
___
Python tracker
<http://bugs.python.org/issue7652>
___
___
Python-bugs-list mailing list
Unsubscri
Stefan Krah added the comment:
The totals are +11.5 :) for hashing, +1 for allowing non-contiguous and
-2 for multi-dimensional. I'll update the docs soon.
--
___
Python tracker
<http://bugs.python.org/is
Stefan Krah added the comment:
One of the Windows bots has the same failure in test_parallel_meta_path:
http://buildbot.python.org/all/builders/x86%20Windows7%203.x/builds/5643/steps/test/logs/stdio
--
___
Python tracker
<http://bugs.python.
Stefan Krah added the comment:
I'm convinced now that some systems just can't handle a low switch
interval gracefully. Look at:
http://buildbot.python.org/all/builders/x86%20Windows7%203.x/builds/5642/steps/test/
Stefan Krah added the comment:
The Lion bot also hangs in test_threading:
http://buildbot.python.org/all/builders/AMD64%20Lion%203.x/builds/383/steps/test/logs/stdio
At least the test_threading failures seem to be recent.
The test_threaded_import *slowness* (40 min!) on FreeBSD/KVM with
Stefan Krah added the comment:
Small nitpick: multi-dimensional hashing wasn't really accidental,
it was perfectly aligned with the previous statically typed equality
definition.
When I suggested PyBuffer_Hash = hash(obj.tobytes()) on python-dev for
non-contiguous and multi-dimensional a
Changes by Stefan Behnel :
--
nosy: -scoder
___
Python tracker
<http://bugs.python.org/issue15814>
___
___
Python-bugs-list mailing list
Unsubscribe:
Stefan Krah added the comment:
It's deliberately not implemented (and documented): Since memoryview is a
complete rewrite, I tried go easy on new features in order to facilitate
review.
My plan was to implement struct packing/unpacking in 3.3.1 in the same
manner as in the new equ
Stefan Krah added the comment:
I've left a couple of comments. -- Personally I'd also prefer if all
docstrings go into a separate section. I always perceive docstrings
as noise that takes up precious vertical space, so for _decimal I even
banned them into docstrings.h.
I don't
Stefan Krah added the comment:
'>' in struct syntax implies "standard size" for 'l', which is 4 bytes.
ctypes uses machine size for c_long, so on an LP64 machine you can unpack
the data as:
>>> struct.unpack_from('>qq', a)
(100, 200)
---
Stefan Krah added the comment:
> On the other hand, this is not that important and consolidating the changes
> in one section will make 3.2 to 3.3 merge easier. I'll consolidate and wait
> for someone else to complain. :-)
Thanks! I'm certain someone will complain, pro
Stefan Krah added the comment:
> 1) shape being None for 0-d views in 3.2 is probably a bug.
Probably. I don't know whether it's worth fixing. Several test cases in ctypes
as well as in NumPy rely on it. So I guess we should just leave it for 3.2.
> 2) "a N-dimensional a
New submission from Stefan Krah:
Coverity found a leak in bytesio.c. Patch attached.
--
components: Extension Modules
files: bytesio_leak.diff
keywords: patch
messages: 169907
nosy: pitrou, skrah
priority: normal
severity: normal
status: open
title: leak in bytesio.c
type: resource
Stefan Krah added the comment:
I like the quick search: It works very well for function names.
For the use cases mentioned here I use Google.
I never use the index, so for me personally the positioning of
the quick search box is perfect.
--
nosy: +skrah
Stefan Krah added the comment:
Infinities do not have payloads. It's a bug in decimal.py that it
accepts non-empty coefficients when constructing infinities.
Since there was a corresponding unit test for as_tuple(), I've
kept the wrong representation for _decimal:
# XXX non
Stefan Krah added the comment:
Mark Dickinson wrote:
> On what grounds is it a bug? I might call it a poor design decision,
> but it's clearly intentional. There's even a unit test for it.
Maybe "bug" is not the right word. I'd call it a deviation from the
speci
Stefan Krah added the comment:
Thanks for taking a look!
--
resolution: -> fixed
stage: -> committed/rejected
status: open -> closed
versions: +Python 3.2
___
Python tracker
<http://bugs.python.or
Stefan Krah added the comment:
It's an implementation detail of decimal.py to represent infinity with
a coefficient of '0' instead of the empty string:
>>> Decimal("inf")._int
'0'
So IMO the tuple representation exposes that implementation detail.
Stefan Krah added the comment:
Mark Dickinson wrote:
> > So, I'd really like to deprecate the whole interface in favor of either a
> > read-only interface:
>
> But I think that's a different issue.
Yep. I was using all this as a rationale that the current tuple i
Stefan Krah added the comment:
Aaron, did you encounter this problem in a real world application?
I'm asking because it would be interesting to know if people use this.
Google is pretty silent on "Decimal((0, (0,), 'F'))", but perhaps
that is not an ideal search term
Stefan Krah added the comment:
Georg, thanks for including all changes that I've asked for!
If you still have the patience for the constant stream of memoryview
doc changes, there are three new ones that might be worth including:
3b2597c1fe35
c9c9d890400c
ca81b9a
New submission from Stefan Krah:
All Windows bots have multiple failures in test_mmap:
==
ERROR: test_entire_file (test.test_mmap.MmapTests)
--
Traceback (most
Stefan Krah added the comment:
Duplicate.
--
resolution: -> duplicate
stage: needs patch -> committed/rejected
status: open -> closed
superseder: -> mmap: add empty file check prior to offset check
___
Python tracker
<http://
Stefan Krah added the comment:
For some reason all Windows buildbots are failing since f962ec8e47a1:
==
ERROR: test_entire_file (test.test_mmap.MmapTests
Stefan Krah added the comment:
This gets rid of the permission error:
diff -r f962ec8e47a1 Lib/test/test_mmap.py
--- a/Lib/test/test_mmap.py Mon Sep 10 01:23:05 2012 +0200
+++ b/Lib
Stefan Krah added the comment:
I think Py_DECREF(m_obj) is missing (at least in 3.3, I didn't
look at the other versions).
--
___
Python tracker
<http://bugs.python.org/is
Stefan Krah added the comment:
Thanks for the report and the patch. I used another approach that still
validates the digits in the coefficient tuple even if it is not used.
decimal.py allows any coefficient:
>>> Decimal((0, ('n', 'a', 'n'), &#x
Changes by Stefan Krah :
--
resolution: -> fixed
stage: -> committed/rejected
___
Python tracker
<http://bugs.python.org/issue15882>
___
___
Python-bugs-
Stefan Krah added the comment:
Even though it's documented, the function is probably a new feature
and should go into 3.4.
Meanwhile, you can call struct.calcsize(format). Any non-trivial
implementation of PyBuffer_SizeFromFormat() would likely do the same.
--
components: +Interp
Stefan Krah added the comment:
The segfault occurs in a path in import.c that has a comment "XXX this
this should really not happen.":
Program received signal SIGSEGV, Segmentation fault.
0x0047d733 in type_dealloc (type=0x8381e0) at Objects/typeobject.c
Stefan Krah added the comment:
Maybe related: If you increase the number of passes in Modules/_testembed.c,
pass 9 fails:
--- Pass 9 ---
_testembed: Objects/typeobject.c:2693: type_dealloc: Assertion `type->tp_flags
& (1L<<9)
Changes by Stefan Krah :
--
components: +Interpreter Core
stage: -> needs patch
type: -> crash
___
Python tracker
<http://bugs.python.org/issue15926>
___
__
Stefan Krah added the comment:
buf[1] contains NUL if SIZE_OF_WCHAR_T is 4.
The man page says:
size_t wcstombs(char *dest, const wchar_t *src, size_t n)
The conversion can stop for three reasons:
3. The wide-character string has been completely converted, including the
terminating L&#x
Stefan Krah added the comment:
I'm convinced that this is a false positive:
size_t wcstombs(char *dest, const wchar_t *src, size_t n);
We have:
1) buf[0] = *wstr and buf[1] = 0.
So:
2) wcstombs(NULL, buf, 0) <= 4.
Then the man page says:
"... the progr
Stefan Krah added the comment:
I was hesitating because the other FreeBSD bots don't have that problem.
The other option would be to kick out the FreeBSD/kvm bot in favor of
FreeBSD/Virtualbox.
http://buildbot.python.org/all/buildslaves/koobs-freebsd-clang
seems to be very stable. I could
Stefan Krah added the comment:
OK, I'll commit the patch soon.
--
___
Python tracker
<http://bugs.python.org/issue15599>
___
___
Python-bugs-list m
Stefan Krah added the comment:
0-dim memory is indexed by x[()]. The ctypes example has an additional
problem, because format=">> x = ndarray(3.14, shape=[], format='d', flags=ND_WRITABLE)
>>> x[()]
3.14
>>> tau = 6.28
>>> x[()] = tau
>>&
Stefan Krah added the comment:
BTW, if c_double means "native machine double", then ctypes should
fill in Py_buffer.format with "d" and not "
<http://bugs.python.org/issue15944>
___
__
Stefan Krah added the comment:
The decision was made in order to be able to cast back and forth between
known formats. Otherwise one would be able to cast from '>> import ctypes, struct
>>> d = ctypes.c_double()
>>> m = memoryview(d)
>>>
Stefan Krah added the comment:
So you want to be able to segfault the core interpreter using the
builtins?
--
___
Python tracker
<http://bugs.python.org/issue15
Stefan Krah added the comment:
Please read msg170482. It even contains a copy and paste example!
--
___
Python tracker
<http://bugs.python.org/issue15944>
___
___
Stefan Krah added the comment:
What is the expected outcome? memoryviews can't be resized, so
this scenario isn't possible:
>>> bytearray([1,2,3]) + b'123'
bytearray(b'\x01\x02\x03123')
--
nosy: +skrah
___
Stefan Krah added the comment:
So the problem is that readinto(view) might result in several references
to view? I don't think that can be solved on the memoryview side.
One could do:
view = PyMemoryView_FromObject(b);
// Lie about writability
((PyMemoryViewObject *
Stefan Krah added the comment:
Richard Oudkerk wrote:
> PyObject_GetBuffer(b, &buf, PyBUF_WRITABLE);
> view = PyMemoryView_FromBuffer(&buf);
> // readinto view
> PyBuffer_Release(&buf);
>
> Would attempts to access a "leaked" refer
Stefan Krah added the comment:
Richard Oudkerk wrote:
> The documentation is not very helpful. It just says that calls
> to PyObject_GetBuffer() must be matched with calls to PyBuffer_Release().
Yes, we need to sort that out, see
Stefan Krah added the comment:
Richard Oudkerk wrote:
> Should PyMemoryView_Release() release the _PyManagedBufferObject by doing
> mbuf_release(view->mbuf) even if view->mbuf->exports > 0?
No, I think it should really just be a wrapper:
diff --git a/Objects/memory
Stefan Krah added the comment:
Antoine Pitrou wrote:
> I don't think we want to expose a mutable bytes object to outside code,
> so IMO PyMemoryView_FromMemory() is preferrable.
I agree, but PyMemoryView_FromMemory(PyBytes_AS_STRING(b), n, PyBUF_WRITE)
just hides the fact that a mu
Stefan Krah added the comment:
Reproducible also on Linux with Python 3.3.
--
components: +Extension Modules -Macintosh
nosy: +belopolsky, skrah
stage: -> needs patch
versions: +Python 3.3
___
Python tracker
<http://bugs.python.org/issu
Stefan Krah added the comment:
The segfault does not occur in a debug build. The stack trace
suggests that timezone_richcompare() accesses other->offset
of the None object:
(gdb) f 2
#2 0x0041d4e9 in do_richcompare (v=None, w=,
op=) at Objects/object.c:563
563
Stefan Krah added the comment:
As I understand it, you prefer memoryviews where the format is
purely informational, whereas we now have typed memoryviews.
Typed memoryviews are certainly useful, in fact they are
present in Cython, see here for examples:
http://docs.cython.org/src/userguide
Changes by Stefan Krah :
--
components: Interpreter Core
nosy: dabeaz, skrah
priority: normal
severity: normal
stage: needs patch
status: open
title: memoryview: expose 'buf' attribute
type: enhancement
versions: Python 3.4
___
Python trac
Stefan Krah added the comment:
Looks good. It would be nice to have this in 3.3.0. There are a couple
of blockers open, so perhaps this could go in, too.
Georg, are we going to have an rc3 anyway?
--
nosy: +georg.brandl
___
Python tracker
<h
New submission from Stefan Krah:
I've installed 3.3.0-rc2 on Windows-7 64-bit using the msi installer.
I'm getting these failures in test_buffer, but I can *not* reproduce
them when I build Win-32/pgo/python.exe f
New submission from Stefan Krah:
This is similar to #15993: With the installed Python from the rc2-msi
test_lzma fails. I cannot reproduce the failure with python.exe (PGO)
compiled from source
Stefan Krah added the comment:
Both lzma and memoryview use PyLong_AsUnsignedLongLong() in the
affected code paths. I get this with the msi installed python.exe:
>>> import array
>>> a = array.array('Q', [1,2,3,4])
>>> m = memoryview(a)
>>> m[0]
Stefan Krah added the comment:
The high and low words of the 64-bit value are switched:
>>> a = array.array('Q', [1])
>>> m = memoryview(a)
>>> m[0]= 2**32+5
>>> m[0]
21474836481
>>> struct.unpack_from('8s', m, 0)
(b'\x01\
Stefan Krah added the comment:
This should be the same as #15993. Closing.
--
resolution: -> duplicate
stage: -> committed/rejected
status: open -> closed
superseder: -> Windows: 3.3.0-rc2.msi: test_buffer fails
___
Python tr
Changes by Stefan Krah :
--
nosy: +nadeem.vawda
___
Python tracker
<http://bugs.python.org/issue15993>
___
___
Python-bugs-list mailing list
Unsubscribe:
Stefan Krah added the comment:
Georg, *if* there is an rc3, could you add 97e53273f423 to it?
The commit is a backwards compatibility fix for constructing
infinities from tuples with arbitrary coefficients.
If there's no rc3, the issue can just be closed as fixed.
--
assignee:
Stefan Krah added the comment:
STINNER Victor wrote:
> Which version should I try? Ultimate? Premium? Professional?
Try Ultimate, it's AFAIK the only version these days that supports PGO.
--
___
Python tracker
<http://bugs.python.org
Stefan Krah added the comment:
Martin v. Löwis wrote:
> For the record, the released binary is not just a PGO build, but has
> been trained with the attached training script.
Thanks. Now I can reproduce the issue with a source build.
--
___
Stefan Krah added the comment:
It's a bit late here already, but unless I'm missing something I think
this is an optimizer bug. I'm attaching a workaround for memoryview.c
and _lzmamodule.c.
--
keywords: +patch
Added file: http://bugs.python.org/file27253/
Stefan Krah added the comment:
This one: 63cb0a642c84
--
___
Python tracker
<http://bugs.python.org/issue15973>
___
___
Python-bugs-list mailing list
Unsubscribe:
Stefan Krah added the comment:
System: Windows 7 64-bit
Build (unpatched): PCBuild\Win32-pgo\python.exe, trained with profile.bat
In the unpatched version, I stepped through this test case in the debugger:
import array
a = array.array('Q', [1])
m = memoryview(a)
m[0] = 1
Stefan Krah added the comment:
Sorry, the line numbers are messed up. They should be:
Objects/memoryobject.c:1569
Objects/memoryobject.c:1776
--
___
Python tracker
<http://bugs.python.org/issue15
Stefan Krah added the comment:
We would need to release the buffers and also check for format 'B'.
With issue15958-2.diff this is possible:
>>> import array
>>> a = array.array('d', [1.2345])
>>> b''.join([b'ABC', a])
b
Stefan Krah added the comment:
Also, perhaps we can keep a fast path for bytes and bytearray, but I
didn't time the difference.
--
___
Python tracker
<http://bugs.python.org/is
Stefan Krah added the comment:
> Given all the problems, I'll stop using PGO on all branches and
> as the compiler apparently generates bad code.
That is probably the best solution.
The problem in memoryview.c:pack_single() is that Visual Studio optimizes the
memcpy() to mov i
2701 - 2800 of 4951 matches
Mail list logo
