Kristján Valur Jónsson added the comment:
Great throwback.
As far as I know, context managers are still not first class citizens. You
cannot _compose_ two context managers into a new one programmatically in the
language, in the same way that you can, for instance, compose two functions
Kristján Valur Jónsson added the comment:
Having given this some thougt, years laters, I believe it _is_ possible to
write nested() (and nested_delayed()) in a correct way in python, without the
ContextManagerExit function.
Behold!
import contextlib
@contextlib.contextmanager
def
Kristján Valur Jónsson added the comment:
The problem with QueryPerformanceCounter is that it drifts. It has high
resolution, but can drift far out of sync with GetTickCount64.
The best solutions on windows combine the two, but that's tricky to impolement.
QPC will wrap, but only af
Kristján Valur Jónsson added the comment:
I'm afraid I had put this matter _far_ out of my head :) Seeing the amount of
discussion on that other defect (stuff I had already come across and scrathced
my head over) I think there is a lot of catching up that I'd need to do and I
am
Kristján Valur Jónsson added the comment:
That was a bit abrupt, want't it? Denver is well aware of this and said:
"but sometimes using the context manager approach adds unnecessary bloat to
already long-winded unit tests."
I happen to agree with him and don't see why w
New submission from Kristján Valur Jónsson :
One of the problems with the profiling modules provided with Python is that
they are not useful in the presence of multiple threads. This is because time
spent in a different thread may be falsely attributed to some random place in a
thread being
Kristján Valur Jónsson added the comment:
Pehaps it hasn't been demonstrated before, but just for the sake of argument
(and because I'm a persistant bugger), here are the two different cases:
current:
ctxt = self.assertRaises(MyException)
with ctxt:
foo()
self.a
New submission from Kristján Valur Jónsson :
issue 9609 updates _lsprof.c to be multi-stack aware. This allows
cProfile.Profile() objects to be shared by many threads and provide meaningfull
results.
This update makes it more convenient to profile running, multi-threaded,
applications. By
Kristján Valur Jónsson added the comment:
Good question. Is that simple to do? I didn't think to look if there were a
central list of all python TLS states.
--
___
Python tracker
<http://bugs.python.org/i
Kristján Valur Jónsson added the comment:
Sounds like a much simpler change. I'll try that out instead.
--
___
Python tracker
<http://bugs.python.org/i
Kristján Valur Jónsson added the comment:
Here is a new, much simpler patch, which simply sets the flags on all the
threads (accessable as a linked list of PyThreadState objects) as suggested by
Antoine.
(Note that neither of the patches has the necessary enhancements to _lsprof.c
to make
Kristján Valur Jónsson added the comment:
Indeed it is. This is a remnant of the previous method. I will fix it, and
add documentation. I also think I'll leave the change to _lsprof out of this
so that this change can stand on its own, irrespective of the profiling engine
being
Kristján Valur Jónsson added the comment:
I just realized that this is probably a redundant change.
We have C apis to get all the Thread states in an interpreter state (I didn't
even know there was such a thing as multiple interpreter states, but there!)
This i
Kristján Valur Jónsson added the comment:
Setting this to invalid, since there already are APIs to do this, at least from
.py code.
--
resolution: -> invalid
___
Python tracker
<http://bugs.python.org/iss
Kristján Valur Jónsson added the comment:
Yes. in #1552880 I tried to make as minimal a change as possible. This
particular patch is still in use in EVE Online, which is installed in various
strange and exotic paths in the orient..
The trick I employed there was to encode everything to utf
Kristján Valur Jónsson added the comment:
Possibly. I made a comment in issue 9425 explaining the particular trick that
this here patch makes (using utf-8 as an intermediate form to avoid having to
change all the machinery in import.c
Kristján Valur Jónsson added the comment:
I conffess that I didn't follow the utf-8/surrogate discussion.
But the utf-8 encoding can encode all valid unicode characters:
UTF-8 may only legally be used to encode valid Unicode scalar values. According
to the Unicode standard the high an
Kristján Valur Jónsson added the comment:
> Yes, but in Python, U+DC80..D+DCFF range is used to store undecodable bytes.
> Eg. 'abc\xff'.decode('ascii', 'surrogateescape') gives 'abc\udcff'.
That's an inventive way of breaking the unicode
Kristján Valur Jónsson added the comment:
Well, it was, originally, but it met with so little interest that I couldn't be
bothered to polish it to inclusion standards. Anyway, there was the
incompatibility problem of what to do with the __file__ attribute, and the fact
that the patc
New submission from Kristján Valur Jónsson :
The following patch adds native TLS implementation for pthreads, avoiding the
relatively slow and clunky default tls implemented in thread.c
--
components: Interpreter Core
files: pthread_tls.patch
keywords: needs review, patch, patch
New submission from Kristján Valur Jónsson :
Holding the "keymutex" lock during malloc and free operations is not a good
idea. The reason is, that custom implementations of malloc and free, can use
the TLS themselves. This is, for example, true in embedded situations, where
on
Kristján Valur Jónsson added the comment:
The code may need a little bit of extra work: The pthreads machine that I have
to work with is a PS3 :).
And, despite the documentation not saying, I suspect that pthread_getspecific()
actually does mess with "errno", making it necessary t
Kristján Valur Jónsson added the comment:
You're right.
Added a new version of the patch.
--
Added file: http://bugs.python.org/file18793/tlspatch.patch
___
Python tracker
<http://bugs.python.org/i
Kristján Valur Jónsson added the comment:
"errno" is preserved by PyEval_RestoreThread(), so this isn't the issue.
What you are probably seeing is latent race conditions in the test suite, made
apparent by a non-locking TLS implementation. The test suite isn't free
Kristján Valur Jónsson added the comment:
Hm, both the test you mention are using the (non-recursive) lock to synchronize
threads. I can't see anything wrong there.
Could you please try to replace the cod in pthread_getspecific() with this:
int err = errno
void *result = pthread_getspe
Kristján Valur Jónsson added the comment:
Ah, good to hear.
--
___
Python tracker
<http://bugs.python.org/issue9786>
___
___
Python-bugs-list mailing list
Unsub
Kristján Valur Jónsson added the comment:
Here is a new patch. When 'allthreads' is specified to
cProfile.Profile.enable(), profling is enabled on all threads.
The testsuite tests to see that all threads do indeed register, it does not
attempt to validate the timings.
It turns it
Kristján Valur Jónsson added the comment:
I left the ifdef in for a quick and easy way to disable this code for those
interested, but I'm happy to remove it if it makes for greater synergy.
--
___
Python tracker
<http://bugs.python.org/i
Kristján Valur Jónsson added the comment:
You may find this hard to believe, but we do in fact embed python into other
applications. In this case, it is UnrealEngine, to drive a complex, console
based game. Yes, embedding python is much harder than it need be and I'll
submit some pa
Kristján Valur Jónsson added the comment:
I forgot to add: The API that our (instrumented) malloc implementation is
calling is:
PyGILState_GetThisThreadState();
--
___
Python tracker
<http://bugs.python.org/issue9
Kristján Valur Jónsson added the comment:
Added a new patch with #ifdef remvoved, for greater harmony.
--
Added file: http://bugs.python.org/file18864/pthread_tls.patch
___
Python tracker
<http://bugs.python.org/issue9
Kristján Valur Jónsson added the comment:
>The malloc
>implementation just has no business looking at the thread state.
Of course it does, if it you want to have any hope of instrumenting your python
memory usage with detailed python runtime information.
Your statement islike sayi
Kristján Valur Jónsson added the comment:
I've changed the function as you suggest, although there are in fact no failure
detection semantics defined for PyThread_create_key(). See e.g. thread.c:294
/* Return a new key. This must be called before any other functions in
* this family
Kristján Valur Jónsson added the comment:
Ok, here is a patch. key creation returns -1 on error, and the caller can
detect this and raise a fatal error.
--
Added file: http://bugs.python.org/file18906/pthread_tls.patch
___
Python tracker
<h
Kristján Valur Jónsson added the comment:
Committed as revision 84914
--
resolution: -> accepted
status: open -> closed
___
Python tracker
<http://bugs.python.org/
New submission from Kristján Valur Jónsson :
When calling Py_Initialize() from an embedding application, there is currently
no way for the application to override Python's initial settin g of sys.path.
An elaborate mechanism in getpathp.c kicks in, guessing the path based on
several cri
Kristján Valur Jónsson added the comment:
Right, forgot about getpath.c.
Added documentation.
See the latest version of the patch.
Please note that I was unable to test the non-windows compile.
--
Added file: http://bugs.python.org/file19003/py_setpath.patch
Changes by Kristján Valur Jónsson :
--
title: Add Py_SetPath API for embeddint python -> Add Py_SetPath API for
embedding python
___
Python tracker
<http://bugs.python.org/iss
Kristján Valur Jónsson added the comment:
Sorry, my mistake, I work in a mixed environment. If there are no objections,
I´ll submit this (with fixed whitespace)
--
___
Python tracker
<http://bugs.python.org/issue9
Kristján Valur Jónsson added the comment:
Completed in revision 85028
--
resolution: -> accepted
status: open -> closed
___
Python tracker
<http://bugs.python.org/
Changes by Kristján Valur Jónsson :
--
nosy: +krisvale
___
Python tracker
<http://bugs.python.org/issue9974>
___
___
Python-bugs-list mailing list
Unsubscribe:
New submission from Kristján Valur Jónsson :
make_buildinfo currently creates temporary files getbuildinfo2.c and
getbuildinfo.o in the current build directory. This update allows the caller
to specify a temp directory to put those files in.
The PCBuild pythoncore.vcproj now makes use of this
Kristján Valur Jónsson added the comment:
Ooops, here is the patch.
--
keywords: +patch
Added file: http://bugs.python.org/file19056/make_buildinfo.patch
___
Python tracker
<http://bugs.python.org/issue9
Kristján Valur Jónsson added the comment:
Interesting, is that a separate defect of doctest?
--
___
Python tracker
<http://bugs.python.org/issue9974>
___
___
Kristján Valur Jónsson added the comment:
Hi, I had forgotten about this.
I went back to the drawing board and had almost completed a new version.
Looking at the Java barrier shows how one can go overboard with stuff. My
though with the barrier is to provide a simple synchronization
Kristján Valur Jónsson added the comment:
Okay, here is a new submission.
I've redesigned it to be more reminiscent of the Java version, by allowing the
barrier to have a "Broken" state and raising a BrokenBarrierError.
I've also redesigned the mechanism from a simple pe
New submission from Kristján Valur Jónsson :
The BufferObject in 2.7 doesn't support the new buffer interface.
This makes it useless for use with the new memoryview object.
This simple patch adds that support.
--
components: Interpreter Core
files: buffer_newbuf.patch
keywords:
New submission from Kristján Valur Jónsson :
When doing socket IO, it is beneficial to use a bytearra() and then using
sock.recv_into() to avoid moving data about.
However, many useful functions still don't accept new style buffers, such as
the bytearray and memoryview. In particular
Kristján Valur Jónsson added the comment:
Do you mean that we should disable writing for the new style buffer interface?
Currently the patch respects the Buffer object's read-only flag
(self->b_readonly):
static int buffer_getbuffer(PyBufferObject *self, Py_buffer *buf, i
Kristján Valur Jónsson added the comment:
I disagree. It's not a new feature. We're merely completing an old feature
(adding new-style buffers from 3.x to 2.7) that wasn't fully implemented. by
the core. The new buffer isn't accepted in a lot of places where you'
Kristján Valur Jónsson added the comment:
ping?
--
___
Python tracker
<http://bugs.python.org/issue8777>
___
___
Python-bugs-list mailing list
Unsubscribe:
Kristján Valur Jónsson added the comment:
Right. The condition object change is necessary to have timeout work. I can
remove that feature, and slate it for another day. Add a separate patch for a
Condition.wait() return value. All of the other apis are able to let the
caller know whether
New submission from Kristján Valur Jónsson :
Add a return value to Condition.wait() that can be used to tell if
Condition.wait() returns due to a timeout effect. This mirrors other wait apis
in threading.py
--
components: Library (Lib)
files: condwait.patch
keywords: patch
messages
Kristján Valur Jónsson added the comment:
Here is an updated patch. It contains documentation.
ReStructured isn't my Forte, and I don't know how to verify that it is correct,
so please review it for me.
--
dependencies: +Add a return value to threading.Condition.wait()
Kristján Valur Jónsson added the comment:
Right.
I've provided more text for the return value and provided an example.
I´ve changed all three to properties, the locking wasn't really required for
waiting().
I added some extra tests for the properties.
--
Added
Kristján Valur Jónsson added the comment:
Committed as revision 85878
--
resolution: -> accepted
status: open -> closed
___
Python tracker
<http://bugs.python.org/
Kristján Valur Jónsson added the comment:
For backwards compatibility, say with other build configurations (there is
still rudimentary support for VS2005, 2003.)
--
___
Python tracker
<http://bugs.python.org/issue9
New submission from Kristján Valur Jónsson :
In a recent email exchange on python-dev, Antoine Pitrou mentioned that slicing
memoryview objects (lazy slices) wasn't necessarily very efficient when dealing
with short slices. The data he posted was:
$ ./python -m timeit -s "x = b&#x
Kristján Valur Jónsson added the comment:
Oh dear. Here it is.
--
Added file: http://bugs.python.org/file19410/memoryobj.patch
___
Python tracker
<http://bugs.python.org/issue10
Kristján Valur Jónsson added the comment:
But then, perhaps implementing the sequence protocol for memoryviews might be
more efficient still.
--
___
Python tracker
<http://bugs.python.org/issue10
Kristján Valur Jónsson added the comment:
As an additional point: the PyMemoryObject has a "base" member that I think is
redundant. the "view.obj" should be sufficient.
--
___
Python tracker
<http://bug
Kristján Valur Jónsson added the comment:
In 2.x, strings are sliced using PySequence_GetSlice(). ceval.c in 3.0 is
different, there is no apply_slice there (despite comments to that effect).
I'd have to take another look with the profiler to figure out how bytes slicing
in 3.0 works
Kristján Valur Jónsson added the comment:
Well then, its back to the profiler for 3.2. I did all of the profiling with
2.7 for practical reasons (it was the only version I had available at the time)
and then ported the change to 3.2 today. But obviously there are different
rules in 3.2
Kristján Valur Jónsson added the comment:
This is a timeout issue, probably encountered on a slow machine.
Checked in revision 85964 increasing the default timeout to cater to slower
machines.
However, I also see that the timeout mechanism used by barrier isn't very
robust. I'l
Kristján Valur Jónsson added the comment:
Silly me, changing the default timeout invalidated the unittest for it.
Fixed in revision 86018
--
___
Python tracker
<http://bugs.python.org/issue10
New submission from Kristján Valur Jónsson :
The attached patch adds a wait_for method to condition objects.
It can simplify code that uses condition variables since it relieves the user
from writing a condition loop. In addition it simplifies timeout logic which
otherwise has to correctly
Kristján Valur Jónsson added the comment:
The benchmarks are from 3.2
Also, I'll do a more relevant profiling session for 3.2. This patch is based
on profiling results from 2.7 so there might be more relevant optimization
cases i
Kristján Valur Jónsson added the comment:
In case I'm not clear enough:
The patch is for 3.2, the benchmarks are 3.2, but it was created based on 2.7
results, which may not fully apply for 3.2
--
___
Python tracker
<http://bugs.py
Kristján Valur Jónsson added the comment:
Yes, I took the liberty of removing the superfluous build configuration as
well, thereby simplifying the .sln and the build dependency graph.
The patch is for python 2.7. Even though it's technically not a python patch
but a build patch, I'
Kristján Valur Jónsson added the comment:
The wait_for() method is basically a distillation of the Semaphore.acquire()
method, which tries to intelligently handle a non-trivial timeout.
With this method it is now possible to simplify Semaphore.acquire, although I
didn't want to do so in
New submission from Kristján Valur Jónsson :
If measuring time across blocking calls, such as thread synchronization, one
currently must time.time(). This is because time.clock() measures cpu seconds
on unix. On windows, however, time.clock() would be more appropriate because
it measures
Kristján Valur Jónsson added the comment:
Certainly.
I was going to put this simple code in time.py when I realized that time was a
C module.
The main point, as Antoine points out, is that time.clock() means two seriously
different things on the two main platforms, and time.clock() is
Kristján Valur Jónsson added the comment:
Good points, Jeffrey. Documentation can be improved and cond.wait_for(pred) is
indeed not guaranteed to wake up when predicate is true unless someone calls
notifyAll.
I spent some time thinking of a name. I tried wait_predicate and
predicate_wait
Kristján Valur Jónsson added the comment:
Well, that is sort of what I'm trying to achieve. Note that you probably want
to use QueryPerformaceCounter on windows (or simply time.clock()) or at least
GetTickCount64 which doesn't wrap around after 50 days :). Also, I'm try
New submission from Kristján Valur Jónsson :
The PyMemoryObject has a "base" member which appears to be obsolete.
Furthermore, the function do_release() attempt to perform some obsolete-looking
woodo with base if it happens to be a tuple. Looks dangerous.
--
components: I
New submission from Kristján Valur Jónsson :
These two socket modules link to a different winsock api, thus pulling two
potentially incompatible dlls in to the process. There is no guarantee that
they interact.
I'll see if there is a simple patch for select module
--
component
Kristján Valur Jónsson added the comment:
Here is a patch
--
keywords: +patch
Added file: http://bugs.python.org/file19475/select.patch
___
Python tracker
<http://bugs.python.org/issue10
New submission from Kristján Valur Jónsson :
import ctypes
ctypes.windll.kernel32.DebugBreak()
This used to be a handy way to attach a debugger to a running program, by way
of JIT debugging. Now ctypes catches and handles this exception so a debugger
is never invoked. Bummer
Kristján Valur Jónsson added the comment:
committed as revision 86136
--
resolution: -> accepted
status: open -> closed
___
Python tracker
<http://bugs.python.org/i
Kristján Valur Jónsson added the comment:
Committed to 3.2 in revision 86137
--
resolution: -> accepted
status: open -> closed
versions: +Python 3.2 -Python 2.7
___
Python tracker
<http://bugs.python.org/
Kristján Valur Jónsson added the comment:
Ah, the SHADOW member...
Weird.
Anyway, I have been hacking around in the memory view. One thing that it does,
and makes me uncomfortable since I think it is breaking the new buffer
protocol, is to
a) PyObject_GetBuffer()
b) Modify the resulting
Kristján Valur Jónsson added the comment:
Ok, here is a new patch which slightly expands the documentation and improves
the timeout unittest.
If there are no objections I'll then commit this shortly.
--
Added file: http://bugs.python.org/file19594/wait_for2.
Kristján Valur Jónsson added the comment:
Good point, Antoine. I'm always trying to keep those timeouts low, however, to
avoid having the testsuite duration grow too much with every test :)
I think we can probably fix the issue by having the lock_tests.Bunch() function
only return whe
New submission from Kristján Valur Jónsson :
in shamodule.c, the digest() method just creates a simple bytes string of the
digest. The digest is stored as an array of 32 bit integers in the native
representation. Therefore, the digest will be different on big- and
little-endian machines
Kristján Valur Jónsson added the comment:
Closing this as fixed.
--
resolution: -> fixed
status: open -> closed
___
Python tracker
<http://bugs.python.org/i
Kristján Valur Jónsson added the comment:
Yes, the race condition with the timeout is a problem.
Here is a patch that implements this lock using a condition variable.
I agree that one must consider performance/simplicity when doing this.
--
Added file: http://bugs.python.org/file21322
Kristján Valur Jónsson added the comment:
I'm just providing this as a fast alternative to the Semaphore, which as far as
I know, will cause a kernel call every time.
Complicated is relative. In terms of the condition variable api, I wouldn't
say that it is. But given the fact th
Kristján Valur Jónsson added the comment:
Emulating condition variables on windows became easy once Semaphores were
provided by the OS because they provide a way around the lost wakeup problem.
The current implementation in cpython was submitted by me :) The source
material is provided for
Kristján Valur Jónsson added the comment:
Btw, the locktimeout.patch appears to have a race condition.
LeaveNonRecursiveMutex may SetEvent when there is no thread waiting (because a
timeout just occurred, but the thread on which it happened is still somewhere
around line #62 ). This will
Kristján Valur Jónsson added the comment:
Antoine: I agree, the semaphore is the quick and robust solution.
sbt: I see your point. Still, I think we still may have a flaw: The statement
that (owned-timeouts) is never an under-estimate isn't true on modern
architectures, I think.
Kristján Valur Jónsson added the comment:
There is no barrier in use on the read part. I realize that this is a subtle
point, but in fact, the atomic functions make no memory barrier guarantees
either (I think). And even if they did, you are not using a memory barrier
when you read the
Kristján Valur Jónsson added the comment:
Antoine: I notice that even the fast path contains a ResetEvent() call. I
think this is a kernel call and so just as expensive as directly using a
semaphore :). Otherwise, the logic looks robust, although ResetEvent() and
Event objects always give
Kristján Valur Jónsson added the comment:
Martin: I wouldn't worry too much about replacing a "Mutex" with a "Semaphore".
There is no reason to believe that they behave in any way different scheduling
wise, and if they did, then any python code that this would a
Kristján Valur Jónsson added the comment:
Sbt: I re-read the code and while I still maintain that the evaluation in line
50 is meaningless, I agree that the worst that can happen is an incorrect
timeout.
It is probably harmless because this state is only encountered for timeout==0,
and it is
Kristján Valur Jónsson added the comment:
I don't think it is necessary to have the initial call to PeekNamedPipe. It is
a kernel call just like WFMO and so just as expensive. It also has some
strange semantics, (such as being possibly blocking in a multithreaded
application. rea
Kristján Valur Jónsson added the comment:
What is the line that the parent process is executing? Line numbers don't seem
to match any more.
And is it possible to set a breakpoint in the child process where the fatal
error is triggered? It would be good to know what is being run at
New submission from Kristján Valur Jónsson :
I found this defect at PyCon 2011 after spending a lot of type fidgeting with
ssl. The test_ssl.py unittest was using timeout on accept sockets and it was
working fine there, but not with the code I was working out. Turns out that
_ssl.py resets
Changes by Kristján Valur Jónsson :
--
type: -> behavior
___
Python tracker
<http://bugs.python.org/issue11721>
___
___
Python-bugs-list mailing list
Unsubscri
Kristján Valur Jónsson added the comment:
I cannot see from the link to which branch that was committed, or what
revision. But I assume it is the default branch. I can confirm that this
appears to be fixed.
The corresponding defect has a long (and bothersome) discussion. I am,
however
Kristján Valur Jónsson added the comment:
Please see issue 11721 where I was commenting on the same.
I don't think the documentation makes it clear that socket.gettimeout() can be
incorrect (i.e. return None when the socket is non-blocking).
I also don't think there is a porta
1 - 100 of 950 matches
Mail list logo
