Re: [Python-Dev] Problem with signals in a single threaded application
Greg Ewing schrieb: >> Please >> try to come up with a patch (e.g. by putting a while(is_tripped) loop >> around the for loop). > > That isn't going to fix it. Why not? > What's needed is to somehow > atomically test and clear is_tripped at the beginning. How would that help? The case in question was a signal raised inside a signal handler. With my proposed solution, that would be processed in the next while loop; if it is cleared at the beginning, the call will wait for the next CheckSignals invocation. Also, why does it need to clear is_tripped atomically? If it is only cleared if it is set, I see no need to make the test and the clearing atomic. Regards, Martin ___ Python-Dev mailing list [email protected] http://mail.python.org/mailman/listinfo/python-dev Unsubscribe: http://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com
Re: [Python-Dev] Object creation hook
Kristján V. Jónsson schrieb:
> We have been using gc.get_objects() but it has several problems:
> 1) It returns all objects in the system.
Actually, it doesn't. It only returns objects that participate
in cyclic GC (i.e. container objects).
> 2) There is no way to frame certain operations and get just those
> objects that were created during their execution. In our case, we
> would like to get the server cluster running, then frame a certain
> operation to get a callback for all created objects, so that we could
> track that they were later destroyed correctly. I have done this
> previously by storing the id()s of all objects returned from
> gc.get_objects() and comparing them "before" and "after" but this
> suffers from 1) above, and the ambiguity of id() in the face of
> objects being created and destroyed.
As Neal says, people are typically successful in debugging memory
leaks by not looking at the entire list of objects, but looking
first at the number of objects (perhaps by type). It's very easy
to take snapshots of these numbers and then check whether they
grow. Only when you see that the numbers grow in an unreasonable
way, you can try to find out why a specific type has too many
objects created.
> void PyCCP_CreationHookFunc(PyObject * obj) { if (PyCCP_CreationHook)
> { PyObject *result, *tmp = PyCCP_CreationHook; PyCCP_CreationHook =
> 0; //guard against recursion result =
> PyObject_CallFunctionObjArgs(PyCCP_CreationHook, obj, 0);
> Py_XDECREF(result); if (!result) PyErr_Clear(); PyCCP_CreationHook =
> tmp; } }
I think this is not thread-safe.
Regards,
Martin
___
Python-Dev mailing list
[email protected]
http://mail.python.org/mailman/listinfo/python-dev
Unsubscribe:
http://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com
Re: [Python-Dev] Python's C interface for types
Tim Peters schrieb: >> Does that mean that they are unstable, in the sense that they may >> change behaviour in new versions of Python? > > They /may/ change, but they won't (== only common sense guarantees > they won't change ;-)). That, of course, is true for any API. For the documented API, there is certainly a stronger desire to provide backwards compatibility in the face of changes, but in some cases, it just means that the change also gets documented. Regards, Martin ___ Python-Dev mailing list [email protected] http://mail.python.org/mailman/listinfo/python-dev Unsubscribe: http://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com
Re: [Python-Dev] Problem with signals in a single threaded application
I apologise for going off-topic, but this is an explanation of why I said that signal handling is not reliable. The only relevance to Python is that Python should avoid relying on signals if possible, and try to be a little defensive if not. Signals will USUALLY do what is expected, but not always :-( Anything further by Email, please. Greg Ewing <[EMAIL PROTECTED]> wrote: > > > This one looks like an oversight in Python code, and so is a bug, > > but it is important to note that signals do NOT work reliably under > > any Unix or Microsoft system. > > That's a rather pessimistic way of putting it. In my > experience, signals in Unix mostly do what they're > meant to do quite reliably -- it's just a matter of > understanding what they're meant to do. Yes, it is pessimistic, but I am afraid that my experience is that it is so :-( That doesn't deny your point that they MOSTLY do 'work', but car drivers MOSTLY don't need to wear seat belts, either. I am talking about high-RAS objectives, and ones where very rare failure modes can become common (e.g. HPC and other specialist uses). More commonly, there are plain bugs in the implementations which are sanctioned by the standards (Linux is relatively disdainful of such legalistic games). Because they say that everything is undefined behaviour, many vendors' support mechanisms will refuse to accept bug reports unless you push like hell. And, as some are DIABOLICALLY difficult to explain, let alone demonstrate, they can remain lurking for years or decades. > There may be bugs in certain systems that cause > signals to get lost under obscure circumstances, but > that's no reason for Python to make the situation > worse by introducing bugs of its own. 100% agreed. > > Two related signals received between two 'checkpoints' (i.e. when > > the signal is tested and cleared). You may only get one of them, > > and 'related' does not mean 'the same'. > > I wasn't aware that this could happen between > different signals. If it can, there must be some > rationale as to why the second signal is considered > redundant. Otherwise there's a bug in either the > design or the implementation. Nope. There is often a clash between POSIX and the hardware, or a cause where a 'superior' signal overrides an 'inferior' one. I have seen SIGKILL flush some other signals, for example. And, on some systems, SIGFPE may be divided into the basic hardware exceptions. If you catch SIGFPE as such, all of those may be cleared. I don't think that many (any?) current systems do that. And it is actually specified to occur for the SISSTOP, SIGTSTP, SIGTTIN, SIGTTOU, SIGCONT group. > > A second signal received while the first is being 'handled' by the > > operating system or language run-time system. > > That one sounds odd to me. I would expect a signal > received during the execution of a handler to be > flagged and cause the handler to be called again > after it returns. But then I'm used to the BSD > signal model, which is relatively sane. It's nothing to do with the BSD model, which may be saner but still isn't 100% reliable, but occurs at a lower layer. At the VERY lowest level, when a genuine hardware event causes an interrupt, the FLIH (first-level interrupt handler) runs in God mode (EVERYTHING disabled) until it classifies what is going on. This is a ubiquitous misdesign of modern hardware, but that is off-topic. Hardware 'signals' from other CPUs/devices may well get lost if they occur in that window. And there are other, but less extreme, causes at higher levels in the operating system. Unix and Microsoft do NOT have a reliable signal delivery model, where the sender of a signal checks if the recipient has got it and retries if not. Some operating systems do - but I don't think that BSD does. > > A signal sent while the operating system is doing certain things to > > the application (including, sometimes, when it is swapped out or > > deep in I/O.) > > That sounds like an outright bug. I can't think > of any earthly reason why the handler shouldn't > be called eventually, if it remains installed and > the process lives long enough. See above. It gets lost at a low level. That is why you can cause serious time drift on an "IBM PC" (most modern ones) by hammering the video card or generating streams of floating-point fixups. Most people don't notice, because xntp or equivalent fixes it up. And there are worse problems. I could start on cross-CPU TLB and ECC handling on large shared memory systems. I managed to get an Origin in a state where it wouldn't even power down from the power-off button, and I had to flip breakers, due to THAT one! I have reason to believe that all largish SMP systems have similar problems. Again, it is possible to design an operating system to avoid those issues, but we are talking about mainstream ones, and they don't. Regards, Nick Maclaren, University of Cambridge Computing Service, New Museums Site, Pembroke Street, Cambrid
Re: [Python-Dev] Python's C interface for types
=?ISO-8859-1?Q?=22Martin_v=2E_L=F6wis=22?= <[EMAIL PROTECTED]> wrote:
>
> [not sure what "And so it goes" means in English]
I apologise. I try to restrain myself from using excessive idiom,
but sometimes I forget. It means "That is how things are, and there
is and will be more of the same."
> It may be a bit problematic to implement, but I think a clean
> specification is possible. If a and b are numbers, and a==b,
> then hash(a)==hash(b). I'm not sure whether "approximately 5.0"
> equals 5 or not: if it does, it should hash the same as 5,
> if it doesn't, it may or may not hash the same (whatever is
> easier to implement).
> For 0: hash(+0.0)==hash(-0.0)==hash(0)=hash(0L)=0
Unfortunately, that assumes that equality is transitive. With the
advanced floating-point models, it may not be. For example, if you
want to avoid the loss of error information, exact infinity and
approximate infinity (the result of overflow) have different
semantics. Similarly with infinitesimals.
Even at present, Python's float (Decimal probably more so) doesn't
allow you to do some things that are quite reasonable. For example,
let us say that I am implementing a special function and want to
distinguish -0.0 and +0.0. Why can't I use a dictionary?
>>> a = float("+0.0")
>>> b = float("-0.0")
>>> print a, b
0.0 -0.0
>>> c = {a: "+0.0", b: "-0.0"}
>>> print c[a], c[b]
-0.0 -0.0
Well, we all know why. But it is not what some quite reasonable
programmers will expect. And Decimal (with its cohorts and variant
precisions) has this problem quite badly - as do I.
No, I don't have an answer. You are damned if you do, and damned
if you don't. It is an insoluble problem, and CURRENTLY doesn't
justify two hashing mechanisms (i.e. ANY difference and EQUALITY
difference).
Regards,
Nick Maclaren,
University of Cambridge Computing Service,
New Museums Site, Pembroke Street, Cambridge CB2 3QH, England.
Email: [EMAIL PROTECTED]
Tel.: +44 1223 334761Fax: +44 1223 334679
___
Python-Dev mailing list
[email protected]
http://mail.python.org/mailman/listinfo/python-dev
Unsubscribe:
http://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com
Re: [Python-Dev] Problem with signals in a single threaded application
Martin v. Löwis wrote:
> Greg Ewing schrieb:
>
>>>Please
>>>try to come up with a patch (e.g. by putting a while(is_tripped) loop
>>>around the for loop).
>>
>>That isn't going to fix it.
>
> Why not?
Correct me if I'm wrong, but what I got from the OP
was that the current method does
if (is_tripped) {
for each signal {
if the signal has occurred, call its handler
}
is_tripped = 0;
}
and the problem is that any setting of is_tripped that
occurs in the midst of calling the handlers gets
wiped out at the end.
Changing this to
while (is_tripped) {
for each signal {
if the signal has occurred, call its handler
}
is_tripped = 0;
}
doesn't solve that, because is_tripped still gets
set to 0 before it's tested again.
> Also, why does it need to clear is_tripped atomically?
Thinking about it more, probably it doesn't. What's
important is to clear it *before* testing whether any
handlers need to be called, i.e.
if (is_tripped) {
is_tripped = 0;
for each signal {
if the signal has occurred, call its handler
}
}
If you really care, you can make that a while instead
of an if so that you don't have to wait until the next
CheckSignals. But if the signal had arrived a few
microseconds later you'd have to do that anyway, so
I don't see it as a big deal.
--
Greg
___
Python-Dev mailing list
[email protected]
http://mail.python.org/mailman/listinfo/python-dev
Unsubscribe:
http://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com
Re: [Python-Dev] Problem with signals in a single threaded application
On 1/27/07, Greg Ewing <[EMAIL PROTECTED]> wrote:
> > Why not?
>
> Correct me if I'm wrong, but what I got from the OP
> was that the current method does
>
>if (is_tripped) {
> for each signal {
>if the signal has occurred, call its handler
> }
> is_tripped = 0;
>}
>
> and the problem is that any setting of is_tripped that
> occurs in the midst of calling the handlers gets
> wiped out at the end.
>
> Changing this to
>
>while (is_tripped) {
> for each signal {
>if the signal has occurred, call its handler
> }
> is_tripped = 0;
>}
>
> doesn't solve that, because is_tripped still gets
> set to 0 before it's tested again.
Agreed.
> Thinking about it more, probably it doesn't. What's
> important is to clear it *before* testing whether any
> handlers need to be called, i.e.
>
>if (is_tripped) {
> is_tripped = 0;
> for each signal {
>if the signal has occurred, call its handler
> }
>}
>
That's exactly what my patch does as you can see here:
http://www.python.org/sf/1643738
Regards,
-- Ulisses
___
Python-Dev mailing list
[email protected]
http://mail.python.org/mailman/listinfo/python-dev
Unsubscribe:
http://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com
