[Python-Dev] Default SIGINT handling dangerous?

2013-12-14 Thread Jurko Gospodnetić 

  Hi all.

  Is there any way to disable the default asynchronous exception 
raising SIGINT signal handling? Or a plan to add support for this? My 
searches on the net found several bug reports and discussions regarding 
this, but nothing seems final... :-(


  I believe the current Python default SIGINT handling by raising an 
asynchronous KeyboardInterrupt exception is dangerous, and can directly 
cause Python interpreter crashes even if no user code is involved, or 
any sort of other 'undefined behaviour' if it is.


  On Windows at least (using Python 3.3.3 at the moment, but seen with 
earlier versions as well), I occasionally get 'python.exe has stopped 
working' dialogs when Ctrl-C is pressed (most often if pressed during 
Python's initialization - loading site.py or earlier).


  The problem with asynchronous exceptions is, of course, that they can 
occur at _any_ time, even at the start of a finally: block, effectively 
causing cleanup code to be skipped even though the programmer intended 
it to be called even in case of exceptions.


  My scripts replace the default SIGINT/SIGBREAK signal handlers as 
soon as possible, and everything works great after that, but things get 
ugly if Ctrl-C is pressed before the script gets a chance to do this. I 
could even live with an 'exit with an ugly traceback', but having the 
process hang or fail so that Windows reports it as non-responding or 
reports it as 'stopped working' and offers to send failure information 
to Microsoft? That just leaves a bad taste in my mouth. :-(


  Also, Python documentation should not helpfully state or infer in 
several places that user can handle KeyboardInterrupt exceptions to 
implement Ctrl-C handling. Even if you do manage to catch it, you must 
never ever ignore it and must terminate your application since it might 
have already been left in an inconsistent state internally (e.g. some 
important finally clause got skipped, even one located deep in the 
Python standard library internals). Doing anything else can only be 
considered a partially working hack. Another problem is that that 
multiple SIGINT signals might occur in a sequence and any such 
KeyboardInterrupt handling can itself be interrupted in the same way as 
the original code. You can say that this is 'unlikely', or even add 
additional code to make this even more unlikely, but it still smells 
bad. :-(


  Hope this helps.

  Best regards,
Jurko Gospodnetić


  Below, I've included a few script outputs (tracebacks included) from 
instances where Python interpreter crashed due to pressing Ctrl-C soon 
after an empty Python script has been run.


In the first of these instances I got the Microsoft's error reporting 
dialog. In all the later instances I checked the Python process's debug 
output and every time it included the message: 'Microsoft Visual Studio 
C Runtime Library has detected a fatal error in python.exe.'.


Using:
  - OS: Windows 7 SP1 x64
  - Python 3.3.3 (64-it)
  - default site.py

--
Occurrence #1:
--

D:\Workplace>run.py
Fatal Python error: Py_Initialize: can't initialize sys standard streams
Traceback (most recent call last):
  File "", line 755, in _open_registry
FileNotFoundError: [WinError 2] The system cannot find the file specified

During handling of the above exception, another exception occurred:

Traceback (most recent call last):
  File "C:\Program Files\Python\Python333\lib\encodings\__init__.py", line 98, 
in search_function
level=0)
  File "", line 1565, in _find_and_load
  File "", line 1523, in_find_and_load_unlocked
  File "", line 1477, in _find_module
  File "", line 777, in find_module
  File "", line 768, in _search_registry
  File "", line 755, in _open_registry
KeyboardInterrupt



--
Occurrence #2:
--

D:\Workplace>run.py
Fatal Python error: Py_Initialize: can't initialize sys standard streams
Traceback (most recent call last):
  File "C:\Program Files\Python\Python333\lib\codecs.py", line 165, in __init__
def __init__(self, errors='strict'):
KeyboardInterrupt



--
Occurrence #3:
--

D:\Workplace>run.py
Fatal Python error: Py_Initialize: can't initialize sys standard streams
Traceback (most recent call last):
  File "", line 1565, in _find_and_load
  File "", line 1523, in _find_and_load_unlocked
  File "", line 1477, in _find_module
  File "", line 1309, in find_module
  File "", line 1288, in _get_loader
  File "", line 1387, in find_loader
  File "", line 110, in _path_isfile
  File "", line 101, in _path_is_mode_type
KeyboardInterrupt



--
Occurrence #4:
--

D:\Workplace>run.py
Fatal Python error: Py_Initialize: can't initialize sys standard streams
Traceback (most recent call last):
  File "C:\Program Files\Python\Python333\lib\io.py", line 52, in 
import abc
  File "", line 1565, in _find_and_load
  File "", line 1533, in _find_and_load_unlocked
  File "", line 486, in _verbose_message
KeyboardInt

Re: [Python-Dev] Default SIGINT handling dangerous?

2013-12-14 Thread R. David Murray 
On Sat, 14 Dec 2013 15:14:10 +0100, "Jurko Gospodneti" 
 wrote:
>My scripts replace the default SIGINT/SIGBREAK signal handlers as 
> soon as possible, and everything works great after that, but things get 
> ugly if Ctrl-C is pressed before the script gets a chance to do this. I 
> could even live with an 'exit with an ugly traceback', but having the 
> process hang or fail so that Windows reports it as non-responding or 
> reports it as 'stopped working' and offers to send failure information 
> to Microsoft? That just leaves a bad taste in my mouth. :-(

This sounds like a more troubling variation on the issue raised in
issue 14288 (http://bugs.python.org/issue14228).  The solution proposed 
there would appear to be something that would help you, so you might
want to chime in on that issue.

(I'll be curious to see the responses to the other points you raise.)

--David
___
Python-Dev mailing list
[email protected]
https://mail.python.org/mailman/listinfo/python-dev
Unsubscribe: 
https://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com

Re: [Python-Dev] Default SIGINT handling dangerous?

2013-12-14 Thread Guido van Rossum 
Your subject should probably have mentioned Windows. SIGINT handling
on UNIX is well-behaved. Yes, you can interrupt a finally clause, but
this by itself doesn't threaten the integrity of the interpreter and
the standard data types.

On Windows, "signal" handling is some feeble emulation done by the C
library wrappers and all bets are, as you've discovered, off. My own
experience is the opposite of yours -- I often end up with
uninterruptable runaway code that can only be stopped by ctrl-BRK,
which takes e.g. Powershell with it. :-(

That said, I'd be fine with a command-line flag to skip the default
SIGINT handler setup.

-- 
--Guido van Rossum (python.org/~guido)
___
Python-Dev mailing list
[email protected]
https://mail.python.org/mailman/listinfo/python-dev
Unsubscribe: 
https://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com

Re: [Python-Dev] How long the wrong type of argument should we limit (or not) in the error message (C-api)?

2013-12-14 Thread Greg Ewing 

David Hutto wrote:
Being that python is, to me, a prototyping language, then every possible 
outcome should be presented to the end user.


So we should produce a quantum superposition of
error messages? :-)

(Sorry, I've been watching Susskind's lectures on
QM and I've got quantum on the brain at the moment.)

--
Greg
___
Python-Dev mailing list
[email protected]
https://mail.python.org/mailman/listinfo/python-dev
Unsubscribe: 
https://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com

Re: [Python-Dev] Default SIGINT handling dangerous?

2013-12-14 Thread Jurko Gospodnetić 

  Hi.

On 14.12.2013. 17:14, R. David Murray wrote:

On Sat, 14 Dec 2013 15:14:10 +0100, "Jurko Gospodneti" 
 wrote:

My scripts replace the default SIGINT/SIGBREAK signal handlers as
soon as possible, and everything works great after that, but things get
ugly if Ctrl-C is pressed before the script gets a chance to do this. I
could even live with an 'exit with an ugly traceback', but having the

process hang or fail so that Windows reports it as non-responding or

reports it as 'stopped working' and offers to send failure information
to Microsoft? That just leaves a bad taste in my mouth. :-(


This sounds like a more troubling variation on the issue raised in
issue 14288 (http://bugs.python.org/issue14228).  The solution proposed
there would appear to be something that would help you, so you might
want to chime in on that issue.


  Thanks for the tip. I've seen the bug report you mention before, but 
thought to first inquire about the issue on the mailing list. I'll chime 
in what I know to the bug report.


  Final solution proposed there is to add a command-line option to 
delay enabling current default Python SIGINT handling until after the 
site.py package has been loaded.


  That would most likely avoid the startup crashes I mentioned in the 
original post.


  On Unix it would make Ctrl-C silently terminate the process if it 
occurs before default Python signal handling is enabled. I do not know 
what effect this would have on Windows - possibly the signal would 
simply be ignored & lost.


  It would also still leave a slight window between when Python sets up 
its default SIGINT handling and when user code has a chance to set up 
its own.


  My first instinct is to not do that and instead add an option to 
block SIGINT handling and allow user code to enable its own or default 
Python handling as it wishes and then unblock SIGINT handling. Note that 
by 'blocking' a signal I do not mean losing/ignoring it but delaying its 
handling until signal handling is unblocked.


  Best regards,
Jurko Gospodnetić


___
Python-Dev mailing list
[email protected]
https://mail.python.org/mailman/listinfo/python-dev
Unsubscribe: 
https://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com

Re: [Python-Dev] How long the wrong type of argument should we limit (or not) in the error message (C-api)?

2013-12-14 Thread Nick Coghlan 
On 14 December 2013 14:56, Vajrasky Kok  wrote:
> Greetings,
>
> When fixing/adding error message for wrong type of argument in C code,
> I am always confused, how long the wrong type is the ideal?
>
> The type of error message that I am talking about:
>
> "Blabla() argument 1 must be integer not wrong_type".
>
> We have inconsistency in CPython code, for example:
>
> Python/sysmodule.c
> ===
> PyErr_Format(PyExc_TypeError,
> "can't intern %.400s", s->ob_type->tp_name);
>
> Modules/_json.c
> 
> PyErr_Format(PyExc_TypeError,
>  "first argument must be a string, not %.80s",
>  Py_TYPE(pystr)->tp_name);
>
>
> Objects/typeobject.c
> ===
> PyErr_Format(PyExc_TypeError,
>  "can only assign string to %s.__name__, not '%s'",
>  type->tp_name, Py_TYPE(value)->tp_name);
>
> So is it %.400s or %.80s or %s? I vote for %s.
>
> Other thing is which one is more preferable? Py_TYPE(value)->tp_name
> or value->ob_type->tp_name? I vote for Py_TYPE(value)->tp_name.
>
> Or this is just a matter of taste?

The idiom has shifted over time, but the preference more recently is
definitely for length limiting user provided identifiers (which are
generally type names) to limit the maximum length of error messages
(to add another variant to the mix, PEP 7 has "%.100s" in an example
about breaking long lines that happens to include reporting
TypeError).

The question should probably be addressed directly in PEP 7, and I'd
be inclined to just bless the "%.400s" variant for future code.

Cheers,
Nick.

-- 
Nick Coghlan   |   [email protected]   |   Brisbane, Australia
___
Python-Dev mailing list
[email protected]
https://mail.python.org/mailman/listinfo/python-dev
Unsubscribe: 
https://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com

Re: [Python-Dev] How long the wrong type of argument should we limit (or not) in the error message (C-api)?

2013-12-14 Thread Antoine Pitrou 
On Sun, 15 Dec 2013 09:10:08 +1000
Nick Coghlan  wrote:
> 
> The question should probably be addressed directly in PEP 7, and I'd
> be inclined to just bless the "%.400s" variant for future code.

Shouldn't we have a special "%T" shortcut instead of trying to
harmonize all the occurrences of `"%.400s", Py_TYPE(self)->tp_name` ?

Sprinkling the same magic number / convention everywhere doesn't sound
very future-proof, nor convenient.

Regards

Antoine.


___
Python-Dev mailing list
[email protected]
https://mail.python.org/mailman/listinfo/python-dev
Unsubscribe: 
https://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com

Re: [Python-Dev] Default SIGINT handling dangerous?

2013-12-14 Thread Antoine Pitrou 
On Sat, 14 Dec 2013 15:14:10 +0100
Jurko Gospodnetić  wrote:
> 
>I believe the current Python default SIGINT handling by raising an 
> asynchronous KeyboardInterrupt exception is dangerous, and can directly 
> cause Python interpreter crashes even if no user code is involved, or 
> any sort of other 'undefined behaviour' if it is.

It would be nice if you could get an actual C traceback, and open an
issue on the bug tracker. Python isn't supposed to crash willy-nilly
when Ctrl+C is pressed.

(also, which Python version is this?)

Regards

Antoine.


___
Python-Dev mailing list
[email protected]
https://mail.python.org/mailman/listinfo/python-dev
Unsubscribe: 
https://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com

Re: [Python-Dev] How long the wrong type of argument should we limit (or not) in the error message (C-api)?

2013-12-14 Thread Victor Stinner 
2013/12/15 Antoine Pitrou :
> Shouldn't we have a special "%T" shortcut instead of trying to
> harmonize all the occurrences of `"%.400s", Py_TYPE(self)->tp_name` ?

Oh, I like this proposition! The following pattern is very common in Python:

"... %.400s ...", Py_TYPE(self)->tp_name

Victor
___
Python-Dev mailing list
[email protected]
https://mail.python.org/mailman/listinfo/python-dev
Unsubscribe: 
https://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com

Re: [Python-Dev] How long the wrong type of argument should we limit (or not) in the error message (C-api)?

2013-12-14 Thread Nick Coghlan 
On 15 December 2013 09:52, Victor Stinner  wrote:
> 2013/12/15 Antoine Pitrou :
>> Shouldn't we have a special "%T" shortcut instead of trying to
>> harmonize all the occurrences of `"%.400s", Py_TYPE(self)->tp_name` ?
>
> Oh, I like this proposition! The following pattern is very common in Python:
>
> "... %.400s ...", Py_TYPE(self)->tp_name

Oh, yes, a %T shortcut for "length limited type name of the supplied
object" would be brilliant. We need this frequently for C level error
messages, and I almost always have to look at an existing example to
remember the exact incantation :)

Cheers,
Nick.

-- 
Nick Coghlan   |   [email protected]   |   Brisbane, Australia
___
Python-Dev mailing list
[email protected]
https://mail.python.org/mailman/listinfo/python-dev
Unsubscribe: 
https://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com

Re: [Python-Dev] Default SIGINT handling dangerous?

2013-12-14 Thread Jurko Gospodnetić 

  Hi.

On 15.12.2013. 0:19, Antoine Pitrou wrote:

It would be nice if you could get an actual C traceback

> and open an issue on the bug tracker.

  Done. C traceback and related information collected and attached to a 
new Python issue report available at:

  http://bugs.python.org/issue19983



Python isn't supposed to crash willy-nilly when Ctrl+C is
pressed.


 :-D



(also, which Python version is this?)


  - Python 3.3.3 (64-bit) downloaded from:
  http://www.python.org/download/releases/3.3.3
  - OS: Windows 7 SP1 x64
  - default site.py


  Hope this helps.

  Best regards,
Jurko Gospodnetić


___
Python-Dev mailing list
[email protected]
https://mail.python.org/mailman/listinfo/python-dev
Unsubscribe: 
https://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com

Re: [Python-Dev] How long the wrong type of argument should we limit (or not) in the error message (C-api)?

2013-12-14 Thread Steven D'Aprano 
On Sun, Dec 15, 2013 at 11:25:10AM +1000, Nick Coghlan wrote:

> Oh, yes, a %T shortcut for "length limited type name of the supplied
> object" would be brilliant. We need this frequently for C level error
> messages, and I almost always have to look at an existing example to
> remember the exact incantation :)

What are the chances that could be made available from pure Python too? 
Having to extract the name of the type is a very common need for error 
messages, and I never know whether I ought to write type(obj).__name__ 
or obj.__class__.__name__. A %T and/or {:T} format code could be the One 
Obvious Way to include the type name in strings:

raise TypeError("Expected int but got '{:T}'".format(obj))

looks nicer to me than either of:

raise TypeError("Expected int but got '{}'".format(type(obj).__name__))
raise TypeError("Expected int but got '{}'".format(obj.__class__.__name__))


-- 
Steven
___
Python-Dev mailing list
[email protected]
https://mail.python.org/mailman/listinfo/python-dev
Unsubscribe: 
https://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com

Re: [Python-Dev] How long the wrong type of argument should we limit (or not) in the error message (C-api)?

2013-12-14 Thread David Hutto 
Susskinds...Me too, but the refinement of the error messages is the point.
We should be looking at the full assessment of the error, which the
prototyping of python should present.

I've seen others reply that python wouldn't be around, or that theree are
other forms I've seen before that will take the forefront.

The point should be to align the prototyping of python with the updates in
technology taking place.

It should be like it usually is, line for lineerror assessments, even
followed by further info to inform the prototyper that is looking to
translate to a lower level language.


On Sat, Dec 14, 2013 at 4:07 PM, Greg Ewing wrote:

> David Hutto wrote:
>
>> Being that python is, to me, a prototyping language, then every possible
>> outcome should be presented to the end user.
>>
>
> So we should produce a quantum superposition of
> error messages? :-)
>
> (Sorry, I've been watching Susskind's lectures on
> QM and I've got quantum on the brain at the moment.)
>
> --
> Greg
>
> ___
> Python-Dev mailing list
> [email protected]
> https://mail.python.org/mailman/listinfo/python-dev
> Unsubscribe: https://mail.python.org/mailman/options/python-dev/
> dwightdhutto%40gmail.com
>



-- 
Best Regards,
David Hutto
*CEO:* *http://www.hitwebdevelopment.com *
___
Python-Dev mailing list
[email protected]
https://mail.python.org/mailman/listinfo/python-dev
Unsubscribe: 
https://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com

Re: [Python-Dev] How long the wrong type of argument should we limit (or not) in the error message (C-api)?

2013-12-14 Thread David Hutto 
We all strive to be python programmers, and some of the responses are that
it might not be around in the future.

Now we all probably speak conversational in other langs, but I'm thinking
of keeping around a great prototyping language.


So the topic becomes how too integrate it with the not just the expected,
but the unexpected technologiesDespite the topic is error messages, it
should apply to all possibilities that could be derived from a prototyping
language like python.


On Sat, Dec 14, 2013 at 11:09 PM, David Hutto wrote:

> Susskinds...Me too, but the refinement of the error messages is the point.
> We should be looking at the full assessment of the error, which the
> prototyping of python should present.
>
> I've seen others reply that python wouldn't be around, or that theree are
> other forms I've seen before that will take the forefront.
>
> The point should be to align the prototyping of python with the updates in
> technology taking place.
>
> It should be like it usually is, line for lineerror assessments, even
> followed by further info to inform the prototyper that is looking to
> translate to a lower level language.
>
>
> On Sat, Dec 14, 2013 at 4:07 PM, Greg Ewing 
> wrote:
>
>> David Hutto wrote:
>>
>>> Being that python is, to me, a prototyping language, then every possible
>>> outcome should be presented to the end user.
>>>
>>
>> So we should produce a quantum superposition of
>> error messages? :-)
>>
>> (Sorry, I've been watching Susskind's lectures on
>> QM and I've got quantum on the brain at the moment.)
>>
>> --
>> Greg
>>
>> ___
>> Python-Dev mailing list
>> [email protected]
>> https://mail.python.org/mailman/listinfo/python-dev
>> Unsubscribe: https://mail.python.org/mailman/options/python-dev/
>> dwightdhutto%40gmail.com
>>
>
>
>
> --
> Best Regards,
> David Hutto
> *CEO:* *http://www.hitwebdevelopment.com
> *
>



-- 
Best Regards,
David Hutto
*CEO:* *http://www.hitwebdevelopment.com *
___
Python-Dev mailing list
[email protected]
https://mail.python.org/mailman/listinfo/python-dev
Unsubscribe: 
https://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com

Re: [Python-Dev] How long the wrong type of argument should we limit (or not) in the error message (C-api)?

2013-12-14 Thread Vajrasky Kok 
On Sun, Dec 15, 2013 at 7:52 AM, Victor Stinner
 wrote:
>
> Oh, I like this proposition! The following pattern is very common in Python:
>
> "... %.400s ...", Py_TYPE(self)->tp_name
>

Okay, I have created ticket (and preliminary patch) for this
enhancement: http://bugs.python.org/issue19984

Vajrasky Kok
___
Python-Dev mailing list
[email protected]
https://mail.python.org/mailman/listinfo/python-dev
Unsubscribe: 
https://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com