[ python-Bugs-1552726 ] Python polls unnecessarily every 0.1 second when interactive
Bugs item #1552726, was opened at 2006-09-05 14:42
Message generated for change (Comment added) made by richardb
You can respond by visiting:
https://sourceforge.net/tracker/?func=detail&atid=105470&aid=1552726&group_id=5470
Please note that this message will contain a full copy of the comment thread,
including the initial issue submission, for this request,
not just the latest update.
Category: Python Library
Group: None
Status: Open
Resolution: Fixed
Priority: 5
Submitted By: Richard Boulton (richardb)
Assigned to: A.M. Kuchling (akuchling)
Summary: Python polls unnecessarily every 0.1 second when interactive
Initial Comment:
When python is running an interactive session, and is
idle, it calls "select" with a timeout of 0.1 seconds
repeatedly. This is intended to allow PyOS_InputHook()
to be called every 0.1 seconds, but happens even if
PyOS_InputHook() isn't being used (ie, is NULL).
To reproduce:
- start a python session
- attach to it using strace -p PID
- observe that python repeatedly
This isn't a significant problem, since it only affects
idle interactive python sessions and uses only a tiny
bit of CPU, but people are whinging about it (though
some appear to be doing so tongue-in-cheek) and it
would be nice to fix it.
The attached patch (against Python-2.5c1) modifies the
readline.c module so that the polling doesn't happen
unless PyOS_InputHook is not NULL.
--
>Comment By: Richard Boulton (richardb)
Date: 2006-09-08 09:34
Message:
Logged In: YES
user_id=9565
HAVE_READLINE_CALLBACK is defined by configure.in whenever
the readline library on the platform supports the
rl_callback_handler_install() function. I'm using Ubuntu
Dapper, and have libreadline 4 and 5 installed (more
precisely, 4.3-18 and 5.1-7build1), but only the -dev
package for 5.1-7build1. "info readline" describes
rl_callback_handler_install(), and configure.in finds it, so
I'm surprised it wasn't found on akuchling's machine.
I agree that the code looks buggy on platforms in which
signals don't necessarily get delivered to the main thread,
but looks no more buggy with the patch than without.
--
Comment By: A.M. Kuchling (akuchling)
Date: 2006-09-07 14:38
Message:
Logged In: YES
user_id=11375
On looking at the readline code, I think this patch makes no
difference to signals.
The code in readline.c for the callbacks looks like this:
has_input = 0;
while (!has_input) {
...
has_input = select.select(rl_input);
}
if (has_input > 0) {read character}
elif (errno == EINTR) {check signals}
So I think that, if a signal is delivered to a thread and
select() in the main thread doesn't return EINTR, the old
code is just as problematic as the code with this patch.
The (while !has_input) loop doesn't check for signals at all
as an exit condition.
I'm not sure what to do at this point. I think the new code
is no worse than the old code with regard to signals. Maybe
this loop is buggy w.r.t. to signals, but I don't know how
to test that.
--
Comment By: A.M. Kuchling (akuchling)
Date: 2006-09-07 14:17
Message:
Logged In: YES
user_id=11375
HAVE_READLINE_CALLBACK was not defined with readline 5.1 on
Ubuntu Dapper, until I did the configure/CFLAG trick.
I didn't think of a possible interaction with signals, and
will re-open the bug while trying to work up a test case.
--
Comment By: Michael Hudson (mwh)
Date: 2006-09-07 14:12
Message:
Logged In: YES
user_id=6656
I'd be cautious about applying this to 2.5: we could end up with the same
problem currently entertaining python-dev, i.e. a signal gets delivered to a
non-
main thread but the main thread is sitting in a select with no timeout so any
python signal handler doesn't run until the user hits a key.
HAVE_READLINE_CALLBACK is defined when readline is 2.1 *or newer* I think...
--
Comment By: A.M. Kuchling (akuchling)
Date: 2006-09-07 14:02
Message:
Logged In: YES
user_id=11375
Recent versions of readline can still support callbacks if
READLINE_CALLBACK is defined, so I could test the patch by
running 'CFLAGS=-DREADLINE_CALLBACK' and re-running configure.
Applied as rev. 51815 to the trunk, so the fix will be in
Python 2.6. The 2.5 release manager needs to decide if it
should be applied to the 2.5 branch.
--
Comment By: A.M. Kuchling (akuchling)
Date: 2006-09-07 13:24
Message:
Logged In: YES
user_id=11375
Original report:
http://perkypants.org/blog/2006/09/02/rfte-python
This is tied to the version of readline being used; the
select code is only used if HAVE_RL_CALLBACK is defined, and
a comment in Python's configure.in claims it's only defin
[ python-Bugs-1552726 ] Python polls unnecessarily every 0.1 second when interactive
Bugs item #1552726, was opened at 2006-09-05 10:42
Message generated for change (Comment added) made by akuchling
You can respond by visiting:
https://sourceforge.net/tracker/?func=detail&atid=105470&aid=1552726&group_id=5470
Please note that this message will contain a full copy of the comment thread,
including the initial issue submission, for this request,
not just the latest update.
Category: Python Library
Group: None
Status: Open
Resolution: Fixed
Priority: 5
Submitted By: Richard Boulton (richardb)
Assigned to: A.M. Kuchling (akuchling)
Summary: Python polls unnecessarily every 0.1 second when interactive
Initial Comment:
When python is running an interactive session, and is
idle, it calls "select" with a timeout of 0.1 seconds
repeatedly. This is intended to allow PyOS_InputHook()
to be called every 0.1 seconds, but happens even if
PyOS_InputHook() isn't being used (ie, is NULL).
To reproduce:
- start a python session
- attach to it using strace -p PID
- observe that python repeatedly
This isn't a significant problem, since it only affects
idle interactive python sessions and uses only a tiny
bit of CPU, but people are whinging about it (though
some appear to be doing so tongue-in-cheek) and it
would be nice to fix it.
The attached patch (against Python-2.5c1) modifies the
readline.c module so that the polling doesn't happen
unless PyOS_InputHook is not NULL.
--
>Comment By: A.M. Kuchling (akuchling)
Date: 2006-09-08 09:12
Message:
Logged In: YES
user_id=11375
That's exactly my setup. I don't think there is a -dev
package for readline 4.
I do note that READLINE_CALLBACKS is defined in
/usr/include/readline/rlconf.h, but Python's readline.c
doesn't include this file, and none of the readline headers
include it. So I don't know why you're finding the function!
--
Comment By: Richard Boulton (richardb)
Date: 2006-09-08 05:34
Message:
Logged In: YES
user_id=9565
HAVE_READLINE_CALLBACK is defined by configure.in whenever
the readline library on the platform supports the
rl_callback_handler_install() function. I'm using Ubuntu
Dapper, and have libreadline 4 and 5 installed (more
precisely, 4.3-18 and 5.1-7build1), but only the -dev
package for 5.1-7build1. "info readline" describes
rl_callback_handler_install(), and configure.in finds it, so
I'm surprised it wasn't found on akuchling's machine.
I agree that the code looks buggy on platforms in which
signals don't necessarily get delivered to the main thread,
but looks no more buggy with the patch than without.
--
Comment By: A.M. Kuchling (akuchling)
Date: 2006-09-07 10:38
Message:
Logged In: YES
user_id=11375
On looking at the readline code, I think this patch makes no
difference to signals.
The code in readline.c for the callbacks looks like this:
has_input = 0;
while (!has_input) {
...
has_input = select.select(rl_input);
}
if (has_input > 0) {read character}
elif (errno == EINTR) {check signals}
So I think that, if a signal is delivered to a thread and
select() in the main thread doesn't return EINTR, the old
code is just as problematic as the code with this patch.
The (while !has_input) loop doesn't check for signals at all
as an exit condition.
I'm not sure what to do at this point. I think the new code
is no worse than the old code with regard to signals. Maybe
this loop is buggy w.r.t. to signals, but I don't know how
to test that.
--
Comment By: A.M. Kuchling (akuchling)
Date: 2006-09-07 10:17
Message:
Logged In: YES
user_id=11375
HAVE_READLINE_CALLBACK was not defined with readline 5.1 on
Ubuntu Dapper, until I did the configure/CFLAG trick.
I didn't think of a possible interaction with signals, and
will re-open the bug while trying to work up a test case.
--
Comment By: Michael Hudson (mwh)
Date: 2006-09-07 10:12
Message:
Logged In: YES
user_id=6656
I'd be cautious about applying this to 2.5: we could end up with the same
problem currently entertaining python-dev, i.e. a signal gets delivered to a
non-
main thread but the main thread is sitting in a select with no timeout so any
python signal handler doesn't run until the user hits a key.
HAVE_READLINE_CALLBACK is defined when readline is 2.1 *or newer* I think...
--
Comment By: A.M. Kuchling (akuchling)
Date: 2006-09-07 10:02
Message:
Logged In: YES
user_id=11375
Recent versions of readline can still support callbacks if
READLINE_CALLBACK is defined, so I could test the patch by
running 'CFLAGS=-DREADLINE_CALLBACK' and re-running configure.
Applied as rev. 51815 to the trunk, so the fix will be in
Python 2.6. The 2.5 release m
[ python-Bugs-1552726 ] Python polls unnecessarily every 0.1 second when interactive
Bugs item #1552726, was opened at 2006-09-05 14:42
Message generated for change (Comment added) made by richardb
You can respond by visiting:
https://sourceforge.net/tracker/?func=detail&atid=105470&aid=1552726&group_id=5470
Please note that this message will contain a full copy of the comment thread,
including the initial issue submission, for this request,
not just the latest update.
Category: Python Library
Group: None
Status: Open
Resolution: Fixed
Priority: 5
Submitted By: Richard Boulton (richardb)
Assigned to: A.M. Kuchling (akuchling)
Summary: Python polls unnecessarily every 0.1 second when interactive
Initial Comment:
When python is running an interactive session, and is
idle, it calls "select" with a timeout of 0.1 seconds
repeatedly. This is intended to allow PyOS_InputHook()
to be called every 0.1 seconds, but happens even if
PyOS_InputHook() isn't being used (ie, is NULL).
To reproduce:
- start a python session
- attach to it using strace -p PID
- observe that python repeatedly
This isn't a significant problem, since it only affects
idle interactive python sessions and uses only a tiny
bit of CPU, but people are whinging about it (though
some appear to be doing so tongue-in-cheek) and it
would be nice to fix it.
The attached patch (against Python-2.5c1) modifies the
readline.c module so that the polling doesn't happen
unless PyOS_InputHook is not NULL.
--
>Comment By: Richard Boulton (richardb)
Date: 2006-09-08 14:30
Message:
Logged In: YES
user_id=9565
I'm finding the function because it's defined in the
compiled library - the header files aren't examined by
configure when testing for this function. (this is because
configure.in uses AC_CHECK_LIB to check for
rl_callback_handler_install, which just tries to link the
named function against the library). Presumably, rlconf.h
is the configuration used when the readline library was
compiled, so if READLINE_CALLBACKS is defined in it, I would
expect the relevant functions to be present in the compiled
library.
In any case, this isn't desperately important, since you've
managed to hack around the test anyway.
--
Comment By: A.M. Kuchling (akuchling)
Date: 2006-09-08 13:12
Message:
Logged In: YES
user_id=11375
That's exactly my setup. I don't think there is a -dev
package for readline 4.
I do note that READLINE_CALLBACKS is defined in
/usr/include/readline/rlconf.h, but Python's readline.c
doesn't include this file, and none of the readline headers
include it. So I don't know why you're finding the function!
--
Comment By: Richard Boulton (richardb)
Date: 2006-09-08 09:34
Message:
Logged In: YES
user_id=9565
HAVE_READLINE_CALLBACK is defined by configure.in whenever
the readline library on the platform supports the
rl_callback_handler_install() function. I'm using Ubuntu
Dapper, and have libreadline 4 and 5 installed (more
precisely, 4.3-18 and 5.1-7build1), but only the -dev
package for 5.1-7build1. "info readline" describes
rl_callback_handler_install(), and configure.in finds it, so
I'm surprised it wasn't found on akuchling's machine.
I agree that the code looks buggy on platforms in which
signals don't necessarily get delivered to the main thread,
but looks no more buggy with the patch than without.
--
Comment By: A.M. Kuchling (akuchling)
Date: 2006-09-07 14:38
Message:
Logged In: YES
user_id=11375
On looking at the readline code, I think this patch makes no
difference to signals.
The code in readline.c for the callbacks looks like this:
has_input = 0;
while (!has_input) {
...
has_input = select.select(rl_input);
}
if (has_input > 0) {read character}
elif (errno == EINTR) {check signals}
So I think that, if a signal is delivered to a thread and
select() in the main thread doesn't return EINTR, the old
code is just as problematic as the code with this patch.
The (while !has_input) loop doesn't check for signals at all
as an exit condition.
I'm not sure what to do at this point. I think the new code
is no worse than the old code with regard to signals. Maybe
this loop is buggy w.r.t. to signals, but I don't know how
to test that.
--
Comment By: A.M. Kuchling (akuchling)
Date: 2006-09-07 14:17
Message:
Logged In: YES
user_id=11375
HAVE_READLINE_CALLBACK was not defined with readline 5.1 on
Ubuntu Dapper, until I did the configure/CFLAG trick.
I didn't think of a possible interaction with signals, and
will re-open the bug while trying to work up a test case.
--
Comment By: Michael Hudson (mwh)
Date: 2006-09-07 14:12
Message:
Logged In: YES
user_id=6656
I'd be cautious about applying
[ python-Bugs-1553577 ] datetime.datetime.now() mangles tzinfo
Bugs item #1553577, was opened at 2006-09-06 11:11
Message generated for change (Comment added) made by nnorwitz
You can respond by visiting:
https://sourceforge.net/tracker/?func=detail&atid=105470&aid=1553577&group_id=5470
Please note that this message will contain a full copy of the comment thread,
including the initial issue submission, for this request,
not just the latest update.
Category: Python Library
>Group: 3rd Party
>Status: Closed
>Resolution: Invalid
Priority: 5
Submitted By: Skip Montanaro (montanaro)
>Assigned to: Neal Norwitz (nnorwitz)
Summary: datetime.datetime.now() mangles tzinfo
Initial Comment:
When using the pytz package (http://pytz.sf.net/) to create
timezone info objects datetime.datetime.now() behaves
differently than the regular datetime.datetime()
contstructor. Here's an example:
>>> import pytz
>>> info = pytz.timezone("US/Central")
>>> info
>>> import datetime
>>> now = datetime.datetime.now(tz=info)
>>> now
datetime.datetime(2006, 9, 6, 12, 44, 18, 983849,
tzinfo=)
>>> t2 = datetime.datetime(2006, 9, 6, 12, 44, 18,
983849, tzinfo=info)
>>> t2
datetime.datetime(2006, 9, 6, 12, 44, 18, 983849,
tzinfo=)
>>> now.tzinfo == info
False
>>> t2.tzinfo == info
True
It appears that datetime.datetime.now() makes an
off-by-one-hour copy of the timezone info it was passed.
I've reproduced this on 2.4.3 and 2.5c1 as of August 17.
(It's also a little annoying that the timezone arg for
datetime.datetime.now() is "tz" while the timezone arg for
datetime.datetime() is "tzinfo". Is there a good
reason for
them to be different?)
Skip
--
>Comment By: Neal Norwitz (nnorwitz)
Date: 2006-09-08 19:08
Message:
Logged In: YES
user_id=33168
Based on Stuart's comment, I'm closing this. Skip, if
there's any part of this that you think is a bug, re-open
this with a comment about the precise issue or open a new
bug report. Thanks.
--
Comment By: Stuart Bishop (zenzen)
Date: 2006-09-07 21:36
Message:
Logged In: YES
user_id=46639
This is a pytz issue, and a result of me abusing Tim's code
in ways he never intended. Tim is quite correct in that
there are actually several tzinfo instances under the
covers. In order to to unambiguous localtime calculations,
an extra bit of information needs to be known (the is_dst
flag in most datetime libraries). pytz uses the tzinfo
instance to store this bit of information. The side affects
of doing this are the behavior you noticed, and confusion as
constructing datetime instances needs to be done as per
pytz's README rather than the documented method in the
Python Library Reference.
>>> import pytz
>>> info = pytz.timezone('US/Central')
>>> info
>>> from datetime import datetime
>>> now = info.localize(datetime.now(), is_dst=True)
>>> now
datetime.datetime(2006, 9, 8, 11, 19, 29, 587943,
tzinfo=)
>>> t2 = info.localize(datetime(2006, 9, 8, 11, 19, 29, 587943))
>>> t2
datetime.datetime(2006, 9, 8, 11, 19, 29, 587943,
tzinfo=)
>>> now.tzinfo == info
False
>>> t2.tzinfo == info
False
>>> now.tzinfo == t2.tzinfo
True
Last time I tried, it seemed impossible to support both
pytz's goals and the datetime construction API specified in
the Python Library Reference without extending the Python
datetime module (and I have yet to specify what would be
required).
If I was to add an __eq__ method to the tzinfo classes, I'm
not actually sure what the correct behavior should be.
Should US/Eastern Daylight Savings Time really equate to
US/Eastern Standard Time? Should US/Eastern Daylight Savings
Time in 2002 really equate to US/Eastern Daylight Savings
Time in 2007? The umbrella timezone might be the same, but
the UTC offsets or switchover dates are different.
The pytz bugtracker is at
https://launchpad.net/products/pytz/+bugs
--
Comment By: Tim Peters (tim_one)
Date: 2006-09-07 10:11
Message:
Logged In: YES
user_id=31435
`tzinfo` is the name of a datetime data attribute, and the
same name (i.e., "tzinfo") is generally used for arguments
that mindlessly attach a subclass of the `tzinfo` class to
an object as the value of its `tzinfo` data attribute. The
datetime constructor is an example of that. `tz` is
generally used when the time zone info is /actively
applied/, as now() does.
In contrast, the datetime constructor never makes any
attempt at conversion; if a tzinfo argument is passed, it's
merely tacked on to the datetime object.
Beyond that, I have no idea why the pytz class passed to
now() isn't showing up as the resulting datetime object's
tzinfo member. For example, that's not what happens if you
try this in the Python sandbox "datetime" directory:
>>> from US import Eastern
>>> from datetime import datetime
>>> now = datetime.now(Eastern)
>>> now
datetime.datetime(2006, 9, 7, 12, 49
