[issue13342] input() builtin always uses "strict" error handler
New submission from stefanholek : The input builtin always uses "strict" error handling for Unicode conversions. This means that when I enter a latin-1 string in a utf-8 environment, input breaks with a UnicodeDecodeError. Now don't tell me not to do that, I have a valid use-case. ;-) While "strict" may be a good default choice, it is clearly not sufficient. I would like to propose an optional 'errors' argument to input, similar to the 'errors' argument the decode and encode methods have. I have in fact implemented such an input method for my own use: https://github.com/stefanholek/rl/blob/surrogate-input/rl/input.c While this solves my immediate needs, the fact that my implementation is basically just a copy of bltinmode.input with one additional argument, makes me think that this could be fixed in Python proper. There cannot be a reason input() should be confined to "strict", or can there? ;-) -- components: Unicode messages: 147005 nosy: ezio.melotti, stefanholek priority: normal severity: normal status: open title: input() builtin always uses "strict" error handler type: behavior versions: Python 3.2, Python 3.3, Python 3.4 ___ Python tracker <http://bugs.python.org/issue13342> ___ ___ Python-bugs-list mailing list Unsubscribe: http://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com
[issue13342] input() builtin always uses "strict" error handler
stefanholek added the comment: I am not quite sure how I would write a custom, readline-using input function in Python (access to PyOS_Readline seems required), that's why I did it in C. Have an example? -- ___ Python tracker <http://bugs.python.org/issue13342> ___ ___ Python-bugs-list mailing list Unsubscribe: http://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com
[issue13342] input() builtin always uses "strict" error handler
stefanholek added the comment: Thank you Antoine, this looks good. However when I try your example I get sys.stdin = io.TextIOWrapper( sys.stdin.detach(), 'ascii', 'replace') ValueError: underlying buffer has been detached -- ___ Python tracker <http://bugs.python.org/issue13342> ___ ___ Python-bugs-list mailing list Unsubscribe: http://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com
[issue12186] readline.replace_history_item still leaks memory
New submission from stefanholek :
This is a continuation of issue #9450.
The 'data' element of a history entry may point to an undo list for the entry.
When freeing the entry the associated undo list must be freed as well, and
'free(data)' alone does not cut it. I have not found any other use of the
'data' element in all of GNU Readline, so it is safe to assume it is either
NULL or an undo list.
diff --git a/rl/readline.c b/rl/readline.c
index 26ac1e2..c8efd5b 100644
--- a/rl/readline.c
+++ b/rl/readline.c
@@ -541,8 +541,18 @@
static void
_py_free_history_entry(HIST_ENTRY *entry)
{
- histdata_t data = free_history_entry(entry);
- free(data);
+ UNDO_LIST *undo_list;
+ UNDO_LIST *release;
+
+ /* A history entry may have an undo_list attached */
+ undo_list = (UNDO_LIST *)free_history_entry(entry);
+ while (undo_list) {
+ release = undo_list;
+ undo_list = undo_list->next;
+ if (release->what == UNDO_DELETE)
+ free(release->text);
+ free(release);
+ }
}
--
components: Library (Lib)
messages: 136951
nosy: stefanholek
priority: normal
severity: normal
status: open
title: readline.replace_history_item still leaks memory
type: resource usage
___
Python tracker
<http://bugs.python.org/issue12186>
___
___
Python-bugs-list mailing list
Unsubscribe:
http://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com
[issue12186] readline.replace_history_item still leaks memory
stefanholek added the comment: These undo lists come into existence when history entries are edited interactively (Arrow-Up, edit line, Arrow-Up, edit line, Enter -> undo list of first history entry leaks). -- components: +Extension Modules versions: +Python 2.6, Python 2.7, Python 3.1, Python 3.2, Python 3.3, Python 3.4 ___ Python tracker <http://bugs.python.org/issue12186> ___ ___ Python-bugs-list mailing list Unsubscribe: http://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com
[issue6953] readline documenation needs work
stefanholek added the comment:
To be zero-based, get_history_item would need to look like:
diff --git a/rl/readline.c b/rl/readline.c
index 33e9905..800bc00 100644
--- a/rl/readline.c
+++ b/rl/readline.c
@@ -559,7 +559,7 @@ get_history_item(PyObject *self, PyObject *args)
if (!PyArg_ParseTuple(args, "i:index", &idx))
return NULL;
- if ((hist_ent = history_get(idx)))
+ if ((hist_ent = history_get(history_base + idx)))
return PyString_FromString(hist_ent->line);
else {
Py_RETURN_NONE;
--
___
Python tracker
<http://bugs.python.org/issue6953>
___
___
Python-bugs-list mailing list
Unsubscribe:
http://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com
[issue6953] readline documenation needs work
stefanholek added the comment: I have read the readline source code, and it does mean just that. :-) Anyway, this does not really apply to the stdlib because unless someone implements 'stifle_history' and friends 'history_base' is going to be 1 at all times. -- ___ Python tracker <http://bugs.python.org/issue6953> ___ ___ Python-bugs-list mailing list Unsubscribe: http://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com
