[issue13386] Document documentation conventions for optional args
Baptiste Carvello added the comment: Hi all, here is a relevant user story. I'm afraid it won't help you much, but it highlights the importance of consistent conventions in doc. My girlfriend is learning Python with no prior programing experience. She quite naturally got used to calling help(function), and noted the following: 1) she naturally understood the meaning of the [opt] notation 2) she did not understand the opt=default notation, as she didn't have a sufficient experience with Python to recognize the syntax 3) even after learning what it meant, she still found that notation obscure and unappealing 4) she got annoyed that two completely different notations where used for two very close concepts 5) she got annoyed that there was no user-discoverable and user-understandable document introducing those notations (if there is one, my mistake :-) I have no ovious solutions to the annoyances. Regarding 4), maybe the [opt=default] notation has something good after all: that it reminds of the [opt] one. And regarding 5), if there is a canonical document about documentation conventions, I could try to summarize it in a language aimed at beginners. -- nosy: +baptiste.carvello ___ Python tracker <http://bugs.python.org/issue13386> ___ ___ Python-bugs-list mailing list Unsubscribe: http://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com
[issue13386] Document documentation conventions for optional args
Baptiste Carvello added the comment: Le 14/11/2011 13:40, Ezio Melotti a écrit : >> 1) she naturally understood the meaning of the [opt] notation > > I guess this depends on her background, I've seen people trying to use [] in > function calls because they saw them in the doc or confusing them for lists, > so I guess that each notation has its pros and cons. > agreed, the [] notation also has its dangers. But the current situation doesn't avoid them, because users will meet both notations. >> 2) she did not understand the opt=default notation, as she didn't >> have a sufficient experience with Python to recognize the syntax > > I agree that at the beginning it could be a bit confusing, but keyword > arguments are an important part of Python and it's among the first things > that one should learn. After that it should be even more natural than []. > the thing is, beginners need to use other people's functions before they really get into writing their own. You need some practice with a syntax before you are able to recognize it in another context. >> 3) even after learning what it meant, she still found that notation >> obscure and unappealing > > ...or maybe not. Can she say what in particular is obscure and unappealing? > I'd say the fact that the main information (that the argument is optional) is not highlighted and only appears as a side-effect of having a default. Inversely, a lot of importance is given to the value of the default, which most users can ignore at first. >> 4) she got annoyed that two completely different notations where used >> for two very close concepts > > This is a good point, and we are trying to move to the arg=default notation. > Unfortunately there are still places that use the old notation. C functions > that have optional arguments but don't accept keyword arguments are a bit > unusual, and IIUC in most of the cases that's an implementation detail that > could be removed. > That would would solve the problem for the stdlib, but other C libraries also have optional arguments which don't accept keyword arguments (for example NumPy ufuncs). Will converting to a keyword argument work for all of them? >> 5) she got annoyed that there was no user-discoverable and >> user-understandable document introducing those notations (if there is > one, >> my mistake :-) > > This brings ups another interesting point. These conventions will probably > end up in the "documenting" section, that is aimed to doc writers. Do we > need an introductory page aimed to the readers that explains the conventions > used in the doc? > I would say we need one. It should probably also be part of the "help()" tool, as the function prototype is the first information that help(function) displays. Cheers, Baptiste -- ___ Python tracker <http://bugs.python.org/issue13386> ___ ___ Python-bugs-list mailing list Unsubscribe: http://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com
[issue13386] Document documentation conventions for optional args
Baptiste Carvello added the comment: Le 14/11/2011 20:51, Eric Snow a écrit : > > So would it be worth the effort to identify each such place in the > built-ins/stdlib and eventually change them all? I've seen support for doing > so in other tracker issues and think it's a good idea personally. > I ran a few grep searches from the root of a recent hg tip: 1) grep -n -r --include=*.py --include=*.c --exclude="topics.py" -E '.+\(.*\[[[:space:]]*,.*\].*\)' . This looks for variants of "function(args [, opt])". There were 231 hits, I caught no false positives. 2) grep -n -r --include=*.py --include=*.c --exclude="topics.py" -E '.+\(.*\[.*,[[:space:]]*\].*\)' . As this pattern is valid Python syntax, I got mostly false positives, but also a few interesting cases such as "range([start,] stop[, step])" or "islice(seq, [start,] stop [, step])" I'm afraid those last examples cannot be described with valid Python syntax. -- ___ Python tracker <http://bugs.python.org/issue13386> ___ ___ Python-bugs-list mailing list Unsubscribe: http://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com
[issue13386] Document documentation conventions for optional args
Baptiste Carvello added the comment: Le 18/11/2011 05:29, Terry J. Reedy a écrit : > > In the following, I give objections to this PO (position only) rule and > suggest an alternative ND (no default) rule: use 'name=default' when there is > a default and '[name]' when there is not'. > > The issue of whether an argument is required or optional is orthogonal to > whether it can be passed by both position and name, only by name, or only by > position. With this logic, you would need to use '[name=default]' when an argument is optional *and* can be passed by name. Sure, this notation is inherently redundant, but is has the advantage of conveying both informations immediately to the user. It is also more coherent with '[name]'. But this is a big change from the current philosophy... -- ___ Python tracker <http://bugs.python.org/issue13386> ___ ___ Python-bugs-list mailing list Unsubscribe: http://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com
[issue12055] doctest not working on nested functions
Baptiste Carvello added the comment: Hello, the attached patch adds the requested feature to 3.2 (the patch is based on 6d67931c63f9), with appropriate doc changes and tests for the new behaviour. It does not apply to 2.7, so I'll send another patch for that soon. -- keywords: +patch nosy: +baptiste.carvello type: feature request -> versions: +Python 2.6, Python 2.7 -Python 3.3 Added file: http://bugs.python.org/file22083/issue12055-32.diff ___ Python tracker <http://bugs.python.org/issue12055> ___ ___ Python-bugs-list mailing list Unsubscribe: http://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com
[issue12055] doctest not working on nested functions
Baptiste Carvello added the comment:
here is the patch for 2.7.
Dave: I don't know if or when the patch will be accepted, as this is a new
feature. In the meantime, you can easily patch your system. As the code changes
are all in Python, you don't need to recompile. Going to your standard library
directory and running a command like the following should do the trick:
filterdiff -i '*/doctest.py' issue12055-27.diff | patch
--
Added file: http://bugs.python.org/file22085/issue12055-27.diff
___
Python tracker
<http://bugs.python.org/issue12055>
___diff --git a/Doc/library/doctest.rst b/Doc/library/doctest.rst
--- a/Doc/library/doctest.rst
+++ b/Doc/library/doctest.rst
@@ -280,7 +280,7 @@
searched. Objects imported into the module are not searched.
In addition, if ``M.__test__`` exists and "is true", it must be a dict, and
each
-entry maps a (string) name to a function object, class object, or string.
+entry maps a (string) name to a function object, code object, class object, or
string.
Function and class object docstrings found from ``M.__test__`` are searched,
and
strings are treated as if they were docstrings. In output, a key ``K`` in
``M.__test__`` appears with name ::
@@ -293,6 +293,12 @@
.. versionchanged:: 2.4
A "private name" concept is deprecated and no longer documented.
+Any functions found are recursively searched similarly, to test docstrings in
+their contained nested functions (nested functions exist as a code object
constant).
+
+Any code objects found, be it in ``M.__test__`` or nested in a function, are
recursively
+searched similarly, to test docstrings in their contained nested functions.
+
.. _doctest-finding-examples:
diff --git a/Lib/doctest.py b/Lib/doctest.py
--- a/Lib/doctest.py
+++ b/Lib/doctest.py
@@ -98,6 +98,7 @@
import sys, traceback, inspect, linecache, os, re
import unittest, difflib, pdb, tempfile
import warnings
+import types
from StringIO import StringIO
from collections import namedtuple
@@ -814,7 +815,10 @@
"""
# If name was not specified, then extract it from the object.
if name is None:
-name = getattr(obj, '__name__', None)
+if type(obj) != types.CodeType:
+name = getattr(obj, '__name__', None)
+else:
+name = getattr(obj, 'co_name', None)
if name is None:
raise ValueError("DocTestFinder.find: name must be given "
"when obj.__name__ doesn't exist: %r" %
@@ -925,17 +929,34 @@
raise ValueError("DocTestFinder.find: __test__ keys "
"must be strings: %r" %
(type(valname),))
-if not (inspect.isfunction(val) or inspect.isclass(val) or
-inspect.ismethod(val) or inspect.ismodule(val) or
-isinstance(val, basestring)):
+if not (inspect.isfunction(val) or inspect.iscode(val) or
+inspect.isclass(val) or inspect.ismethod(val) or
+inspect.ismodule(val) or isinstance(val, basestring)):
raise ValueError("DocTestFinder.find: __test__ values "
- "must be strings, functions, methods, "
+ "must be strings, functions, "
+ "code objects, methods, "
"classes, or modules: %r" %
(type(val),))
valname = '%s.__test__.%s' % (name, valname)
self._find(tests, val, valname, module, source_lines,
globs, seen)
+# Look for tests in a function's contained objects.
+if inspect.isfunction(obj) and self._recurse:
+for val in obj.__code__.co_consts:
+if inspect.iscode(val):
+valname = '%s.%s' % (name, val.co_name)
+self._find(tests, val, valname, module, source_lines,
+ globs, seen)
+
+# Look for tests in a code object's contained objects.
+if inspect.iscode(obj) and self._recurse:
+for val in obj.co_consts:
+if inspect.iscode(val):
+valname = '%s.%s' % (name, val.co_name)
+self._find(tests, val, valname, module, source_lines,
+ globs, seen)
+
# Look for tests in a class's contained objects.
if inspect.isclass(obj) and self._recurse:
for valname, val in obj.__dic
[issue12055] doctest not working on nested functions
Baptiste Carvello added the comment: Eric: my bad, I guess I was living in the past, before 3.2 was released :-) Anyway, my 3.2 patch applies to default (6354b4ceba1d), with just a one-line offset for test_doctest.py. All tests pass. By the way, my 2.7 patch was based on 76e5fe8e21fd. -- ___ Python tracker <http://bugs.python.org/issue12055> ___ ___ Python-bugs-list mailing list Unsubscribe: http://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com
[issue6011] python doesn't build if prefix contains non-ascii characters
Baptiste Carvello added the comment: Hello, I just tried your patch on latest svn (r84707), but I found out that the problem I reported can no more be reproduced. First, '_locale' seems now to be built earlier. Also, a fallback has been introduced in 'locale.getpreferredencoding'. When '_locale' cannot be imported, the encoding is now parsed from the environment variables (cf 'Lib/locale.py', line 558 and below). It looks like 'locale.getpreferredencoding' is now no more likely to fail than 'sys.getfilesystemencoding'. So I'm not sure if a patch is still needed at all. In case a patch still makes sense, pay attention that there now also is a 'Lib/sysconfig.py', which also has a '_parse_makefile' function. This function uses a logic similar to the one in 'Lib/distutils/sysconfig.py', even thought it uses the builtin 'open', so it would need the same fix, if one is needed. Cheers, B. -- ___ Python tracker <http://bugs.python.org/issue6011> ___ ___ Python-bugs-list mailing list Unsubscribe: http://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com
[issue6011] python doesn't build if prefix contains non-ascii characters
Baptiste Carvello added the comment: Eric: the bug does not exist with 2.7, as the Makefile is read as bytes. It exists with 3.1.2. By the way, when I say the bug is solved on 3.2, I only mean the narrow problem of using a non-ascii prefix that *is* decodable with the current locale. I do not mean the more general problem that arises, for example, when building with the 'C' locale, as is discussed in issue9561. With a 'C' locale, the build fails also with 3.2. -- ___ Python tracker <http://bugs.python.org/issue6011> ___ ___ Python-bugs-list mailing list Unsubscribe: http://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com
[issue6011] python doesn't build if prefix contains non-ascii characters
Baptiste Carvello added the comment: Hello, I can reproduce the exact same error as Éric. The end of the output is a little bit more informative here: Could not find platform dependent libraries Consider setting $PYTHONHOME to [:] Fatal Python error: Py_Initialize: Unable to get the locale encoding SystemError: NULL result without error in PyObject_Call /bin/sh: line 1: 13513 Aborted CC='gcc -pthread' LDSHARED='gcc -pthread -shared ' OPT='-DNDEBUG -g -fwrapv -O3 -Wall -Wstrict-prototypes' ./python -E ./setup.py build make: *** [sharedmods] Error 134 I only kept the part of the log that is output if I rerun "make". The number 13513 must be a PID number. It is not stable across invocations. Running just "./python -E ./setup.py build", or just "./python setup.py build" does also abort with Could not find platform dependent libraries Consider setting $PYTHONHOME to [:] Fatal Python error: Py_Initialize: Unable to get the locale encoding SystemError: NULL result without error in PyObject_Call Aborted The instructions to reproduce, used in my home directory (which has a ASCII path), are: svn co http://svn.python.org/projects/python/branches/py3k cd py3k/ export LC_ALL=C export LANG=C ./configure --prefix=/home/baptiste/Desktop/Téléchargements/PyInstall make make ./python -E ./setup.py build ./python setup.py build The svn revision pulled was 85926. Hope this helps, Baptiste -- ___ Python tracker <http://bugs.python.org/issue6011> ___ ___ Python-bugs-list mailing list Unsubscribe: http://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com
[issue6011] python doesn't build if prefix contains non-ascii characters
Baptiste Carvello added the comment: A little bit more information: the error message comes from Python/pythonrun.c, line 736, in function initfsencoding. This part of the code is protected with a preprocessor #if: #if defined(HAVE_LANGINFO_H) && defined(CODESET) so I tried replacing that with #if 0. However, the function then fails on line 750. The comment on line 749 states: /* Such error can only occurs in critical situations: no more * memory, import a module of the standard library failed, * etc. */ It looks like it is not the case, and that Py_FileSystemDefaultEncoding has no reasonable default when "python" is called from the build directory with the "C" locale. For the record, when running the system "python" with the "C" locale, the filesystemencoding is gets set to 'ANSI_X3.4-1968'. Last thing, in case it is of any use, all my testing is done on an amd64 Debian stable system. Cheers, Baptiste -- ___ Python tracker <http://bugs.python.org/issue6011> ___ ___ Python-bugs-list mailing list Unsubscribe: http://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com
[issue6011] python doesn't build if prefix contains non-ascii characters
Baptiste Carvello added the comment:
Indeed, I retried with 534a9e274d88 (that was the tip of 3.2 sometime
yesterday) and my original problem is solved. Thank you.
While I was at it, I ran "make test", and got 3 unusual skips and 1
failure.
The skips are test_sax, test_xml_etree and test_xml_etree_c and they are
skipped on purpose when the example XML filename is not encodable to
utf8. No problem here.
The failure is for test_distutils. 3 individual tests are failing:
test_simple_built, test_debug_mode and test_record. The cause of this
failure is that the "install" command installs a test distribution to a
path containing sys.prefix. This is not a problem per se, but later
test_simple_built tries to zip this distribution, and cannot construct a
valid archive name. A similar problem happens when test_record tries to
write the distribution's filenames to a record file (and test_debug_mode
fails because of test_record).
Imho those failures cannot be fixed, so the only possible improvement is
to skip those tests. The attached trivial patch does just that, but I'm
not sure if it's worth patching distutils for that.
Cheers,
Baptiste
--
Added file: http://bugs.python.org/file21992/test_distutils_surrogateescape.diff
___
Python tracker
<http://bugs.python.org/issue6011>
___diff --git a/Lib/distutils/tests/test_bdist_dumb.py
b/Lib/distutils/tests/test_bdist_dumb.py
--- a/Lib/distutils/tests/test_bdist_dumb.py
+++ b/Lib/distutils/tests/test_bdist_dumb.py
@@ -24,6 +24,12 @@
except ImportError:
ZLIB_SUPPORT = False
+try:
+os.path.normpath(sys.prefix).encode("utf8")
+PREFIX_NOT_UTF8 = False
+except UnicodeEncodeError:
+PREFIX_NOT_UTF8 = True
+
class BuildDumbTestCase(support.TempdirManager,
support.LoggingSilencer,
@@ -42,6 +48,7 @@
super(BuildDumbTestCase, self).tearDown()
@unittest.skipUnless(ZLIB_SUPPORT, 'Need zlib support to run')
+@unittest.skipIf(PREFIX_NOT_UTF8, 'prefix is not encodable to utf8')
def test_simple_built(self):
# let's create a simple package
diff --git a/Lib/distutils/tests/test_install.py
b/Lib/distutils/tests/test_install.py
--- a/Lib/distutils/tests/test_install.py
+++ b/Lib/distutils/tests/test_install.py
@@ -16,6 +16,13 @@
from distutils.tests import support
+try:
+os.path.normpath(sys.prefix).encode("utf8")
+PREFIX_NOT_UTF8 = False
+except UnicodeEncodeError:
+PREFIX_NOT_UTF8 = True
+
+
class InstallTestCase(support.TempdirManager,
support.EnvironGuard,
support.LoggingSilencer,
@@ -166,6 +173,7 @@
cmd.user = 'user'
self.assertRaises(DistutilsOptionError, cmd.finalize_options)
+@unittest.skipIf(PREFIX_NOT_UTF8, 'prefix is not encodable to utf8')
def test_record(self):
install_dir = self.mkdtemp()
___
Python-bugs-list mailing list
Unsubscribe:
http://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com
[issue6011] python doesn't build if prefix contains non-ascii characters
Baptiste Carvello added the comment: Hello, the patch solves the bug for me as well (using locale "C", the filesystem encoding is utf-8). However, I do not understand why the patch checks that the shebang line decodes with both utf-8 and the file's encoding. The shebang line is only used by the kernel to locate the interpreter, so none of these should matter. Or have I misuderstood the patch? Cheers, Baptiste -- ___ Python tracker <http://bugs.python.org/issue6011> ___ ___ Python-bugs-list mailing list Unsubscribe: http://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com
[issue3187] os.listdir can return byte strings
Baptiste Carvello <[EMAIL PROTECTED]> added the comment: If, as I understand, it is the application's job to call listdir with bytes or unicode depending on the platform, it might be useful to have a function in the os module telling whether the filesystem is bytes of unicode-native. That way, the idiom for doing something to all files in a given directory would read: >>> directory="my_direcory" ... if not os.is_filesystem_unicode(): ... directory=directory.encode(sys.stdin.encoding) ... for name in os.listdir(directory): ... f=open(name) ... # do something ... f.close() and it would work on all platforms, even if one of the filenames is not in the locale's normal encoding. If this idiom is correct, it could also be useful to include it in the module documentation so that application authors know what they are supposed to do. -- nosy: +zegreek ___ Python tracker <[EMAIL PROTECTED]> <http://bugs.python.org/issue3187> ___ ___ Python-bugs-list mailing list Unsubscribe: http://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com
[issue6011] python doesn't build if prefix contains non-ascii characters
New submission from Baptiste Carvello : I have tried to build python (version 3.1 beta 1) on linux and install it to a non-standard prefix which contains non-ascii utf-8 characters (my locale being utf-8). The build directory's path is ascii-only. The exact configure line is given in the attached file 'tb.txt'. Then the 'make' command fails at the stage where python extensions are built, with the traceback displayed in file tb.txt (in short: UnicodeDecodeError: 'ascii' codec can't decode byte ... ). The problem is triggered when 'distutils.sysconfig.get_config_vars' tries to parse the Makefile. The Makefile is opened with 'distutils.text_file.TextFile', which in turns calls 'io.open' with no 'encoding' parameter. At this stage of the build, the 'locale' module is not available (due to '_collections' not being), so that 'locale.getprefferedencoding' cannot be called and the encoding falls back to ascii (a quick look to 'Modules/_io/textio.c' suggests that this fallback mechanism is already designed for being used at build time). The solution I propose would be to use 'sys.getfilesystemencoding' as a fallback first, as it is defined during build time on most systems: windows, mac and on posix if 'CODESET' exists in 'langinfo.h'. Given that in build routines, non-ascii characters are only likely to be encountered in filesystem paths, this seems a reasonable behavior. The attached patch 'text_file.diff' implements this strategy in 'distutils.text_file', and then calls 'io.open' with the appropriate 'encoding' parameter. It could be argued, however, that this new fallback is of general interest and should be implemented directly in 'Modules/_io/textio.c'. If you deem so, I could try to come up with a new patch. The attached patch solves the problem on my system, and does not introduce test failures (which is expected, as the new fallback should only make a difference at build time). Cheers, Baptiste -- assignee: tarek components: Distutils files: tb.txt messages: 87674 nosy: tarek, zegreek severity: normal status: open title: python doesn't build if prefix contains non-ascii characters type: behavior versions: Python 3.1 Added file: http://bugs.python.org/file13974/tb.txt ___ Python tracker <http://bugs.python.org/issue6011> ___ ___ Python-bugs-list mailing list Unsubscribe: http://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com
[issue6011] python doesn't build if prefix contains non-ascii characters
Baptiste Carvello added the comment: And here comes the patch -- keywords: +patch Added file: http://bugs.python.org/file13975/text_file.diff ___ Python tracker <http://bugs.python.org/issue6011> ___ ___ Python-bugs-list mailing list Unsubscribe: http://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com
[issue6011] python doesn't build if prefix contains non-ascii characters
Baptiste Carvello added the comment: OK, here is also the patch to 'Modules/_io/textio.c', as it is in fact quite trivial. Choose which one you prefer :-) Baptiste -- Added file: http://bugs.python.org/file13984/textio.diff ___ Python tracker <http://bugs.python.org/issue6011> ___ ___ Python-bugs-list mailing list Unsubscribe: http://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com
